N=1 trial website

I think that n=1 experiments can be scientific if done properly. There are a lot of anecdotes on the internet of people posting their results on probiotics and supplements, but its really hard from that data to determine what might work for a given person. Additionally, there are effects for which simple reviews (like on amazon) arent scientific. For example, given the research on probiotics and the minimum CFU to see a therapeutic effects, many of the probiotics on amazon don’t meet this requirement yet most of them have very high reviews and a high number of reviewers. Could this be placebo effect?

I have an idea for a website where i’d like to harness the data collected from n=1 experiments people are already doing while trying to remove the biases of placebo. Here are some random thoughts:

  1. Have a website where folks register their experiment (lets call it a trial hereafter) and others can critique the experiment from a statistical point of view. Maybe they need a longer washout period or should get bloodwork done before starting their experiment. It could be like a sort of stackoverflow interface. Trials that people are interested in could get upvoted or people could donate a few dollars to help the tester get the tests done. The goal of this stage would be to have well designed experiments.

  2. Everyone who does a trial has a profile page where they can record some metadata about themselves like sex, weight, height, fitness level, genetic profile, microbiome profiles, etc. This would help tease out phenotype issues whereby some probiotics (just an example) work for some but not for others. Someone on the internet could enter their metadata in and find trials that match their phenotype and browse results.

  3. Somehow karma is assigned to those who complete the trials, provide a well written writeup of results and provide their data. This is the part I am the least sure about how to incentivize. This encourages folks to share results and most importantly, complete the trial.

  4. I think the karma system would encourage others to read highly upvoted trials so they design their experiment properly so that they too can be upvoted. I think this gets to higher quality data. By posting the phenotype data, folks can find others who have a similar makeup and see how they might respond to the trial. You can generate statistics once many people do the trials to see trends. For example, say there is a probiotic company shipping from the northeast. Perhaps folks in the northeast react different to the probiotic than those on the west coast purely because the probiotic expires in the heat when shipped to the west coast. You could remove these effects and the data would be available in a way that the academic community could control for these in large trials.

  5. If you want to show off your visualization or statistical skills, there could be a section where you can post visualizations of trial results or show stats (and other trends) from several completed trials. Somehow you could provide karma to these folks too to encourage participation.

That’s all i have for now.

I think that a clear taxonomy is important so that such a database can be used to verify and repeat the results of others, indicate where more formal blinded and controlled studies need to be done, and can help create a framework to explain the results in a broader context. Patient by patient heterogeneity becomes more of an issue with individualized network, but common attributes of individual results will lead to medical advances.

There’s no shortage of places for people to share their experiments and get feedback (e.g. this forum). I’m not sure there are enough people doing experiments at the moment to get much benefit from karma points etc.

Having something like GoFundMe or Experiment.com for self-experiments would be nice. In addition to money, people and companies could contribute free (or discounted) lab tests, gadgets, help with data processing etc.

Not sure about the business model for such a service. The opposite (i.e. a service that helps companies who sell supplements etc recruit volunteers for specific experiments) might be easier to make work.

@IsaacGerg @madprime Perhaps OpenHumans.org can be a suitable platform for “harnessing data from n=1 experiments”?