It depends on a number of factors. Ultimately, you probably want to use a concept called statistical power to calculate the relevant parameters for your experiment. A couple of considerations:
What effect size are you expecting?
Imagine your experiment has a huge effect on your measure (e.g. your bodyweight goes from 50kg -> 80kg). You will need a smaller sample size to confidently assert that your experiment caused a change
This article recommends using a Hedge’s g test for single-subject experiments. You could research how to calculate power for this test to help with your question!
Thanks for the reply!
So according to statistical power there are 3 determines of power. Significance level, effect size and sample size.
The problem is I’m trying to calculate the sample size (Amount of days I experiment), but I don’t know the effect size in advance.
Pretty much! An alternative was of thinking about it would be:
With N sample size, I’m able to detect a X change in (your measure) with Y% level of statistical significance
As part of your experiment design, you will know that it can only detect significant changes in effects of a certain size or larger. If you end up with less sample size than planned, you can still learn from the experience, just with less confidence (stat sig.) or less precision (detect larger effects only).
None of these statistical methods have anything to say about correctness; all they do is tell you how often you’d find a similar effect in random data sets…
Sample size is somewhat relevant for coin-flip experiments: You can randomly drink or skip coffee each day, for example, and measure some effect.
But many things (like neck exercises) are cumulative, so the more important question is how long do you need to stick with each treatment… This you need to ask a physiotherapist, not a statistician
