Tools / free calculator
Effective sample size calculator
Fifty thousand rows from three thousand people is not fifty thousand observations. When measurements come from the same person, ward or school they repeat information, and the sample is worth less than its row count. This works out the design effect, what your data is really worth, and how far out your confidence intervals are if you ignore it.
Knowing your effective sample size does not fix an analysis that assumed independence. The repair is a model that accounts for the clustering, a mixed effects model or GEE, and which of those you want depends on whether you care about individuals or about the population average.
Why clustering changes everything →In short
What is effective sample size?
The number of independent observations your data is genuinely worth. Divide your raw count by the design effect, which is 1 + (m − 1) × ICC, where m is the average number of observations per cluster. Ten measurements each from 100 people with an ICC of 0.5 gives 1,000 rows, a design effect of 5.5 and an effective sample size of 182. Analysing those 1,000 rows as though they were independent makes every confidence interval about 2.3 times narrower than it should be.
Common questions
Answers before you ask
What is effective sample size?
The number of independent observations your data is actually worth. When observations are clustered, for example several measurements from the same person or several patients from the same ward, they carry overlapping information. The effective sample size is the raw count divided by the design effect, and it is the figure your standard errors should be based on.
How do you calculate the design effect?
The design effect is 1 + (m − 1) × ICC, where m is the average number of observations per cluster and ICC is the intraclass correlation. With 10 observations per person and an ICC of 0.5, the design effect is 5.5, so 1,000 observations are worth about 182 independent ones.
What ICC should I assume if I do not know it?
Estimate it from your own data where you can, since it varies enormously by outcome. As a rough guide, repeated self-reported symptom or mood measures from the same person often sit between 0.4 and 0.7, patient outcomes within a GP practice or ward typically fall between 0.01 and 0.1, and pupils within a school commonly sit between 0.1 and 0.2. Report the value you used and test how sensitive your conclusions are to it.
Why does my effective sample size stop growing?
Because there is a hard ceiling. As the number of observations per cluster rises, the effective sample size approaches the number of clusters divided by the ICC and never passes it. With 100 participants and an ICC of 0.2 you cannot exceed 500 effective observations, however many measurements each person provides. More clusters raise the ceiling; more observations per cluster only approach it.
What happens if I ignore clustering?
Your standard errors come out too small by a factor of the square root of the design effect, your confidence intervals too narrow by the same factor, and your p-values too small. A design effect of 9 means intervals three times narrower than they should be, which turns ordinary noise into apparently significant findings. It is the most common reason an observational analysis fails statistical peer review.
Does this apply to repeated measures as well as clusters?
Yes. The arithmetic is the same whether the cluster is a ward, a school, a household or a single person measured many times. For app, wearable or diary data the participant is the cluster and each entry is an observation within it.
Is my data stored?
No. The calculator runs entirely in your browser. Nothing you type is transmitted or stored anywhere, and there is no account.
Keep reading
Related
Clustered data and the unit of analysis
Why repeated measures shrink your sample, and how to choose the right unit.
Sample size calculator
The standard calculation for independent observations, with the written justification.
Independent statistical review
A fixed-price check of analysis already done, before it reaches a journal or a board.