Guides / sampling
Sampling methods
How you select people decides what your results mean, and no amount of analysis afterwards can repair a bad selection. This guide covers the probability methods and the non-probability ones, works through stratified allocation with real arithmetic, shows what cluster sampling costs you in precision, and is honest about when stratifying buys almost nothing.
They divide into two families. Probability methods give every member of the population a known, non-zero chance of selection: simple random, systematic, stratified, cluster and multistage. Because the selection probability is known, you can calculate how far your estimate is likely to be from the truth. Non-probability methods, including convenience, purposive, quota and snowball sampling, do not. They are often the only practical option and are entirely legitimate, but they cannot support a confidence interval that means what it appears to mean.
Before any method
The sampling frame comes first
The frame is the list you sample from. A patient register, a staff database, an electoral roll, a customer file. Every property of your sample is inherited from it, which is why the frame deserves more attention than the method.
Your conclusions apply to the frame, not to the population you had in mind. State both and describe the gap. "All adults in the borough" and "all adults registered with a GP in the borough" are not the same group, and the people in the difference are usually the ones the study most wanted to reach.
Known selection probability
The probability methods
| Method | How it works | Use when | Cost |
|---|---|---|---|
| Simple random | Every member equally likely, drawn at random | You have a complete frame and the population is reachable | Can be expensive and geographically scattered |
| Systematic | Every kth member after a random start | The frame is ordered and long | Dangerous if the list has a cycle matching k |
| Stratified | Split into groups, sample within each | Groups differ on the outcome and you know who is in which | Needs the grouping variable on the frame |
| Cluster | Sample whole groups, measure everyone inside | The population is naturally grouped and travel is costly | Substantial loss of precision |
| Multistage | Clusters, then units within them | National surveys | Complex weighting and variance estimation |
Systematic sampling, and its trap
The interval is the population divided by the sample.
k = N / n = 10,000 / 600 = 16.67, so k = 17
Pick a random start between 1 and 17, then take every 17th record. It is quick, it needs no random number generator at the point of selection, and in practice it is often easier to audit than a random draw because anyone can reproduce it from the start value.
It fails in one specific way. Taking every 10th shift from a rota where shifts cycle weekly will select the same weekday every time. The method is sound; the problem is an interval that resonates with a pattern already in the list. Check how the frame is ordered before choosing k. If the order is arbitrary, systematic sampling is a perfectly good substitute for simple random. If the order relates to the outcome, say a patient list sorted by admission date, it behaves like an implicit stratification and usually improves precision slightly.
The finite population correction
Standard error formulas assume you are drawing from an effectively infinite population. When your sample is a large fraction of a small population, that assumption is pessimistic and you are overstating your uncertainty.
fpc = √((N − n) / (N − 1))
Multiply the standard error by this factor.
| Population | Sample | Fraction | Correction factor |
|---|---|---|---|
| 10,000 | 600 | 6% | 0.970, a 3% reduction |
| 400 | 200 | 50% | 0.708, a 29% reduction |
At 6% it is barely worth applying. At 50% it is substantial and ignoring it means reporting intervals almost a third wider than they should be. The usual rule is to apply it once the sample exceeds about 5% of the population, which happens more often than people expect in audits of single departments and surveys of small professional bodies.
Unknown selection probability
The non-probability methods
| Method | How it works | Legitimate use |
|---|---|---|
| Convenience | Whoever is available | Piloting an instrument, early exploratory work |
| Purposive | Deliberately chosen for a characteristic | Qualitative work where you want information-rich cases, not representativeness |
| Quota | Fill preset counts per subgroup, selection within them non-random | Market research against time and budget limits |
| Snowball | Participants recruit others | Hidden or hard-to-reach populations with no frame |
None of these is a lesser method. Purposive sampling is the correct choice for most qualitative research, where maximum variation or extreme cases tell you more than a random draw would. Snowball sampling may be the only way to reach a population that no list describes.
A confidence interval assumes a known probability of selection. Without it the interval still computes and still prints, but it no longer describes how far your estimate might be from the population value. Report descriptive statistics and be explicit that inference to a wider population is not supported. A quota sample of 1,000 is not a random sample of 1,000, however similar the demographic profile looks.
Quota sampling is the one most often mistaken for probability sampling, because matching the population on age and sex makes it feel representative. It controls the variables you quota on and nothing else, which is the same limitation that applies to weighting after the fact.
The four purposive variants worth naming
| Variant | Select for | Answers |
|---|---|---|
| Maximum variation | The widest spread of the characteristic | What is common across very different cases |
| Homogeneous | A narrowly defined shared position | Depth within one group, typical of focus groups |
| Extreme or deviant case | The outliers | What happens at the edges, and why |
| Critical case | The case where the mechanism is clearest | If it does not hold here, it holds nowhere |
Naming the variant in a methods chapter is worth doing. "Purposive sampling" alone reads as a label; "maximum variation purposive sampling, to capture the range of departmental practice" reads as a decision. Examiners notice the difference.
How many for a qualitative study
There is no calculation, and anyone who offers you one is overselling. The working answer is information power: enough cases that further interviews stop producing new codes. Commonly cited ranges are 6 to 12 for a narrowly defined homogeneous group and 20 to 30 for a broader interview study, but these are conventions rather than thresholds.
Say how you judged it. "Recruitment continued until two consecutive interviews produced no new codes, which occurred at 14" is a defensible sentence. "Saturation was reached" on its own is not, because it describes a conclusion without describing how you arrived at it.
Worked example one
Stratified allocation
A survey of 10,000 businesses, sample of 600. Three size strata, and the outcome varies by size.
| Stratum | N | Expected p | Proportional | Neyman |
|---|---|---|---|---|
| Small | 6,000 | 0.22 | 360 | 334 |
| Medium | 3,000 | 0.45 | 180 | 200 |
| Large | 1,000 | 0.60 | 60 | 66 |
Step 1. Proportional allocation
Each stratum gets a share matching its share of the population.
Small: 600 × 6,000 / 10,000 = 360
Step 2. Neyman allocation
Sample more heavily where there is more variability. For a proportion, the standard deviation is the square root of p(1−p), which is largest near 0.5.
nₕ ∝ Nₕ × √(pₕ(1 − pₕ))
Medium and large businesses have proportions closer to 0.5, so they are more variable and get a slightly larger share.
Step 3. Compare the two
SE under proportional allocation = 1.831 points
SE under Neyman allocation = 1.824 points
Neyman is 0.8% more efficient. That is almost nothing, and saying so is more useful than presenting Neyman as a technique worth the trouble. It pays off when strata differ sharply in variability, and here they do not differ much.
Step 4. What stratification bought at all
A simple random sample of the same 600 gives an SE of 1.915 points. Proportional stratification gives 1.831, an improvement of 4.4%.
Modest, and typical. Stratification pays when the strata differ substantially on the outcome. Here they differ moderately, so the gain is real but small. Its other benefit is often more valuable: it guarantees you will have enough large businesses to report on separately, which a simple random sample does not.
Worked example two
What clustering costs you
Cluster sampling saves money by concentrating fieldwork. It is paid for in precision, and the bill is larger than most people expect.
Take the same 600 respondents, now reached by sampling whole workplaces.
| Per cluster | ICC | Design effect | Effective sample |
|---|---|---|---|
| 10 | 0.05 | 1.45 | 414 |
| 25 | 0.05 | 2.20 | 273 |
| 10 | 0.20 | 2.80 | 214 |
| 25 | 0.20 | 5.80 | 103 |
design effect = 1 + (m − 1) × ICC, and effective n = 600 / design effect
Reading the bottom row
Twenty-five people from each of 24 workplaces, with an intracluster correlation of 0.20, gives 600 interviews worth about 103 independent observations. You collected six times more data than you can use.
The lever that matters
Look at the first and third rows. The ICC quadrupled and the design effect nearly doubled. Now compare rows one and two: holding ICC fixed and raising cluster size from 10 to 25 also inflates it sharply. More clusters with fewer people in each is almost always better than fewer clusters with more, for the same total sample and the same cost per interview.
If a power calculation says 400 and you intend to cluster, you need 400 multiplied by the design effect. At an ICC of 0.05 with 10 per cluster that is 580. Discovering this after fieldwork is not recoverable, which makes the design effect one of the few calculations that must be done first.
The same arithmetic governs the analysis, not just the design. Clustered data analysed as if independent produces standard errors that are too small, which is covered in the guide to clustered data and the unit of analysis.
Decision
Choosing between them
| Situation | Choose |
|---|---|
| Complete frame, population reachable remotely | Simple random, or systematic if the list is long and unordered |
| Subgroups differ on the outcome and must be reported separately | Stratified, proportional unless variability differs sharply |
| Population is geographically dispersed and visits are costly | Cluster, with many small clusters and an inflated sample |
| No frame exists and the population is hidden | Snowball, reported as non-probability |
| Qualitative study seeking depth | Purposive, chosen for information richness |
| Fixed budget, demographic balance required, no frame | Quota, reported honestly as non-probability |
Most real studies combine them. A national survey might stratify regions, cluster within them, then sample systematically from each cluster. That is multistage sampling, and its variance calculation has to account for every stage.
What non-response does to a good design
A well-chosen sample is still only a plan. Response rates almost never come back flat across strata, and the differential is what causes the damage.
Return to the business survey. Suppose the response rates differ by size, as they usually do.
| Stratum | Sampled | Responded | Rate | Design weight |
|---|---|---|---|---|
| Small | 360 | 198 | 55% | 1.127 |
| Medium | 180 | 126 | 70% | 0.886 |
| Large | 60 | 48 | 80% | 0.775 |
372 responses from 600. Now compute the headline figure two ways.
unweighted across respondents = 0.3469
weighted back to population shares = 0.3270
Two percentage points, in the direction of the group that answered most readily. Large businesses were 10% of the population but 13% of the respondents, and they had the highest value on the outcome, so the raw figure drifts upward. The weights above simply restore each stratum to its real share of the population.
Weighting repairs who is in your data. It cannot repair the possibility that non-responders within a stratum differ from responders in the same stratum. A small business that ignored the survey because it is struggling is not represented by a small business that replied. Report the response rate by stratum, state that you weighted, and do not claim the weighting removed non-response bias. The only real defence is a high response rate, which is a fieldwork problem rather than a statistical one.
Doing it
In R, SPSS and Stata
R
Drawing the sample, then analysing it with the design respected.
set.seed(2026)
srs <- frame[sample(nrow(frame), 600), ]
strat <- frame %>% group_by(size) %>% slice_sample(n = c(360, 180, 60)[cur_group_id()])
The analysis is where it matters. A stratified or clustered sample analysed with ordinary commands gives wrong standard errors.
library(survey)
d <- svydesign(ids = ~workplace, strata = ~size, weights = ~wt, data = samp)
svymean(~outcome, d)
ids declares the clusters and strata the strata. Omit ids on clustered data and the intervals come out far too narrow.
SPSS
Drawing a simple random sample is built in.
SAMPLE 600 FROM 10000.
For a stratified draw, sort and use SELECT IF within strata, or the Complex Samples module's sampling wizard. Analysis of a complex design needs that module: CSPLAN ANALYSIS to define the design, then CSDESCRIPTIVES or CSGLM. Ordinary FREQUENCIES on a clustered sample will understate the uncertainty.
Stata
Stata handles survey designs best of the three.
sample 6
svyset workplace [pweight = wt], strata(size) fpc(Nh)
svy: mean outcome
svy: proportion outcome, over(region)
svyset is declared once and every subsequent svy: command respects it. The fpc() option applies the finite population correction, which matters when you have sampled a large fraction of a small population and will otherwise overstate your uncertainty.
Any random draw should be reproducible. Set a seed, write it in the methods, and the exact sample can be regenerated if anyone asks how those particular units were chosen. It takes one line and it answers a question that is awkward to answer any other way.
Avoid these
Six mistakes that cost marks
| Mistake | Why it matters |
|---|---|
| Calling a convenience sample random | "Randomly approached shoppers" is convenience sampling. Random has a precise meaning and examiners check it. |
| Ignoring the design in the analysis | A clustered sample analysed as independent gives intervals that can be half the correct width. |
| Not inflating for the design effect | 400 required with clustering at ICC 0.05 and 10 per cluster means recruiting 580. This cannot be fixed afterwards. |
| Systematic sampling into a cycle | Every 10th shift on a weekly rota selects the same weekday every time. |
| Treating quota as probability | Matching on age and sex controls age and sex. Nothing else. |
| Describing the frame as the population | Conclusions apply to the list you sampled from. Name the gap between it and the population you meant. |
The second one is the most expensive, because it is invisible. Nothing in the output warns you. A clustered sample analysed with ordinary commands produces a p value, a confidence interval and a tidy table, all of which look exactly like correct output and are simply too narrow. Findings reported that way have been retracted.
The third is the most final. Every other item on this list can be addressed in the write-up by describing the limitation honestly. An under-powered clustered study cannot, because the data does not exist and fieldwork has ended.
A power calculation returning 400 means recruiting 580 once clustering is accounted for, and that is not something you can fix after fieldwork. Send the design and you get the target, the allocation, and an analysis plan that matches how you sampled.
See power analysis and sample size
Elaine HalliburtonProfessor of Applied StatisticsWriting it up
How to report it
For non-probability designs add one sentence placing the limit on your own claims, for example: "Participants were recruited purposively for maximum variation. Findings describe this sample and are not generalisable to the wider population."
Common questions
Frequently asked questions
What is the difference between probability and non-probability sampling?
Probability sampling gives every member of the population a known, non-zero chance of selection, which is what allows a confidence interval to describe how far your estimate might be from the truth. Non-probability sampling does not, so the interval still computes but no longer means what it appears to mean.
What is stratified sampling?
Dividing the population into groups that differ on the outcome, then sampling within each. It guarantees representation of every group and usually improves precision, though often modestly: in a worked example it reduced the standard error by 4.4% against a simple random sample of the same size.
Should I use proportional or Neyman allocation?
Proportional unless the strata differ sharply in how variable the outcome is. Neyman samples more heavily where there is more variation, but when variability is similar across strata the gain is tiny; in the worked example it was 0.8%, which rarely justifies the extra complexity.
What is purposive sampling?
Deliberately selecting participants for a characteristic rather than at random, usually to obtain information-rich cases. It is the standard and correct approach in qualitative research, where maximum variation or extreme cases tell you more than a random draw. It does not support generalisation to a wider population and should not claim to.
Why does cluster sampling need a larger sample?
People within a cluster resemble each other, so each additional person in the same cluster adds less new information. The design effect is 1 + (m − 1) × ICC. With 25 per cluster and an ICC of 0.20 the design effect is 5.80, so 600 interviews are worth about 103 independent observations.
Is snowball sampling acceptable in academic research?
Yes, where no sampling frame exists, which is common for hidden or stigmatised populations. It is a recognised method with a substantial literature. Report it as non-probability, describe how the chains started and how long they ran, and do not attach inferential claims about the wider population.
How large should my sample be?
That is a power question rather than a sampling one, and it depends on the effect you need to detect and the precision you need. Whatever the calculation returns, inflate it for the design effect if you are clustering, and again for expected non-response.
Related services
If the design has to survive review
Power analysis & sample size
A priori power analysis with the justification an ethics committee will accept.
From £180Statistical consultancy
Design, analysis and reporting for charities, universities, the NHS and business.
From £950Independent statistical review
A fixed-price check of analysis already done, before it reaches a journal or a board.
Fixed quoteEvery piece of work is done and signed by a named statistician. Send the data or describe the project and you get a price and a deadline, usually within one working day.