Guides / service evaluation
Service evaluation: a practical guide
A service evaluation describes what a service is actually achieving — its activity, who it reaches, what changes and what using it is like. This guide covers how it differs from audit and research, how to design one around a decision, what to measure, and how to report findings without claiming more than the data supports.
Written and reviewed by Sidra Khan, Lead BiostatisticianA service evaluation describes what a service is currently achieving, without measuring against a pre-set standard and without trying to establish what the standard should be. It sits between clinical audit, which compares practice to an existing standard, and research, which generates new knowledge intended to apply beyond the service studied. Service evaluation normally falls outside research ethics approval, but the governance decision should be recorded before data collection begins.
Definition
What service evaluation is
A service evaluation asks what a service is actually achieving. It describes current practice and its results, without comparing them to a pre-set standard and without trying to establish what the standard ought to be.
That middle position is the whole point. Clinical audit measures against a standard that already exists. Research generates new knowledge about what works. Service evaluation sits between them: it tells an organisation what is happening now, so it can decide what to do next.
Audit asks "are we meeting the standard?". Research asks "what should the standard be?". Service evaluation asks "what is this service doing, and for whom?" — a question neither of the others answers.
Comparison
How it differs from audit and research
Getting this wrong is the most common governance error in NHS and local authority projects, because the three have different approval requirements.
| Service evaluation | Clinical audit | Research | |
|---|---|---|---|
| Question | What is this service achieving? | Are we meeting the standard? | What should the standard be? |
| Standard | None — describes what is | Compares against an existing one | Seeks to establish one |
| Who it applies to | This service only | This service only | Intended to generalise |
| Allocation | Never | Never | May allocate or randomise |
| Treatment | Normal care only | Normal care only | May depart from normal care |
| REC approval | Not normally required | Not normally required | Normally required |
| Typical output | Report to commissioners or the board | Audit report and action plan | Publication |
The practical test has two parts. Are you allocating anyone to anything? If yes, it is research. Is the answer meant to change practice beyond your own organisation? If yes, it is probably research. If neither, you are almost certainly in evaluation or audit territory.
The clinical audit guide covers the audit side in full.
Use cases
When a service evaluation is the right choice
Conversely, if a recognised standard already exists and the question is simply whether you meet it, run an audit. It is quicker, better understood, and the action that follows is clearer.
Design
Designing one that answers the question
Most weak evaluations fail at the first step: nobody wrote down the question. "Evaluate the service" is not a question, and a report answering it will be a description nobody acts on.
Who will read this, and what will they do differently depending on what it says? If nobody can answer, stop until they can.
"Is the service reaching the population it was commissioned for?" is answerable. "Is the service effective?" is not, without saying effective at what.
Compared to what — the service before, a similar service elsewhere, the population it was meant to serve? Without a comparison you can describe a level but never interpret it.
Routinely collected data is cheaper, more complete and less disruptive than new collection. Only collect new data where the routine data genuinely cannot answer the question.
Which patients, over what window, and out of how many eligible. Deciding this afterwards invites the suspicion that it was chosen to flatter.
Measurement
Choosing what to measure
A useful evaluation usually needs all four of these. Reporting only the first is the commonest shortcoming.
| Domain | What it tells you | Example measure |
|---|---|---|
| Activity | What the service did | Referrals received, appointments delivered, contacts made |
| Reach | Who it got to — and who it missed | Uptake by deprivation quintile, ethnicity, age band |
| Outcome | What changed for those people | Symptom scores, readmission, employment, wellbeing measure |
| Experience | What it was like to use | Structured feedback, free-text comments, complaints themes |
Reach is the one most often missed and the one commissioners ask about. A service can deliver its full activity target while systematically failing the people it was commissioned to help — and only a breakdown by population group will show it. Reporting totals alone conceals exactly the thing the evaluation exists to find.
Use a validated instrument where one exists for the thing you care about. A locally invented five-point scale cannot be compared to anything and will not survive scrutiny from a commissioner who has seen the standard measure used elsewhere.
Analysis
Analysing and reporting
Service evaluation analysis is mostly descriptive, and the discipline is in being honest about what description can and cannot support.
Always report denominators and uncertainty. "72% of users improved" needs the count behind it and an interval around it. Without both, the reader cannot tell a real finding from a small-sample artefact.
Break results down by group, not just in total. The headline figure is rarely the useful one. Differences by deprivation, ethnicity, age or referral route are what change commissioning decisions.
Be explicit about what you cannot claim. Without a comparison group, an improvement over time is not evidence the service caused it — people recover, seasons change, and the most unwell are referred first. Saying so plainly strengthens a report rather than weakening it, because the alternative is a reader noticing it for you.
If people enter a service because they are at their worst, some will improve regardless of what the service does. Any evaluation of a service with a severity threshold at entry has to acknowledge this, and ideally account for it. It is the single most common reason an evaluation overstates benefit.
Governance
Governance and approvals
Service evaluation does not normally require research ethics approval in the UK, but it is not approval-free.
Worked examples
Three worked examples
| Community mental health | Social prescribing | Virtual ward | |
|---|---|---|---|
| Decision it supports | Whether to extend the pilot | Whether to recommission | Whether to expand capacity |
| Activity | Referrals, contacts per user | Link worker sessions delivered | Patients managed, bed-days saved |
| Reach | Uptake by deprivation quintile | Referral source and ethnicity | Age and comorbidity distribution |
| Outcome | Change in a validated wellbeing score | GP attendance before and after | Readmission within 30 days |
| Experience | Structured feedback at discharge | Free-text coded thematically | Patient and carer interviews |
| Main threat to the conclusion | Regression to the mean — entry at crisis point | Selection: motivated users self-refer | Case mix: healthier patients selected |
Each has a threat to the conclusion that no amount of data collection removes. Naming it in the report is what distinguishes an evaluation a commissioner can rely on from one that simply reports a favourable number.
Pitfalls
Six mistakes that waste the effort
Answers
Frequently asked questions
What is a service evaluation?
A service evaluation describes what a service is currently achieving — its activity, who it reaches, what changes for those people and what using it is like — without measuring against a pre-set standard and without trying to establish what the standard should be. It answers 'what is this service doing?' rather than 'are we meeting the standard?'.
What is the difference between service evaluation and clinical audit?
Clinical audit compares practice against a standard that already exists and expects a change and a re-audit. Service evaluation has no such standard; it describes current practice and results so that a decision can be made. Neither normally requires research ethics approval.
What is the difference between service evaluation and research?
Research sets out to generate new knowledge intended to apply beyond the service studied, and may allocate people to different treatments. Service evaluation examines only the service in question, never allocates, and uses normal care. Research normally requires REC approval; service evaluation normally does not.
Does service evaluation need ethics approval?
Not normally in the UK. It still needs a recorded governance decision — use the HRA decision tool, keep the output, and have it confirmed by your R&D or governance lead before data collection. Identifiable data will usually also need a lawful basis and a DPIA.
What should a service evaluation measure?
Four domains: activity (what the service did), reach (who it got to and who it missed), outcome (what changed for them) and experience (what it was like to use). Reach is the one most often omitted and the one commissioners most often ask about.
Can a service evaluation prove the service worked?
Rarely on its own. Without a comparison group, an improvement over time could be the service, could be natural recovery, or could be regression to the mean where people enter at their worst. A good evaluation states this plainly rather than implying causation it cannot support.
Who should carry out a service evaluation?
Someone far enough from the service to report an unwelcome finding. Teams can and do evaluate their own services well, but where the result affects funding or continuation, independence is worth having — both for the quality of the analysis and for how the report is received.
Keep reading
Related tools and services
Clinical audit
The six-stage cycle, setting standards, sample size and closing the loop.
Free toolClinical audit sample size calculator
How many records you need, and whether a change would be detectable.
ServiceNHS data analysis
Independent statistical support for audit, evaluation and service review.