Passport to Success — An Independent Evaluation of Social-Emotional Learning in English Primary Schools

Health Research
UK
2021-04-23
Active

Passport to Success — An Independent Evaluation of Social-Emotional Learning in English Primary Schools

Knowledge Gap

Children’s mental health is a growing public health concern in England. An estimated 16 per cent of children aged 5–16 live with mental health difficulties, and research shows that the majority of adult mental health conditions originate in childhood and adolescence. Early intervention — particularly in schools — is therefore a priority in British health policy.

Social-emotional learning (SEL) is one of the most widely used approaches. International meta-analyses show that such programmes have a positive average effect, and they are in widespread use in English schools. But the overarching research conceals important blind spots:

  • Who do the programmes work for? Effects vary across gender, socioeconomic background, ethnicity, and the presence of special educational needs.
  • How and why do they work? The mechanisms behind effects are poorly understood, making it difficult to improve or adapt programmes.
  • Do effects last? Most studies measure outcomes immediately after the intervention. What happens one or two years later?
  • Are programmes cost-effective? Schools have limited resources. The question of what a programme costs — in money and time — is rarely answered.
  • Have they been independently evaluated? Many programmes used in British schools have only been evaluated by their developers.

Passport: Skills for Life represented exactly such a gap. The programme was in use in English schools and had documentation from its developer — but no independent, randomised evaluation.

Executive Summary

This project conducted the first independent randomised controlled trial (RCT) of Passport: Skills for Life — a universal social-emotional learning programme for children aged 9–11, designed for use in English primary schools.

62 schools in Greater Manchester and surrounding areas participated. Over 2,400 children are included in the study. Half the schools delivered the Passport programme; the rest continued as normal. Children were assessed at three time points: at the start, at the end, and twelve months after completion.

Key finding:

Passport had no statistically significant effect on internalising symptoms (anxiety, depression) or any of the secondary outcome measures, at either post-intervention or one year later.

The economic analysis found that the programme was not cost-effective. The qualitative process evaluation revealed that teachers valued the comic book format and the opportunities for emotional reflection, but that the programme created challenges around differentiation and inclusion of pupils with special educational needs and disabilities (SEND).

Despite null findings on effectiveness, the study delivers substantial new knowledge: methodological, about implementation processes, and about the conditions for cost-effectiveness. The findings have been published in open-access, peer-reviewed journals, shared with schools, and used in dialogue with policy makers.

Method

The study used a two-arm parallel cluster randomised controlled design, in which schools — not individual children — were randomised to either an intervention or a control group. This is the strongest possible design for testing the effect of school-based programmes, equivalent to the gold standard in medical research.

Sample and Recruitment

62 primary schools were recruited from Greater Manchester and surrounding areas. A total of 2,425 pupils are included in the intention-to-treat (ITT) sample — that is, all who were included at baseline, regardless of actual programme completion.

Intervention

Schools in the intervention group implemented Passport: Skills for Life, a 10-week universal SEL programme for 9–11-year-olds, delivered by class teachers through weekly lessons based on comic books and structured activities.

Outcomes

  • Primary: Internalising symptoms (anxiety, depression) measured with the Strengths and Difficulties Questionnaire (SDQ)
  • Secondary: Emotion regulation, wellbeing, loneliness, bullying (as victim and perpetrator), and perceived peer support

Assessment Time Points

  • T1: Baseline (before intervention)
  • T2: Post-intervention (immediately after the programme)
  • T3: 12-month follow-up

Process Evaluation

A mixed-methods implementation and process evaluation supplemented the RCT, including teacher surveys, qualitative case studies, and classroom observations. Young people from the organisation Common Room contributed actively throughout the study.

Statistical Analysis

Analyses followed a pre-registered statistical plan. In addition to a standard ITT analysis, a multiverse CACE analysis (Complier Average Causal Effect) was conducted to examine how different definitions of “programme compliance” affect conclusions.

Findings

Main Findings

1. No effect on mental health or social functioning

Passport: Skills for Life had no statistically significant effect on internalising symptoms at either post-intervention (effect size d = −0.04) or at the 12-month follow-up. Equivalent null findings were observed across all secondary outcomes: emotion regulation, wellbeing, loneliness, bullying, and peer support.

This is a robust finding. The effect size is very small and statistically negligible. The study’s scale (over 2,400 children, 62 schools) provides sufficient statistical power to detect even moderate effects, had they existed.

2. Not cost-effective

The health economics analysis found that the programme is not cost-effective. The analysis also reveals that schools’ decision-making processes around programme adoption are multidimensional: they are driven not only by cost and evidence, but by values, capacity, and institutional identity.

Secondary Findings

3. Implementation is not the same as intention

The process evaluation identified a consistent pattern: complex instructional elements of the Passport programme were systematically simplified and shortened in classroom practice. A core components analysis linking the programme theory to actual delivery found that the elements theoretically designed to produce change were precisely those most often reduced or cut.

This is an important observation: null findings may mean that a programme does not work — but they may also mean that it was never delivered as intended.

4. Challenges around inclusion

Teachers reported that the programme struggled to differentiate adequately for pupils with special educational needs and disabilities (SEND). One of the submitted articles examines possible iatrogenic effects — the potential for the programme to have had a negative impact on SEND pupils.

Editorial note: This finding is under peer review and should not be included in external communications until publication is confirmed.

5. Methodological contribution: Multiverse CACE analysis

The study introduced a methodological approach with broad relevance: a multiverse analysis of complier average causal effects (CACE). The analysis systematically demonstrated how different choices of compliance definition affect conclusions about effectiveness — an important contribution to the methods literature.

6. Teachers’ perspectives

The qualitative evaluation showed that teachers consistently valued the comic book format and the structured opportunities for emotional reflection. Challenges were primarily linked to differentiation and to the programme placing greater demands on teachers than expected.

Conclusion

The study concludes that Passport: Skills for Life should not be recommended for wider implementation in English primary schools based on available evidence. The programme showed no effect on mental health or social functioning in a well-conducted, independent RCT with strong statistical power.

This is an important signal for policy makers: popularity and developer-funded documentation are not sufficient grounds for investing in school-based programmes. Independent evaluation is essential.

The study nonetheless contributes knowledge that extends well beyond the Passport programme itself:

  • It demonstrates that implementation quality is critical, and that even well-intentioned programmes can deteriorate in practice.
  • It documents that universal programmes can have different — and potentially negative — effects for vulnerable subgroups.
  • It introduces a new methodological standard for compliance analysis in RCTs.
  • It shows that school-based cost-benefit assessments are more complex than the health sector assumes.

For Kavli Trust, this project represents an investment in the kind of knowledge that is rarely produced elsewhere: independent, rigorous, and willing to deliver uncomfortable answers. It is precisely this type of research that makes it possible to distinguish between what works and what merely looks good.

Three PhD students have built their doctoral theses on data from the study. 11 scientific articles have been published, are under peer review, or are in progress. All participating schools have received individualised reports. The findings are actively communicated to policy makers in British education.

All publications from the project are available open access. Project OSF page: https://osf.io/8znvx/overview