The TextNow Transition Programme aimed to improve the reading comprehension skills of pupils at the transition from primary to secondary school by encouraging engagement in, and enjoyment of, reading. The programme was delivered by Unitas, a national charity that helps young people access, participate, and progress in mainstream education and training.
Participating students received 20-minute one to one sessions with a volunteer coach each weekday for five weeks at the end of primary school and for a further 10 weeks at the start of secondary school. Children were expected to read independently for a further 20 minutes per day, and were rewarded for attendance with credits that could be used to buy books online.
The trial examined the impact of the programme on 501 pupils in 96 schools across England who had been identified as unlikely to achieve Level 4a or above by the end of Key Stage 2. Pupils who were not likely to gain at least Level 2 were not included in the trial.
The study was funded by the Education Endowment Foundation as one of 24 projects in a themed round on literacy catch-up at the primary-secondary transition. Projects funded within this round aimed to identify effective ways to support pupils not achieving Level 4 in English at the end of Key Stage 2. The project was one of four funded with a particular focus on reading for pleasure.
Testing the impact of a volunteer-led one to one reading programme.
Queen's University Belfast, Sheffield Hallam University
Organising your school
Language and literacy
The following conclusions summarise the project outcome
The trial has not provided any evidence that the TextNow Transition Programme improved reading comprehension or attitudes towards reading for pleasure over the transition from primary to secondary school.
On average, pupils who participated in the programme made slightly less progress than similar pupils who did not. However, this finding was not statistically significant, meaning that it could have occurred by chance.
The programme was found to have a differential effect for pupils eligible for free school meals compared to their peers. A small positive (but not significant) effect was found for pupils eligible for free school meals, while a negative (and statistically significant) effect was detected for pupils not eligible for free school meals. It is unclear why this differential effect was found.
Higher attendance at the 20-minute daily coaching sessions was found to have a positive impact on reading comprehension. However, this was only found to be statistically significant for attendance at sessions in secondary schools. Attendance of the coaching sessions was not found to have an impact on the secondary outcomes (liking reading and motivation to read).
The programme appeared to be more effective when coaches were highly trained, enthusiastic and committed, and when secondary schools worked closely with feeder primaries to coordinate all elements of the programme.
What is the impact?
On average, pupils participating in the programme made less progress than those who did not. The size of the difference was small, and can be envisaged as saying that pupils who participated in the programme made approximately one months less progress than those who did not. However, the finding was not statistically significant, meaning that it is not possible to conclude with confidence that the observed effect was due to the programme rather than chance.
The programme appeared to have a different impact on pupils eligible for and claiming free school meals compared to their peers. To look at this difference more closely, additional, separate analyses were conducted on pupils eligible for free school meals and their peers. This revealed a positive effect for pupils eligible for free school meals. However, this effect was not statistically significant, meaning that it could have occurred by chance. A negative effect equivalent to three months’ less progress was found for pupils not eligible for free school meals. The effect was found to be statistically significant, which means that is it unlikely to have occurred by chance.
The programme did not appear to have a differential effect according to gender or prior attainment.
In addition to attainment, pupils’ enjoyment of reading and motivation to read was assessed. No statistically significant difference was detected between participating pupils and those in the comparison group on either measure.
Overall attendance at sessions was good, but higher attendance rates at the 20-minute daily coaching sessions were found to have a positive impact on reading comprehension. However, this was only found to be statistically significant for attendance at sessions in secondary schools. The programme appeared to be more effective when secondary schools worked closely with feeder primaries to co-ordinate all elements of the programme. In addition, more effective coaches were highly trained, enthusiastic and committed.
|GROUP||NO. OF PUPILS||EFFECT SIZE (95% CONFIDENCE INTERVALS)||ESTIMATED MONTHS’ PROGRESS||IS THIS FINDING STATISTICALLY SIGNIFICANT?||EVIDENCE STRENGTH||COST|
|Intervention vs. Control||391||-0.06 (-0.22, +0.09)||-1 month||No|
|Intervention vs. Control (FSM only)||116||+ 0.18 (-0.13, +0.48)||+2 months||No||N/A|
|Intervention vs. Control (Non-FSM only)||275||-0.19 (-0.01, -0.36)||-3 months||Yes||N/A|
How secure is the finding?
Overall, the evaluation findings are judged to be of moderate security. This assessment takes into account a number of factors including the study’s design, size and the level of drop-out.
This evaluation was set up as an effectiveness trial. Effectiveness trials aim to test the intervention under realistic conditions in a large number of schools. A randomised controlled trial design was employed to compare outcomes of pupils receiving the intervention to similar pupils who did not.
This study had some limitations with regard to recruitment and attrition (e.g. where pupils dropped out of the intervention, or where test data for participating pupils was not available). 501 pupils were initially recruited compared to the desired sample size of 600 and of these, 391 provided full data for analysis of the primary outcome. However, despite this attrition, pre-test comparisons showed the comparison and intervention groups were well balanced. The study was large enough to detect an effect size of 0.2 (equivalent to approximately three months’ additional progress). This means that any impact at or above this level could have been detected with a high level of confidence.
The sample of schools in the study was broadly comparable with other English schools with above average levels of disadvantage and thus the results are somewhat generalisable beyond the immediate context.
Generally, the literature suggests there is potential in tutoring programmes for improving both academic and social and emotional outcomes. However, existing evidence also suggests that highly structured tutoring programmes provide the best effects on outcomes (i.e. where tutors are well trained, lessons focus on specific reading skills and sessions are timetabled systematically). This recommendation may be counter to the TextNow underpinning model of ‘choice – enjoyment – comprehension’. Furthermore, there is lack of systematic evidence in the wider academic literature for a pathway of causality between choice and comprehension.
The study provides important insights into the feasibility of the programme in terms of: its potential effects on outcomes; an estimate of these potential effect sizes (providing valuable information on sample sizes for future study); the impacts on different groups (particularly with regard to free school meal eligibility); and key facilitators and barriers to implementation (from the process evaluation and the exploratory analysis of dosage).
How much does it cost?
Based on estimates from Unitas, the direct cost that schools would be required to pay for TextNow programme is £112 per pupil. This estimate includes resources (£50 per pupil), salary costs (£46), administration and other (£16). The estimate assumes a minimum cohort size of 10 and that sufficient schools are in the programme to make it viable. In addition, to run the TextNow programme each school must appoint a coordinator to recruit and manage volunteer coaches, manage programme delivery and undertake quality assurance and reporting. The programme also requires volunteer coach time of 20 minutes per pupil per day for 75 days, plus time for initial training, preparation for coaching sessions and keeping records.
This estimate does not include estimates of the cost of co-ordinators, coaches and other resources in kind.