ThinkForward’s Social Impact Bond was one of the first in the world to successfully complete. Thanks to this investment we not only transforming the lives of over a thousand of the most disengaged young people in London but investors in ThinkForward’s programme also reaped a financial return.

Since completing the bond we have identified some key lessons highlighted in Philanthropy Impact’s December 2016 issue. To view the full article click here or read the below.

In the UK and continental Europe, SIBs are being used primarily to experiment with new interventions, such as the programmes commissioned by the UK’s Department for Work and Pensions Innovation Fund to support disadvantaged young people. In the US, however, ‘pay for success’ initiatives are generally used to scale existing evidence-based programmes. Following the launch of the first SIB in the UK in 2010, much was written to describe the mechanics, structures and challenges of implementing new deals. Over 40 SIBs have been launched since then – in the UK, US, Australia, Canada, Netherlands, Belgium, Portugal and Germany. As investors crowd-in and the first round of SIBs have been completed, the buzz is less about how to design them and more about whether they have worked. What are the lessons from the first SIB deals? What should investors and advisors know about measuring success and risks within SIBs? As investors crowd-in and the first round of SIBs have been completed, the buzz is less about how to design them and more about whether they have worked. Based on our experience developing SIBs, we have identified some key lessons. Firstly, investors should be aware of the risk that SIBs may not reach completion as planned. The first SIB in the world was launched in the UK in 2010 with the One Service in Peterborough Prison. It was designed to fill a gap in the UK criminal justice system by supporting short-term prisoners and preventing their reoffending. Social investors invested £5m in a project meant to run for seven years. But in 2014 the Ministry of Justice announced a decision to restructure the provision of probation services nationally and the Peterborough pilot was brought to an early close.

The New York Times (Wall St. Money Meets Social Policy at Rikers Island) reported a similar outcome for the first SIB in the US, launched in 2012 to support high-risk adolescents detained at Rikers Island, the New York City jail. In this case, it wasn’t the change of heart of policymakers but implementation issues: the control group fell apart as wardens ‘could not separate teenagers who were to participate in a course of cognitive behavioural therapy from those who were not supposed to attend’, and then there was the failure to recruit the teachers required to carry out the therapeutic programme after the city’s Education Department pulled out. Was this a total failure? Investors will certainly be disappointed: Goldman Sachs had invested $9.6m with a 75% guarantee from Bloomberg Philanthropies. But from the perspective of the City of New York, it wasn’t a disaster: the city had tested a new intervention at scale with the benefit of a rigorous evaluation and then walked away without spending any taxpayer money. So another key lesson for investors is the importance of focusing part of their due diligence around their partner’s record of, and ability to, recruit beneficiaries and frontline workers. At ThinkForward SIB, our programme in London helps more than 1,000 at risk young people aged 14 to 19 years to complete their education and transition into work. The challenge of recruiting beneficiaries was partially solved by working inside schools. ThinkForward SIB carefully selected partner schools where teachers were committed to refer, and give access, to students. And when it came to staff, ThinkForward investors took a conscious decision to pay a competitive salary to frontline workers in order to secure the most talented applicants. When SIBs reach maturity and investors receive payments linked to outcomes, a further set of questions arises: how to compare results across SIB deals? What is the relationship between financial return and social impact? The promise of SIBs (and impact investing in general) is to align financial return with social impact, but one does not always imply the other.

Last year the New York Times (Success Metrics Questioned in School Program Funded by Goldman) reported how early-education experts questioned the validity of the impact results associated with a Utah pre-school programme designed to help 109 ‘atrisk’ kindergarten children avoid special education. Goldman Sachs reported a $260,000 payout from the SIB but, according to the NYT, critics argued that the programme’s unusually high success rate (99%) was based on a ‘faulty assumption that many of the children in the programme would have needed special education without the preschool, despite there being little evidence or previous research to indicate that this was the case’.This issue arises when validation of outcomes is not dependent on a control group or counterfactual assessment and there is the possibility of ‘creaming’, that is, enrolling participants who are easiest to work with and to achieve outcomes. As not all SIBs will run a control group – it is not always practical, ethical or cost-effective – impact investors must appreciate the limitations of linking the achievement of an outcome (say avoiding special education) to impact (the degree to which the outcome resulted from the intervention). In the case of the ThinkForward SIB, while the payments were not linked to a counterfactual, we worked with the Education Endowment Foundation to carry out a pilot randomised controlled trial (RCT) with randomisation at school and pupil level. This evaluation, which is separate from the SIB work, aims to identify quantitative evidence that the ThinkForward intervention is having a positive impact on education attainment and other outcomes. A clear relationship between financial return and impact is also complicated by some pricing strategies.

Sometimes commissioners link payments to a set of outcomes. In the case of the DWP Innovation Fund, for example, its rate card set a price for ten outcomes per participant. DWP would pay for improved attendance at school (up to £1,400 per participant), entry into employment (up to £3,500 per participant) and for other outcomes linked to improved employability. The outcomes in the rate card are important, but not all are equally relevant. A sophisticated impact investor will want to know if the financial return is driven by less important outcomes, say passing one GCSE at school (DWP pays up to £1,100 per participant), or is the result of achieving more socially meaningfully outcomes such as securing employment for 26 weeks (DWP pays up to £2,000 per participant). Another challenge with some payment structures is that they pay for the same outcomes, even if the programmes or the people the programmes target are very different. For example, a 12-week employability programme may be effective with young people who are work ready, but those further from the labour market may require longer interventions to achieve the same employability outcomes. The pricing structures don’t always recognise these nuances. Over time we hope that commissioners’ pricing will be better targeted, reflecting different levels of support needed to get different groups of people to the same outcome. Until then, investors who want to support the hardest to reach may need to accept lower financial returns. Social impact bonds have opened up government funding streams to delivery partners that would otherwise not have been able to access payment by outcomes financing, and nudged up expectations of managing to impact. However, the challenges reviewed above suggest that SIBs should not be seen as the perfect tool for measuring social impact. Ultimately, investors, commissioners and delivery organisations must define and measure success, and hold themselves accountable to the objectives laid in their own organisational missions.