- July 21, 2023
The Department for Education has just published the Final Report from a longitudinal evaluation of three SIBs that aimed to improve outcomes for young people leaving local authority care, with a particular emphasis on employment, education and training (EET) outcomes. ATQ Director Neil Stanworth was part of the evaluation team, which has been evaluating the three projects (In South London, Greater Bristol and Sheffield) since 2018.
Commenting on the report, Neil said “We’re really pleased that this report has finally been published, after a number of delays. It shows how social outcomes contracts can make a real difference to the lives of some very vulnerable young people, and also how social investment funding enables real flexibility in the way complex support is delivered.”
- June 29, 2022
Social Outcomes Contracts (or Social Impact Bonds – SIBs) have been a big part of ATQ’s work since we formed almost exactly 10 years ago, in August 2012. We are proud to have supported many organisations to set up and implement contracts, and in recent years have become heavily involved in research and evaluation of such contracts through work such as this for the Department for Culture, Media and Sport.
Last week saw the publication of what might be the most high-profile piece of research we have ever done, into the value created by Social Outcomes Contracts (SOCs) in the UK since they first started more than a decade ago. Our report was part of broader work by Big Society Capital (BSC) to reflect on and celebrate the growth of SOCs over the last ten years.
The statistic that has gained most attention from our report, and was mentioned at Prime Minister’s Questions last week, is that every pound spent on SOCs in outcome payments has generated more that £10 of value. In other words, and in the technical language of cost benefit analysis, the Benefit Cost Ratio was 10.2. Even if you accept that some of this value would have happened anyway (as it probably would in some contracts, but by no means all) we are still looking at a return to government and society of many times the initial outlay.
As one of many organisations that have been grappling with SOCs and SIBs, and whether they are value for money for many years, we hope this report will provoke further debate about whether and when such contracts are a useful tool in the commissioner’s armoury. Indeed it looks like that debate has already started, and we are happy to be part of it. For now, I would offer three observations on this piece of work.
First, our report is and always was intended to be entirely factual. It takes data on the outcomes that have been achieved by SOCs – as measured and validated by those contracts – and attempts to put a value on them based on the improvements they make to people’s lives. This is not an exact science, and so we were deliberately cautious – see below; but we are making no judgement on the efficacy of SOCs, and nor are we comparing the performance of projects with each other or with other types of contract. We have recently made a major contribution to another research report – the third update on the evaluation of the Commissioning Better Outcomes Fund – which does explore the strengths and weaknesses of SOCs in some depth, but this report does not do that.
Second, I am slightly puzzled by suggestions in some quarters that our value estimates are too big, and therefore somehow less credible. Puzzled because we were deliberately conservative in our assumptions, as we explain in some detail in our report. This is our usual practice when undertaking cost benefit analysis because we know that inflating ‘savings’ – either consciously or unconsciously – is self-defeating. Our clients sometimes challenge our caution, and ask us to make more optimistic assumptions, but we tend to resist. In this case we were even more cautious than usual, consistently using low estimates of unit costs saved when larger, well-evidenced estimates were available; assuming no sustainment of outcomes such as periods of employment; and leaving many outcomes which potentially have value out of our analysis altogether. The truth is that whenever we have done this type of exercise, the estimates we produce tend to be large because the costs of adverse outcomes – children in care, young people long-term NEET, older people needing hospitalisation for conditions that can be managed better at home – are themselves large, and much greater than many realise.
Finally, both the work we have done and the wider analysis by BSC seem to me to confirm that, whatever their other benefits and defects, these types of contract are delivering a pretty good ‘bang for buck’ and have leveraged a lot of value for not much spending. This is not just because of the benefit cost ratios outlined above, but also due to two other factors. First, according to BSC’s figures these results have been achieved with the injection of around £71m in social investment. This is much less than many expected, and there has been criticism in some quarters that investment in SOCs has fallen well short of projections. But a ’glass half full’ view would be that this is a good thing. The amount of working capital needed to smooth the wheels of these contracts (mainly to make the payment by results mechanism work for social sector organisations) has been much less than the total contract values, and this makes them pretty efficient.
The second point is that government and other bodies (notably the National Lottery Community Fund) have funded a high proportion of outcome payments made through SOCs, leading to criticism that such contracts only exist because of such funding. This is likely true (there have been very few contracts implemented without some form of subsidy from one or other so-called Outcomes Funds) but it does look like a reasonable investment, and it looks like a very good investment when one considers the particular case of the Commissioning Better Outcomes Fund and Life Chances Fund, which typically cover 20-30% of total outcome payments – with the rest coming from local commissioners. So the effective leverage of these contracts is maybe 3-4 times what it would be if they were wholly funded by government or the Community Fund.
And since many commissioners and providers have said that they would not have put up their own money without this pump-prime funding, what’s not to like?
Sponsored Content
- July 21, 2015
Below is a blog ATQ wrote for the Commissioning Better Outcomes (CBO) Evaluation site. ATQ is working with Ecorys on the CBO evaluation and co-wrote the Deep Dive report referred to.
How do we improve the health of individuals, in a way that prevents health issues arising in the first place or getting worse? And how can Social Impact Bonds (SIBs) be developed in health and other areas where the benefits are longer term, with less immediate results and savings to the public purse?
These questions were at the heart of an event held last Friday (10 July) in Birmingham. It brought together a range of stakeholders with a strong interest in improving the nation’s health, especially through the use of SIBs.
The event was hosted by the Big Lottery Fund, whose Commissioning Better Outcomes (CBO) Fund looks set to play a major role in encouraging more SIBs in the next few years. It was held to discuss a new SIB which breaks new ground in health and has just been examined in depth as part of the evaluation of…
View original post 701 more words
- October 1, 2013
Last week I went to the Local Government Association’s headquarters in Smith Square for the launch of the Commissioning Better Outcomes Fund. This is a new £40 million fund from the BIG Lottery, and aims to encourage and support commissioners (and especially local authorities) who want to explore the use of Social Impact Bonds (SIBs) to achieve better social outcomes and ultimately reduce costs.
The new fund is intended to be complementary to the Cabinet Office’s existing £20m Social Outcomes Fund, which means there will be around £60 million available to support new SIBs (less a bit that the Cabinet Office has already committed to existing projects). The two Funds have a slightly different focus, but essentially aim to provide some part funding for SIBs in order to improve their financial viability – especially if the commissioner does not think they can cash enough savings from the SIB to make it self-funding.
What is exciting about the new fund is that part of it (between £3m and £4m) has been earmarked for development grants through which commissioners can buy in the specialist advice they might need to get a SIB of the ground – for example, to carry out feasibility work. There will be a common expression of interest process for the two funds, and if an EoI is accepted a commissioner can then apply for a grant to do the more detailed work needed to submit a full application to one of the Funds – and hopefully get a significant contribution to the funding of the project as a whole.
Having been involved in the development of SIBs and similar social investment backed initiatives for around three years now, we think this is a potentially game-changing development. Many commissioners would like to explore the use of SIBs, but in the current financial climate, the cost of initial feasibility work and specialist advice and support is a significant barrier.
The only snag is that, as a first step, commissioners will need to have done some prefeasibility work and got to the point where they have a proposition that will get through the expression of interest process. The good news is that many local authorities and other commissioners already have ideas for improving outcomes in such areas as looked after children, family intervention, adult social care and the new challenge of preventative health interventions. The initial work to turn such ideas into credible propositions should not be too onerous, in our experience, and advisors such as us are available to help. With the opportunity to access development grants downstream, the time is right to turn concepts into reality.
If you would like to know more about how ATQ could help you make an expression of interest to the new funds, potentially at no cost to you, please contact us via our website here.
- August 6, 2013
Some advocates and supporters of social impact bonds appear to have shifted their position on the importance of cashable savings. My colleague (Eileen Robinson) attended a seminar a few weeks ago where a spokesperson from Social Finance argue strongly that SIBs did not need to be underpinned by cashable savings. Now an interesting piece by Adrian Brown has argued that a strong focus on cashable savings is holding back the development of SIBs and the social investment market generally. He argues that SIBs should be mainly about innovation, not about financial returns.
Simply put, the case for identifying cashable savings is that they enable SIBs to be self funding – effectively an extension of the invest to save principle. This was always one of the main attractions of the SIB approach, especially for a government looking for solutions to unprecedented financial pressure on public services. The case against – eloquently put by Adrian Brown – is that an over-emphasis on the cashability of savings may restrict both the outcomes that might be funded through a SIB, and the total funding made available. In essence, commissioners may look for projects simply because they enable savings to be cashed, rather than because they achieve the most important outcomes, or enable them to test new ways of doing things.
I have no problem with the argument that we should not obsess about cashable savings, or that we should focus mainly on the value of the outcomes to be achieved. However, I am concerned that we risk throwing the baby out with the bathwater if we start to argue that cashable savings are not important. Here are three reasons why I think they should remain at the centre of thinking.
First, while some commissioners may want to use SIBs solely to test innovation, most cannot afford to do this. Their main motivation is to find ways to fund outcomes that would otherwise not be funded – and usually to fund preventative or early intervention which reduces the need for (more expensive) crisis intervention. Not all the savings achieved will be cashable, but some will be, and often enough will be cashable to make the whole project viable.
Second, there are some outcomes which are socially worthwhile and do achieve early, cashable payback. This is particularly the case when dealing with an existing issue which imposes high costs, such as children in residential care, and vulnerable adults with high care needs. If interventions can be commissioned which enable such groups to live in the community, there are likely to be improved outcomes for them as individuals AND cashable savings to pay for the services. In these cases, what’s not to like?
The final argument is about the rigour that a SIB or PbR process imposes on commissioners. Adrian Brown suggests that instead of trying to match outcomes to savings, commissioners should “regularly look across the portfolio of services they buy and ask themselves which are the least effective. Decommissioning these services frees up current spending that can be used to fund SIBs in policy areas where innovation is most urgently needed”.
But we see little evidence that commissioners have either the data available, or review processes in place, to enable them to do this rationally. In our experience of working with commissioners over the last three years, a key benefit of the SIB development process is that it allows the commissioner to consider in detail what it costs to provide a service, what is the impact of that service – in both financial and non-financial terms – and whether an alternative service will achieve better returns – both social and financial. And if their financial analysis shows that they can improve outcomes to the point where they can decommission a service to achieve cashable savings, that is surely better than making less well informed and perhaps arbitrary decisions about what services should no longer be funded.
So if commissioners want to see SIBs as a way to test innovation and have the funding to use them in that way, fine by me,. If they want to use them to save money and introduce much greater rigour into their commissioning processes, why not?
- March 7, 2013
Last Monday the Cabinet Office launched “What Works” a new initiative to improve evidence-based policy development. The Cabinet Office Press Release, with detailed documents attached, is here. A useful summary from the Guardian is here.
As the press release explains “The What Works Network, a key action in the Civil Service reform plan, will consist of two existing centres of excellence – the National Institute for Health and Clinical Excellence (NICE) and the Educational Endowment Foundation – plus four new independent institutions responsible for gathering, assessing and sharing the most robust evidence to inform policy and service delivery in tackling crime, promoting active and independent ageing, effective early intervention, and fostering local economic growth.”
In simple terms, the aim is to do for evidence-based programmes and interventions, what NICE does for drugs. If NICE approves a drug or treatment, the NHS will prescribe it (provided local commissioners can afford it). If an intervention has a similar seal of approval, commissioners and others are more likely to use it.
As the Guardian observes, this has been something of a pet project for the Cabinet Secretary, Sir Jeremy Heywood, since he was merely head of the Cabinet Office. It is an important element in payment by results and social impact bonds, where it is hard to design such schemes (and especially assess the risk for providers and investors) if you do not have a reasonable confidence that the interventions proposed will improve outcomes and achieve the results required.
But it seems to me that this has much wider implications. First, as critics have already observed, it should make it harder for ministers to base policy on whim or ideology. There is a long standing joke among Whitehall watchers that Ministers prefer policy-based evidence making to evidence-based policy-making – i.e. getting the facts to fit the policy rather than the other way round. If the new policy centres do start to become as established as NICE, this will probably be harder to do.
But something else that intrigues me is whether it will become much more the norm to attach a monetary value to social policy. This is very much the way that NICE works – which leads to increasing controversy when it rules against the use of an expensive drug that might extend the life of a loved one for a few weeks or months, on the grounds that it is not worth the money.
In social policy, there is still a strong view among many providers that we should not concern ourselves with whether something works in financial terms, but should intervene simply because it is “a good thing” or “the right thing to do”.
I have always found this a difficult argument to sustain if the thing that is being done requires public money. There are only two reasons for any intervention which requires investment by the state. It achieves a better result – for the person themselves and/or for society as a whole – than doing nothing. This requires measurement against what would have happened without the intervention, through some form of control or baselining.
The other is that doing something costs less than doing nothing, usually because it avoids higher costs later. On almost a daily basis, it seems that we have a new study or report which argues the case for public investment on these grounds. Yet many of those who advocate such policies seem reluctant to go the extra mile and resist attempts to properly measure the value that they add in financial terms.
But if “What Works” mirrors existing, similar work in the US – notably by Washington State Institute for Public Policy – it will become very much about this intervention costs X and saves Y, that intervention costs A and saves B – as well as about whether it is effective in reducing or solving a social problem.
No, we cannot reduce everything to pounds and pence. But if the public sector is being asked to put money to work to make things happen, It seems reasonable to ask for some evidence that those things add value.
- February 13, 2013
There is a famous, almost certainly apocryphal, story about a 19th century recipe for cooking jugged hare. Before getting into the detail of how to cook it, the recipe advises the reader: “First, catch your hare”
I’m reminded of this when I read some of the advice that is starting to become widely available on how to build a case for investing in better social outcomes. The advice usually relates to social impact bonds (SIBs) or payment by results (PbR) – but it also comes up in the wider context of making the case for any early or preventative intervention which reduces service demand.
Some of this advice is excellent, and nearly all of it is useful. But what most of it has in common – and why it reminds me of the challenge of catching a hare before you can cook it – is that it tells you what to do but is often short on detail of how to do it. For example, there is advice to collect data on costs and benefits, undertake cost benefit appraisal, build a financial model, estimate whether savings are cashable, etc. But more detailed guidance on how you do these things is thin on the ground. Basically, much of the guidance is saying “First, build your business case”
At ATQ we have been through this pain. In a previous life we worked alongside the Cabinet Office with four local authorities on the feasibility of using payment by results or social impact bonds to fund interventions for troubled families. The effort involved in testing the feasibility of intervention – and whether it was wholly or partly self-funding as result of future savings – was, to put it mildly, significant. And at every stage of the process, we kept saying to ourselves – we must make sure we record how we did this, so that others can benefit from what we have learnt and do not have to do the same things from scratch.
So last week (somewhat later than we had originally intended) we published a joint Cabinet Office/ATQ guide to building a business case for intervention which improves outcomes. You will find this on the Cabinet Office website as a How to Guide for commissioners of Social Impact Bonds, and on our website as a Payment by Results Toolkit. But both these short titles are somewhat misleading. The methodology and tools we used do not work only in a PbR or SIB context. They can also be used whenever a public sector body wants to know whether and how costs will change if it takes action which addresses a social problem and ultimately reduces service demand. They should help anyone facing these challenges to:
- identify what data they need to undertake a cost appraisal;
- collect that data in a structured way; and
- model and analyse the data once they have it (especially if they use the financial model we are making available).
We are not saying that the methodology we worked out is the only one available, or that we have all the answers. Much good work is being done by many public sector bodies, such as local authorities which have been involved in Whole Place Community Budget pilots or are at the forefront of work with troubled families. And the tools do not solve the basic challenge of collecting good data – though the Cabinet Office and others are working on making better data available
But, hopefully, they will help at least some of those who are grappling with these challenges and finding it a bit like chasing a hare round a field.
- November 28, 2012
On November 23rd the Cabinet Office launched two initiatives designed to stimulate the market in social impact bonds (SIBs) and similar payment by results (PBR) schemes: the Centre for Social Impact Bonds (an online resource for those considering how to do this stuff); and the Social Outcomes Fund – which may be of more interest to commissioners and others since it involves government putting up some money to stimulate the market in SIBs.
We played a small part in the development of the Social Outcomes Fund, since it was one of the recommendations made in the report we co-authored with the Cabinet Office (in a previous life) on the feasibility of applying SIBs/PBR to troubled families. The principle of a SIB is that interventions to solve a social problem are funded from future savings to the commissioner and others. One of our findings was that it is difficult to persuade those who benefit from a successful SIB or PBR scheme and are not the main commissioner to make a contribution.
This was also the experience of Essex County Council when trying to get contributions to its SIB aimed at keeping older children out of care or custody (the launch of which, along with another SIB aimed at homeless rough sleepers in London, was also announced at the same event)
So the purpose of the fund – as made clear by Ministers at the launch event – is to enable those wanting to set up a SIB to apply for the fund if they cannot get contributions direct from other bodies. The Cabinet Office also hope that the Fund will help grow the market for SIBs and similar schemes – which have been slower to take off than many hoped Other sources of funding are starting to emerge which have similar aims – such as the impending Results Fund being set up by Big Society Capital.
In our view (we could hardly say otherwise) the Social Outcomes Fund is undoubtedly a good thing. The developing debate is whether SIBs and similar schemes will ever get to the sort of scale where they can make a real difference to social outcomes and – by extension – start to really impact on the costs of social problems. The general view at the launch from all sides (investors, commissioners and policy makers) seemed to be that things have moved on a good deal over the past couple of years, but there is still a way to go.