Tag Archives: social impact measurement

You owe me five farthings (and a social impact report)

The February 2015 OECD report Social Impact Investment – Building The Evidence Base described ‘Social Impact Investment’ as:  “the provision of finance to organisations addressing social needs with the explicit expectation of a measurable social, as well as financial, return”.  On that basis, there’s not much ‘Social Impact Investment’ going on in the UK social investment market.

Oranges & Lemons, written by Investing for Good and funded by Big Society Capital and the Esmee Fairbairn Foundation, is an assessment of ‘The State of Play of Impact Measurement among UK Social Investment Finance Intermediaries’ (SIFIs).

The report, published earlier this month, is probably not one that will attract significant numbers of readers working for frontline charities and social enterprises; many face a tough choice about whether to ignore it based on their scepticism about social investment or their scepticism about impact measurement but, nevertheless, it’s a useful and timely contribution to UK social investment’s understanding of itself.

The research is based on interviews with ten leading UK SIFIs. Unfortunately, while interviewees were prepared to talk about their approaches to impact measurement when investing in  social organisations* they were either unwilling or unable to provide any data related to specific deals. This means the research is primarily anecdotal: it’s essentially a report on what SIFIs say about their impact measurement processes.

Five of the key points emerging from the report are:

1. It is not clear what impact measurement is for (in the social investment market)

In the conclusions and recommendations section of the report, the authors pose the question: “If a SIFI were to ask itself, ‘What if I don’t do an impact report?’, ‘What if I don’t include x in my impact report?’… In reality, there is often little by way of consequence.

Instead the assumption is that impact reporting is ‘a noble thing to do, and good practice recommends it‘.

Even before getting into the detail, there is no clear consensus on whether the broad, overall point of impact measurement in social investment is to measure (and hopefully ultimately improve) the social impact of organisations and initiatives funded via social investment or to measure (and improve) the impact of social investment on those organisations. Currently, it seems to be a loosely conceptualised combination of the two.

2. It is not clear who measurement is for 

Most social impact investors# are not, on the evidence of this report, actively interested in social impact measurement. The authors note that: “SIFI’s investors were largely found to be satisified, and not to be actively demanding more information, specific kinds of information, or more standardisation, even though the reports themselves varied considerably.

While not addressed in the report, there is a wider issue of the public accountability of participants in the social investment market. Of the 10 SIFIs interviewed for the report, 8*** have received some public money – either directly or through Big Society Capital – to fund their investments. At least 3, receive or have received the majority of their funding from either UK or devolved government.

It seems strange that as we approach the first £1 billion** of UK government funding for the social investment market, SIFIs (as a group) are not actively seeking to make themselves accountable to the public for the money that supports their industry and have no apparent mechanisms for demonstrating the social impact of the public money they receive. Principles aside, that’s likely to be a practical problem for them if/when social investment becomes less politically popular.

3. Lots of pre-deal work, not much post deal 

The majority of the impact assessment work currently undertaken by SIFIs takes place at the pre-deal stages. The approaches are different, some SIFIs operate general funds, others map particular social sectors and seek investments in those markets. When analysing specific bids, the approaches range from discussion-based ‘social due diligence’ – talking to organisations about what they do – to using a standard impact rating methodology for analysing all deals.

When it comes to deal-making, 2 out of 10 SIFIs don’t request investees set any specific Key Performance Indicators (KPIs) at all while, at the other end of the spectrum, 4 SIFIs ask investees to agree KPIs from a menu based on outcomes they (the SIFI) is seeking to achieve.

There is no evidence that impact is used as a basis for choosing one investment over another. The authors note that: “In the majority of cases, if an opportunity reaches the investment committee but doesn’t win investment, the reasons are predominantly financial.

The overall effect is that: “Pre-deal analysis can more as form of screen or impact hurdle, with financial considerations coming to the fore once it has been cleared.

Tellingly: “There were no cases of ‘impact defaults’, in which funds were withdrawn or investments written down on account either of insufficient impact data being reported, or of the impact data being reported showing that insufficient impact was being generated.

The point is not that ‘impact defaults’ are desirable but if they are not possible that’s in unclear how SIFIs investments can meaningfully be described as ‘impact investments’ when as the authors note, on this basis ‘impact is not an investment concern‘.

4. No connection between impact and price 

The authors note that: “There is little evidence for impact explicitly or directly moving price.” While there’s a sense that: “Investors may be prepared to take an extra risk for high impact” there’s no evidence of: “clear processes by which e.g a high impact organisation can be charged 3% for a loan whereas a lower impact organisation would pay 7%

This point is simultaneously vitally important to our explanation and understanding of ‘impact investment’ but also a potential red herring. Professor Alex Nicholls at Said Business School, in his work on ‘impact risk’, has usefully asked whether it’s correct to assume that organisations with potential for high social impact will necessarily offer a bigger financial risk than those with a lower potential impact.

It’s not necessarily logical or useful for SIFIs to offer ‘impact discounts’ -particularly given the challenges the authors highlight about generating independently audited data- but the rhetoric on social investment from politicians and some social investment leaders, is that they do.

It’s one of many issues where there’s a yawning gap between the social investment hype machine and the reality experienced by charities and social enterprises. On the one hand there’s a theoretical vision of a market where SIFIs demand that organisations achieve (and report on) measurable social outcomes and receive better investment deals as a result of doing so, on the other you have the reality where most SIFIs are using potential social impact as an enhanced screening process and asking for information back so that they can prepare case study-led impact reports for their funders.

The idea that ‘you can get a cheap loan if you can demonstrate your social impact’ is a pernicious, cynicism-inducing myth that SIFIs and social investment leaders can all play their part in squashing at every conceivable opportunity.

5. Everyone has a different system 

Perhaps unsurprisingly: “No off-the-shelf metholodologies are being used by SIFIs to measure their impact, and instead all are individually and unique.

This apparently means both that different SIFIs measure impact in different ways but also that individual SIFIs allow different investees to report their impact in different ways.

This is not necessarily wrong in either case. It doesn’t seem useful, either in terms of the responsibilities placed on the investee or the likely value of the data generated, to use the same model to assess the impact of a £25,000 bridging loan and a £2.5 million quasi-equity investment.

But the fact that all social investments are not usefully comparable does not mean that no social investments are usefully comparable at all. Oranges and Lemons recommends: (i) that more work is done to understand the best ways of measuring the impact of different types of investment and (ii) that SIFIs form a peer group ‘for the reading and reviewing of impact reports’.

These are sensible recommendations. It would be great if they helped SIFIs generate and publish more meaningful data (not necessarily more data). It would be even better if they helped SIFIs develop clearer ideas about who they’re generating data for and why.

*The report does not cover Social Impact Bonds (SIBs) involve a direct specific, relationship between outcomes and returns it isn’t useful to compare them to situation where SIFIs are choosing whether and how to combine impact measurement and finance.

#Investors providing money for SIFIs to invest

**Cumulative total of grants, investment funds and tax breaks since 2003 – mostly spent since 2010.

*** Correction – an earlier version said all 10 organisations had received public funding – CAF Venturesome and Esmee Fairbairn Foundation had not received public funding for social investment at time of writing.


Filed under Uncategorized