You owe me five farthings (and a social impact report)

The February 2015 OECD report Social Impact Investment – Building The Evidence Base described ‘Social Impact Investment’ as:  “the provision of finance to organisations addressing social needs with the explicit expectation of a measurable social, as well as financial, return”.  On that basis, there’s not much ‘Social Impact Investment’ going on in the UK social investment market.

Oranges & Lemons, written by Investing for Good and funded by Big Society Capital and the Esmee Fairbairn Foundation, is an assessment of ‘The State of Play of Impact Measurement among UK Social Investment Finance Intermediaries’ (SIFIs).

The report, published earlier this month, is probably not one that will attract significant numbers of readers working for frontline charities and social enterprises; many face a tough choice about whether to ignore it based on their scepticism about social investment or their scepticism about impact measurement but, nevertheless, it’s a useful and timely contribution to UK social investment’s understanding of itself.

The research is based on interviews with ten leading UK SIFIs. Unfortunately, while interviewees were prepared to talk about their approaches to impact measurement when investing in  social organisations* they were either unwilling or unable to provide any data related to specific deals. This means the research is primarily anecdotal: it’s essentially a report on what SIFIs say about their impact measurement processes.

Five of the key points emerging from the report are:

1. It is not clear what impact measurement is for (in the social investment market)

In the conclusions and recommendations section of the report, the authors pose the question: “If a SIFI were to ask itself, ‘What if I don’t do an impact report?’, ‘What if I don’t include x in my impact report?’… In reality, there is often little by way of consequence.

Instead the assumption is that impact reporting is ‘a noble thing to do, and good practice recommends it‘.

Even before getting into the detail, there is no clear consensus on whether the broad, overall point of impact measurement in social investment is to measure (and hopefully ultimately improve) the social impact of organisations and initiatives funded via social investment or to measure (and improve) the impact of social investment on those organisations. Currently, it seems to be a loosely conceptualised combination of the two.

2. It is not clear who measurement is for 

Most social impact investors# are not, on the evidence of this report, actively interested in social impact measurement. The authors note that: “SIFI’s investors were largely found to be satisified, and not to be actively demanding more information, specific kinds of information, or more standardisation, even though the reports themselves varied considerably.

While not addressed in the report, there is a wider issue of the public accountability of participants in the social investment market. Of the 10 SIFIs interviewed for the report, 8*** have received some public money – either directly or through Big Society Capital – to fund their investments. At least 3, receive or have received the majority of their funding from either UK or devolved government.

It seems strange that as we approach the first £1 billion** of UK government funding for the social investment market, SIFIs (as a group) are not actively seeking to make themselves accountable to the public for the money that supports their industry and have no apparent mechanisms for demonstrating the social impact of the public money they receive. Principles aside, that’s likely to be a practical problem for them if/when social investment becomes less politically popular.

3. Lots of pre-deal work, not much post deal 

The majority of the impact assessment work currently undertaken by SIFIs takes place at the pre-deal stages. The approaches are different, some SIFIs operate general funds, others map particular social sectors and seek investments in those markets. When analysing specific bids, the approaches range from discussion-based ‘social due diligence’ – talking to organisations about what they do – to using a standard impact rating methodology for analysing all deals.

When it comes to deal-making, 2 out of 10 SIFIs don’t request investees set any specific Key Performance Indicators (KPIs) at all while, at the other end of the spectrum, 4 SIFIs ask investees to agree KPIs from a menu based on outcomes they (the SIFI) is seeking to achieve.

There is no evidence that impact is used as a basis for choosing one investment over another. The authors note that: “In the majority of cases, if an opportunity reaches the investment committee but doesn’t win investment, the reasons are predominantly financial.

The overall effect is that: “Pre-deal analysis can more as form of screen or impact hurdle, with financial considerations coming to the fore once it has been cleared.

Tellingly: “There were no cases of ‘impact defaults’, in which funds were withdrawn or investments written down on account either of insufficient impact data being reported, or of the impact data being reported showing that insufficient impact was being generated.

The point is not that ‘impact defaults’ are desirable but if they are not possible that’s in unclear how SIFIs investments can meaningfully be described as ‘impact investments’ when as the authors note, on this basis ‘impact is not an investment concern‘.

4. No connection between impact and price 

The authors note that: “There is little evidence for impact explicitly or directly moving price.” While there’s a sense that: “Investors may be prepared to take an extra risk for high impact” there’s no evidence of: “clear processes by which e.g a high impact organisation can be charged 3% for a loan whereas a lower impact organisation would pay 7%

This point is simultaneously vitally important to our explanation and understanding of ‘impact investment’ but also a potential red herring. Professor Alex Nicholls at Said Business School, in his work on ‘impact risk’, has usefully asked whether it’s correct to assume that organisations with potential for high social impact will necessarily offer a bigger financial risk than those with a lower potential impact.

It’s not necessarily logical or useful for SIFIs to offer ‘impact discounts’ -particularly given the challenges the authors highlight about generating independently audited data- but the rhetoric on social investment from politicians and some social investment leaders, is that they do.

It’s one of many issues where there’s a yawning gap between the social investment hype machine and the reality experienced by charities and social enterprises. On the one hand there’s a theoretical vision of a market where SIFIs demand that organisations achieve (and report on) measurable social outcomes and receive better investment deals as a result of doing so, on the other you have the reality where most SIFIs are using potential social impact as an enhanced screening process and asking for information back so that they can prepare case study-led impact reports for their funders.

The idea that ‘you can get a cheap loan if you can demonstrate your social impact’ is a pernicious, cynicism-inducing myth that SIFIs and social investment leaders can all play their part in squashing at every conceivable opportunity.

5. Everyone has a different system 

Perhaps unsurprisingly: “No off-the-shelf metholodologies are being used by SIFIs to measure their impact, and instead all are individually and unique.

This apparently means both that different SIFIs measure impact in different ways but also that individual SIFIs allow different investees to report their impact in different ways.

This is not necessarily wrong in either case. It doesn’t seem useful, either in terms of the responsibilities placed on the investee or the likely value of the data generated, to use the same model to assess the impact of a £25,000 bridging loan and a £2.5 million quasi-equity investment.

But the fact that all social investments are not usefully comparable does not mean that no social investments are usefully comparable at all. Oranges and Lemons recommends: (i) that more work is done to understand the best ways of measuring the impact of different types of investment and (ii) that SIFIs form a peer group ‘for the reading and reviewing of impact reports’.

These are sensible recommendations. It would be great if they helped SIFIs generate and publish more meaningful data (not necessarily more data). It would be even better if they helped SIFIs develop clearer ideas about who they’re generating data for and why.

*The report does not cover Social Impact Bonds (SIBs) involve a direct specific, relationship between outcomes and returns it isn’t useful to compare them to situation where SIFIs are choosing whether and how to combine impact measurement and finance.

#Investors providing money for SIFIs to invest

**Cumulative total of grants, investment funds and tax breaks since 2003 – mostly spent since 2010.

*** Correction – an earlier version said all 10 organisations had received public funding – CAF Venturesome and Esmee Fairbairn Foundation had not received public funding for social investment at time of writing.


Filed under Uncategorized

7 responses to “You owe me five farthings (and a social impact report)

  1. A billion pounds from business and individuals who pay taxes David. So did we at least get a financial return on our investment? Otherwise it would seem that those using their profit for social benefit contribute twice.

    The inclusion of Porter and Kramer in this report was perhaps inevitable. What they’d argued in Creating Shared Value was that the opportunity to apply capitalism for social benefit had been overllooked. That I knew not to be true. You soon learn that denial is a bif part of social innovation.

    The emphasis of Creating Shared Value is returning a profit with social benefit, whereas social enterprise as we saw it was a matter of putting people first. When Mark Kramer declared that corporations profit from solving social problems i disagreed. We saw the risk of the usual suspects undermining the social objective.

    When finally, the crisis we tried to avert turned to violent conflict, the usual suspects gathered at Davos, If you look at the backdrop, even now the emphasis is on profit first, rather than “those in need being helped first, not secondarily, along the way or by the way” .


  2. Beanbags admin

    “So did we at least get a financial return on our investment?”

    None of the government cash that’s gone into social investment has been put in on the basis the government – and, by extension, the taxpayer – will get a financial return.

    In some cases, as with Futurebuilders, the government has got some money back and is now putting that money into more social investment support (via Access).

    If there is an expectation of return – either financial or social – beyond ‘the government supports social investment because it’s a good thing’ then it hasn’t yet been articulated.


  3. Great blog and thanks for bringing this report to light – it totally passed me by.

    Social investors/intermediaries are shooting themselves in the foot by sticking to anonymity in reports like these.

    There may be some good reasons for social investors to take different approaches to measurement. For instance, Charity Bank is essentially a lender of last resort to charities who need loans that high street banks can’t or won’t provide. Nesta Impact Investments does equity. Both are Big Society Capital-funded intermediaries.

    It would seem unnecessary to make Charity Bank investees conform to high standards of impact measurement when the sole purpose is to provide charities with loans.

    On the other hand, as Nesta are providing equity investments it’s right that they have a much higher standard of evidence as presumably some of their investees are for-profit, company limited by shares. If these organisations are to take social investment they need to damn well prove they’re making a difference. (For similar reasons, the SE100 tends to show company ltd by shares are much better at impact measurement than charities. this obviously doesn’t mean they’re making a bigger difference to society)

    I have no idea if either Charity Bank or Nesta Impact Investments was involved in this report. If the context was available *maybe* some valid reason would come to light for the variations. A shabby-looking approach to measurement might look a tad better if you could see the actual portfolio.

    It would be nice if any who contributed to the report wanted to respond or clarify their approach to measurement. I’m sure they’re all reading this blog!


  4. Beanbags admin

    Hi Matt,

    Thanks for this. I should make clear there is a public list of SIFIs who took part in the report: Big Issue Invest, Bridges, Charity Bank, Esmee Fairbairn, Nesta Impact Investments, Impact Ventures, Reasonance, SIB, SIS and Venturesome.

    But not only is there no information on the actual deals in SIFIs’ portfolios, there isn’t even attribution of anecdotes to particular SIFIs.

    In the case of portfolio data, based on the report and what was said at the launch, that’s definitely unavailable because SIFIs were either unwilling or unable to make the information available.

    I don’t know whether SIFIs were asked whether they’d happy to be quoted explaining their differing processes in the abstract and demanded to remain anonymous, or whether they weren’t asked.

    Agree with you on differing approaches to measurement. The problem isn’t the existence of the differing approaches. (While some convergence between some SIFIs doing similar things might be useful, particular to them) it’s the lack of transparency about what the approaches are and vagueness about what they’re for that’s the big problem.


  5. Pingback: Finance Matters | Impact measurement: asking some difficult questions

  6. “Impact investment must not become the tail that wags the dog” said Peter Holbrook last week. Yet I see it happening repeatedly. Today even the most prominent of social enterprise won’t be found at a conference on Inclusive Capitalism.
    This as I see it, is a reframing of the argument for business with social objectives, i.e social enterprise. It can’t be done without excluding others and the causes they support. I’m not even sure whose side Oxfam is on.


  7. Hello all, just to clarify, Big Society Capital were not investors in all the deals covered in the Oranges and Lemons report – we co-commissioned it to get a sense of impact measurement practice across the whole social investment market.

    Our own approach to impact assessment and monitoring is outlined in our first impact report which states that the level of evidence we expect varies according to the size and scope of a deal. So what we expect for a social impact bond or an equity fund can differ from a backstop facility or community shares underwriting fund. For information our impact report is here: Our theory of change and impact priority is outlined in the report too which says that we want social investment to deliver outcomes for vulnerable and disadvantaged people across the UK. Happy to hear any feedback about the content of our report too.

    We are also gathering and posting examples of impact reports from intermediaries and front line organisations which are publicly available under the social impact content of our website as we believe that one of the best ways to promote better impact practice is to support better peer to peer learning by showcasing examples from intermediaries and front line organisations. We hope that more organisations will follow suit and commit to publishing their impact evidence so that others can learn from it.

    Separately, we have recently published a document about transparency which outlines further what we would like to do in this area which picks up on some of the points above and would again welcome responses to it:

    Lastly, you may be interested to know that NESTA published a great report last week about practical lessons about impact measurement based on learning from their fund last week which is available here: It shows how impact evidence can be a really helpful tool to both improve your work and leverage further income.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s