BRENNEN

Industries / Government, education and community / Nonprofits / Foundations

Measuring results

Foundations try to find out whether their grants worked, by asking grantees, commissioning evaluations and, in some cases, paying only for results.

4

Milestones achieved, from 2003 to 2018.

1

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 2003

    Grantees grade their funders

    The Center for Effective Philanthropy built the Grantee Perception Report, a survey of a foundation's grantees whose results are compared with those of peer funders. The Hewlett Foundation has commissioned one regularly since 2003, and CEP says more than 300 foundations have used it.

    hewlett.orgcep.org

    Credit

    Center for Effective Philanthropy; William and Flora Hewlett Foundation

  2. 2016

    Evaluations kept in-house

    In a 2016 benchmarking survey of 127 senior evaluation or program staff, run by the Center for Evaluation Innovation and CEP, only 57% said their foundations share evaluation findings outside their own institutions, and 71% said their foundations invest too little in sharing evaluation data.

    LessonFoundations' own evaluation staff said they share too little of what they learn.

    philanthropy.com

    Credit

    Center for Evaluation Innovation; Center for Effective Philanthropy

  3. 2018

    Paid only for results

    In what Brookings calls the world's first development impact bond for education, the UBS Optimus Foundation put up $270,000 for the charity Educate Girls, and the Children's Investment Fund Foundation (CIFF) agreed to repay it, with a 15% internal rate of return (a yearly return), if results were met. The evaluator IDinsight, using a randomized trial to measure learning, found that by year three 768 girls, 92% of those identified, had enrolled, and learning gains reached 160% of the target.

    LessonBrookings noted that the results show the program worked, not that paying by results beats other ways of paying for it.

    brookings.edu

    Credit

    UBS Optimus Foundation; Children's Investment Fund Foundation; Educate Girls; Instiglio; IDinsight

  4. 2018

    Better measures, same results

    From 2009 through 2016, three school districts and four charter school groups spent $597 million on the Gates Foundation's Intensive Partnerships for Effective Teaching, $237 million of it from Gates grants. RAND and the American Institutes for Research found that the sites improved how teachers were evaluated but that the initiative "failed to achieve its goals for improved student achievement and graduation".

    LessonThe evaluators pointed to incomplete implementation without proven models to copy, conflicting goals for teacher evaluation, and changing state and local conditions.

    rand.org

    Credit

    Bill & Melinda Gates Foundation; RAND Corporation and American Institutes for Research (evaluation)

Watch for

  1. An AI reader for reports

    The GitLab Foundation says a staff member built an internal AI tool to help review grantee reports and research literature, and that it "has reduced manual analysis time by up to five hours per grantee application". No check of the tool's accuracy was published.

    Achieved whenThe foundation or an outside party publishes how often the tool's summaries match a person's reading of the same reports.

    gitlabfoundation.org

    Claimed by

    GitLab Foundation, June 2026

Open

  1. 01

    What the evaluations found

    An open index of evaluations that foundations have commissioned and published, coded by program, funder, country, method (randomized trial, comparison group, interviews only) and main result (improved, no clear effect, worse), starting from Candid's Issue Lab collection and adding reports from foundation websites.

    The brief
    Credit · unclaimed
  2. 02

    Reports sized to grants

    A dataset of what 150 foundations ask grantees to report, from their published reporting guidance and grant terms: how often, how many questions, what data and attachments, set against each foundation's typical grant size.

    The brief
    Credit · unclaimed

Open 01 · the brief

What the evaluations found

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

An open index of evaluations that foundations have commissioned and published, coded by program, funder, country, method (randomized trial, comparison group, interviews only) and main result (improved, no clear effect, worse), starting from Candid's Issue Lab collection and adding reports from foundation websites.

Why it is still open

Issue Lab holds more than 36,000 research reports, case studies and toolkits and is moving into the permanent collection of the Rockefeller Archive Center, but Candid's description of it mentions no coding of what the reports found. In 2016 only 57% of foundation evaluation staff surveyed said their foundations share findings outside. This research found no dataset that codes the results of published foundation evaluations.

candid.orgphilanthropy.com

Built with what exists

An AI model that reads long PDF reports, such as Claude, Gemini or Muse Ember (see the Note "The jobs agents repeat"), to draft the codes; a coding guide modeled on those used in systematic reviews; two human coders; Python.

  1. First step

    Pull 50 evaluations on one topic, such as education, from Issue Lab and code them by hand.

  2. Done looks like

    1,000 evaluations coded; two coders agree on the main-result code for at least 85% of a 200-report sample; the share of null results reported by method and field; data published under an open license.

Open 02 · the brief

Reports sized to grants

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A dataset of what 150 foundations ask grantees to report, from their published reporting guidance and grant terms: how often, how many questions, what data and attachments, set against each foundation's typical grant size.

Why it is still open

In 2020, 64% of foundations CEP surveyed said they had reduced what they ask of grantees, a self-report made during the pandemic. Giving Evidence found that few foundations track the costs their processes create for charities. This research found no public comparison of reporting demands with grant size.

cep.orggiving-evidence.com

Built with what exists

A web agent such as Browser Use to collect public reporting guidance and templates (see the Note "Deleting 80% to start over"); an AI model to code the demands; 360Giving's GrantNav and US tax return data for grant sizes; a spreadsheet.

  1. First step

    Gather the reporting guidance of 20 funders and code it by hand.

  2. Done looks like

    150 foundations coded by two people who agree on at least 90% of codes; a published chart of reporting demands against median grant size; the count repeated after a year to see whether demands fall.