Industries / Government, education and community / Lobbying
Data
Lobbying data work turns filings, meeting logs and bill texts into databases that reporters, researchers and the public can search and compare.
6
Milestones achieved, from 1998 to 2026.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1998
Federal filings become a database
The Center for Responsive Politics, now OpenSecrets, built a database of US federal lobbying filings starting with 1998; it standardises the spelling of names, rolls subsidiaries up to their parent companies and links reports to bills and agencies.
LessonThe raw filings need heavy cleaning before they can be compared.
Credit
Center for Responsive Politics (OpenSecrets)
2013
LobbyPlag matches lobby texts
Volunteers compared leaked lobby papers with amendments proposed in European Parliament committees on the EU's data protection law, the General Data Protection Regulation, to show "which changes proposed by lobbyists went straight into amendments", and asked the public to help check about 1,700 more.
LessonThe tracing worked only because papers had leaked and volunteers checked them by hand.
Credit
OpenDataCity; Europe versus Facebook; La Quadrature du Net and others for the papers
2014
EU meeting logs go public
From 1 December 2014, EU Commissioners and their offices must publish their meetings with lobbyists within two weeks, under a Commission decision. LobbyFacts, run by the campaign groups Corporate Europe Observatory and LobbyControl, collects these logs and the Transparency Register and keeps register history back to February 2012 that it says "is no longer available anywhere else".
Credit
European Commission; Corporate Europe Observatory and LobbyControl
2019
Copycat bills traced at scale
Reporters ran nearly 1 million bills from all 50 US states and Congress through a text-similarity program and found at least 10,000 bills almost entirely copied from model legislation over eight years, more than 2,100 of which became law.
LessonModel bills appear in no lobbying disclosure, and the method caught only near-verbatim copying.
Credit
USA TODAY, The Arizona Republic and the Center for Public Integrity
2020
Filings linked to bills
In Song Kim and Dmitriy Kunisky published a dataset linking US bills and their sponsors to the interest groups that lobbied on them, built from more than two million text descriptions in lobbying reports; Kim's LobbyView site at MIT now lists more than a million reports. The same year the Sunlight Foundation, which built open-government data tools and says it served "more than 4.2 billion API calls", shut down.
LessonLobbying data depends on grant-funded labs and nonprofits, which can close.
Credit
In Song Kim (MIT); Dmitriy Kunisky
2026
One US filing site
The Senate's lobbying data page announced in early 2026 that it would close after 30 June 2026, and search for federal lobbying reports moved at the end of July 2026 to LDA.gov, a joint House and Senate site whose public API serves every filing as structured data.
Credit
the Clerk of the House; the Secretary of the Senate
Watch for
EU register cross-checks payments
Since 5 June 2025 the EU Transparency Register automatically compares the amounts clients declare paying consultancies with what the consultancies declare receiving, and its 2025 annual report says the linking "has greatly helped with consistency and compliance".
Achieved whenAn outside body, such as the European Ombudsman, the European Court of Auditors or an academic study, measures fewer mismatches.
transparency-register.europa.eutransparency-register.europa.eu
Claimed by
Joint Transparency Register Secretariat, June 2025 and May 2026
State data joins OpenSecrets
OpenSecrets was formed in June 2021 by merging the Center for Responsive Politics with the National Institute on Money in Politics, whose FollowTheMoney site held US state data, in order to combine federal, state and local data. FollowTheMoney now says its data runs through the 2024 election year and that the site "isn't maintained as we integrate with OpenSecrets".
Achieved whenOpenSecrets publishes state lobbying data for 2025 and later.
Claimed by
OpenSecrets, June 2021, restated on FollowTheMoney in November 2026
Open
01
One format for lobby registers
An open, documented data format for lobby registers, converters that put the US, EU, Canadian, Irish, French and German registers into it, and a match of the same organisations across them.
The briefCredit · unclaimed02
Ground truth for lobby footprints
A public set of pairs, each linking a passage in a lobby group's position paper to an amendment in the European Parliament, labelled by two reviewers as copied, reworded or unrelated, so that methods for tracing lobby text into law can be scored.
The briefCredit · unclaimed
Open 01 · the brief
One format for lobby registers
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
An open, documented data format for lobby registers, converters that put the US, EU, Canadian, Irish, French and German registers into it, and a match of the same organisations across them.
Why it is still open
International IDEA assessed 16 registers in 2024 and found that most let the public download their data but that "formal coordination between lobbying registers is of pivotal importance – and so far largely unaddressed". Transparency International EU found in 2024 that the countries it studied lack unique identifiers for officials and companies, "which renders these datasets almost impossible to link".
Built with what exists
The LDA.gov API and the other registers' downloads; company identifiers such as the Legal Entity Identifier (LEI) where they exist; an AI model to match organisation names across languages; a spreadsheet for two reviewers.
First step
Map the fields of the US and EU registers to a draft format and match the 100 largest US lobbying clients to EU register entries.
Done looks like
A published format with converters for six registers; organisations matched across registers with at least 95% precision on 300 pairs checked by two reviewers; the combined data refreshed monthly for six months.
Open 02 · the brief
Ground truth for lobby footprints
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public set of pairs, each linking a passage in a lobby group's position paper to an amendment in the European Parliament, labelled by two reviewers as copied, reworded or unrelated, so that methods for tracing lobby text into law can be scored.
Why it is still open
Researchers at EPFL in Switzerland who traced alignments between lobby groups and members of the European Parliament wrote in 2024 that "there exists no large database of verified MEP-lobby alignments", so they could check their methods only indirectly, against retweets and meeting records.
Built with what exists
Position papers from the EU Transparency Register and lobby groups' websites; Parliament's published committee amendments; LobbyPlag's GDPR data as a first source of known copies; an AI model to propose candidate pairs; a spreadsheet for two reviewers.
First step
Label 100 candidate pairs drawn from LobbyPlag's GDPR amendments and measure how often two reviewers agree.
Done looks like
2,000 labelled pairs across at least five laws; the two reviewers agree on at least 80% of labels; existing methods scored against the set; data and guidelines published.