BRENNEN

Industries / Technology / Social media

Data access for researchers

This work decides how people outside a platform can study what happens on it: through the platform's own data tools, by collecting public posts themselves, or under laws that require access.

7

Milestones achieved, from 2011 to 2026.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 2011

    A shared tweet archive closes

    Twitter forced TwapperKeeper.com, a service that gathered tweets by keyword or hashtag on request and made the resulting datasets public, to close in 2011, as the company began selling access to its data through resellers such as Gnip, which it later bought. Datasets were afterwards shared more and more "through non-public, grey channels," the media scholar Axel Bruns wrote in 2019.

    LessonAccess granted under a platform's terms can be withdrawn whenever the platform chooses.

    eprints.qut.edu.au

    Credit

    Twitter; account by Axel Bruns

  2. 2018

    APIs cut after a scandal

    On 4 April 2018, after the Cambridge Analytica reports, Facebook's chief technology officer Mike Schroepfer announced limits, effective that day, on the Events, Groups and Pages APIs; until then "any app could use the Pages API to read posts or comments from any Page." Facebook also brought the end of the Instagram Platform API forward from 31 July to the same day. Netvizz, a widely used research tool, then failed Facebook's new app review, and its developer, Bernhard Rieder of the University of Amsterdam, stepped back from it. An API, or application programming interface, is the channel through which outside software requests data.

    LessonPrivacy fixes made without consulting researchers broke public-interest research tools along with the abusive ones.

    about.fb.comeprints.qut.edu.au

    Credit

    Facebook; account by Axel Bruns

  3. 2020

    Platform data with added noise

    On 13 February 2020 Facebook and Social Science One, a project at Harvard's Institute for Quantitative Social Science, published the first codebook for a dataset on how people viewed and shared web links on Facebook. It is protected by differential privacy, which adds random noise to the counts so that no individual can be picked out. By its tenth version, in April 2023, it covered about 68 million URLs shared from January 2017 to October 2022, in over 3.1 trillion rows. Access is only through Social Science One's proposal process, and the codebook warns that the noise biases statistical results, most often toward zero.

    dataverse.harvard.edudataverse.harvard.edu

    Credit

    Facebook; Social Science One, described by Gary King of Harvard and Nathaniel Persily of Stanford

  4. 2024

    Suit against researchers dismissed

    X Corp. sued the Center for Countering Digital Hate, a nonprofit that had published reports on hate speech on X, arguing that its researchers broke X's terms of service by compiling public posts and that the reports cost it advertisers. On 25 March 2024 US District Judge Charles Breyer dismissed the suit, writing that it was "unabashedly and vociferously about one thing," punishing the group for its speech, and that X had failed to "allege losses based on technological harms" from the scraping. X said it would appeal.

    ksat.com

    Credit

    Judge Charles Breyer

  5. 2024

    CrowdTangle switched off

    Meta, which bought CrowdTangle in 2016, shut down the free tool for tracking public posts on Facebook and Instagram on 14 August 2024. Mozilla and, separately, dozens of groups had asked it to keep the tool running through at least January 2025, past the US presidential election. Its replacement, the Meta Content Library, is open only to academic researchers and nonprofits, which leaves out most news organizations.

    LessonA monitoring tool owned by the platform it monitors can be withdrawn just before the events it was used to watch.

    lasvegassun.com

    Credit

    Meta

  6. 2025

    First fine over research access

    On 5 December 2025 the European Commission fined X €120 million in the first decision that a platform had broken the EU's Digital Services Act (DSA). One of the three breaches was that X's terms barred independent collection of public data, such as scraping, and its access procedures put up further hurdles, against Article 40(12), which requires the largest platforms to give researchers access to public data. X was given 90 working days to submit an action plan for its ad repository and researcher access.

    eucrim.eu

    Credit

    European Commission

  7. 2026

    A court orders data access

    On 17 February 2026 the Higher Regional Court of Berlin ordered X to give Democracy Reporting International, a nonprofit, access to its public data through its API, free of quota limits, until 30 June 2026, for research ahead of Hungary's elections. Overruling a lower court on jurisdiction, it held that Article 40(12) gives researchers a right they can enforce in court themselves, not only through the Commission.

    techpolicy.press

    Credit

    Democracy Reporting International; Gesellschaft für Freiheitsrechte (Society for Civil Rights)

Watch for

  1. Final rulings for TikTok, Meta

    The Commission said on 24 October 2025 that it had preliminarily found TikTok, and Meta for Facebook and Instagram, in breach of their duty to give researchers adequate access to public data, which brought to four the platforms it had told so.

    Achieved whenthe Commission adopts final non-compliance decisions, as it did for X.

    digital-strategy.ec.europa.eu

    Claimed by

    European Commission, October 2025

  2. Internal data for vetted researchers

    The Commission said its delegated act of 2 July 2025 "enables researchers to obtain unprecedented access to platforms' internal data," with national regulators vetting the first researchers from October 2025. The DSA40 Data Access Collaboratory, which collects researchers' own reports of their applications, had by 8 September 2026 logged 70 complete applications, with 24 accepted and 27 rejected, but no accepted request for non-public data.

    Achieved whena vetted researcher publishes results from non-public data delivered under Article 40(4).

    digital-strategy.ec.europa.eudsa40collaboratory.eu

    Claimed by

    European Commission, July 2025

Open

  1. 01

    Canary posts for research APIs

    A test of whether platforms' research data tools return what was actually posted, using only a team's own posts as the answer key. The team publishes about 50 clearly labeled test posts per platform from its own accounts on Facebook, Instagram, TikTok, YouTube and Bluesky, logs each post's ID and time, then queries each research tool daily for 60 days: the Meta Content Library API, TikTok's Research API, YouTube's Data API under its researcher program, and Bluesky's open firehose. It records whether each post is found, how soon, and how the tool's view and like counts compare with the account's own dashboard.

    The brief
    Credit · unclaimed
  2. 02

    Practice copies of research APIs

    Mock versions of three research data tools, TikTok's Research API, the Meta Content Library API and X's API, that answer the same queries in the same documented formats with entirely synthetic posts and accounts, so researchers can build and test their code, and reviewers can rerun it, before or without access to real data.

    The brief
    Credit · unclaimed

Open 01 · the brief

Canary posts for research APIs

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A test of whether platforms' research data tools return what was actually posted, using only a team's own posts as the answer key. The team publishes about 50 clearly labeled test posts per platform from its own accounts on Facebook, Instagram, TikTok, YouTube and Bluesky, logs each post's ID and time, then queries each research tool daily for 60 days: the Meta Content Library API, TikTok's Research API, YouTube's Data API under its researcher program, and Bluesky's open firehose. It records whether each post is found, how soon, and how the tool's view and like counts compare with the account's own dashboard.

Why it is still open

Earlier audits measured the tools against other people's posts, so they had no exact record of what had been posted. AI Forensics reported in June 2025 that TikTok's Research API returned no metadata for one in eight of 260,000 videos gathered through data donations. Luka Bekavac and Simon Mayer rebuilt the feeds two test accounts saw on TikTok and Instagram during two elections and found that the TikTok Research API and the Meta Content Library left out up to about 50% of those posts and stripped up to about 83% of their context (preprint, revised September 2026). Both audits covered only TikTok's and Meta's tools and measured them against posts the researchers had not written; no test was found that used researchers' own posts as a known answer key, or that covered YouTube or Bluesky.

aiforensics.orgarxiv.org

Built with what exists

Approved researcher access to each tool; TikTok's official research API wrapper and Cybersecurity for Democracy's open-source tiktok-library; Python and a scheduler; no engagement bought or faked.

  1. First step

    Apply for researcher access to TikTok's Research API and the Meta Content Library, and fix the log format for posts and queries.

  2. Done looks like

    For each platform, the share of test posts found, the median delay before a post appears and the error in view and like counts, published with the code and raw query logs.

Open 02 · the brief

Practice copies of research APIs

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

Mock versions of three research data tools, TikTok's Research API, the Meta Content Library API and X's API, that answer the same queries in the same documented formats with entirely synthetic posts and accounts, so researchers can build and test their code, and reviewers can rerun it, before or without access to real data.

Why it is still open

Access takes months. The DSA40 Data Access Collaboratory's tracker reports an average of 138 days for three public-data requests to Facebook and 63 days for decisions at X, and a GitHub search for mock or emulated copies of these research APIs found none.

dsa40collaboratory.eu

Built with what exists

Each tool's public documentation written out as an OpenAPI description, a standard format for describing an API; a mock server such as Prism or FastAPI; synthetic posts written by an open-weight model such as Mistral Large 4 (see the Note "A trillion open parameters"); TikTok's official wrapper as a test client.

  1. First step

    Write an OpenAPI description of the TikTok Research API's video query endpoint from its public documentation.

  2. Done looks like

    Unmodified official client libraries complete every documented example query against the mocks with matching field names, paging and error codes, and the published code of three studies runs end to end on the synthetic data.