Industries / Technology / Software and SaaS
Developer tools
Developer tools are the programs that programmers use to write code, keep track of its versions, test it, turn it into running software and watch it at work, from text editors and compilers to code hosts, build servers, monitoring services and AI coding assistants.
1952
Grace Hopper had the A-0 system running on the UNIVAC I computer. It found numbered routines on a tape and joined them into a program, and Hopper called it "the first compiler", a program that turns a programmer's notation into the computer's own instructions.
40
Milestones achieved across 6 domains.
13
To watch for: announced, not yet confirmed outside.
13
Open deliverables. Nobody has shipped them.
In one minute
Where it started
In 1952 Grace Hopper's A-0 system let the UNIVAC I assemble a program from routines stored on tape, and Yale University's biography of her calls it the first compiler. In the 1960s and 1970s the basic kit was built at MIT, Bell Labs and Berkeley: editors that search by pattern (QED), version control that stores each change (SCCS, 1972), a tool that rebuilds only what changed (Make, 1978) and a checker for likely bugs (lint, 1978). In 1983 Borland's Turbo Pascal put an editor and a compiler in one program that sold for $49.95.
Where it stands
Most of the kit is free and shared. By GitHub's count more than 180 million developers worked on its site in 2025, pushed nearly 1 billion commits and merged about 43 million pull requests a month; in Stack Overflow's 2024 survey 74% of respondents used Microsoft's free VS Code editor. Code is built and tested by services such as GitHub Actions, packed into Docker containers and watched through open standards such as OpenTelemetry. AI agents now write and review code too: one count found more than 456,000 pull requests opened by agents by mid-2025, accepted less often than people's.
What is open
Many current claims about developer tools, from the accuracy of AI code reviewers to compilers and browsers written by teams of agents, come from the companies that make them. The open jobs below are outside tests on public code: typing delay measured with a light sensor, what AI editors send off a machine, SHA-256 support before Git 3.0, AI reviewers beside rule-based checkers, the cost and overhead of monitoring, and whether agent-built software passes the standard test suites. Each can be started this month with free tools and current AI models.
Domains
AchievedWatch forOpen
Domain
Progress
Last achieved
Watch for
Open
Code editors
The programs in which programmers write and change code, from plain text editors to integrated development environments (IDEs), single programs that also build, run and check the code.
7 · 1 · 2
Last achievedA new IDE withdrawn2025
Watch forAny agent in any editor
2
openVersion control and review
Keeping every past version of a project's code, merging the work of many people, and checking each change before it goes in.
7 · 2 · 2
Last achievedMercurial repositories switched off2020
Watch forSHA-256 by default
2
openTesting and checking
Tools that look for mistakes in code, either by reading it without running it (static analysis) or by watching it while it runs, and tools that drive a program the way a user would to see whether it still works.
5 · 3 · 3
Last achievedTests that cannot be trusted2016
Watch forEach maker wins its test
3
openBuild and deploy
Turning source code into a program that runs, fetching the outside code it depends on, and running those steps automatically on shared machines each time the code changes.
7 · 2 · 2
Last achievedA moved tag leaks secrets2025
Watch forHardened base images, now free
2
openDebugging and monitoring
Seeing what a running program actually does, from stopping it to inspect it line by line to collecting logs, measurements and traces from programs in use, so that faults can be found and fixed.
7 · 2 · 2
Last achievedA $65 million monitoring bill2022
Watch forAn AI debugger's own score
2
openAI coding
AI models that write, change, test and review code, from suggestions inside a code editor to agents that work through a task on their own and hand back a finished change.
7 · 3 · 2
Last achievedA browser by agents, judged2026
Watch forA compiler from 16 agents
2
open
Advances to build with
3 research notes

GitHub Copilot Fusion
Effort, not model
GitHub Copilot Fusion picks the AI models for each request, and in a test GitHub paid for, its top setting scored 2 points below a rival at about half the cost.

Cursor Agent SDK
Findings that cite notes
Cursor released a kit for building AI agents, and its example, a code reviewer that cites the note behind each finding, is known only from Cursor's account.

Inkling 2 Small
Checked without being asked
The maker of Inkling 2 Small, Thinking Machines, says the AI model checks its work without being asked, a habit that no published test measures.
Open in developer tools
3 of 13 open deliverables
Open · Code editors
Keystroke delay, measured alike
A public table of typing latency, the time from pressing a key to seeing the character on screen, for ten editors: VS Code, Cursor, Zed, Visual Studio, IntelliJ IDEA, Sublime Text, Neovim and Emacs in a terminal, Emacs in its own window, and Notepad as a baseline. Each is timed in a small file and in a 50,000-line project, with AI features on and off, on Windows, macOS and Linux. A small board plays the keyboard and a light sensor on the screen sees the character appear, so no editor's own clock is trusted.
Built with what exists
Typometer for a software-only cross-check; a microcontroller board that can act as a USB keyboard, such as an Arduino Leonardo; a photodiode, a light sensor, read by the same board; a monitor running at 120 Hz or more; Python for the statistics.
Open · Version control and review
SHA-256 support, tested
A public table, rerun every month until Git 3.0 ships, that tests by script whether each code host, Git library and tool works with a SHA-256 repository. Eight operations: create, push, clone, fetch, open a pull or merge request, run a CI job (an automatic build and test), show line history in an editor, and mirror to a second host. Hosts: GitHub, GitLab, Bitbucket, Azure DevOps, Codeberg and a self-run Gitea. Libraries that programs use to read Git data: libgit2, JGit, go-git, gitoxide and Dulwich. Tools: GitHub Actions' checkout step, GitLab CI, Jenkins' Git plugin, and the Git views in VS Code and a JetBrains IDE.
Built with what exists
Git 2.56 or later (git init --object-format=sha256 makes a SHA-256 repository); free accounts on each host; a shell or Python test harness; a scheduled CI job to rerun it each month.
Open · Testing and checking
AI reviewers beside static analyzers
A public test that gives the same 100 code changes to AI code reviewers (CodeRabbit, GitHub Copilot code review, Cursor Bugbot, Greptile) and to rule-based static analyzers (Infer, Semgrep, CodeQL). Half the changes are known to introduce a bug and half have no known bug, so false alarms can be counted as well as catches. Every comment from every tool is published.
Built with what exists
Defects4J, a collection of 854 reproducible bugs from open-source Java projects, each with its fix, from which a change that reintroduces the bug can be made; forks of those projects on GitHub; free or trial plans of each AI reviewer (see the Note "Findings that cite notes" for the kit Cursor builds its agents with, and the Note "Effort, not model" for GitHub Copilot's current setup); Infer, Semgrep and CodeQL, which all analyze Java.
Where it sits
Broader