Software and SaaS
The software industry writes computer programs and sells them, as products or as subscriptions to services it runs over the internet (software as a service, or SaaS), and keeps them working, secure and supported for the businesses and people who use them.
1955
John Sheldon and Elmer Kubie, who had worked at IBM, set up Computer Usage Company to write programs for clients for a fee. Kubie later titled his account of it "Recollections of the First Software Company".
48
Milestones achieved across 7 domains.
15
To watch for: announced, not yet confirmed outside.
14
Open deliverables. Nobody has shipped them.
In one minute
Where it started
In 1955 Computer Usage Company began writing programs for clients for a fee, and IBM's customers formed SHARE to swap the programs they wrote themselves. In 1969, while the US Justice Department pursued an antitrust case against it, IBM began to charge for much of its software separately, and the Computer History Museum says software "changed from a giveaway to a competitive commercial product" overnight. In 2000 Salesforce began renting its software over the web by subscription, the model now called software as a service.
Where it stands
Code is kept in shared version control, built and tested automatically many times a day, and released in stages to computers that are often rented from cloud companies; in the Cloud Native Computing Foundation's 2024 survey, 80% of respondents ran Kubernetes, a system that runs programs across many machines, in production. AI tools are routine: 84% of respondents to Stack Overflow's 2025 survey used them or planned to, more distrusted their accuracy than trusted it, and a 2025 randomized trial found experienced developers took 19% longer with them. Large failures still come from small steps, such as CrowdStrike's 2024 content update and an AI agent that deleted a live database in 2025.
What is open
Companies now say AI agents write most of their new code, handle half their support conversations and investigate their incidents, and most of those figures are their own. The open jobs below are independent tests and public tables that would check such claims: how agent teams scale, how good AI-written tests are, whether release gates catch bad releases, how support bots answer. Each can be started this month with open-source software, public data and current AI models, on systems the builder owns.
Domains
AchievedWatch forOpen
Domain
Progress
Last achieved
Watch for
Open
Writing code
Turning what a program should do into instructions a computer can run, done by people and now also by AI tools.
7 · 2 · 2
Last achievedSlower, but felt faster2025
Watch forThree quarters of new code
2
openTesting
Checking that software does what it should, and finding where it does not, before its users do.
7 · 2 · 2
Last achievedThe test had leaks2024
Watch forAn agent that writes tests
2
openReleasing
Getting a finished change from the programmers' shared copy of the code to the people who use it: building it, checking it, and switching it on, for a few users first or for everyone at once.
6 · 2 · 2
Last achievedOne update, 8.5 million crashes2024
Watch forRisk scores replace code freezes
2
openRunning it in production
Keeping software up and answering for the people who use it: the computers it runs on, the people on call, and what happens when it fails.
7 · 2 · 2
Last achievedLocked out by its outage2021
Watch forAn SRE agent across Microsoft
2
openSecurity
Keeping software, and the data it holds, safe from attackers: finding flaws, fixing them, getting the fixes to users and catching break-ins.
7 · 3 · 2
Last achievedAI finds and patches flaws2025
Watch forThousands of flaws, few patched
2
openIn depthCybersecurity
Data
Storing customers' data, keeping it from being lost, answering questions from it, and handling it as the law requires.
7 · 2 · 2
Last achievedAn AI agent deletes production2025
Watch forAgents create most databases
2
openCustomer support
Answering customers' questions and fixing their problems after they buy, by staff, through help pages and public forums, and now by AI agents.
7 · 2 · 2
Last achievedA support bot invents policy2025
Watch forHalf of support by agents
2
open
Advances to build with
4 research notes

Gemini 4 Carbon
Cyber defenders first
Google says cyber defenders got Gemini 4 Carbon first, and its 88.2% flaw-finding score is Google's own run on a 183-task subset with guardrails off.

GitHub Copilot Fusion
Effort, not model
GitHub Copilot Fusion picks the AI models for each request, and in a test GitHub paid for, its top setting scored 2 points below a rival at about half the cost.

Cursor Agent SDK
Findings that cite notes
Cursor released a kit for building AI agents, and its example, a code reviewer that cites the note behind each finding, is known only from Cursor's account.

Inkling 2 Small
Checked without being asked
The maker of Inkling 2 Small, Thinking Machines, says the AI model checks its work without being asked, a habit that no published test measures.
Open in software and saas
3 of 14 open deliverables
Open · Writing code
Brooks's law, tested on agents
A public experiment that gives the same coding tasks to teams of 1, 2, 4, 8 and 16 AI coding agents working in parallel on one codebase. It records how long each team takes, what it costs, and whether the result passes the project's tests. A second condition adds agents halfway through a task, the case Brooks wrote about.
Built with what exists
coding agents that run from the command line, such as Claude Code or GitHub Copilot's agent (see the Note "Effort, not model"); the Cursor Agent SDK to script teams of agents (see the Note "Findings that cite notes"); git worktrees, which give each agent its own copy of the files; tasks and tests from CooperBench.
Open · Testing
Mutation scores in public
A public table, updated monthly, that shows for the 100 most-downloaded Python packages what share of planted bugs (mutants) each package's own tests catch. Then, for the ten weakest, use a coding agent to write tests that catch the surviving mutants, and offer them to the maintainers as pull requests.
Built with what exists
mutmut or Cosmic Ray, mutation testing tools for Python; each package's own test suite; PyPI download counts to choose the packages; Hypothesis for property-based tests; a coding agent such as Claude Code or Inkling 2 Small (see the Note "Checked without being asked") to write the new tests.
Open · Releasing
A test for canary gates
A public test bench for canary analysis. That is the check that runs while a new release serves a small share of users, say 10%, and decides whether to send it to everyone or roll it back. Plant 30 bad releases in an open-source demo shop (crashes, slow pages, failed payments, a memory leak that shows only after 20 minutes) and 30 good ones. Run every release through a rule-based gate and an AI gate, and publish what each one caught and missed.
Built with what exists
the OpenTelemetry Demo, an open-source online shop made of many small services, whose built-in feature flags already cause failures such as payment errors, a memory leak and slow images; a Kubernetes cluster (kind on a laptop, or a small cloud cluster); Argo Rollouts with Prometheus metrics for the rule-based gate; the argoproj-labs metric-ai plugin with a current model such as Claude or GPT for the AI gate; NVIDIA OpenShell to keep the agent inside the test cluster (see the Note "Stopped in milliseconds").