Industries / Finance and business services / Payroll
Tax withholding and filing
This is the work of taking income tax and social contributions out of each pay, paying them to the state, and reporting to the tax office what was paid and taken.
5
Milestones achieved, from 1943 to 2020.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1943
Tax taken at the source
Until 1943 a US worker paid income tax in the year after earning the income. The Current Tax Payment Act made employers withhold income tax from wages from 1 July 1943, at 20% of pay above a weekly exemption, of which 3% covered the wartime Victory tax, by the Senate Finance Committee's report on the bill. So that nobody paid two years of tax in one, the final law canceled 75% of the lower of each person's 1942 or 1943 tax; this was a modified version of the "pay-as-you-go" plan of Beardsley Ruml, treasurer of the R. H. Macy department store and chairman of the Federal Reserve Bank of New York, joined to a Treasury plan for withholding, by the committee's 1946 legislative history.
Credit
US Congress; the US Treasury; Beardsley Ruml
1944
Pay as you earn
Britain's Pay As You Earn system (PAYE) began on 6 April 1944. Each employee gets a code number from the tax office, and the tax is cumulative: each payday's deduction is worked out on pay so far in the tax year, so allowances unused in a low week are not lost and tax is not over-deducted; a leaver's P45 form passes the running totals to the next employer. By the end of January 1944, 15 million people earning more than £100 a year had received their code notices, by HMRC's account, and Taxation magazine counts about one million employers and 16 million employees who had to be taught the system. More than 30.5 million people now pay income tax and National Insurance through PAYE, by HMRC's count.
Credit
The Inland Revenue and its secretary, S. Paul Chambers, by Taxation magazine's account
2013
Reported on every payday
From April 2013 UK employers have sent HMRC the details of each payment "on or before" payday, through Real Time Information (RTI), which HMRC calls the biggest change to PAYE since 1944. More than 1.5 million employers moved over in stages. HMRC's 2017 review counts more than 40 million employees and pensioners and over 100 million transactions a month, and £672 million saved from 2013–14 to 2016–17 through fewer overpaid tax credits. The same review found that duplicate employment records had caused up to 80% of the charges employers disputed, and that £390 million of possibly spurious employer debt was open on 31 March 2017.
LessonWhen employers cannot see the records the tax office holds about their staff, small mismatches turn into disputed bills that take years to clear.
Credit
HMRC, with the Department for Work and Pensions, whose Universal Credit benefit uses the earnings feed
2019
Real-time reporting spreads
From 2019 Irish employers must, on or before each payment, work out deductions from the latest Revenue Payroll Notification for each employee and send pay and tax details to Revenue, Ireland's tax office; the annual P35 return was discontinued. Australia extended Single Touch Payroll, under which employers report payroll and superannuation (retirement savings) information to the tax office when wages are paid, to all employers regardless of size from 1 July 2019.
Credit
Revenue (Ireland); the Australian Parliament and the Australian Taxation Office
2020
The W-4 drops allowances
Form W-4 is the form a US employee fills in to tell the employer how much federal income tax to withhold. For 2020 the IRS redesigned it: the old form counted "withholding allowances" whose value depended on the personal exemption, which a change in law had removed, and the new form instead asks in five steps for filing status, other jobs, credits for children and other income. Employees with a form from before 2020 did not have to file a new one.
Credit
Internal Revenue Service
Watch for
Tips and overtime on W-2s
A 2025 US law, Public Law 119-21, lets workers deduct up to $25,000 of qualified tips and up to $12,500 ($25,000 for married couples filing jointly) of qualified overtime from income for tax years 2025 to 2028. Qualified overtime is only the premium the Fair Labor Standards Act requires, the "half" of time and a half. From tax year 2026 employers must report these on the W-2, the annual pay and tax statement: box 12 code TT for overtime, code TP for cash tips, and a Treasury tipped occupation code in a new box 14b.
Achieved whenThe 2026 W-2s, due 1 February 2027, have been filed and the IRS or another outside party, such as the Treasury Inspector General for Tax Administration, reports how many carried the new codes and how many had to be corrected.
Claimed by
Internal Revenue Service, in its 2026 W-2 instructions, January 2026
Benefits taxed through payroll
HMRC says that from 6 April 2027 UK employers must report company cars and their fuel, company vans and their fuel, and medical insurance through payroll software in real time, instead of on the annual P11D form, and that other benefits, except loans and accommodation, must follow from 6 April 2028.
Achieved whenThe April 2027 start goes ahead and HMRC reports how many employers report benefits through payroll and how far P11D filings fell.
Claimed by
HMRC, guidance published November 2026
Open
01
W-2 tips and overtime, tested
A public set of at least 150 invented pay years for hourly and tipped workers, each with its pay records and the correct 2026 W-2 entries: qualified overtime under box 12 code TT, cash tips reported to the employer under code TP, and the Treasury tipped occupation codes in box 14b. Then a published score for payroll products and free calculators run against it.
The briefCredit · unclaimed02
An open payroll benchmark
A benchmark of at least 300 withholding tasks with fixed answers, covering US federal, state and local withholding and the worked examples that the tax offices of the UK, Ireland and Australia publish, graded by a script to the cent, released in full, and a published run of several AI models, three times each.
The briefCredit · unclaimed
Open 01 · the brief
W-2 tips and overtime, tested
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public set of at least 150 invented pay years for hourly and tipped workers, each with its pay records and the correct 2026 W-2 entries: qualified overtime under box 12 code TT, cash tips reported to the employer under code TP, and the Treasury tipped occupation codes in box 14b. Then a published score for payroll products and free calculators run against it.
Why it is still open
The first W-2s with these entries are due 1 February 2027, and the IRS instructions leave room for error: only the "half" of time and a half counts as qualified overtime, box 14b takes up to two occupation codes, and "000" must be entered if any tips came from an occupation not on Treasury's list. Two small tools already compute the overtime figure from timesheet exports, the Qualified Overtime Checker and Tallyhalf; each lists the cases it handles and the ones it leaves out, such as hospital 8-and-80 schedules, but neither publishes test cases with answer keys, and this research found no public test set.
Built with what exists
The IRS's 2026 W-2 instructions; the Labor Department's overtime rules (29 CFR part 778); Python for the answer keys; the two existing tools as first subjects; an AI coding agent to write the case generator, checked only by the hand-worked answers, the way AWS checked an emulator its agents wrote (see the Note "Code no person read").
First step
Write 20 cases from the edge cases the two existing tools list, work the answers by hand, and run both tools on them.
Done looks like
150 pay years covering at least ten named edge cases (two pay rates in a week, a bonus spread back over the weeks it was earned, state daily overtime that does not count, a week that straddles the year end, tips in three occupations, tips in an occupation not on the list), each answer citing its rule, released under an open license, and a public table scoring at least three tools, with every mismatch explained.
Open 02 · the brief
An open payroll benchmark
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A benchmark of at least 300 withholding tasks with fixed answers, covering US federal, state and local withholding and the worked examples that the tax offices of the UK, Ireland and Australia publish, graded by a script to the cent, released in full, and a published run of several AI models, three times each.
Why it is still open
Remote, an international payroll firm, published PayMark in October 2026: 2,000 payslips from 50 countries on which AI models must work out employees' statutory deductions and employers' contributions. The best of 12 systems tested got 26.0% of payslips fully right, by Remote's own run, and its post links no data, so nobody else can rerun it. The largest public test this research found has ten tasks, graded by hand (velvetchief/payroll-llm-eval, October 2026). Data to grade against now exists: one GitHub repository holds 2026 withholding tables for 51 US jurisdictions with 140 published worked examples they are tested against, and another tests federal withholding against every row of the IRS biweekly wage bracket table. Both are self-described and less than a year old.
Built with what exists
Those repositories; IRS Publication 15-T; a simple agent harness; models such as Claude Opus 5.5 and Meta's Muse Ember, which matched Claude Haiku 5.5 on an outside firm's tests at a third of the cost per task (see the Note "The jobs agents repeat").
First step
Turn the 140 worked examples in the open tables repository into question-and-answer pairs and run one model on them.
Done looks like
300 tasks, each traced to a published example or table, graded by script; at least four models run three times each; every task, failed answer and raw output public, so that another team gets the same scores.