Industries / Industrial equipment and services / IoT and industrial automation / SCADA integration
Storing plant data
Historians are the databases that keep every reading a SCADA system collects, often for years, so that engineers can look back at what happened and analysts can work from the record.
4
Milestones achieved, from 1985 to 2024.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1985
One database for plant readings
Oil Systems Inc., founded in October 1980 by the chemical engineer J. Patrick Kennedy and later renamed OSIsoft, released the first version of its PI System ("Plant Information System") in 1985, POWER magazine reports. It gathers readings from instruments, controllers and other software into one store that people query from spreadsheets and browsers; OSIsoft said in 2020 that it ran at more than 19,000 sites, and AVEVA, a UK industrial software company, completed its purchase of OSIsoft, valued at $5 billion, in March 2021.
Credit
J. Patrick Kennedy, OSIsoft
1987
A door that drops points
Edgar H. Bristol of the Foxboro Company patented a way to shrink a stream of process readings: from the last stored point a corridor of set width opens, readings are discarded for as long as one straight line through the corridor still fits them, and only the corridor's end points are kept. Known as the swinging door method, it is the compression step in AVEVA's PI System, which first drops changes smaller than a set deadband (the "exception" test), by AVEVA's 2023 account.
Credit
Edgar H. Bristol, The Foxboro Company (patent filed October 1985, granted May 1987)
2004
Compression spoils the analysis
Nina Thornhill, Shoukat Choudhury and Sirish Shah showed that the compression many commercial historians apply before archiving "interferes with many types of data-driven analyses", such as statistical monitoring, control loop benchmarking and fault detection, and gave a test that tells from stored data whether it was compressed. Engineers at Saudi Aramco reached the same result in 2025 for machine-learning models: compression that was too aggressive missed 35% of short anomalies in their test.
LessonSettings chosen to save disk space decide which questions the record can answer later.
Credit
Thornhill, Choudhury and Shah (Journal of Process Control); Reham Faqehi and colleagues, Saudi Aramco
2024
A cloud historian retires
Microsoft retired Azure Time Series Insights, a cloud service for collecting, storing and querying industrial sensor data over long periods, on 7 July 2024, and told users to move their data to other services.
LessonHistory kept in a vendor's cloud service has to be moved, at the owner's cost, when the vendor withdraws the service.
Credit
Microsoft
Watch for
Plain-language questions to plant data
AVEVA says its Industrial AI Assistant, built with Microsoft and OpenAI, lets users ask questions in ordinary language about data stored in its CONNECT cloud platform and summarize data over a period of time. Its service terms give "no guarantee of the accuracy" of the answers.
Achieved whenan outside group publishes how often its answers match the stored data on a fixed set of questions.
Claimed by
AVEVA, service description revised April 2026
An open database as historian
Inductive Automation says the Enterprise Historian it is building with Tiger Data on TimescaleDB, an open-source time-series extension of the PostgreSQL database, took in data more than 10 times faster, answered history queries about 10 times faster and used more than 10 times less disk than standard PostgreSQL, in its own test on 2.1 billion data points. General release was targeted for February 2027.
Achieved whenthe product ships and a user or outside tester reports similar figures on plant data.
Claimed by
Inductive Automation and Tiger Data, September 2026
Open
01
A shared compression test
A public benchmark that compresses the same real plant data with the methods historians use (the exception deadband and the swinging door, each at several settings) and with lossless methods, then reruns standard analyses on each version: mean and spread, anomaly detection, and detection of oscillating control loops. It reports file size, error and how much each analysis result moves.
The briefCredit · unclaimed02
Late data after link outages
A test rig that cuts the network link between a simulated remote site and a historian for set periods, from one minute to three days, lets the site's store-and-forward buffer (which holds readings until the link returns) refill the historian, and checks every stored point and every hourly average against a copy of the original readings.
The briefCredit · unclaimed
Open 01 · the brief
A shared compression test
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public benchmark that compresses the same real plant data with the methods historians use (the exception deadband and the swinging door, each at several settings) and with lossless methods, then reruns standard analyses on each version: mean and spread, anomaly detection, and detection of oscillating control loops. It reports file size, error and how much each analysis result moves.
Why it is still open
Each study uses its own data. The Saudi Aramco engineers who tested swinging-door compression in 2025, on a synthetic signal and on their own temperature and vibration data, wrote that "there are no standardized metrics to evaluate compression's impact on analytical accuracy across different industrial applications."
Built with what exists
Petrobras's 3W dataset of real and simulated oil-well sensor readings, whose data files carry a CC BY 4.0 licence; open swinging-door code such as gfoidl/DataCompression on GitHub; Python with pandas and NumPy; Parquet files with zstd compression as the lossless baseline.
First step
Code the swinging door from Bristol's 1987 patent, run it on ten 3W sensor channels at five corridor widths, and plot compression ratio against error.
Done looks like
A public repository with at least 5 datasets, 4 methods at 5 settings each, and a table of compression ratio, largest error and change in each analysis result, rerun by one other group.
Open 02 · the brief
Late data after link outages
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A test rig that cuts the network link between a simulated remote site and a historian for set periods, from one minute to three days, lets the site's store-and-forward buffer (which holds readings until the link returns) refill the historian, and checks every stored point and every hourly average against a copy of the original readings.
Why it is still open
The Time Series Benchmark Suite (TSBS), a common public benchmark for time-series databases, includes trucks that go offline and send batched, out-of-order data later, but it measures loading and query speed, not whether the stored record is complete and correct. Systems also treat late data differently: AVEVA's own guidance says out-of-order data bypasses the PI System's compression step.
Built with what exists
Docker; TimescaleDB, InfluxDB, QuestDB and openHistorian, an MIT-licensed historian from the Grid Protection Alliance whose storage accepts out-of-order inserts; Linux's tc netem tool to cut and slow links; a Python publisher with an on-disk buffer; the Eclipse Mosquitto MQTT broker.
First step
Run TimescaleDB and InfluxDB in Docker, cut a simulated site's link for one hour with netem, and count missing and duplicate points after it reconnects.
Done looks like
A public table, for each historian and each outage length, of points lost, duplicated and stored out of order, and of errors in hourly averages, from at least 20 runs each.