Your instrument export changed. The parser said everything was fine.
A firmware update adds one header line, every value shifts a row and the import stays green. Keep the raw file, carry the run context and hold unknown layouts.
On this page
- The import that succeeded on the wrong rows
- List the instruments and the exports people actually use
- Use supported exports before scraping anything
- Decide what every result must carry
- Keep the raw file and refuse unknown layouts
- Test the files that break parsers
- Check your data platform’s instrument coverage first
- Scope one instrument workflow you can accept and operate
- Questions we hear
The short version
- A parser that finishes without errors has not proved that any value landed in the right well.
- Keep every raw instrument file untouched, with a fingerprint, beside the results made from it.
- Carry well, sample, unit, method, instrument and software version with each value, or two runs cannot be compared.
- Use supported exports, existing standards and the platforms you already license, then build only the gap.
The import that succeeded on the wrong rows
On Monday the plate reader in the assay lab gets a firmware update. On Tuesday its export has one extra header line: the reader now records chamber temperature before the data block. The parser skips a fixed number of lines, so it now starts one line early and reads the column labels as row A. Every row shifts down one: row A’s absorbance lands against row B’s wells, row H’s readings are dropped and row A is stored as zeros. Most values look plausible. The job is green.
Two weeks later a scientist asks why the curves from run R-0412 on plate PL-118 look wrong. Nobody can say which runs used the new layout: nothing recorded the firmware version or kept the original file.
The file changed by one line. Every result moved by one row.
PL-118 and R-0412 are illustrative, not client data. The pattern is common: plate readers, qPCR systems, HPLC and flow cytometers each export their own CSV, XLSX or binary layouts, which can change with firmware, software or method.
List the instruments and the exports people actually use
For each instrument, record the model, the controlling software and version, the export people actually use and where the file goes next. Then watch a scientist prepare one run: which block they copy into a template, where they re-key the plate map, which units they add by hand and which settings never leave the instrument PC.
That shows why two instruments measuring the same thing still can’t be compared. One qPCR system reports Cq, another Ct; one writes Undetermined, another leaves a blank. One plate reader labels wells A1, another A01. Each difference is a decision someone makes by hand today, not always the same way.
- Blocks copied from the export into a template
- Plate map re-keyed from the ELN
- Units added from memory
- Instrument settings left on the instrument PC
- A layout change noticed only when a curve looks wrong
- Raw file kept with a content fingerprint
- Plate map linked from the ELN or LIMS
- Units and method stored with every value
- Instrument ID and software version recorded per run
- Unknown layouts held for review
Use supported exports before scraping anything
Ask each vendor what your installation supports: a documented export, an SDK or an API in the controlling software. Prefer those to screen-scraping or reverse-engineering a proprietary binary. A supported interface has a version history; a scraper breaks silently at the next update.
Device control is a separate question. SiLA 2 is a vendor-independent standard in which an instrument exposes its features as commands and properties to any SiLA client. SiLA can serve data as well as take commands, but where a supported export works, a first data project does not need it. A supported protocol also does not prove a physical method ran correctly.
Decide what every result must carry
FDA’s data integrity guidance puts it plainly: the number 23 is meaningless without its unit, and metadata can include the acquisition timestamp, the person who ran the test and the instrument ID. That guidance covers drug manufacturing, but the point holds for any cross-instrument comparison.
Agree a normalized result with the scientists who use it. At minimum:
- Result: value, unit, qualifier and measurement type, with the original kept beside any conversion.
- Position: plate ID and well, in the source notation and a canonical form mapped through the labware definition.
- Context: sample, condition, replicate and control role, joined from the ELN or LIMS plate map by explicit identifiers, never a typed filename.
- Run: instrument ID, software and firmware versions, method and version, key settings, operator and timestamps.
- Lineage: the raw file’s fingerprint, the parser version and the mapping version.
Before inventing a schema, look at the Allotrope Simple Model, which the Allotrope Foundation describes as a JSON representation of laboratory data that people and software can both read. If a model fits your technique, target it and note the fields it lacks.
- Capture
Store the raw file, fingerprint and run identity.
- Parse
Read a known layout into the agreed model.
- Contextualize
Join samples and plate map from the ELN or LIMS.
- Publish
Deliver reviewed results with their lineage.
Keep the raw file and refuse unknown layouts
Store every original file unchanged, with a content fingerprint, its instrument and its arrival time. When the parser improves, reprocess from the source, not from someone’s template.
Give every supported export a layout signature: expected header lines, block markers, column names, well count and software version. A file that does not match is held, not parsed by best guess. In the opening example, a signature check would have stopped R-0412 at the extra header line, before any value reached a well.
- Detect
Compare the file with known layout signatures.
- Hold
Stop the run and alert the instrument owner.
- Qualify
Test a new parser version on real exports.
- Release
Reprocess held runs under the approved version.
The instrument owner decides whether output from new firmware is scientifically equivalent. The pipeline applies that decision and records which parser version produced each result.
Test the files that break parsers
Use historical exports from before and after a software update. Before a pilot goes live, it should pass checks like these:
- Add a header line to a known export. The file is held as an unknown layout and no result is created.
- Import the same file twice under different names. The fingerprint recognizes the duplicate.
- Export one plate with A01 notation and one with A1. Both map to the same wells.
- Remove the plate map. Results wait for context instead of publishing without samples.
- Leave one well empty and mark another Undetermined. Each keeps its own state; neither becomes zero.
- Interrupt publication. The pipeline reconciles what arrived before retrying.
Then compare a normalized run with the scientist’s own spreadsheet for the same plate. Matching the right values to the right wells is the acceptance test; a green job status is not.
Check your data platform’s instrument coverage first
Platforms already cover much of this. TetraScience documents agents and connectors that pull files from Windows-based instruments and data from chromatography systems such as Empower and Chromeleon, and notes that some need extra third-party licenses. If you own such a platform, check its coverage of your instruments and versions first. Configure before you build.
The remaining gap is usually narrower: an unconfigured instrument, plate maps that never reach the results, a layout check, a review screen for held runs or the link to your experiment record. We build those pieces and hand them over with a runbook.
Scope one instrument workflow you can accept and operate
- Name one instrument family, method, destination and scientific owner.
- Record instrument models, software versions and the exports in use.
- Check vendor exports, SDKs and platform parsers before writing one.
- Agree the result model: well, sample, unit, method and run metadata.
- Keep raw files unchanged with a fingerprint and parser version.
- Hold unknown layouts and define who approves a new version.
- Test duplicates, notation changes, missing plate maps and empty wells.
- Measure preparation time per run and name the operating owner.
Request a call from this article and describe the instrument export your team prepares by hand most often. The first conversation needs no data; confidential files can wait for an NDA or your approved access process. See how we work with biotech teams. If your results arrive from an external partner rather than your own instruments, read the CRO data integration guide.
Questions we hear
Should we convert everything to the Allotrope Simple Model?
Only where a model fits the technique and your downstream tools can use it. It helps most when several instruments measure the same thing. Keep the raw file either way.
Do we need SiLA 2 or instrument control for this?
No. Most comparison problems are solved by reading supported exports reliably and adding context. Instrument control is a separate automation project.
Is this GxP software?
The intended use decides, and your quality team owns that call. Discovery research is often outside GxP. If results support regulated work, 21 CFR Part 11 and validation scope are your quality team’s decisions; we work to the requirements they set and provide the test evidence, change history and documentation they ask for. We do not certify compliance.
Sources
- Our Technology: the Allotrope FrameworkAllotrope Foundation
- SiLA StandardsSiLA Consortium
- Tetra IntegrationsTetraScience Developer Documentation
- Data Integrity and Compliance With Drug CGMP: Questions and AnswersU.S. Food and Drug Administration, December 2018