Trustworthy Computer Vision Integrity Assurance for Data, Models and Inference Outputs in Multi-Contributor Pipelines

Official description

2. Description. 2.1. Background. Operational computer vision pipelines may combine training data from multiple contributors, pretrained or vendor-supplied models, and inference outputs consumed by downstream systems. This creates distinct integrity and assurance risks across the data, model and inference lifecycle. Data may contain deliberately or inadvertently mislabelled samples, duplicated content, out-of-distribution material or trigger-based backdoors. A model may be substituted, modified or contain hidden behaviour that is not apparent during routine validation. Inference records may also be replayed, replaced or altered after generation unless they are cryptographically linked to the exact input, model and processing chain that produced them. Existing controls often address only individual parts of this lifecycle. The challenge is therefore to create a unified, evidence-based assurance layer that can assess these risks without assuming that every contributing source is trusted. of Fos 2.2. Detailed Description. Teams are required to build an extensible computer-vision assurance framework that can evaluate a contributed dataset, a trained model and associated inference records, and produce an evidence-based assessment of integrity and risk. The solution should not be hard-coded to a single model architecture or dataset. Five core capabilities are required. 2.2.1. Training-Data Integrity. Identify suspicious or anomalous samples associated with trigger injection, label flipping, systematic mislabelling, near-duplicate flooding and out-of-distribution insertion. Where contributor, batch or source metadata is available, the system should aggregate sample-level evidence into a source-level risk assessment rather than flagging samples in isolation. 2.2.2. Model Integrity. Assess whether a supplied model exhibits anomalous, substituted or backdoor-like behaviour using methods appropriate to the level of access available. Approaches may include behavioural fingerprinting, trigger search or reconstruction, parameter or activation statistics, and comparison against a defined reference battery. The system must state the access assumptions, confidence and limitations of its assessment. 2.2.3. Inference Provenance and Output Integrity. Create a verifiable cryptographic binding among the input image, model identifier or weight digest, preprocessing and inference configuration, and resulting output. The design should make post-hoc alteration, substitution or replay of protected inference records detectable through hashes, signatures and appropriate sequence, timestamp or nonce controls. 2.2.4. Distribution-Shift and Anomaly Assessment. Detect material deviation from a declared reference distribution, including changes caused by terrain, season, sensor, illumination or acquisition conditions. The system should characterise the observed shift, provide a calibrated risk or confidence score, and distinguish probable operational drift from suspicious manipulation where the available evidence supports such a distinction. 2.2.5. Analyst-Facing Assurance and Governance. Every flag must include a human-readable reason, supporting evidence, confidence or severity, the affected asset, and a recommended disposition such as accept, review or quarantine. The solution must maintain a tamper-evident audit trail and explicitly declare attack classes or conditions that it does not support. 2.2.6. Constraints, the complete evaluation workflow must operate offline and in an air-gapped environment with no dependency on cloud services or external APIs. The solution should ingest common computer-vision dataset formats, including COCO and YOLO, and support the organiser-defined reference model formats, including ONNX and PyTorch/TorchScript. Baseline integrity assessment must not require retraining of the contributed model, although optional remediation may use retraining. Methods that require white-box access must fall back gracefully or clearly report that the relevant assessment is unavailable when only black-box access is provided. 2.3 Expected Solution. Teams are expected to develop a model-agnostic assurance system for assessing the integrity of training data, trained computer-vision models and inference outputs. The solution shall use publicly available or team-generated datasets and models, with teams developing reproducible methods to introduce representative poisoning, backdoor, substitution and tampering scenarios for testing. The system should identify suspicious data or contributor behaviour, assess model integrity, detect tampering of inference records, provide supporting evidence for each finding, and generate a clear assurance report stating confidence, limitations and recommended action. Each team must submit the source code, architecture and setup notes, the assurance-report schema, a reproducible audit log, and a clear coverage statement identifying supported attack classes, assumptions and known limitations.