Challenge 4

Generative AI for Automatic Test Case Generation from Crash Databases & Standards_Siemens Industry Software NV 

Team: 

Aalen University of Applied Sciences – Human in Command Research Group (coordinator), with Starwit Technologies GmbH, Germany

Proposal title: 

TRACE – Traceable Reconstruction and Agentic Crash-to-Test Engineering

Organisation

Aalen University of Applied Sciences (Hochschule Aalen), Germany, coordinates this consortium through its Human in Command research group, led by Prof. Dr. Doris Aschenbrenner, which works on applied, human-centred and trustworthy AI. Technology partner Starwit Technologies GmbH builds open-source, camera-based perception for urban traffic.

Project info

TRACE builds an end-to-end, evidence-grounded pipeline from European crash records to simulation-ready test cases. Source-specific adapters harmonise crash databases, investigation reports, images, video, maps and V2X messages into a shared 6-Layer Structured Scenario Representation in which every element is a claim with provenance, confidence and epistemic status. Missing values stay visible, and generated information cannot silently become ground truth.

Open-weight LLMs extract structured claims from coded and narrative records using grammar-constrained JSON output and retrieval over codebooks and standards; vision-language models interpret imagery and video only where they add information; OpenStreetMap and, where available, CAM/DENM messages provide deterministic context. A knowledge graph with missingness-aware multi-view embeddings finds comparable cases, so missing parameters are derived as empirical ranges from better-documented crashes rather than guessed. Recurring patterns become Functional Scenarios, are enriched into uncertainty-aware Logical Scenarios, checked for kinematic and traffic-rule plausibility in SUMO and, where useful, concretised in CARLA or Simcenter Prescan. The catalogue is matched against Euro NCAP- and UNECE-aligned references to expose coverage gaps.

Targets include Functional Scenario macro-F1 of about 0.85, over 90% of attributes with resolvable evidence pointers, and 30–40% less manual scenario-authoring effort.

Details

European crash evidence is scattered across national databases, investigation reports, images, video, maps, infrastructure data and V2X messages. Engineers must interpret these sources by hand, infer missing parameters and rebuild scenarios, which limits both the scale and the traceability of automated-driving validation.

TRACE, developed by Aalen University’s Human in Command research group with Starwit Technologies, changes the unit of AI reasoning from documents to evidence-linked claims. Incomplete crashes remain partial: conflicting evidence can coexist and missing values stay visible. Language and vision-language models interpret unstructured evidence, while deterministic sources such as OpenStreetMap supply geometry wherever machine-readable facts exist. Parameters missing from one record are estimated as ranges from structurally similar, better-documented cases. An agentic orchestrator coordinates the tools, and the SUMO traffic simulator rejects physically inconsistent scenarios before higher-fidelity simulation in CARLA. Starwit’s camera-based traffic data from municipal sites lets the team compare accident-derived scenarios with observed traffic behaviour.

Starting from a TRL 2 proof of concept, the project will reach TRL 5 with a containerised pipeline that runs reproducibly on multi-country crash data, exports ASAM OpenSCENARIO where applicable, passes automated plausibility checks and has its traceability and coverage-gap results validated by domain experts. Targets include Functional Scenario macro-F1 of about 0.85, inferred parameter ranges containing the reference value in about 80% of cases, over 90% of exported attributes with a resolvable evidence pointer, catalogue-matching precision and recall of about 0.8, and 30–40% less manual scenario-authoring effort. The conflict patterns identified could also support early-warning systems in cities that already run traffic perception.