Challenge 4
Generative AI for Automatic Test Case Generation from Crash Databases & Standards_Siemens Industry Software NV
Team:
SmartRoads, Portugal
Proposal title:
The Road as a Witness: From European Crash Records and Street-Level Evidence to Calibrated Validation Scenarios
Organisation
SmartRoads is a Portuguese SME that runs a production vision-language AI pipeline rating road infrastructure from imagery for municipalities. Its founders combine road engineering, with nine years at Portugal’s National Laboratory for Civil Engineering (LNEC), with applied AI, and it won EIT Jumpstarter 2025 (Urban Mobility & Smart Cities).
Project info
SmartRoads starts from a simple idea: a crash record says what happened, but the road at that location shows where and why. The four named databases (STATS19, BAAC, StatBel and Barcelona) are coded tables without narratives or photographs, often lacking speed limits and junction geometry.
For Functional Scenarios, the EU-built EuroLLM-22B maps each country’s fields and codebooks onto one PEGASUS-style taxonomy. For Logical Scenarios, the design centre of the proposal, every record is geocoded to its OpenStreetMap road segment, paired with the nearest street-level image from Mapillary or Panoramax plus map topology, and read by an open vision-language model (Qwen3-VL, with Mistral Small benchmarked) prompted with iRAP road-assessment rubrics. Each attribute, such as lanes, cycle infrastructure, sightline or lighting, carries a value, an evidence crop, a source and a confidence calibrated per attribute class by conformal prediction. Rule-based and embedding matching against Euro NCAP-aligned references produces a gap report, and scenarios are exported as OpenSCENARIO 1.3 files that run in Simcenter Prescan and CARLA.
All models are open-weight, self-hosted in the EU and need no training. Targets include ten times less expert time per record and at least 90% of attributes with a checkable evidence pointer.
Details
Europe’s open crash databases record what happened, but as coded tables with no narrative or photographs, and often without the speed limit or junction layout. Scenarios built from them miss the road infrastructure that shaped each crash, which matters especially for the cyclists and pedestrians who account for most road deaths in cities.
SmartRoads proposes to treat the road itself as a witness. Each crash record is placed on its OpenStreetMap road segment and paired with street-level imagery of the site, which an open vision-language model reads against established road-engineering rubrics. The result is a Logical Scenario that inherits the real lanes, cycle infrastructure and sightlines of its location, with an evidence crop, a source and a calibrated confidence for every attribute. Scenarios are matched against Euro NCAP-aligned references to show which infrastructure-related conflicts are not yet tested, and exported to OpenSCENARIO for Simcenter Prescan and CARLA. The extraction engine already runs in production for municipal customers.
Starting from TRL 3, the team will join existing components into one containerised pipeline, reaching TRL 4 by the mid-term checkpoint and TRL 5 with the full pipeline running on four national databases and scored on a 200-record blind test set. Targets include at least 85% expert agreement on attributes corroborated by OpenStreetMap, at least 90% of attributes with a checkable evidence pointer, 80% precision and 70% recall when matching to Euro NCAP-aligned references, calibrated confidence within 5 points of nominal coverage, and ten times less expert time per record. The same evidence layer also serves road authorities planning for Vision Zero.
