Framework for field evaluation of signal cooperative intelligent transport system use cases as based on the Ipswich Connected Vehicle Pilot

Cooperative Intelligent Transport Systems (C-ITS) have the potential to greatly increase traffic safety on roads. To ensure C-ITS implementation is effective, the equipment within the ecosystem must communicate using standardised, available and reliable messages that can be trusted for use in end-user warnings. If the warnings displayed to the end-user are incorrect, untimely or inappropriate the system may have the reverse effect on user safety instead.
Queensland Department of Transport and Main Roads (TMR) identified the need for government investment in C-ITS roadside stations (R-ITS-S) and through the Cooperative and Automated Vehicle Initiative’s (CAVI) Ipswich Connected Vehicle Pilot (ICVP) installed and studied the performance of the equipment as part of a larger behavioural study. This paper describes a framework created for the assessment of R-ITS-S suitability against ten key criteria, with a focus on reuse in future Australian deployments of C-ITS, projects and pilots. These criteria were proposed after reviewing current literature of European and U.S standards and guidelines.
The assessment criteria within the framework includes: 1) Roadside Coverage – the effective range of the roadside unit (RSU); 2) Availability –the percentage of time the RSU and its substituent components are operational; 3) Reliability – a sub-category of availability, finding the Mean-Time-Between-Failures (MTBF), Mean-Time-To-Repair (MTTR) and the total number of failures; 4) Latency –the total time for communicating messages between sent and received timestamps; 5) Usability – the accuracy of the MAP Extended Message (MAPEM) file transmitted by the RSU with respect to the road centreline; 6) Security – the percent of security certificates that fail under conditions where it is expected that certificates should be passed; 7) Refreshment Rate – the assessment of updated signal statuses between the field processor and RSU; 8) Classification Correctness – the assessment of how correct the presented signal use case messages are; 9) Integrity – the assessment of instances where the integrity of the system is compromised; and 10) Conformance – a judgement of the conformance of the ICVP against the European standards.