Ensuring the reliability, performance, and security of collaborative tools is critical in the automotive and cybersecurity CCAM domains. Within SELFY project, we developed a toolbox to designed to address future threats, risks, and attacks, providing a structured approach to tackling emerging cybersecurity challenges.
The diverse range of tools developed within SELFY presented unique testing requirements, making it difficult to standardize approaches. Maintaining consistency across simulation, laboratory, and real-world environments required careful coordination and alignment among partners.
To address this, testing was integrated early in the development, following industry best practices. Our approach was based on the ISO/IEC/IEEE 29119-1:2022 and the ASAM Test Specification Study Group Report 2022.
Comprehensive testing matrix
One of the key achievements in the validation phase is the development of a comprehensive testing matrix. This matrix provides a structured approach for test planning. The matrix allows a comparison of different tools and interfaces, defining planned tests, including differences and similarities in their testing methods.
The testing is structured around two primary elements:
- Test objects: assets that need to be tested for a specific tool or combination. Examples are data, interfaces, or software.
- Test methods: the approach to take to test a given object. Examples are reliability testing, security testing, functional testing, and integration testing.

Each SELFY partner contributed to the matrix, listing planned tests relevant to their respective tools. This matrix is a living document, providing updated information on the validation and verification status of the developed SELFY tools. By streamlining the testing process, the matrix enhances transparency, efficiency, and collaboration among stakeholders.
Standardised testing report
Another significant accomplishment is the introduction of a standardised testing report skeleton. This document provides a uniform template across SELFY tool test reports, making comparisons easier and meaningful.
The testing report includes essential information such as the testing matrix, scenarios, use and test cases, expected results, actual results, deployment details (set up used for the test), implementation specifics, and data-sharing highlights (data involved).
Similar to the testing matrix, each partner of a SELFY tool includes the test results of their executed tests. For example, the VSOC interfaces were tested automatically, and all endpoints sent the correct response. Such results were systematically recorded, ensuring comprehensive validation and verification.
This structured format supports transparent communication and collaboration, enabling stakeholders to track progress to ensure all tools meet standards. The unified report format also addresses the need for transparency and traceability in testing, enabling all stakeholders to have access to the same up-to-date information.
To sum up
The testing matrix and skeleton provide a clear and concise overview during the testing phase, making it easier to identify the status of the SELFY tools’ testing process.
These tools ensure that the significant test artifacts of the various SELFY tools within the CCAM ecosystem are managed in a consistent and easily comparable format. By fostering collaboration, transparency, and comparability, SELFY’s testing framework contributes to the ongoing advancement of secure and resilient CCAM technologies.
Authors: Technische Hochschule Ingolstadt