Software component qualification (ISO26262-8, clause 12) is an activity widely used with the intent of reusing existing software for Safety. In our first article, we had discussed the big picture view of component qualification with the analogy of a jigsaw puzzle. In this second part of the article, we will cover the following topics:
- The broad steps that need to be carried out for SW component qualification
- Challenges in qualification and possible solutions
The broad steps that need to be carried out for SW component qualification
Let us consider the example of a component qualification performed for a Math library that is proposed to be used in an ADAS system that has Safety goals which demand Math libraries to be safety relevant. Let’s assume that only the code for the Math library exists.
The Qualification activity could be broadly broken into 3 steps.
- Defining safety-relevant requirements and testing them
- Defining the Integration/User guide and performing Integration testing
- Verification of the software component at the unit level
The steps need not be necessarily carried out in the same sequence as mentioned here. The scheduling of these steps must be planned based on the context of the program and the as-is state of the software component.
Step 1: Defining Safety-Relevant Requirements and Testing Them
The first step in component qualification involves understanding the role of the software component in the safety-critical system. In an ISO 26262 context, this means:
- Identifying which technical safety requirements the component will help fulfil.
- Defining software safety requirements that the component must meet.
The broader intention here is to test the component against these defined safety requirements and ensure that it works as per the requirements. Defining safety requirements and testing them are non-negotiable activities in Software component qualification.
Consider a Math library that provides trigonometric functions such as Sine and Cosine, used in an ADAS system for path planning and sensor fusion. The system may require highly accurate conversions from polar coordinates like distance and angle, to cartesian coordinates x,y,z, essential for real-time decision-making.
The Math library may implement several trigonometric functions, logarithmic functions, rounding functions and other arithmetic operations but not all of these may be of interest in the context of safety. In this example, let us assume that only the Sine and Cosine functions are safety relevant, and the rest of the functions implemented by the Math library are not safety relevant.
It is imperative to give due importance to this stage of defining high-quality software requirements. Requirements must be written preferably at a detailed level considering various aspects of the component’s behaviour. For example, besides only defining functional requirements, the Engineer carrying out the qualification needs to think about behavioural requirements in case of errors or overload (robustness requirements). Response time requirements and resource usage aspects (if that’s crucial to the component and the system in which it is used) must also be considered.
The defined safety requirements may include:
- Accuracy: What should be the acceptable tolerance for trigonometric function calculations?
- Range: What range of inputs (e.g., angles in radians or degrees) must the functions handle?
- Error handling: How should the system respond to invalid inputs like infinity or values outside the acceptable range?
- Performance: What is the required response time? For ADAS path planning, this may range from 20-80 µs, while sensor fusion could demand response times below 10 µs.
- Ensuring that there are no discontinuities or jumps in the computed values of the sine(x) as the x changes
- Ensuring compliance with technical standards such as IEEE 754 for floating point arithmetic
In a qualification activity, it is important to go deeper into the requirements with an application perspective to understand how exactly the component should support to achieve the safety requirements. This makes a seemingly application independent math library to be viewed from special lenses worn afresh for each application.
Another important aspect is to perform the testing on the actual target HW in which this existing SW component is used to fulfil safety requirements. The component may have worked well without issues on several projects or systems before, but that is still on a different hardware. Given that the Component qualification is now aimed at a new program that has a safety context, it must be tested on the target hardware of this system. Even if the component is an HW independent component like a BSW component, this step is still crucial. Also, testing on the target HW with the object file of the SW component built with that specific compiler used, ensures that the risk of errors introduced due to optimization flags, architecture of the CPU, compiler settings which may lead to producing slightly different values than expected, are eliminated.
Upon completing this step, we have shaved off several of the tiny edges of the piece and are coming closer and closer to a perfect fitment.
Step 2: Defining the Integration/User Guide and Performing Integration Testing
After defining the safety requirements, the next step is ensuring that the integration process is well understood and documented. Think of it as a step of putting together a user guide or integration guide for the SW component if one doesn’t exist, or to adapt the existing one for the context of a FuSa use case.
Some of the questions we should ask at this step are:
- How should the component be integrated into the new system?
- What are the interfaces, resources, and configurations required?
For example, if using a trigonometric function from a Math library, the integration should think about:
- Whether the function accepts input in radians or degrees.
- What resources, such as memory or runtime, are required.
- Whether the component is thread-safe, allowing for concurrent use across multiple applications.
It’s important to conduct integration testing against the software architecture of the ASIL System, to verify that the software component is integrated with its safety software callers of the ASIL system as per the integration guidelines, thereby sanding off the edges even further!
Step 3: Verification of the Software Component at the Unit Level
At the unit level, the component must be verified through:
- Code reviews against the safety requirements.
- Static analysis ensuring compliance with atleast the Mandatory and Required rules of the MISRA standard and the MISRA rules related to design principles of ISO26262
- Other types of analysis (such as control flow/data flow/dynamic analysis, scheduling analysis etc) as required
- Unit testing, designed not merely to meet code coverage (SC, BC, MCDC) requirements but mainly to test the component’s design, inputs, and outputs rigorously.
For example, when verifying the trigonometric functions, unit testing should cover various input values (both valid and invalid) and ensure that the outputs match the expected results.
Challenges in Software Component Qualification
The flow chart below describes the flow of a qualification activity. This describes the steps that were described above and the consequences at a high level.
While software component qualification can save development time and cost, one may end up finding gaps in the reused SW during the process.
- Major Gaps: Example, a component fails during the testing of the requirements revealing some bugs in the reused software or it has major MISRA violations that require several code changes
- Minor Gaps: Example, an advisory MISRA violation or a minor code bug identified during review or testing
To address the gaps, a risk analysis must be performed to identify what is the impact due to this gap at a safety goal level or vehicle level. There are several risk analysis methods that can be used, a common one being the RPN, a quantitative approach of using severity, occurrence and detection as parameters to compute the risk no. Then the Organization must evaluate if the if the risk no is higher than what is acceptable for that system.
If risks are identified that cannot be accepted by the system – there are only 2 choices – either find alternative solutions to mitigate the risk or accept the safety risk responsibly. In case of alternative solutions to mitigate the risk, the typical options are either
- to change the reused SW if it is a minor change,
- to identify external safety mechanisms that can compensate for the component’s limitations.
- Replace the component with an alternative that meets ASIL requirements.
There is no unified solution for all risks and each risk must be evaluated on a case-by-case basis. Typically making wrong reuse decisions such as reusing an immature SW from an older program or reusing a SW component that does not fulfil all requirements of the ASIL system could bite badly at the later stage, undermining the very purpose of qualification.
Conclusion
Software component qualification must not be viewed as a complete solution to reuse any SW for ASIL. It is rather a risk analysis and mitigation activity that is performed to consciously take a reuse decision. There is also not a standard definition or rule book for what needs to be done for qualification since this depends a lot on the context of the system, the required ASIL level, the purpose of the component and its state of maturity. An in-depth technical analysis and a good risk analysis methodology are key vital investments to perform a high-quality qualification.