DO-330 Introduction – Tool Qualification
DO-330 Tool Qualification whitepaper applies to avionics software tools for DO-178C, DO-254, DO-278. This DO-330 whitepaper describes the five tool categories versus criticality levels (DALs) and how to truly qualify avionics software tools.DO-330 Design Tool and DO-330 Verification Tool information is provided. DO-330 Tool Qualification Levels (TQL’s) 1 – 5 are described as are DO-330 Tool Qualification Criteria (TQC) 1 – 3.
Software and hardware engineering tools are computer programs that help engineers create, analyze, verify, track, modify, produce or specify the application programs being developed. Such tools and programs have been in use since the beginning of computing. Tools aid the improvement of efficiency and effectiveness in the development process by automating mundane operations while bringing the level of abstraction and understanding closer to the developer.
What is a Tool?
Webster’s dictionary defines a tool as an instrument; anything used as a means to an end, something used in the performance of an operation, anything regarded as necessary to carrying out one’s occupation or profession, or a person that is used or manipulated by another (subject of another discussion, but an interesting analogy). RTCA/DO-178C, RTCA/DO-254 and FAA Order 8110.49 define a tool as a computer program used to develop, test, analyze, produce, or modify another program or its documentation. So for avionics, a tool consists of software itself used somewhere within the lifecycle of avionics systems. Consider the following common types of tools used to develop software in the figure below, and ask yourself if they need to be qualified (to be answered in the next section):
Each of the tools in the above figure plays a key role in avionics engineering and each is available ready-to-use, as “commercial off-the-shelf” (COTS) software which can be purchased directly from any of dozens of software tool vendors. Under DO-178B, tools were simply classified as “development” tools or “verification” tools. However, DO-178C does away with such a simple classification because technical advances have allowed for hybrid tools which perform verification while also reducing subsequent development activities; this is explained later herein via tool “criteria”.
Tools used during engineering exist in all project phases: requirements specification, software design and code, integration, configuration management, and verification. Although it is possible to develop a safety-critical application in the aviation industry with the use of only implementation tools (compiler, assembler, and linker), this is increasingly unlikely given the complexity and enormity of electronic systems and modern avionics in this era.
When is It Necessary to Qualify a Tool via DO-330?
Tool qualification perDO-330 is required whenever the design assurance process(es) described in RTCA/DO-178C or RTCA/DO-254 are eliminated, reduced, or automated by the use of the tool unless the output of the tool is verified. Verification of the tool’s output must be accomplished through the verification process as defined by RTCA/DO-178C Section 6. (Tool Qual is dealt with via DO-330, the subject of this AFuzion paper). Remember, in avionics development, “Verification” has a specific meaning (as the following is not official FAA/EASA policy, the author calls it the “Hilderman Verification Equation”):
DO-178B Versus DO-178C Tool Qualification
Under the former DO-178B (which of course is eclipsed by DO-178C), tool qualification was addressed simply within DO-178B itself and clarified via the FAA’s ubiquitous 8110.49: tools were categorized as simply one of the following:
- Development Tools: capable of inserting an error within operational flight software; or
- Verification Tools: incapable of inserting an error, but potentially capable of failing to detect an error in the flight software.
However, DO-178B was released in 1992, in the earlier days of advanced software development, which was before associated guidelines for complex electronic hardware (DO-254) and CNS/ATM (DO-278A) were released. The all-too-brief (less than four pages) tool qualification guidelines within DO-178B were just that: often too brief. So new avionics software tool qualification guidance was needed for multiple reasons as summarized in the following figure:
As depicted above, there were four key reasons for the introduction of DO-330 “Software Tool Qualification Considerations” to provide the necessary supplementary tool qualification information in one document. Determining whether any tool needs to be qualified is accomplished by assessing the outcome of three questions regardless of the tool category.
- Can the tool insert an error into the airborne software/hardware or fail to detect an existing error in the software/hardware within the scope of its intended usage?
- Will the tool’s output not be verified or confirmed by other verification activities as specified within for example, Section 6 of DO-178C for software?
- Will the output of the tool be used to either meet an objective or replace an objective of RTCA/DO-178C, DO-254, DO-278A, etc.?
If the answer to all three questions is YES then the tool will most likely be required to be qualified. The figure below poses these questions via a classic flow chart. It should be noted that the answer to the first and third question is almost always “Yes”, therefore the real question of tool qualification necessity normally comes down to just one question: “Will the tool output be verified?” If the answer is “No” then qualification is almost always required. A simple flow-chart for these questions is depicted below in Figure 4. To be honest, most tools can either insert an error or fail to detect an error; also most tools eliminate, reduce, or automate avionics development/verification processes. Therefore the real question to be answered in determining if a tool needs to be qualified is “Is the output of the tool otherwise verified?” If the output of an DO-178C, DO-254, or DO-278A tool is not verified then almost always that tool must be qualified. Once you determine that a tool needs to be qualified, then you determine the tool’s TQL for your application and apply DO-330 objectives to that tool instance.
There are three tool criteria, meaning a tool’s usage is assessed to fall within one of the following three criteria categories (see AFuzion’s Advance DO-178C and DO-254 Training for added details; info here: https://afuzion.com/private-training/avionics-software-advanced-do-178c-training-class/) :
- Criteria 1: A tool whose output is part of the airborne software and thus could insert an error.
- Example: code generation tool which automatically generates source code from models
- Criteria 2: A tool that automates verification process(es) and thus could fail to detect an error, and whose output is used to justify the elimination or reduction of:
- Verification process(es) other than that automated by the tool, or
- Development process(es) that could have an impact on the airborne software.
- Example: model-checking tool which verifies completeness while also checking coverage
- Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error.
- Example: structural coverage tool that assesses code coverage
Quick question: can quality be built-in to a product after it’s developed? Of course not: true product quality relies upon high-quality planning, implementation per plan, and assessment of implementation along with supporting processes. Just as DO-178C requires lifecycle processes for avionics software, DO-330 defines such a lifecycle for qualified tools as shown below in the following Figure. As shown in that following figure, the tool qualification lifecycle consists of three key activities: 1) Tool Planning, 2) Tool Development, and 3) Tool Verification; these activities must be performed sequentially, starting with Planning, then Development, and finally Verification. But continuously performed in the background during each of these activities are the corresponding Integral Processes of Tool Configuration Management, Tool Quality Assurance, and Tool Qualification Liaison.
Planning the Tool Qualification.
Now, the TQL has been formalized, your certification authority has formally approved your plans or you are reasonably sure they will, so actual tool qualification can begin. The required tool qualification objectives that must be satisfied are detailed in the ten Annex A tables at the back of DO-330. These objectives depend upon the TQL, as summarized in the following figure: note that the required tool qualification objectives increase as the TQL advances from the least rigorous (TQL 5) to the most rigorous (TQL 1):
To download the remaining 12+ pages of this technical DO-330 AFuzion Tool Qualification whitepaper, please download below:
Free: Download Remaining 10+ Page Paper Here