DO-330 Introduction – Tool Qualification

DO-330 Introduction – Tool Qualification

Each of the tools in the above figure plays a key role in avionics engineering and each is available ready-to-use, as “commercial off-the-shelf” (COTS) software which can be purchased directly from any of dozens of software tool vendors.  Under DO-178B, tools were simply classified as “development” tools or “verification” tools.  However, DO-178C does away with such a simple classification because technical advances have allowed for hybrid tools which perform verification while also reducing subsequent development activities; this is explained later herein via tool “criteria”.

Tools used during engineering exist in all project phases: requirements specification, software design and code, integration, configuration management, and verification.   Although it is possible to develop a safety-critical application in the aviation industry with the use of only implementation tools (compiler, assembler, and linker), this is increasingly unlikely given the complexity and enormity of electronic systems and modern avionics in this era.

When is It Necessary to Qualify a Tool via DO-330?

Tool qualification perDO-330 is required whenever the design assurance process(es) described in RTCA/DO-178C or RTCA/DO-254 are eliminated, reduced, or automated by the use of the tool unless the output of the tool is verified. Verification of the tool’s output must be accomplished through the verification process as defined by RTCA/DO-178C Section 6. (Tool Qual is dealt with via DO-330, the subject of this AFuzion paper). Remember, in avionics development, “Verification” has a specific meaning (as the following is not official FAA/EASA policy, the author calls it the “Hilderman Verification Equation”):

DO-178B Versus DO-178C Tool Qualification

 

Under the former DO-178B (which of course is eclipsed by DO-178C), tool qualification was addressed simply within DO-178B itself and clarified via the FAA’s ubiquitous 8110.49:  tools were categorized as simply one of the following:

1. Development Tools: capable of inserting an error within operational flight software; or
2. Verification Tools: incapable of inserting an error, but potentially capable of failing to detect an error in the flight software.

However, DO-178B was released in 1992, in the earlier days of advanced software development, which was before associated guidelines for complex electronic hardware (DO-254) and CNS/ATM (DO-278A) were released.  The all-too-brief (less than four pages) tool qualification guidelines within DO-178B were just that: often too brief.  So new avionics software tool qualification guidance was needed for multiple reasons as summarized in the following figure:

As depicted above, there were four key reasons for the introduction of DO-330 “Software Tool Qualification Considerations” to provide the necessary supplementary tool qualification information in one document. Determining whether any tool needs to be qualified is accomplished by assessing the outcome of three questions regardless of the tool category.

1. Can the tool insert an error into the airborne software/hardware or fail to detect an existing error in the software/hardware within the scope of its intended usage?
2. Will the tool’s output not be verified or confirmed by other verification activities as specified within for example, Section 6 of DO-178C for software?
3. Will the output of the tool be used to either meet an objective or replace an objective of RTCA/DO-178C, DO-254, DO-278A, etc.?

If the answer to all three questions is YES then the tool will most likely be required to be qualified. The figure below poses these questions via a classic flow chart.  It should be noted that the answer to the first and third question is almost always “Yes”, therefore the real question of tool qualification necessity normally comes down to just one question:  “Will the tool output be verified?” If the answer is “No” then qualification is almost always required.   A simple flow-chart for these questions is depicted below in Figure 4.  To be honest, most tools can either insert an error or fail to detect an error; also most tools eliminate, reduce, or automate avionics development/verification processes.  Therefore the real question to be answered in determining if a tool needs to be qualified is “Is the output of the tool otherwise verified?” If the output of an DO-178C, DO-254, or DO-278A tool is not verified then almost always that tool must be qualified.  Once you determine that a tool needs to be qualified, then you determine the tool’s TQL for your application and apply DO-330 objectives to that tool instance.

Why Tool Qualification using DO-330? This author has encountered numerous projects over the decades which “assumed” tool qualification for a given project was not required; without DO-330, subsequent results were disastrous when use of that tool’s output was later disallowed when the tool was unable to be qualified. DO-330 provides for tool qualification activities to be directly related to the potential tool impact; that impact is based upon both the category of the tool and also the DAL (Design Assurance Level) it’s applied to. Therefore, DO-330 introduces five Tool Qualification Levels (TQLs) based upon three Tool Criteria.

There are three tool criteria, meaning a tool’s usage is assessed to fall within one of the following three criteria categories (see AFuzion’s Advance DO-178C and DO-254 Training for added details; info here: https://afuzion.com/private-training/avionics-software-advanced-do-178c-training-class/) :

1. Criteria 1: A tool whose output is part of the airborne software and thus could insert an error.
   • Example: code generation tool which automatically generates source code from models
2. Criteria 2: A tool that automates verification process(es) and thus could fail to detect an error, and whose output is used to justify the elimination or reduction of:
3. Verification process(es) other than that automated by the tool, or
4. Development process(es) that could have an impact on the airborne software.

• Example: model-checking tool which verifies completeness while also checking coverage

3. Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error.

• Example: structural coverage tool that assesses code coverage

Quick question:  can quality be built-in to a product after it’s developed?  Of course not:  true product quality relies upon high-quality planning, implementation per plan, and assessment of implementation along with supporting processes.  Just as DO-178C requires lifecycle processes for avionics software, DO-330 defines such a lifecycle for qualified tools as shown below in the following Figure.  As shown in that following figure, the tool qualification lifecycle consists of three key activities:  1) Tool Planning, 2) Tool Development, and 3) Tool Verification; these activities must be performed sequentially, starting with Planning, then Development, and finally Verification.   But continuously performed in the background during each of these activities are the corresponding Integral Processes of Tool Configuration Management, Tool Quality Assurance, and Tool Qualification Liaison.

Planning the Tool Qualification.

Now, the TQL has been formalized, your certification authority has formally approved your plans or you are reasonably sure they will, so actual tool qualification can begin. The required tool qualification objectives that must be satisfied are detailed in the ten Annex A tables at the back of DO-330.  These objectives depend upon the TQL, as summarized in the following figure: note that the required tool qualification objectives increase as  the TQL advances from the least rigorous (TQL 5) to the most rigorous (TQL 1):

Tool Planning, Development, & Verification per DO-330

There are three key activities for producing qualified avionics tools, processes exhaustively described in our DO-330 whitepaper.

1. Tool Qualification Planning, which includes identifying all applicable DO-330 objectives and defining how each is to be accomplished.
2. Tool Development, which initiates tool implementation (or reverse engineering for pre-existing tools) once those qualification plans have been documented and reviewed.
3. Tool Verification, which under DO-330 requires two sequential processes: tool verification proces

To download the remaining 12+ pages of this technical DO-330 AFuzion Tool Qualification whitepaper, please download below:

Download Full 10+ Page DO-330 Whitepaper