DO-178C Intro, Compliance: Free Tools/ Papers / Resources

DO-178C Introduction White Paper

Required Independence versus DAL for DO-178 & DO-254

DO-178 has specific objectives based upon the criticality level of the software. Higher DALs must satisfy more DO-178 objectives than lower levels. After the software criticality level has been determined, you must examine DO-178 to determine exactly which objectives you need to satisfy for the software. 

All You Need to Know About DO-178 Processes

Once you’ve established the DAL of your software, you are ready for planning how you will comply with DO-178C. This is where DO-178 compliance is similar to building a house: you’ve performed geographic analysis to determine what type of foundation is required — that is your “safety assessment.” Then you need a Planning Process, followed by a Development Process. A concurrent Correctness Process is ongoing throughout both Planning and Development.

DO-178 & DO-254 have three integral processes: Planning, Development, and Correctness:

As you can see in the above figure, the DO-178 Planning process comes first, and when complete is followed by a larger DO-178 Development process. In the background the largest process, Correctness (which involves many different integral processes, including QA, CM, Verification, and the work of an FAA Certification Liaison), is performed continuously. But what is meant by Planning, Development, and Correctness? 

The Planning Process

Just as you need to have plans prior to building a house, DO-178C necessitates a planning process prior to developing new software, or before re-using pre-existing or legacy software.  There are five plans and three standards associated with the Planning Process. The five DO-178C plans are theoretically equally important, but AFuzion ranks them in the following order of importance:

DO-178 Plans:

1. Plan for Software Aspects of Certification (PSAC): an overall synopsis for how your software engineering will comply with DO-178, and the roles for FAA certification and EASA certification.
2. Software Quality Assurance Plan (SQAP): details how you intend to meet DO-178’s quality assurance objectives for the software you’re developing.
3. Software Configuration Management Plan (SCMP): details how DO-178’s change management and baseline/storage objectives will be fulfilled on this project.
4. Software Development Plan (SDP): summarizes how software requirements, design, code, and integration will be performed in conjunction with the usage of associated tools to satisfy DO-178’s development objectives.
5. Software Verification Plan (SVP): summarizes the review, test, and analysis activities, along with associated verification tools, to satisfy DO-178’s verification objectives.

DO-178 Standards:

1. Software Requirements Standard: This standard provides criteria to divide system requirements and relevant assessments into high-level requirements and high-level to low-level requirements, including derived and safety-related requirements.
2. Software Design Standard: provides criteria for defining and assessing the software architecture and design.
3. Software Coding Standard: provides criteria for implementing and assessing the software source-code.

Figure: DO-178C’s Five Plans & Three Standards (reprint from AFuzion’s DO-178C Training)

Each of the above DO-178C Plans typically requires 40 to 50 pages each. DO-178C Plans, Standards, and Checklists are not provided by FAA/EASA since they are tailored to specific systems. 

While established avionics companies have their own plans, AFuzion’s DO-178C Plan templates are used by  300 new and smaller companies and 7,500 engineers worldwide.

The Correctness Process

First Focus:  DO-178 Tests Based Upon Requirements (i.e. Functional Testing)
DO-178C testing is a unique blend of traditional “black-box” (test the requirements, not the implementation) and “white-box” (test the actual software design/code).  DO-178C requires four basic testing techniques as depicted in the following figure:

Figure: DO-178C’s Four Basic Test Categories (reprint from AFuzion’s DO-178C Training)

Back in the days of DO-178B, the requirement was generally for execution-based software testing, meaning the tests were not explicitly traced to specific software requirements. However, the DO-178B upgrade to DO-178C emphasizes requirements-based testing, in which  DO-178C High Level Requirements (HLRs), Low Level Requirements (LLRs), code, and tests need to have verified bi-directional traceability as depicted in the following DO-178C Traceability Figure:

Figure: DO-178C’s Bi-Directional Traceability (reprint from AFuzion’s DO-178C Training)

For additional information on DO-178C traceability, download AFuzion’s “DO-178C Traceability” paper. 

DO-178C also requires proof of deterministic software design, inclusive of software data flow, control flow, and coupling analysis. DO-178C’s data coupling is quite mainstream; however the standard’s control coupling is often misunderstood. AFuzion’s advanced DO-178C training covers all of the data flow, control flow, and DFCF coupling analysis but the following two graphics more fully explain DO-178C control coupling:

It’s imperative to remember that DO-178C requires reviews of software requirements, design, code, and tests. DO-178C reviews should be performed independently for DALs B and A.

You also need to retain the proof of these reviews, including both the DO-178C engineering review and DO-178C Quality Assurance audits with DO-178C transition criteria. In aviation, activities are checklist-based (just ask any pilots), so engineers universally apply checklists for DO-178C reviews. We provide additional details on DO-178C checklists and a free DO-178C checklist sample. 

First, all testing must be based upon specific requirements for DO-178, and under DO-178C, all major code segments should trace to at least one requirement. DO-178 does not provide subjective thresholds for requirement granularity, so testing requirements is solely dependent upon the requirements themselves.

DO-178C also compensates for any requirements that are too weak by additionally stipulating that software must undergo additional robustness testing and structural coverage assessment to meet DALs A through C. If you have good DO-178 requirements, testing those requirements should typically yield 90 percent coverage of the requisite robustness cases and 80 percent of the code for Level B. Good requirements provide a high level of detail for low-level functionality and potential robustness conditions. That way, developers can cover 80 to 90 percent of the necessary conditions just by assessing the requirements and writing test cases specifically suited to them.

However, if you have weak DO-178 requirements, then writing test cases from those weak requirements may only yield 50 to 60 percent of the requisite coverage. In that case, you will discover the missing requirement detail during testing structural coverage activity and be required to go back and add requirements. However, this process is not required for criticality levels D and E. 

Clearly, it’s much more cost-effective in DO-178C to do it well the first time instead of doing it poorly and having to go back and do it all over again. So, if there is a lesson to be learned here: invest in good DO-178 (and DO-254) requirements up front and support them with structured reviews and checklists. This is particularly important for DO-178 FAA certification and ED-12 EASA certification.

Back to the concept of criticality, remember that as the DAL of your software increases, so does the level of software testing required for DO-178C. For initial software development, the criticality level has less impact, but not so for testing. The following figure summarizes the testing required per DAL.

Major Testing Differences Between DO-178 Criticality Levels:

How to Reduce DO-178 Compliance Costs

Remember, each avionics system is assigned one or more levels of criticality, and each component within that system must meet or exceed its assigned criticality level. Since the amount of testing and verification rises along with the increasing DAL, the cost also increases. 

The following graph, and associated table below, depict the actual cost deltas associated with each level of criticality: as a baseline, the additional costs of DO-178C are contrasted with medium-quality software development, e.g. high quality financial or consumer software (where the applicable true CMMI level of development was between Level 2 and 3).

 

Graph: Delta Cost and Schedule of DO-178 per Criticality Level, Above CMMI 3

The cost differential within DO-178C is the most significant between Level D and Level C.  Why? Level C requires the following key objectives which Level D does not and which results in Level C requiring 35 percent more effort than Level D:

• Testing of low-level software requirements
• Ensuring 100 percent coverage of all source code statements
• Assessment of requirements, design, and code to standards
• Greater rigor placed upon reviews
• In many cases more rigorous configuration management

Level B requires additional structural coverage (decision-condition, e.g. all branches in the source code), additional independence in reviews, and tighter configuration management.  At first glance, it would seem that Level B should be significantly more expensive, some 50 percent to 70 percent, than Level Quality software engineering organizations already incorporate a semi-automated and streamlined process which includes independent reviews and tight configuration management; ergo the added cost of those aspects for Level B is largely mollified. 

In Level B (and C) there must be detailed, low-level software requirements and they must be thoroughly tested. Remember, DO-178C requires detailed low level requirement verification beginning with Level C, and those low level requirements will cover the vast majority of software logic decisions. 

During requirements-based testing, most (80 to 90 percent+) of the branches in the source code are already covered and hence require no additional structural coverage testing if test capture and coverage tools are appropriately used during that functional testing. Therefore, the seemingly significant cost increase associated with Level B versus C structural coverage is already mitigated by DO-178C’s greatly enhanced requirements-based testing.  

The reader is well-advised to undergo upfront DO-178 Training and DO-178 Process Improvement to leverage these cost reduction techniques. You can also get the first, and most recently updated, book on DO-178C here from Amazon Kindle.

On the other hand, followers of DO-178B who at some level may have sacrificed more rigorous development and testing methods for efficiency will find even greater cost impact in adhering to DO-178C. This will certainly be true if their “efficiency” was due to taking liberties with DO-178B’s intended, but less enforced, low-level requirement detail. 

Such shortcuts enabled less detailed functional testing with many fewer logic branches verified. While acceptable for DO-178B Level C, it’s unacceptable for DO-178C Level C which mandates greater detail for low level requirements. As a result of that greater detail, DO-178C inadvertently reduced the cost difference between Level C and Level B compared to DO-178B because the decision-condition structural coverage objective of Level B is largely covered already in Level C due to those more detailed low level requirements.  

Also, developers making extensive use of Parameter Data Items (objects or logic external to the main application program) are now required to fully document, review, trace, and test all that data under DO-178C, something they theoretically should have already been doing under the intent of DO-178B. 

Level A is certainly the most critical software level and hence the most expensive. But there are also some myths surrounding Level A, one of which is that “Level A is extremely difficult to achieve and the software will cost at least 30 to 50 percent more than Level B,” which is simply not true. The most significant cost driver in Level A over Level B is the MCDC testing requirement. Level A imposes yet more structural coverage requirements (MCDC testing), source to binary correlation, and more independence within reviews. 

However, with proper application of modern structural coverage tools, personnel training, and thorough requirements based testing, the added cost for Level A can be largely contained, thus Level A software is only slightly more expensive than Level B.  

Although, as previously mentioned, the system/hardware costs will still be higher for Level A than B, due to added redundancy required to meet Level A’s 100x higher required reliability over Level B. For this reason, most COTS products pursue Level B certifiability instead opting for Level A. 

Where does the money go?

The following figure, DO-178C Cost Metrics, shows a typical breakdown of expenditures for a Level C project; Level A of course would have a greater percentage allocated to Verification.

Figure:  DO-178C Cost Metrics

 

Benefits of DO-178C

DO-178C is certainly neither free nor cheap. However, DO-178C can actually be cost-effective when implemented properly, particularly when evaluated over a product lifetime or subsequent product versions when DO-178C efficiency and benefits are most notable. 

The following describes the most common benefits from DO-178C compliance based upon this author’s 25 years of experience consulting on over 250 avionics projects.

1. Greater upfront requirements clarity for developers

DO-178C mandates thorough and detailed software requirements. Such detail, and the necessary discipline, forces answers to be provided up-front instead of being deferred. This method minimizes assumptions in the development process and enhances consistency and testability of requirements. It also reduces fault and missing requirements and any relevant rework. It’s true that other standards and guidelines such as CMMI also mandate such upfront requirements; however, DO-178C is unique in its detailed enforcement of those requirements.

2. Fewer coding iterations

Code iterations, or churn, are the bane of software engineering. New products may require ten, twenty, or even thirty different versions or iterations over the course of development and the product lifetime. With strong engineering processes and discipline, code should be largely correct the first time it is written and should not require dozens of updates to get it right, particularly when using well-advised software modeling. Models and code should be reviewed by analyzing implementation versus documented requirements per DO-178C.

3. Fewer bugs found during module testing

Since DO-178C mandates thorough and testable requirements, in addition to code reviews for Level C and above, far fewer bugs will be found in the later stages of module testing.  The independent code reviews required for Level B and A also further that aim.  This is especially true since DO-178C code reviews require the following completed and configured items prior to writing models or code:

• Standards and checklists
• High and low level requirements
• Design
• Traceability for the above

Remember, DO-178C traceability should be as shown in the following DO-178C Traceability Figure:

 

DO-178C Traceability Figure:

4. Greater consistency within software

Software is like a chain: it can only be as strong as its weakest link. Software that is 99 percent correct is 1 percent incorrect, which means it is unsafe. You can never prove particular software to be perfect, and DO-178C makes no claims that perfect adherence to its objectives yields perfect software. The weakest software module, or software engineer, can potentially threaten software safety. Thankfully, all software must be consistent according to its level of criticality, and DO-178 enforces this.

5. Fewer defects found during integration of new software components

Integration can be a lengthy iterative process during which developers discover major defects and have to make full design changes to fix them. Not so with DO-178, in which integration is typically 50 to 75 percent faster than in non-DO-178 environments. Since all software the  operation of which is potentially affected by hardware must be tested on actual target hardware, integration begins earlier and developers can verify the software sooner in the process. Also, the system upon which the software resides must comply with ARP4754A (or ARP4754B later this year), which mandates similarly strong system processes. Plus, DO-254 ensures that custom hardware logic components will be of similar high quality at integration. Thus, integration begins earlier with inherently fewer flaws than in non-DO-178C environments. See our video on reducing DO-178C Gaps.

6. Improved configuration management

The ability to control and recreate any snapshot of the project is covered by Configuration Management (CM). However, CM also ensures security, backups, completed reviews, problem reporting, and version control. DO-178’s strong CM requirements, coupled with modern tools that automate many CM tasks, ensure higher software quality now and in the future. Literally any version of released DO-178C-compliant software must be able to be recreated and retested in its entirety for 30 years after delivery. That means not only that the application source code must be controlled and captured, but that all the ancillary files including makefiles, build scripts, third party tools, development environments, design and test data, etc. must all be captured.This capture eliminates common problems in software maintenance found within other industries by ensuring software development processes are documented and repeatable at every stage.

7. Easier regression testing

You build your product on the assumption that it will be successful and therefore have a long life. You also know your software will evolve through new applications, installations, and versions. Regression testing can be expensive if it’s extensive or manual; DO-178 provides thorough traceability to determine which modules need changes or analysis and corresponding retesting. Because all versions and changes are traced and documented, developers can often automate retesting, making it a much simpler process. The following graphic shows an optimal DO-178C Testing strategy:

 

Figure:  Optimal DO-178C Testing Strategy:

(Excerpt from AFuzion’s DO-178C Training)

8. More thorough testing

DO-178C requires various deterministic verification aspects, including software structural coverage, 100 percent high- and low-level requirements coverage, and greater robustness coverage with clarity. Parameter Data Items must be particularly well documented and verified under DO-178C. The standard also necessitates removing unused or dead code and verifying and documenting deactivated code, at least for DAL C through A. This significantly reduces the probability of software errors encountered in the field.

9. Improved hardware and system testing

System testing is typically challenging for embedded systems because the development environment is quite removed from the target environment. Also, testing is often performed via multiple engineering teams. That’s why DO-178c mandates that software components that could be affected by hardware be tested on the hardware, e.g. hardware/software interfaces, interrupts, timing, board-level components, BSP/RTOS, etc. The improved determinism and quality of all these components via DO-178 ensures improved hardware integration. Since DO-178C requires complete traceability, it is quite possible to show via traceability that system requirements were actually fully tested during software testing and therefore need not be repeated by additional testing at the system level.

10. Fewer in-the-field defects

In-the-field defects and customer returns can be both highly dangerous in the case of safety-critical systems, and also highly expensive in both the short and long term. However, products developed under DO-178C have been demonstrated to result in 80 to 90 percent fewer returns. For a brief introductory technical one-hour instructional video on DO-178C and also related topics including DO-331 videos and DO-332 videos, view the free video DO-178C Costs &  Gaps Closure one-hour introduction video from AFuzion.

11. Improved reusability

The thorough and consistent documentation required by DO-178 greatly improves reusability for software components and code segments through modularization, enforcement of documented modern engineering principles, and reviews to ensure all the above was achieved. In software, reusability is the Holy Grail, but the reality is that unless a software component is at least 80 percent reusable (i.e. unchanged), then it is quicker and less risky to simply start from scratch. In reality, most software is less than 50 percent “reusable.” With DO-178, and enforcement of design/coding standards, coupled with independent reviews and traceability, most modules should be at least 70 percent reusable. When modeling and C++ are employed, reusability increases even more.

12. Improved management awareness of true development criteria and schedule status

How do you measure software development progress and ensure managerial awareness of the completion status and development criteria for a software component? The answer is via modern and accurately detailed management techniques built around DO-178. The provision for insight, traceability, accurate status on design, development, testing, integration, and reviews is found in DO-178. For a detailed DO- 178 PSAC Review Checklist, see our DO-178C Plans and Checklists page.

Download the rest of this paper for eight additional DO-178C Cost Benefits

DO-178C Compliance Tools

DO-178C requires five DO-178C Plans, three DO-178C Standards, and 20+ DO-178C checklists. Most companies (over 150 companies and 9,000 engineers worldwide) use AFuzion’s DO-178C Planning templates, DO-178C Standards templates, and DO-178C checklist templates described below. Once you’ve acquired these resources from AFuzion and customized them for the first project, your engineers will retain the expertise to create, customize and re-use as appropriate on future DO-178C projects. 

DO-178C Plans and Checklist Templates cover all phases of the system’s software project lifecycle and are developed with DO-178C in mind. The users of these templates would need to have some basic understanding of DO-178C, such as attendance at AFuzion’s DO-178C training.

The regulatory agencies require that most airborne commercial systems operating within commercial airspace comply with DO-178C and DO-254. The planning and processes for the engineering lifecycle are required for any DO-178C and DO-254 project, and those processes must be defined before initiating a particular phase and followed during that phase. 

Although checklists are not formally required by DO-178C and DO-254, your regulatory agencies will certainly require that you prove conformance to DO-178C and DO-254 according to your approved processes. Nearly all DO-178C and DO-254 projects use checklists for reviews and proof of conformance.

Although there is no perfect process for all programs, there are distinct areas of each individual project. AFuzion Process Templates are meant to provide the proper framework to meet the system process requirements of DO-178C. Each project will vary somewhat in how you choose to define, implement, or augment the AFuzion process templates. Factors to consider include project Design Assurance Levels, complexity, and expertise of staff, development methodology, tools/environment, and technology. 

The AFuzion process Templates provide the basic elements of an avionics project compliant with DO-178C and DO-254. Further, AFuzion can customize and tailor these processes by the appropriate amount as an outflow of the gap analysis process, upon request as part of the optional first draft delivery.

If there are items in the checklists that are not applicable to your program they could simply be answered with “N/A.” It is also recommended that the checklists be placed under your project’s configuration management (CM) system to ensure checklist integrity.

And remember, under higher DALs, independent reviews are always preferable to reviews done by the developer. 

It should be noted that the checklists should be widely distributed to all personnel developing any avionics lifecycle item, prior to that person beginning such initiation. Thus, the checklist serves as a “report card” through which the originator’s success is measured. When the originator understands what the independent verification reviewer will be evaluating the related software/hardware artifact for, the originator will more productively attain checklist compliance during development of that artifact.  

This applies to requirements, design, implementation, test, etc. The level of “independence for verification” required by DO-178C and DO-254 varies according to Design Assurance level (Levels A through E).   

These DO-178C templates and checklists also help in getting organizations to the goal of higher SEI CMM/CMMI ratings (preferably Level 3 through 4+). Using AFuzion’s process templates and checklists will maximize the probability of project success and quality. More DO-178C Template information is available on our plans and checklists page.

AFuzion’s DO-178C Plans and Standard Templates:

• Plan for Software Aspects of Certification
• Software Quality Assurance Plan
• Software Configuration Management Plan
• Software Development Plan
• Software Verification Plan
• Software Requirement Standard
• Software Design Standard
• Software Coding Standard
• Software Configuration Index / Software Environment Configuration Index

 AFuzion’s DO-178C Checklist Templates:

• Plan for Software Aspects of Certification Checklist
• Software Quality Assurance Plan Checklist
• Software Configuration Management Plan Checklist
• Software Development Plan Checklist
• Software Verification Plan Checklist
• Software Requirements Standard Checklist
• Software Design Standard Checklist
• Software Coding Standard Checklist
• Software Configuration Index Checklist
• Software Requirements Checklist
• Software Design Data Checklist
• Source Code Review Checklist
• Software Test Procedures Checklist
• Software Verification Results Checklist
• Software Verification Analysis (including Traceability) Checklist
• Software Accomplishment Summary Checklist
• Stage of Involvement #1 Checklist
• Stage of Involvement #2 Checklist
• Stage of Involvement #3 Checklist
• Stage of Involvement #4 Checklist

Understanding Military Compliance for DO-178C

By the early 2000s, U.S. military organizations realized that the commercial aerospace sector, particularly the part that is regulated by the FAA via DO-178, maintained certain advantages that were not inherent in the defense establishment. They were faced with a choice:

1. Maintain the status quo and do nothing.
2. Update their own standards to adopt the best aspects of DO-178.
3. Simply adopt DO-178B (and later, DO-254 and DO-178C).

Ultimately, the military chose option three, although the choice was hardly a simple one. As with any established organization, there were myriad opinions, entrenched practices and opposition, added initial transition costs, and politics. Still, military aviation has been gradually adopting not only DO-178, but also DO-254, ARP4754A or the upcoming update ARP4754B, and ARP4761/A. 

Further complicating the military adoption of civilian aviation guidelines were the following aspects:

• The military did not want to relinquish oversight of avionics development for defense to the Federal Aviation Administration (FAA), nor did the FAA have the bandwidth or authorization to intervene with military projects.
• Military organizations were unfamiliar with DO-178 and DO-254 specifics and therefore applied widely varying and subjective criteria. After all, DO-178 and DO-254 tend to be terse and vague­. Specialized training by experts is typically required to apply them in the real world.
• Because of the above, military organizations often further complicated DO-178/DO-254 adoption by requiring simultaneous adherence to their own specialized, military-specific standards in addition to DO-178/DO-254. While well-intentioned, this action is counter-productive since the standards differ and conflict with each other in key areas. DO-178 and DO-254 already have ample ambiguity and subjectivity, and requiring corollary adherence to military aviation standards grossly complicates compliance. Today, most military organizations solely utilize a near-exact adoption of DO-178C and DO-254.
• Military safety was initially commonly built around MIL STD 882, which is somewhat different from ARP4754A/ARP4761 or the new ARP4754B/ARP4761A.

Differences for Military DO-178C Compliance

While the military has essentially adopted DO-178C for avionics software development, there are a few areas in which this adoption varies from the way civil aviation organizations apply the standard. 

• Different emphasis on Safety Analysis; more focus on Mission Success Probability
• Often less redundancy but harsher operational environments
• Many Onboard Mission Systems; less direct DO-178C flight-safety impact but necessary for Mission Success
• Focus upon DO-178 “Military Compliance” versus DO-178C “Certification”
• Military Agency approval: generally not FAA/EASA
• All documents received by (and reviewed by) military/customer; not just PSAC, CI, and SAS.

In addition, military entities traditionally use SSR, CDR, PDR, and TRR for aviation software and legacy DO-178C. However, DO-178C utilizes the Stage of Involvement (even if informally) instead. Download the full DO-178C Introduction paper via the link below for additional details on military compliance for DO-178C.   

Note that the military increasingly adheres to Multi-Core Processing development and certification per AC 20-193 (AMC 20-193) in Europe with more stringent MCP requirements; AFuzion’s CAST-32A/MCP 20-193 Training is a popular option. You can view our DO-178C training for more details. The figure below shows the military DO-178C versus Civil DO-178C meetings and AFuzion’s recommended means of aggregating the two when applying DO-178C on military projects:

Figure: Military DO-178C Compliance Meetings versus DO-178C Civil Meetings/SOI’s

For the remaining 14 pages of this AFuzion DO-178 Introduction Technical Whitepaper, please download below.

← Back To All Whitepapers