DO-178 Introduction Whitepaper
Avionics certification explained
FOR ENGINEERS AND MANAGERS
Avionics certification explained – the big picture. The roles of Safety (ARP4761A), Systems (ARP4754A) and the avionics development ecosystem are fully described along DO-178C’s and DO-254’s relationship facts.
DO-178 has an innocuous title: ”Software Considerations in Airborne Systems and Equipment Certification.” But it’s best not to judge the book by its cover. In reality. ‘178,’ as Industry slang calls it. is largely considered the bible of avionics software development. Interestingly, since it was first developed in the 80’s (a time in which there was relatively little software in safety-critical systems), it has become the de facto embodiment for much of those systems. In fact, a careful examination of standards within Military aviation, medical devices, railroads, automotive, and nuclear power will reveal striking similarities with 178. An accidental coincidence? Hardly. Was 178 the first such standard? That answer hardly matters. except that history is important. And why is history important in the case of DO-178? Because 178 has evolved via three subsequent iterations and it’s important to understand the reasons for those changes.
Remember the parable about the seven blind men and the elephant? Each touched a different part of the elephant and tried to describe what they were touching without knowing anything about the other parts of the elephant. An impossible task indeed, yet it’s an easy trap to tall into within aviation: aircraft and their systems are so complex that no single person can fully understand all the intricacies. In other words. without enlightenment. we’re all little better than the blind men. DO-178 attempts to lay a framework so that development personnel and certification authorities can work with full vision and leave residual blindness behind. 178 does succeed in providing such a framework. However, it doesn’t guarantee clarity of vision and certainly not perfection. What. avionics software is not perfect? Of course not. DO-178 isn’t perfect? Hardly. In fact, paraphrasing Winston Churchill’s famous quotation concerning democracy. ‘DO-178 is the worst standard in the world except tor all the others!”
Development Assurance (Criticality) Levels:
First, you need to understand the Development Assurance Level (DAL) of the software you are developing for DO-178. The rigor applied to planning, development, and correctness of your software is directly associated with its DAL—often referred to as “criticality level.” There are five levels, with increasing rigor from Level E to the most stringent Level A, as depicted below. Note that the assigned level is dependent upon type of aircraft; the following is for Part 25 aircraft, e.g. larger aircraft and applies to both DO-178, DO-254, DO-178B, and DO-178C:
Required Independence versus DAL for DO-178 & DO-254
DO-178 has specific objectives based upon the criticality level of the software. Higher DAL’s must satisfy more DO-178 objectives than lower levels. After the software criticality level has been determined, you examine DO-178 to determine exactly which objectives must be satisfied for the software. Now you are ready for planning. This is where DO-178 is similar to building a house: you’ve performed geographic analysis to determine what type of foundation is required—that is your “safety assessment”. Then you need a Planning Process, followed by a Development Process. A concurrent Correctness Process is ongoing throughout both Planning and Development. Avionics software engineering under DO-178 is thus the same as building a house and follows the same three-phased process approach.
DO-178 & DO-254 have three integral processes: Planning, Development, and Correctness:
As can be seen in the above figure: the DO-178 Planning process comes first, and when complete is followed by a larger DO-178 Development process. In the background the largest process, Correctness (Integral Processes: QA, CM, Verification, FAA Certification Liaison), is performed continuously. What is meant by Planning, Development, and Correctness? Here is a brief summary.
Planning Process. Before development, or before re-using pre-existing or legacy software, you need to plan your activities. Just like building a house, the building inspectors first need to inspect a set of plans followed by regular inspections of the house as it’s being built: foundation, walls, electrical, plumbing, roof, and the finished building. DO-178 is similar. There are five plans and three standards associated with the Planning Process:
- Plan for Software Aspects of Certification (PSAC): an overall synopsis for how your software engineering will comply with DO-178, and the roles for FAA certification and EASA certification.
- Software Quality Assurance Plan (SQAP): details how DO-178’s quality assurance objectives will be met for this project.
- Software Configuration Management Plan (SCMP): details how DO-178’s change management and baseline/storage objectives will be performed on this project.
- Software Development Plan (SDP): summarizes how software requirements, design, code, and integration will be performed in conjunction with the usage of associated tools to satisfy DO-178’s development objectives.
- Software Verification Plan (SVP): summarizes the review, test, and analysis activities, along with associated verification tools, to satisfy DO-178’s verification objectives.
- Software Requirements Standard: provides criteria for the decomposition and assessment of System Requirements into software high-level requirements and high-level to low-level requirements; including derived and safety related requirements.
- Software Design Standard: provides criteria for defining and assessing the software architecture and design.
- Software Coding Standard: provides criteria for implementing and assessing the software source-code.
Figure: DO-178C’s Five Plans & Three Standards (reprint from AFuzion’s DO-178C Training)
DO-178C Plans, Standards, and Checklists are not provided by FAA/EASA since they are tailored for specific systems. DO-178C five plans are theoretically equally important but AFuzion ranks them in the following order of importance:
- 1. PSAC: the DO-178C Plan for Software Aspects of Consideration is the #1 document and always submitted to certification authority.
- 2. SQAP: the DO-178C Software Quality Assurance Plan defines your independent proof of conformance and audit strategy to prove that conformance.
- 3. SCMP: the DO-178C Software Configuration Management Plan documents your strategy to control baselines, versions, changes, and ability to recreate software.
- 4. SDP: the DO-178C Software Development Plan defines your strategy for developing safe software requirements, design, implementation, and integration.
- 5. SVP: the DO-178C Software Verification Plan defines the strategy to review, test, and analysis the corresponding software requirements and code.
Each of the above DO-178C Plans typically requires 40-50 pages each. While established avionics companies have their own plans, over 300 new and smaller companies procured AFuzion’s DO-178C Plan templates which are in used by 7,500 engineers worldwide – more information (and free samples of DO-178C Plans) are available here: https://afuzion.com/plans-checklists/#do-178c-plan-templates
First Focus: DO-178 Tests Based Upon Requirements (E.g. Functional Testing)
DO-178C testing is a unique blend of traditional “black-box” (test the requirements, not the implementation) and “white-box” (test the actual software design/code). DO-178C requires four basic testing techniques as depicted in the following figure:
Figure: DO-178C’s Four Basic Test Categories (reprint from AFuzion’s DO-178C Training)
Under the former DO-178B, software testing was often “execution” based, meaning not explicitly traced to software requirements. However, the DO-178B upgrade to DO-178C emphasizes “requirements-based” testing, where DO-178C High Level Requirements, Low Level Requirements, Code, and Tests have verified bi-directional traceability as depicted in the following DO-178C Traceability Figure:
Figure: DO-178C’s Bi-Directional Traceability (reprint from AFuzion’s DO-178C Training)
For additional information on DO-178C traceability, download AFuzion’s “DO-178C Traceability” paper here: https://afuzion.com/traceability/
DO-178C also requires proof of deterministic software design, inclusive of software data flow, control flow, and coupling analysis. DO-178C’s data coupling is quite mainstream; however DO-178C’s control coupling is often misunderstood. AFuzion’s advanced DO-178C training covers all of data flow, control flow, and DFCF coupling analysis but the following two graphics more fully explain DO-178C control coupling:
It’s imperative to remember that DO-178C requires reviews of software requirements, design, code and tests; DO-178C reviews should be performed independently for DAL’s B and A. But proof of DO-178C reviews needs to be retained including both DO-178C engineering review and DO-178C Quality assurance audits with DO-178C transition criteria. In aviation, activities are checklist based (just ask any pilots!) so checklists are universally applied for DO-178C reviews. For details on DO-178C checklists and a free DO-178C checklist sample, see details here: https://afuzion.com/plans-checklists/#do-178c-checklists
First, testing is based upon requirements for DO-178 (the latest version, DO-178C, goes further by stating all major code segments should trace to at least one requirement). Since DO-178 does not provide subjective thresholds for requirement granularity, testing of requirements is dependent upon the requirements themselves. However, DO-178 compensates for potentially weak requirements by requiring, for Level A through C, software to undergo additional robustness testing and structural coverage assessment. If you have good DO-178 requirements, testing those requirements should typically yield 90% coverage of the requisite robustness cases and 80% of the code for Level B. Why? Because good requirements provide good detail for low-level functionality and potential robustness conditions; such can be gleaned from the requirements thus test cases can cover 80-90% of the necessary conditions just by assessing the requirements and writing test cases for them. However, if you have weak DO-178 requirements, then writing test cases from those weak requirements may only yield 50-60% of the requisite coverage; in that case, you will discover the missing requirement detail during testing structural coverage activity (though not required Level D and E) and be required to go back and add requirements. Clearly, like most things in real-life, it’s much more cost-effective in DO-178C to do it well the first time instead of going back to improve it and do it again. So, if there is a lesson learned to be shared here: invest in good DO-178 (and DO-254) requirements up front and support them with structured reviews and checklists; this is particularly important for DO-178 FAA certification and ED-12 EASA certification.
It’s imperative to note that DO-178’s five criticality levels call for significantly more software testing as the criticality level increases. For the software development, criticality level has less impact but not so for testing as the following figure summarizes the testing required per DAL.
Major Testing Differences Between DO-178 Criticality Levels:
Criticality Levels and Cost.
DO-178 and DO-254 entail five different levels of criticality, ranging from Level A (most critical) to Level E (least critical). Each avionics system is assigned one or more levels of criticality, and each component within that system must meet or exceed its assigned criticality level. As the criticality level increases, so does the degree of rigor associated with documentation, design, reviews, implementation, and verification. Accordingly, cost and schedule increase as well. The following graph, and associated table below, depict the actual cost deltas associated with each level of criticality: as a baseline, the additional costs of DO-178C are contrasted with medium-quality software development, e.g. high quality financial or consumer software (where the applicable true CMMI level of development was between Level 2 and 3).
Graph: Delta Cost and Schedule of DO-178 per Criticality Level, Above CMMI 3
The cost differential within DO-178C is the most significant between Level D and Level C. Why? Level C requires the following key objectives which Level D does not and which results in Level C requiring 35% more effort than Level D:
- Testing of low-level software requirements
- Ensuring 100% coverage of all source code statements
- Assessment of requirements, design, and code to standards
- Greater rigor placed upon reviews
- In many cases more rigorous configuration management
Level B requires additional structural coverage (decision-condition, e.g. all branches in the source code), additional independence in reviews, and tighter configuration management. At first glance, it would seem that Level B should be significantly more expensive, e.g. 50% – 70%, than Level C. In theory, it seems to make sense, but as in many areas of life, common sense overcomes theory. In Level B (and C) there must be detailed, low-level software requirements and they must be thoroughly tested. Remember, DO-178C requires detailed low level requirement verification beginning with Level C and those low level requirements will cover the vast majority of software logic decisions. During requirements-based testing, most (80-90%+) of the branches in the source code are already covered and hence require no additional structural coverage testing if test capture and coverage tools are appropriately used during that functional testing. Therefore, the seemingly significant cost increase associated with Level B versus C structural coverage is already mitigated by DO-178C’s greatly enhanced requirements-based testing. Also, quality software engineering organizations already incorporate a semi-automated and streamlined process which includes independent reviews and tight configuration management; ergo the added cost of those aspects for Level B is largely mollified. The reader is well-advised to undergo upfront DO-178 Training and DO-178 Process Improvement to leverage these cost reduction techniques. For the first, and most recently updated, book on DO-178C procure here from Amazon Kindle: https://www.amazon.com/Avionics-Certification-Complete-DO-178-DO-254-ebook/dp/B07BGF96WM/
Now, “efficient” followers of DO-178B will find even greater cost impact in adhering to DO-178C. Their “efficiency” may have been due to taking liberties with DO-178B’s intended, but less enforced, low-level requirement detail. Such shortcuts enabled less detailed functional testing with many fewer logic branches verified. While acceptable for DO-178B Level C, it’s unacceptable for DO-178C Level C which mandates greater detailed of low level requirements. As a result of that greater detail, DO-178C inadvertently reduced the difference between Level C and Level B because the decision-condition structural coverage objective of Level B is largely covered already in Level C due to those more detailed low level requirements. Also, developers making extensive use of Parameter Data Items (objects or logic external to the main application program) are now required to fully document, review, trace, and test all that data under DO-178C; something they “should” have been doing under the intent of DO-178B. Voilà.
Level A is the most critical software level and hence the most expensive. True. Still another myth exists for Level A, namely, “Level A is extremely difficult to achieve and the software will cost at least 30-50% more than Level B.” False. Level A imposes yet more structural coverage requirements (MCDC testing), source to binary correlation, and more independence within reviews. The most significant cost driver over Level B is the MCDC testing requirement. However, with proper application of modern structural coverage tools, personnel training, and thorough requirements based testing, the added cost for Level A can be largely contained, thus Level A software is only slightly more expensive than Level B. For this reason, most COTS products pursuing Level B certifiability instead opt for Level A. However, as previously mentioned the system/hardware costs will be higher for Level A than B due to added redundancy required to meet Level A’s 100x higher required reliability over Level B.
Where does the money go?
The following Figure, DO-178C Cost Metrics, shows a typical breakdown of expenditures for a Level C project; Level A of course would have a greater percentage allocated to Verification.
Figure: DO-178C Cost Metrics
Benefits of DO-178C.
DO-178C is certainly neither free nor cheap, as cited above. However, DO-178C can actually be cost-effective, when implemented properly. Particularly when evaluated over a product lifetime or subsequent product versions when DO-178C efficiency and benefits are most notable. Why then are so many entities adopting DO-178? Because of the actual DO-178 benefits.
The following describes the most commonly obtained benefits from DO-178C based upon this author’s 25 years’ experience consulting on over 250 avionics projects.
- Greater upfront requirements clarity: DO-178C mandates thorough and detailed software requirements. Such detail, and the necessary discipline, force answers to be provided up-front instead of being deferred. Assumptions are drastically minimized. Consistency of requirements and their testability is greatly enhanced. Iterations and rework due to faulty and missing requirements are greatly reduced. Yes, other standard and guidelines such as CMMI mandate such upfront requirements; however, DO-178C is unique in its enforcement of requirement detail.
- Fewer coding iterations: Code iterations, or churn, are the bane of software engineering. In many cases, ten, twenty, and even thirty versions of evolving code files exist on new products. With strong engineering processes and discipline, code should be largely correct the first time it is written and should not require dozens of updates to get it right, particularly when using well-advised software modeling. Models and code should be reviewed by analyzing implementation versus documented requirements.
- Fewer bugs found during module testing: Since DO-178C mandates thorough and testable requirements, in addition to code reviews for Level C and above, far fewer bugs will be found during module testing. Independent code reviews required for Level B and A also further that aim. And DO-178C code reviews require the following completed and configured items prior to writing models or code:
- Standards & checklists
- High and low level requirements
- Traceability for the above
Remember, DO-178C traceability should be as shown in the following DO-178C Traceability Figure:
DO-178C Traceability Figure:
- Greater consistency within software: Software is like a chain: it can only be as strong as its weakest link. Software which is 99% correct is 1% incorrect, which means it is unsafe. Software is never provably perfect and DO-178C makes no claims that perfect adherence to its objectives yields perfect software. The weakest software module, or software engineer, is on the critical path of software safety. All software must be consistent per its level of criticality and DO-178 enforces such.
- Fewer defects found during integration: Integration can be a lengthy iterative process where major defects requiring design changes are discovered and fixed. Not with DO-178, where integration is typically 50-75% faster than non-DO-178 environments. Since all software whose operation is potentially affected by hardware must be tested on actual target hardware, integration begins earlier and software is verified earlier. Also, the system upon which the software resides must comply with ARP-4754A, which mandates similarly strong system processes; it’s likely the custom hardware logic components will need to meet DO-254 (hardware logic’s corollary to DO-178C) so that hardware will similarly be of higher quality at integration. Thus, integration begins earlier with inherently fewer flaws than non-DO-178C environments.For a video on reducing DO-178C Gaps, see here: https://www.youtube.com/watch?v=7doZAZ5CTMk
- Improved configuration management: The ability to control and recreate any snapshot of the project is covered by Configuration Management (CM). However, CM also ensures security, backups, completed reviews, problem reporting, and version control. DO-178’s strong CM requirements, coupled with modern tools which automate many CM tasks, ensures higher software quality now and in the future. Literally any version of released DO-178C compliant software must be able to be recreated and retested in its entirety for thirty years after delivery. That means not only the application source code must be controlled and captured, but all the ancillary files including makefiles, build scripts, 3rd party tools, development environments, design and test data, etc. must all be captured. This capture eliminates common problems in software maintenance found within other industries.
- Easier regression testing: You build your product on the assumption that it will be successful, and therefore have a long life. You also know your software will evolve through new applications, installations, and versions. Regression testing can be expensive if extensive or manual; DO-178 provides thorough traceability to determine which modules need changes or analysis, and corresponding retesting. Since DO-178C requires thorough and provable regression testing, DO-178C adherents receive the benefit of easier and usually automated software retesting. The following graphic shows optimal DO-178C Testing strategy:
Figure: Optimal DO-178C Testing Strategy:
(Excerpt from AFuzion’s DO-178C Training)
- More thorough testing: 100% high- and low-level requirements coverage, greater robustness coverage with clarity, and software structural coverage are deterministic verification aspects of DO-178C. Parameter Data Items must be particularly well documented and verified under DO-178C. Unused software structures (dead code) must be removed and deactivated code documented and verified for Levels C through A The probability of software errors encountered in the field is vastly reduced, particularly at Levels A and B.
- Improved hardware and system testing: system testing is typically challenging for embedded systems because the development environment is quite removed from the target environment. Also, testing is often performed via different engineering teams. DO-178 mandates that software components which could be affected by hardware be tested on the hardware, e.g. hardware/software interfaces, interrupts, timing, board-level components, BSP/RTOS, etc. And the improved determinism and quality of all these components via DO-178 belies improved hardware integration. Since DO-178C requires complete traceability, it is quite possible to show via traceability that system requirements were actually fully tested during software testing and therefore need not be repeated by additional testing at the system level.
- Fewer in-the-field defects: In-the-field defects and customer returns are hugely expensive in both the short- and long-term. Products developed under DO-178 have been demonstrated to result in 80%-90% fewer returns. For a brief introductory technical 1-hour instructional video on DO-178C and also related topics including DO-331 Videos and DO-332 videos, , view the free Video DO-178C Costs & Gaps Closure 1-hour introduction video from AFuzion at the following link: https://www.youtube.com/watch?v=7doZAZ5CTMk&t=1031s (DO-178C Introduction Video Tutorial – 1 Hour)
- Improved reusability: Via thorough and consistent documentation required by DO-178, modularization, enforcement of documented modern engineering principles, and reviews to ensure all the above was achieved, reusability is greatly improved. In software, reusability is the Holy Grail, but the reality is that unless a software component is at least 80% reusable (i.e. unchanged), then it is quicker and less risky to simply start from scratch. In reality, most software is less than 50% “reusable”. With DO-178, and enforcement of design/coding standards, coupled with independent reviews and traceability, most modules should be at least 70% re-usable. And when modeling and C++ are employed, reusability increases even more.
- Improved management awareness of true development criteria and schedule status: How many software projects report a “99% Complete” status week after week? How is software progress measured? How can management truly ascertain completion status of software? The answer to all these questions is via modern and accurately detailed management techniques built around DO-178. The provision for insight, traceability, accurate status on design, development, testing, integration, and reviews is found in DO-178. For a detailed DO- 178 PSAC Review Checklist, see https://afuzion.com/plans-checklists/#do-178c-plan-templates and the 10-page AFuzion PSAC Review Checklist figure below:
DO-178C PSAC Review Checklist: Page 1 of 10
- Improved completion schedule due to the above: “What gets measured gets done.” DO-178 provides for continuous project completion and quality statusing.
(Download the rest of this paper for seven (7) additional DO-178C Cost Benefits.
DO-178C requires five DO-178C Plans, three DO-178C Standards, and 20+ DO-178C checklists. Most companies (over 150 companies and 9,000 engineers worldwide) use AFuzion’s DO-178C Planning templates, DO-178C standards templates, and DO-178C checklist templates described below. Once acquired from AFuzion and customized on the first project, your DO-178C project will retain the expertise to create, customize and re-use as appropriate on future DO-178C projects. Note that the DO-178C Planning documents (five documents) can be purchased in either Template form or “Initial Draft” form. The Template form option provides the basic templates which you then modify to create an initial draft. The Initial Draft option provides for AFuzion to first create initial drafts of all five planning documents using the same template, but adding the customer’s basic product information to create an initial draft; the customer then must finalize this initial draft to create the first versions of these five planning documents.
DO-178C Plans and Checklist Templates cover all phases of the system’s software project lifecycle and are developed with DO-178C in mind. The users of these templates would need to have some basic understanding of DO-178C, such as attendance at AFuzion DO-178C training (see here: https://afuzion.com/do-178-introduction/ )
The regulatory agencies require that most airborne commercial systems operating within commercial airspace to comply with DO-178C and DO-254 (details can be found in the regulatory website). The planning and processes for the engineering lifecycle are required for any DO-178C and DO-254 project and those processes must be defined before initiating that phase and followed during that phase. Although Checklists are not formally required by DO-178C and DO-254, your regulatory agencies will require that you prove conformance to DO-178C and DO-254 according to your approved processes; this conformance is very difficult to achieve and prove without checklists. In fact, nearly all DO-178C and DO-254 projects use checklists for reviews and proof of such.
AFuzion Process Templates are meant to provide the proper framework for customization to meet the system process requirements of DO-178C. Although there is no perfect process for all programs, there are unique areas of each individual project. Each project will vary somewhat in how it chooses to define, implement, or augment the AFuzion process Templates. Factors to consider include project Design Assurance Levels, complexity, and expertise of staff, development methodology, tools/environment, and technology. The AFuzion process Templates provide the basic elements of an avionics project compliant with DO-178C and DO-254. Further, AFuzion can customize and tailor these processes by the appropriate amount as an outflow of the gap analysis process, upon request as part of the optional first draft delivery.
If there are items in the checklists that are not applicable to your program they could simply be answered with “N/A”. It is also recommended that the checklists be placed under your project’s configuration management (CM) system to ensure checklist integrity.
Independent reviews are always preferable to reviews done by the developer. It should be noted that the checklists should be widely distributed to all personnel developing any avionics lifecycle item, prior to that person beginning such initiation. Thus, the checklist serves as a “report card” whereby the originator’s success is measured. When the originator understands what the independent verification reviewer will be evaluating the related software/hardware artifact for, the originator will more productively attain checklist compliance during development of that artifact. This applies to requirements, design, implementation, test, etc. The level of “independence for verification” required by DO-178C and DO-254 varies according to Design Assurance level (Level A through E). More DO-178C Template information is available here: https://afuzion.com/plans-checklists/
AFuzion’s DO-178C Plan/Standard Templates:
- Plan for Software Aspects of Certification
- Software Quality Assurance Plan
- Software Configuration Management Plan
- Software Development Plan
- Software Verification Plan
- Software Requirement Standard
- Software Design Standard
- Software Coding Standard
- Software Configuration Index / Software Environment Configuration Index
AFuzion’s DO-178C Checklist Templates:
- Plan for Software Aspects of Certification Checklist
- Software Quality Assurance Plan Checklist
- Software Configuration Management Plan Checklist
- Software Development Plan Checklist
- Software Verification Plan Checklist
- Software Requirements Standard Checklist
- Software Design Standard Checklist
- Software Coding Standard Checklist
- Software Configuration Index Checklist
- Software Requirements Checklist
- Software Design Data Checklist
- Source Code Review Checklist
- Software Test Procedures Checklist
- Software Verification Results Checklist
- Software Verification Analysis (including Traceability) Checklist
- Software Accomplishment Summary Checklist
- Stage of Involvement #1 Checklist
- Stage of Involvement #2 Checklist
- Stage of Involvement #3 Checklist
- Stage of Involvement #4 Checklist
These DO-178C templates and checklists also help in getting organizations to the goal of higher SEI CMM/CMMI ratings (preferably Level 3 – 4+). Usage of AFuzion process templates and checklists are intended to maximize the probability of project success and quality.
For the remaining 14 pages of this AFuzion DO-178 Introduction Technical Whitepaper, please download below.
Information Request Form
Please provide the following information to receive your full WhitePaper