AGILE Introduction for Aviation DO-178C & DO-278A

 

Free: Download Remaining 10+ Page Paper Here

As a caveat, in this environment we think it’s necessary to walk away from “pure” Agile, or what has been called “Manifesto Agile”, for safety-critical software.   To satisfy the needs of safety-critical environments, we see it necessary to adapt Agile methods to overcome shortcomings frequently found in Agile applications.  After all, Agile, is designed to be adaptive whereas safety-critical software development is evidentiary-based with proof of formal process adherence.  The result of these modifications is still recognizable as “Agile” but addresses the needs of safety-critical software particularly DO-178C.

Hopefully you will agree, but if not, everyone can still benefit from better understanding Agile and applying at least several of its key attributes.  Recognize that the adaptations assume a nominal development environment and culture.  Given a specific instance, an organization can (and should) fine tune these approaches – either emphasizing or de-emphasizing different aspects, keeping in mind that the ultimate goal is to deliver reliable value to the customer.  The team at AFuzion has had success in helping teams adapt Agile to fit into their environment.

Principles of Agile Development for DO-178C

In the discussion that follows we will refer to DO-178C. Recognize that this document is also published in Europe as ED-12C. The comments apply equally.

Software Development Processes

 

DO-178C objective Table A-2 calls for the project to establish key process elements to be followed for the project. These elements are common to any development. They include understanding and gaining agreement on what is to be done (requirements), deriving requirements and communicating them to stakeholders, defining the software product architecture, delineating low level requirements, translating requirements into source code, and ensuring the resultant executable works in the target environment. These software planning elements in DO-178C are summarized in the graphic below via five Plans plus three Standards for avionics software development:

 

All these activities are needed in an Agile project. While the Agile principles espouse characteristics that would influence the DO-178C tasks (e.g., continuous attention to technical excellence, simplicity), Agile does not call for any of these steps to be abandoned. That’s because they are necessary to delivering a product that will meet the customer’s needs. SCRUM does however include features that enhance these activities.

Before discussing the features, it is useful to elaborate on the low-level requirements and why they are there. Recall that in DO-178C parlance, low level requirements are requirements from which source code can be directly generated. In other words, the inventiveness characteristic of software development has already been accomplished. Decisions on key elements of how the code unit will perform its assigned task were accomplished, recorded, and vetted with stakeholders and experts/peers. These consensus decisions are then available to project members who need to understand them (either now or later). The fact that the decisions are consensus decisions, eliminates the risk that a single developer will make an inappropriate decision.

Tables A3 through A7 are similar in that each of them focuses on the outputs of each development step. Whether these steps are taken on each feature to be implemented or on the feature set as a whole, the steps remain the same. Requirements must be analyzed. Designs must be created. Source code must be generated and integrated. The product must be integrated into its operational environment and the capabilities verified.

The tables enumerate expectations for the results of each step (compliance to standard, traced, consistency, reviewed by peers, etc.)

How does Agile/SCRUM fit in?

First, recall that the power of Agile/SCRUM comes from a team-centric approach.  The team commits to the SPRINT BACKLOG.  The team creates potentially deliverable product.  The team has all the skills necessary to be successful and the team works together to deliver value.  Experience shows that the end result is a higher quality product that more closely aligns with customer needs … (continued on remaining 12 pages of this AFuzion proprietary paper – download below).

Click here to download the remaining pages of this AFuzion Agile DO-178C Whitepaper

BAD Software Health: The Best (or Worst) Ways to Fail …

DO-178C’s 71 objectives are interspersed across ten categories of avionics software engineering activities. Failures can literally occur within any of these objectives or activities. But which objectives are most frequently misunderstood and which activities yield the greatest risk of failure? This paper summarizes the best (or worst!) ways to fail within each of DO-178C’s ten process categories. That’s right! Each of DO-178C’s (and same for DO-254 and ARP4754A) ten process categories can be associated with major failures. The secret is knowing what the potential failures are so that you can formulate advanced strategies to avoid them while maximizing your chances of success. And, of course, maximizing success means minimizing risk, cost, and schedule times while simultaneously maximizing quality. You’ve no doubt heard of the saying: “You can have it fast, or cheap, or with high quality … so pick one.” However, successful avionics projects truly require all three: cost-effective and fast development processes, with acceptably high standards of quality. Here, avoiding the common mistakes within each of DO-178C’s ten software engineering activities is paramount to success.

For free membership in the world’s largest Avionics Software DO-178C User Group, please click here: https://www.linkedin.com/groups/4102798/

DO-178C’s (and DO-254’s Similar) Ten Process Categories

So we know that DO-178C covers ten categories of software engineering activities. These ten categories and the common mistakes associated with each are depicted below:

DO-178C’s Top Ten Failures vs Activities (Similar for DO-254 & DO-278A)

1. Planning to Fail

Aviation, like athletic training, is all about preparation and planning. Before an aircraft takes off, a flight plan is conceived of, detailed, and then filed. Similarly, a competitive athlete makes a training plan and smart, successful athletes use nutritionists and trainers to help them. Success requires planning, and no one intentionally plans to fail. However, DO-178C and DO-254 require an advance planning activity yielding five detailed plans plus three standards; the planning recipe requires sufficient detail on the required ingredients of “what”, “who”, and “when” without too much detail on “how”. As mentioned, avionics software health resembles human health in many ways: many factors come together to contribute to the overall state of health and, for avionics, those factors must be summarized in the five plans and three standards. Remember: Planning includes Safety (ARP4761/A) and Systems (ARP4754A) – these processes are not optional. Do your FHA/PSSA/SSA.

Some athletes find that despite their seemingly well-prepared exercise plans, their improvement quickly fades along with their dreams of success: and that their exercise plan simply lacks the ability to strengthen muscles. Why? Typically, the answer is that they are missing crucial details about diet, intensity and the duration of exercise and, most importantly, how to measure success. Aviation is the same. It is all too easy with DO-178C to draft high-level plans which do not sufficiently define the key detailed aspects of your avionics system necessary for successful certification. Common aspects missing from DO-178C planning documents include:

✔ Use of previously developed software. If used, how will it be certified or brought up to compliance standards

✔ Use of configurable software, where users may “reconfigure” software after certification by modifying off-board configuration data. It is imperative to follow DO-178C’s explicit Parameter Data Item (PDI) criteria for such activities.

✔ Application of DO-330 for Tool Qualification, and identifying the full suite of software engineering tools planned for use then defining where the outputs of each will be verified. If verification is insufficient, formal DO-330 Tool Qualification will likely be required.

✔ Application of Model-Based Development as per DO-331 and, when used, a brief synopsis of applicable Specification Model standard versus Design Model standard (these cannot be the same standard: like high-level requirements and low-level requirements, the Specification Model and Design Model are distinctly different and require sequential verification, in that order).

✔ Description of the means to analyse data and control flow coupling. This coupling analysis is about more than checking a box during a code peer review – it requires proper consideration of software design and the interfacing modules/data. Recent third-party tool improvements will help with coupling analysis but your plans must summarize coupling analysis activities via verification engineers.

✔ Regression analysis: all projects have changes and the means to verify those changes depend on regression analysis. The processes used by verification engineers for regression analysis and retesting must be sufficient for the plan reviewer to assess adequacy – experienced verification engineers know how to apply the tools and techniques to semi-automate this process to ensure it’s done right, first time.

By considering the above common mistakes within your “avionics exercise plan”, your project has a chance to win the gold medal at the first attempt. For free DO-178C training videos, view here:

2. The Requirements of DO-178C and DO-254 Failure

The world’s leading software experts agree: most software defects are due to defective requirements. Yet DO-178C provides scant details for ensuring that you have great requirements. Yes, you could hire or outsource your development to world-class avionics developers, but what can you learn from them about developing great avionics software requirements? First, experts know that the successful longevity of DO-178 is due in no small part to its careful balancing of 71 deterministic objectives versus flexibility on how these 71 objectives are met. Yes, DO-178C could include many more pages of suggestions on how to improve software requirements: but they would be just that – subjective suggestions. Consider that avionics projects span a vast variety of domains, complexity, criticality, and size. No single “How to” guide would suffice. Instead, DO-178C uses structural coverage analysis to assess requirements: for Design Assurance Levels (DALs) where people could be seriously injured or worse (beginning with DAL C in DO-178C, ARP4754A, and DO-254), software structural coverage is required. Simply put, software structural coverage objectives exist for multiple purposes, including assessing the degree of software structural coverage obtained when performing requirements-based functional tests.

In Brooks’ 1975 book Mythical Man Month, written just before the first version of DO-178, it is revealed that a leading cause of software defects is “assumptions”, and that these assumptions are often the result of weak requirements. The first version of DO-178 was developed soon thereafter, fully cognizant of Brooks’ writings. When a software developer cannot explicitly understand a desired result (“software output”), that developer is likely to make implementation assumptions. Of course, the developer should instead strive to achieve an improved requirement. Yet developers are much more likely to simply power on and do what they think best, not necessarily what is right. How do good avionics managers avoid this? They employ the following best practices:

✔ Utilize detailed software requirements standards which are reviewed at SOI #1, and which include examples of good, and not so good, high-level requirements and low-level requirements.

✔ Have software testers review the requirements BEFORE the developers see them. Those testers then try to write deterministic test cases from those requirements without making any assumptions; where that is not obviously clear, feedback their questions to the requirements development process to yield improved requirements.

✔ Use software modelling where systems engineers and software engineers use a shared-modelling platform and formal language (SCADE, SysML, UML, etc.). This shared language minimizes the very assumptions endemic to weak requirements.

The figure below shows critical DO-178C requirements attributes (applicable to DO-254 and DO-278A requirements also):

3. V = R + T + A (The DO-178C and DO-254 Equation!)

Yes, engineers love equations! The above equation is clear, but why is the “A” so small? Again, it’s about health. Specifically, healthy verification of software requirements.

But is avionics software health really as simple as following an equation? Almost. But the equation must be properly understood as depicted in the following figure (detailed over 2 hours via AFuzion’s DO-178C training):

V = R + T + A

Verification = Reviews + Tests + A Very Small Analysis

That’s right, in DO-178C and DO-254, verification is performed via a combination of reviews, tests, and analysis:

➤ Reviews: virtually everything is reviewed. Plans, standards, safety, requirements, design, code, tests.

➤ Tests: requirements and code are tested via ground-based executable tests of flight software.

➤ Analysis: when the above combination of reviews and tests does not completely satisfy DO-178C’s verification objective, additional analysis must be performed.

The DO-178C and DO-254 verification equation is best satisfied when the requirements are sufficiently detailed that two independent verifiers could achieve equivalent assessment results. Remember: verification does NOT directly improve software. The goal of verification is to assess the software, specifically if the objectives of the DO-178C compliant plans and standards were fulfilled. Actual software improvement then comes via the feedback process which then feeds into improving the requirements, design, and software.

It is imperative that DO-178C and DO-254 (and ARP4754A!) requirement granularity be sufficient to devise robustness tests based on those requirements. Robustness should answer the question “does the software behave consistently and deterministically under less benign conditions of error values, boundary values, performance testing, illegal state transitions, off-nominal timing values, etc.? In other words, the “rainy day” scenarios. After robustness testing, DAL C, B, and A verification includes “white-box” tests, where actual software attributes and structural coverage are assessed. One objective is to assess the absence of dead code (code which has no reason or requirement to be there and which could and should be removed) and the sufficiency of mechanisms to ensure non-execution of deactivated code. Experienced systems engineers elicit detailed requirements and expert avionics testers are adept at devising test cases to assess software/requirements robustness and dead/deactivated code.

Voila: V = R + T + A

4. Designing Software Failure per DO-178C and Hardware Failure per DO-254

Design is like that distant uncle you see only once a year at Christmas or Thanksgiving. If you span across the entire life cycle nothing is as forgotten as design is. People often fail at requirements but plenty has been written and debated in conferences and training sessions about the importance of them. The same is true for Software DO-178C Verification and Hardware DO-254 V&V. Code has always been the gravity center – more emotionally than in terms of its actual impact on quality output. What is left over and abandoned then? Design! Indeed, design is often treated as a distant uncle and not a very wealthy one. And to some degree, agile processes (which may be helpful when well applied) are kicking design even further from the road. That is why frameworks and standards like AUTOSAR exist. It is not necessarily just to enable modularity, integration and reusability, but also because most people are bad at design and do not realize just how bad they are. Therefore, DO-178C applied properly can provide a useful framework for acceptable design. In the past, V&V was the subject of software developers’ bigotry – a set of activities left to those that were not sufficiently skilled at programming and were therefore relegated to the back seats in IT’s second class – but for a few years now, even non-safety-critical industries, such as banking, have started to view V&V as very important, explicitly procuring such services. For free training videos on avoiding DO-178C Failures and DO-254 Failures, view here:

Back to our metaphor. Winning athletes focus on optimizing their capacity for physical movement, knowing that they cannot fundamentally alter their body’s “design”. In athletics, the design of the human body is the ultimate equalizer: most bodies have fundamentally similar “designs”. Not so in avionics: each system has an implementation customized to its particular design. Avionics software developers thus depend upon a robust and flexible design to yield the necessary consistency and determinism while affording the possibility of future system evolution. So what are the causes of failure in the design of avionics systems? Over the past 25 years, the engineers at CRITICAL Software and AFuzion have worked on 300+ avionics systems – here are the most common design techniques observed which can result in failure:

✔ Reusing prior designs which are poorly understood or documented. If you are inheriting such a design, first document it, then analyse it to see if it is even worth modifying. Like modifying an old or decrepit building, it may be more cost effective to simply start over, the right way.

✔ Failing to have, and verify conformance to, a DO-178C software design standard that specifies:
o Details for all external and internal interfaces, including full bit patterns, encoding rules, use of any variant records, and source and destination information for all data items (remember: include internal interfaces).
o Rules for defensive design/coding
o Health monitoring details
o Task prioritization schemes
o RTOS and BSP usage and limitations plus a reference to allowable API’s (those designated “DO-178C certifiable”).
o Rules for controlling coupling, which provide sufficient details to perform subsequent coupling analysis.

✔ Failing to encapsulate DO-254 hardware dependencies to yield portability. Remember, for successful avionics programs, the question is not “Will it ever be modified?” but rather “When will it be modified?”. Most successful avionics systems are eventually updated with newer hardware and increased functionality. Only by pre-planning future portability can such future upgrades be accommodated. Plan for common core functionality which should not be changed and partitioned away from hardware-specific functionality which is then encapsulated within its own modules. Minimize the scope of change so that the vast majority of software modules, objects/classes and design elements can remain unchanged and unscathed.

Bad DO-178C and DO-254 design is also the key source of V&V difficulties when it comes to incremental testing, data and control flow analysis, effective development of test stubs and many more issues. What is strange is that one will often see people pointing fingers at the requirements or V&V team without realizing that disastrous design is actually the fundamental cause. For a video on DO-178C Gaps and Failures (and how to avoid), watch here:

Click below to download the remaining 8 pages of this DO-178C and DO-254 improvement paper.

(Note: Details of all the above mentioned details are contained within the following 11 pages of this paper (download to read). For additional DO-178C/DO-254 training, guidance, gap analysis, and mentoring, simply contact AFuzion.)

Free: Download Remaining 10+ Page Paper Here

← Back To All Whitepapers