The FAA’s oft-touted shift to a risk-based oversight system is evident in many of the agency’s programs, but a Transportation Department Office of Inspector General (OIG) report found the FAA’s oversight remains notably absent in one of the areas touted as critical to manage future certification demand.
The agency’s decade-old Organization Designation Authorization (ODA) program was conceived to standardize a process that had been in place since the 1950s: delegating certification tasks to approved “designees” at manufacturers, with the FAA ensuring their work met regulatory standards.
As the FAA grapples with ever-tighter federal budgets and increasing demand from industry, ODA, which approves teams of designees at companies, is seen as key to keeping projects flowing and aircraft flying. Today, about 80 manufacturers are ODAs, with one—Boeing—having a dedicated oversight office because of the volume of work that its ODAs handle. ODA oversight falls under the FAA’s certification offices, which employ about 700 engineers and 250 inspectors. They are responsible for both ODA-designated companies as well as the 1,500 other manufacturers that do not have ODAs.
Despite the uniqueness of the work that ODA teams perform, the agency’s current staffing model—known as the Aviation Safety Staffing Tool and Reporting System—does not include detailed, organizational-level ODA data. It also does not detail how much time its own workforce spends on ODA work versus traditional surveillance.
“As a result, all ODAs, regardless of size and complexity, are treated equally in FAA’s staffing model,” says OIG’s report, released in mid-October. “According to the FAA, ‘it is continuing to work on developing staffing standards to more effectively perform ODA oversight.’”
Part of the improvement is expanding the model so it can make office-level projections. But the model uses average time spent on ODA work as its primary metric. Since ODAs vary in size—a variable the model does not account for—the tool is not very effective for projecting at the office level. Instead, managers determine staffing levels needed on the front lines.
OIG’s report notes that while ODA oversight is evolving, its lack of a systems- or risk-based approach continues to be a major drawback. A 2012 industry working group report cited the lack of a systems-focused approach as a major reason the ODA program was not delivering more benefits such as streamlining the certification process.
The agency’s primary oversight tools are an 18-item supervisory checklist and a team audit. “FAA guidance provides little direction as to how engineers and inspectors should complete their ODA oversight checklist and lacks a focus on risk,” OIG says. “Most checklist items are focused on activities of individual company personnel rather than overseeing the ODA system.”
Such a prescriptive approach sometimes leads the agency to focus on seemingly minor violations, such as incorrect paperwork, rather than bigger-picture systems shortcomings, like inadequate testing procedures to prove a design meets regulatory standards.
“Our review of the biennial team audits conducted by FAA in fiscal years 2013 and 2014 at five ODAs found that roughly half of the 123 findings were minor issues (i.e., paperwork errors),” OIG says. “For example, FAA found that a company was using its marketing name rather than its official name on a technical document.”
The FAA has revised its ODA guidance twice, “but engineers and inspectors still do not have sufficient guidance and risk-based tools to meet program requirements and focus on highest-risk areas,” OIG says. The FAA’s guidance does not address how data should be used to help assess ODA risk. As a result, most oversight offices are not using data as part of their ODA evaluations.
“Shifting to an oversight approach that is systems- and risk- based will take time and require sustained management attention,” OIG explains.
OIG made nine recommendations aimed at addressing the concerns detailed in the report. The main focus is developing risk-based staffing tools and related guidance. The FAA concurred with each recommendation but made changes to three of them. The agency plans to have all the recommendations addressed by mid-2017. OIG has asked for more details on the recommendations the FAA modified but considers the others “resolved, but open pending completion of planned actions.”