Applying natural language processing to aircraft maintenance LexX Technologies
Australia’s LexX Technologies’ diagram shows how it applies NLP for aviation.

Natural Language Processing For Aircraft Maintenance Explained

Big data analytics works great with data, but how can you add written aircraft manuals and mechanic troubleshooting notes to the mix?

With the latest aircraft churning out gigabytes of numerical data, there is plenty of number-crunching to do to enable predictive maintenance. But a great deal of information important to predictions and smarter troubleshooting is not in the form of numbers; it is in words—in manuals, in pilot and mechanic notes—or even in mechanics’ spoken descriptions of immediate problems. For this kind of data, natural language processing (NLP) is the tool of choice.

Humans do four basic things with language that machines also can do, or at least attempt. We speak, and computers are rapidly maturing in the ability to translate voice to text. We read, and optical character recognition by computers, or OCR, is becoming increasingly better at transforming handwritten notes to typed text. We listen, and computers can now easily translate text to voice. But humans also perform a fourth function, understanding language. This is the much more difficult core of NLP—understanding the meaning of words so we can make decisions and act upon them.

Voice-to-text, OCR and text-to-voice software is generally available in plug-and-play modules, although each must be customized for a particular application such as aircraft maintenance. As for NLP capabilities, anyone who has used Apple’s Siri or Amazon’s Alexa should have a feel for them. But these are consumer tools, investment costs for which can be spread over hundreds of millions of paying customers. Developing NLP for a specialized business market like aircraft maintenance is much tougher to justify economically, and the demands may be more technically difficult as well.

These demands include helping mechanics extract portions of massive aircraft, engine and component manuals relevant to maintenance problems and understanding the implications of descriptions of past maintenance problems, remedies attempted and the results of those attempts.

One textbook on the subject defines the understanding core of NLP as translating natural human statements into data—either numbers or very well-defined text—that a computer can use to learn about a specific subject: in our case, aircraft maintenance. The aim is to produce data that can be used to build algorithms.

This requires that computers initially be programmed to digest the grammar and vocabulary of natural human language, including common idioms and how the true meaning of a word depends on its context.

Once translation into numbers or well-defined text is done, other techniques such as machine learning can be applied to extract implications. For example, hundreds of mechanics’ scribbled notes on troubleshooting experience are put in a form that enables statistical analysis of the most effective steps, given specific conditions. Or, in a simpler example, thousands of pages of PDF manuals are put in a form that allows an engineer to quickly locate all sections relevant to a specific problem.

FIRST STEPS

In general, the first step for NLP is gathering the materials to be processed. If pilot or mechanic notes are still on paper, these documents must be scanned into PDFs.
OEM manuals are usually already PDFs or digital. But accessing some manuals may be challenging, as OEMs seek to retain and control them. Even manuals kept by airline or MRO document staff may be jealously protected.

Some very useful language is spoken, either recorded or live, by pilots in cockpits or mechanics working on aircraft. But hangars are noisy, and before mechanics’ comments can be used, filters must be developed to screen out background noise from the useful words of mechanics as they work on aircraft.

The second step, once relevant material has been obtained and scanned, is converting it into text that NLP can work with. Optical character recognition can turn scanned writing into text, so this technique obviously works with PDFs. Speech recognition—sometimes called automatic speech recognition, computer speech recognition or speech to text—does the same for vocal records.

Neither character recognition nor speech recognition software is perfect, but both are maturing rapidly. The more familiar they are with the particular subject of aircraft maintenance and with particular handwriting samples or voices the more effective they can be.

The third step, with the original language data in digital text, is the harder part: understanding this text. NLP first ingests the digital text, which may have originally come from maintenance manuals, component manuals, service bulletins, airworthiness directives and fault-isolation manuals mechanics’ notes or recorded comments. All these sets of text are put into its “memory.”

NLP then breaks down the related sentences of this text and analyzes the vocabulary and grammar. Natural expression of a problem, experience, remedy or other action is converted into a highly structured and precise statement that a computer can treat like a mathematical statement. NLP will also create an index, like a book index, but specific to the aircraft-maintenance domain.

NLP algorithms to do this kind of analysis may be available from generic sources. But these generic algorithms must be tailored to reflect the subject of aircraft maintenance and then be optimized for speed.
The experts in both aircraft maintenance and NLP who can do this well are neither plentiful nor inexpensive. That is one reason that progress in applying NLP to aircraft maintenance has been slow. But each NLP project, even ones with limited objectives, builds tools and capabilities that can support further advances.
The next steps depend on how NLP is to be used, and we are just beginning to see all the possibilities.

NLP APPLICATIONS

One use is information retrieval. Conventional techniques for finding something in a mass of already digitized materials involve looking for a perfect or near-match of the search term in the material. Each word is examined, and a yes-match or no-match is given. The result may be a very incomplete tapping of the available information.

NLP enables a smarter search, the kind a human being would do if he or she had time to examine all possible documents. NLP looks for all the information that is related to or close to the search term or question, based on context rather than perfect matches.

Another use might be question-answering, or troubleshooting in aircraft maintenance terms. Here, an important intermediate step is necessary:

Suppose we want to tap the experience of mechanics who have worked on problems in the past. We would use NLP to convert all that experience into precise statements of faults found, steps taken and results achieved. We would then apply machine learning and statistical techniques to sift the data for the most likely fixes for similar future problems, or at least the optimum sequence of troubleshooting steps to achieve a timely fix.

And if we want to enable the mechanic to interact with the troubleshooting system on, say, a personal device, we might want to apply NLP to the mechanic’s question itself. Instead of requiring the mechanic to select from a pull-down menu of limited options, the mechanic could type in or simply speak into a microphone, the problem he is attempting to resolve. Then NLP would translate this naturally expressed question into more precise terms and find the answer suggested by analysis of past troubleshooting experience.

Another use for NLP could be summarizing text. Engineers or mechanics now write reports and summaries of the research they have conducted or jobs they have closed. An NLP-based generative system could eliminate this tiresome chore.

LexX Technologies

Australia’s LexX Technologies’ diagram shows how it applies NLP for aviation.

NLP could also be useful in finding chronic repair problems. Instead of humans combing through voluminous files of past defect reports, looking for common faults and writing them up, NLP could do the job and focus managers’ attention on the most frequent defects. The system would not be solving the problems, but it could ensure that the most chronic faults receive priority attention.

Some believe that future NLP may even replace humans repetitively writing task cards by providing instructions—after reading and digesting maintenance programs, defect reports, service bulletins and other material—to mechanics on demand.

It is still early in the process of applying NLP to aircraft maintenance or to other highly demanding business applications. But the critical first steps are already being taken.

For example, CaseBank Technologies uses NLP in its ChronicX software that analyzes mechanic and pilot reports to identify repeated problems across an airline’s fleet. Since the original reports are written in natural language, it would be extremely difficult for staff to sift through thousands of written notes and find the repeaters. ChronicX also assigns defects to Air Transport Association of America (ATA) chapters automatically and more accurately than humans could.

Australia’s LexX Technologies is helping bring digital intelligence to aerospace and other sectors in two ways. First, its software can find the relevant sections of repair manuals and related documents rapidly and accurately. Second, LexX is developing better troubleshooting tools by combining NLP with machine learning to draw lessons from repair history.

Larger IT companies are also active in this space. IBM already has helped Korean Air tap historical mechanics’ reports to improve its troubleshooting. And Intel’s Saffron AI Quality and Maintenance Decision Support Suite is beginning to enable aerospace companies to classify defects by ATA classifications much faster and more accurately than humans do.

Hide comments

Comments

  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Publish