Closing in on cockpit AI

Appareo announced June 18 that the Fargo, North Dakota, firm best known for the Stratus line of ADS-B products has purchased Aerovie, developer of the eponymous EFB created by a pilot. The Stratus maker plans to integrate into Aerovie features that Appareo had previously rolled out in 2018 as part of the new Stratus Horizon Pro app.

In the near term, the Aerovie acquisition will end Appareo’s reliance on third-party application developers to harvest data from the Stratus, including attitude and heading reference system (AHRS) data, georeferenced charts, and more, and make it available to pilots. Appareo’s Stratus Horizon Pro features, including radio transmission recording and playback, and radio transcription (a beta feature) that converts ATC speech to text, will be folded into the new EFB, which may eventually get a new name from its new owner.

In the longer term, the benefits of integrating this hardware and software should expand. Neither Siri nor Alexa will be flying with you, but that’s due to a combination of trademarks and technical issues. By whatever name they come up with, the functionality offered by voice command recognition and execution will be similar in aviation to products that have grown increasingly popular in the consumer electronics world. Some of us are already talking to our cars (and getting a response).

Faced with the unique challenges of aviation, the Appareo team opted to develop their own version of the digital personal assistant, one that is flight-worthy.

“Stratus Horizon Pro is an exciting first step toward AI-assisted technology for pilots,” said Appareo AI Systems Director Josh Gelinske in 2018. “We look forward to building on the core of this machine learning technology to reduce workload and increase safety in the cockpit.”

Appareo took on the challenge of building a deep neural network from scratch to interpret aviation speech, including the rapid-fire bursts of jargon that would likely tie Siri and Alexa in knots. An audio cable connecting the aircraft audio panel to a mobile device running Stratus Horizon Pro by way of a headset connection already enables the mobile device to record radio transmissions for playback through the headset. Speech transcription capability is still in beta testing.

Appareo is “training” the foundation of the digital cockpit assistant of the future to cope with regional accents and rapid-fire speech (looking at you, New York Approach), and transform ATC transmissions into text. (GPS integration already refines the current capabilities, enabling the software to interpret the correct fix based on location in case of a duplicate name, for example.)

Accurate speech recognition is a much more complex computing task than some might think. Most consumer electronics with this function today rely on an internet connection, since Siri and Alexa do much of the “recognition” work in the cloud, not on your smartphone. An Appareo spokesperson noted in an email exchange June 19 that the firm is hard at work making speech recognition workable and consistent using only the computing power available in the cockpit, no internet required.

The possibilities are tantalizing. Eventually, with the right mix of hardware and software, future pilots will get used to delegating tasks like frequency changes, programming routes, route changes, and perhaps running checklists or controlling other components to a virtual, digital, “intelligent” co-pilot. One that will never forget to lower the gear. (Insert your own joke about pilot “intelligence” here.) Fumbling with knobs and dials and the task saturation that comes with that will be a thing of the past, a quaint horror relegated to aviation history.

Meanwhile, the Aerovie EFB should soon integrate with older models of the Stratus (the Stratus 3 can already integrate with any EFB app using the GDL 90 protocol). The company will continue to support any EFB that uses the GDL 90 protocol, including ForeFlight and others.

Current Aerovie subscribers (plans cost $6.99 monthly or $69.99 annually) will not see any disruption as the app is transitioned to the new owners. Aerovie was developed by private pilot and software developer Bryan Heitman, and began as an app to distribute pireps. Heitman has added many features since. Today, Aerovie’s selling points include a vertical weather profile tool that allows pilots to assess conditions along a route based on speed and departure time.

“We’re very excited to take Aerovie to the next level by joining the Stratus product family. Our companies have been aligned in our commitment to G.A. over the years," said Heitman in a news release. "We understand the unique needs of a single-pilot operation versus a crew, and we’re committed to making flying safer and more friendly for that G.A. pilot.”