Augmented reality in the cockpit: projections in the sky
In a critical situation, every second counts on an airplane – and retrieving the most important information quickly can be crucial. Pilots have to maintain an overview at all times: of visible obstacles such as airplanes, weather or terrain and, if possible, of invisible elements hidden behind clouds. This is where ZEISS comes in with technologies such as multifunctional smart glass. Thanks to augmented reality, important information is projected directly into the field of view. The result: a safer and more comfortable flying experience that redefines the future of aviation.
Get onboard, take off and leave everything behind: Linda Kotzur experiences this feeling of freedom time and time again. Nevertheless, as a passionate pilot, she also knows that flying comes with great responsibility. After all, every mistake, no matter how small, may have particularly serious consequences. Vision plays a key role. Private pilots are heavily dependent on good visibility when flying according to visual flight rules. You are responsible for the distance to clouds and other aircraft. "Your eyes are constantly moving back and forth between the cockpit instruments and your surroundings – a particular challenge, especially when landing," she explains.
Airplanes of the future: information in the field of vision
In the future, ZEISS will help pilots to cope better with these challenges, especially in poor visibility and emergency situations. For several years now, the company has been working on an advanced head-up display (HUD) that projects essential information directly into the field of vision. In developing the technology, ZEISS draws on the valuable experience of long-standing pilots such as Linda Kotzur and Tino Janke.
Dreaming about flying
Linda Kotzur has been in the cockpit for more than six years: In 2018, she made the decision to get her pilot's license – initially in secret so as not to worry her family. "They would have been worried about me otherwise," she remembers. In addition to her job in internal sales, she works with experts to maintain a website on the subject of "Becoming and being a pilot". Her mission is to make aviation safer. In line with this mission, she has also developed an online aviation weather course – for pilots who fly according to visual flight rules.
She has since reduced the hours in her office job so that she can devote more time to flying. "A pilot never stops learning" is her motto – and that's why she regularly broadens her horizons with the latest developments that make flying safer and easier, such as multifunctional smart glass from ZEISS.
If you fly as a VFR (visual flight rules) private pilot, then you are basically piloting by sight.And if, despite the best flight preparation and weather planning, the weather changes for the worse, it can get unpleasant.
Professional pilot Tino Janke has been flying passenger aircraft for more than 20 years from the Boeing 737 to the Airbus A330neo. The 49-year-old is one of around 10,000 commercial pilots in Germany and is also a passionate powered aircraft and glider pilot.
Single-pilot operation is increasingly being sought in aviation, he explains. "If this trend continues, head-up displays will significantly alleviate the burden placed on pilots." While "pilot flying" and "pilot monitoring" oversee the flight process in today's tried-and-tested "two-man cockpit", electronics will increasingly take over the role of the "monitoring pilot" in the future.. "If commercial aircraft are only controlled by one pilot, as is already the case with some business jets today, efficiency will be paramount."
He sees great potential in multifunctional smart glass technology. "In view of the constant growth in air traffic, airspace will need to be shared with more and more actors. Conflict-free coexistence in airspace will have to be guaranteed not just for aircraft, but also drones and very soon autonomous air cabs as well.
Thanks to its high optical quality, the system overcomes the weaknesses of earlier models. Contrast, glare, visibility and operation are significantly improved.
Dr. Dennis Lehr, part of the ZEISS Microoptics management team with responsiblity for the HUD project, describes these benefits as follows: "Primary flight data such as speed, altitude, position and flight direction will be read directly in the field of vision when looking out of the cockpit. A large field of vision also makes it possible to display information adapted to the respective situation in the interests of efficiency." The technology helps pilots to focus their attention as much as possible on the flight and the world outside.
But how does this technology actually work? An infrared and microwave camera captures the surroundings and projects them as an image directly into the aircraft's field of vision. This means that runways, obstacles or mountains can be recognized even if visibility is poor. The HUD thus minimizes risks and prevents collisions. Unnecessary holding patterns and flight diversions due to bad weather can also be increasingly avoided which benefits the environment.
Compact and cost-effective systems for increased safety
"HUDs have been in use in commercial aviation for years, but are not yet fully established," explains Tino Janke. "Our industry is eagerly awaiting a new generation of this technology," he adds. After all, previous systems stood out primarily due to their sheer size in the cockpit.
Tino Janke has been very much involved in giving training in emergency procedures for years. He thinks HUDs will be introduced universally in general aviation in the distant future too. "Particularly when there is only one pilot in the cockpit, there is a lack of redundancy and a helping hand to reach for the right checklist." Incorrectly completed checklists or ones that have not been completed at all are still often the cause of accidents in aviation."HUDs can already make a major contribution to overcoming these challenges," emphasizes the aviation expert.
The HUD from ZEISS, he believes, will set new standards: "Thanks to its high optical quality, the system overcomes the weaknesses of previous models. Contrast, glare, visibility and operation are significantly improved. We expect not only the simple mirroring of instrument displays, but the entire range of modern display options," he adds.
Our vision is to make this technology accessible to all pilots from commercial pilots to private pilots. We want to take the pressure off them and give them a safe flying experience in all visibility conditions – from take-off to landing.
In order to meet these expectations, ZEISS is relying on its proven holography technology, which has been used successfully for years in ESA and NASA space missions, to develop the HUD. Thanks to continuous development, multifunctional smart glass technology now offers a wide range of new applications in the aviation and automotive industries, including HUDs, transparent displays, lighting systems and sensor systems. In aviation, not only pilots benefit from these innovations, but also passengers: in the future, transparent displays could show information and entertainment directly in the windows and enhance the flight experience.
But what has made this development so difficult so far? "The cost and the size," explains Dr. Dennis Lehr. The doctor of physics explains, "HUDs with conventional optics are particularly large and expensive and take up a lot of space in a comparatively cramped cockpit. This makes them unsuitable for small machines. "The challenge was to make these systems more compact and more cost-effective," he explains.
After years of intensive development work, the ZEISS solution is currently in the test phase and should be ready for the market and for use in business jets and passenger aircraft in around three years' time. "Our vision is to make this technology accessible to all pilots from commercial pilots to private pilots. In this way, we want to take the pressure off them and provide them with a safe flying experience under all visual conditions – from take-off to landing," emphasizes Lehr.
Linda Kotzur is already looking forward to testing the new HUD technology: "I love the technology on the airplane," says the 34-year-old. She sees ZEISS' technology not only as an increase in safety, but also as a relief in the cockpit. "I can hardly wait until HUD systems find their way into small airplanes."
In focus: Augmented reality in the cockpit
-
Augmented reality adds digital image content to our perception of our surroundings. In the airplane, this digital technology is used to project information from the cockpit behind the windshield and directly into the field of view. Pilots no longer have to lower their eyes to read the flight instruments – they can see the relevant information while simply looking out through the windshield.
-
The expansion of reality through virtual or augmented reality (VR or AR) will play a major role in the cockpit of the future. They will make flying safer because pilots will no longer have to look away from the windshield at the measuring instruments to read information. This also allows them to see obstacles that cannot be seen in the real environment. This also makes flying safer. In addition to augmented reality, voice control and assistance systems are also set to change flying. In the airplanes of the future, pilots will be able to call up information or carry out actions by voice command. The system will also be able to give them recommendations for action based on data.
-
While conventional displays are monitors that are permanently installed in the vehicle, holographic head-up displays (HUDs) use screens such as the windshield to display their content. They are projected onto the glass surface using optics and seem to appear behind it – directly in the driver's field of vision. In the past, such solutions as these required a lot of space. That is why it was not possible to install them in smaller aircraft. However, ZEISS has now developed a system that requires less space and is much more powerful.