"Our Ambition: Develop the Best Solution"

It's a fact that people also buy glasses online. However, in quick virtual fittings of off-the-shelf frames, it can happen that details are missing and the quality is inadequate. Not so with the digital frames used in ZEISS Virtual Try-on. Armin Eichert, physicist and project manager for the frame scan at ZEISS, explains how the scanner was developed and what makes ZEISS digital frames so special.

How did the frame scanner come to be developed at ZEISS?

It all started in 2017 with a ZEISS venture, an internal start-up, aimed at producing high-quality, photorealistic 3D scans. As the specific use case the eyeglass frame scanning was identified. Therefore, the ZEISS Vision Care business group was where this venture then evolved. Everyone really pulled together and worked on the scanner and software as if it was a start-up within ZEISS. This was done in collaboration with scientists from the ZEISS Innovation Hub @ KIT, as well as the design department within our R&D division and software developers from ZEISS Vision Technology Solutions in Bangalore. The core team consists of physicists and computer vision scientists, i.e. specialized computer scientists. Right from the start, our ambition was to develop the best solution – to get as close to reality as possible. However, it is not so easy to scan frames, so we really tried out a lot of things and tested various scanning layouts and processes in order to achieve our quality standards. In addition, the scanner system had to be scalable.

I would go into more detail about scalability right there. But first the question: What are the digital twins of the frames actually needed for?

The digital frames are used in ZEISS Virtual Try-On. This is the digital frame try-on that can be used on-site at the specialist retailer, from home or on the go. The digital frames are superimposed onto the user's own 3D avatar. All with a quality that allows you to order glasses, even progressives, from home.

And why is it so difficult to scan frames?

A 3D scan is always a snapshot, so everything has to be just right at the moment when it is captured. Frames are not suitable for a simple scan because the material is either partially transparent or highly reflective. As a result, establishing suitable lighting conditions in the scanner is a complex task. Uniform lighting in the scanner is absolutely essential and this ultimately presented the greatest challenge. Moreover, there is huge variance between frames and their materials, all of which behave differently. Another challenge, for example, was to create suitable environmental conditions for the frame digitization so that the same high-quality capture could be maintained across all frame model types. This was achieved through a great deal of optical know-how, which was tested in various prototypes. Digital frames were demonstrated to customers for the first time at ZEISS Convention 2019.

How did they react to the frame models? Are they already able to use them in their stores now?

They were enthusiastic even back then. Two more scanner prototypes were created in collaboration with the design department within our research and development division. Using the final scanning process, we have so far generated over 5,500 high-quality, precise, and true-to-reality 3D frame models. To offer digital frame try-on as a specialist retailer, ZEISS VISUFIT 1000 Platform and the associated "Virtual Try-on" module are required as a basis. This is currently available in more than 30 countries.

How exactly does frame scanning work at ZEISS?

The details are obviously a trade secret. What we can say, however, is that cameras with ZEISS lenses are used in the process. We perform a variety of measurements on the frames using these cameras and other hardware. The results of these measurements are then automatically transferred to a largely photorealistic 3D model using specially developed software.

Is it fully automated?

No, it is not fully automated at the moment. Once the first near-photorealistic version of the 3D model has been created, the human quality check starts. Experienced graphic designers that form a worldwide network look at the model, improve it, retouch where necessary and ensure that it actually is very close to reality.

What makes ZEISS digital frames so special?

Definitely the degree of detail and precision. We have carried out comparisons and can say that we deliver the highest quality in the market in the field of virtual frames. Specifically, this means that the smallest details, engravings, or surface reliefs, structural changes of the material can be viewed using the 3D model. The reflection of the ambient light on the material is reproduced 1:1. Of course, it is still different from physically holding the frames in your hands. But our models come very close to reality. Another point illustrates the emphasis we place on accuracy: after all, the scanner was developed for use in the industrial environment, where more powerful movements can occur. The housing design meets the highest stability demands in order to keep the cameras absolutely stable in their capturing position. This is because even the smallest changes to the mechanical setup had to be re-calibrated.

Could the scanner be used for other products where a similar level of detail is required?

The scanner we have developed is currently optimized for eyeglass frame scanning exclusively and is designed precisely for this purpose. However, if developed, the technology has the potential to produce photorealistic scans of other objects also.

You said the scanner system should be scalable, what exactly does that mean?

Using pure in-house know-how, we have created a true key technology that can be scaled. This means we have developed something that can be replicated. We are able to scale hardware, software and the processes, basically the entire system, and thus make it available at other locations, too. We are planning additional scanners at various locations so that we can act quickly worldwide. My goal is to populate our frame catalog with thousands of frames per month in the future.

That sounds like an incredibly fast development.

Yes, it was indeed a very fast development, but we had and still have the management's commitment and support and developed the crucial knowledge on our own. It's also really fun to be a pioneer and to work on a new task like this with the team. There was never any silo thinking on this project. Ultimately, our high-quality frame models help our customers in ophthalmology to compete successfully on the market. This has been and continues to be a great experience for all of us.

Thank you for these interesting insights!