I talk about displays in XR a lot. I think many people focus on the resolution and size, but there are way more factors needing improvement to bring VR to a mass market greater than we see today. Retention is still a problem, and to enable lenses which give better comfort: you need extremely more efficient displays. I should note that in my last article: I focused on self-emitting technologies such as OLED and skipped the possibility of micro-lenses on MiniLED backlights to create a directional backlight. This is another thing you see the LCD industry interested in doing for XR. Every OEM is pushing displays first because there is nothing close to a Moore’s Law type situation for optics technologies. But displays are in everything: The fabricators are competitive.
Now that I have finished explaining some of my theoretical concepts on how companies could enable a Holocake lens to be usable without lasers, let’s talk more about Holographic Optical Elements, or “HOEs”. This is the basis of Holocake and actually way more technologies being shown in the Mirror Lake concept.
There are two types of HOEs being pursued. One is called a PPHOE and the other is known as a LCHOE. Each is named after the material that is being used to record the hologram. PP stands for photopolymer. They are extremely sensitive to thewavelength and angles from which the light is being received. This is why Meta uses laser backlights for their Holocake prototypes. LC stands for Liquid Crystal. From what I understand, they are still being recorded onto films with a photopolymer like PPHOEs but are “doped” with LCs. They are more concerned with the polarization state more than anything else. And LCHOEs are especially useful because they can be designed with dynamic properties based on voltages! Anything you’ve seen related to Solid-State Varifocal (such as Half-Dome 3) are a type of LCHOE!
In Part 1, I speculated ways that Tier 1 OEMs such as Apple, Meta, and Valve could collimate the light sources of lambertian LED sources to better work with Holographic Lenses. One thing I have to note in this article, which focuses more on the optics: Is I think LCHOEs are the way to go in the nearer term. If you look at figures b1 and e1 in the image above: You may notice that the incident angle of PPHOEs are intensely small; maybe reaching about 5 degrees. This is very difficult goal to reach with any sort of tricks done to lambertian light sources which always emit in a wide cone (often up to 180 degrees)
LCHOEs are very sensitive to polarization, but allow a much larger incidence angle of light. It leaves the impossibility sector for things such as OLEDs/LEDs. You could easily improve the cone of light emission for something like a OLED microdisplay using the methods I already explained. To use and fabricate high quality LCHOEs, you just need a lot of knowledge on Liquid Crystals and polarizer fabrication: two things that the display/tech entire industry has become VERY good at since LCDs have been around for multiple decades already.
To recap some Tier 1 company interests in LCHOE technologies, I first have to go over the ImagineOptix story that I reported on late last year. I found out that Valve was suing a company in the North Carolina “Research Triangle” Area. First of all, while it is common for companies (including patent trolls) to file lawsuits against Valve, it is very rare for Valve themselves to get litigious. This isn’t only shown by public documents, but former Valve hardware employee Jeri Ellsworth also commented to me that it was rare for them to act aggressively in the courts when she was there (Early 2010s).
This out-of-the-ordinary lawsuit pushed me to do more research on the ImagineOptix company and the agreements that were made between Valve and the Optics company. I learned in 2018, Valve actually invested about 10 million dollars toward ImagineOptix to help build a fab which would be dedicated to producing LCHOEs for Valve. This investment came with a Master Supply Agreement in writing. Sometime around early 2020, however, Valve wanted to cancel this MSA and demanded ImagineOptix would pay back the investment with interest. We only have speculations on what caused sudden change of heart. But a different Tier 1 company promised a 5 million dollar investment toward IO in the same year that the lawsuit started. The new investor was likely Meta who would confirmedly acquire IO in late 2021.
The Valve vs. ImagineOptix lawsuit would eventually be dropped by Valve in early 2021 in exchange for something that was likely settled out of court. What would Valve have gotten out of IO behind closed doors? Taking timelines into account, it seems that Valve filed for a bunch of patents immediately after they and IO settled the suit. All of these patents were related to LC polarization films that could improve a ton of display technologies such as LBS, MiniLED backlights, OLED microdisplays and more. They also included patents with active LC lenses that could enable either solid state varifocal or variable prescription via software. So I think it is a fair guess that ImagineOptix allowed Valve to take or buy the technologies they wanted. And apply them as their own patents.
I will now stop talking about Valve, which is always difficult for me to do. Let’s talk about the final days of ImagineOptix before Meta acquired them. It was pretty clear that after the lawsuit, ImagineOptix was in a poor state. They were hoping to raise more than the 5 million they got promised in 2020. I speculate this rush of needed capital was due to the fact they also had an expensive rent to pay on a lens fab that was nearly finished (the one that was originally planned for Valve). ImagineOptix mentioned in a local news article that their investors actually wanted them to build the fab in Asia to keep production costs down, but they refused and built it in America anyway. They said they could keep the costs down despite that. But I have my doubts on that, and their “2021 SPIE For-Sale Sign presentation” and acquisition is proof enough.
The final patent that ImagineOptix would apply for would happen in the same exact day/week that Valve sent in applications for their dump of Polarizer patents. This IO patent would be very notable, in my opinion. It was titled “OPTICAL ELEMENTS FOR INTEGRATED IR AND VISIBLE CAMERA FOR DEPTH SENSING AND SYSTEMS INCORPORATING THE SAME.” Within this patent, it would detail a LCHOE that could redirect two different wavelengths of light. One being focused on IR wavelengths, and the other on visible color wavelengths. With this system, you could theoretically direct display light to the eyes and direct IR light to a camera stationed wherever you want… sound familiar? Well it might not because not many people talked about the “strangeness” of the eye-tracking system shown on Meta’s Mirror Lake concept.
The Mirror Lake concept wow’d many people in the XR community. They showed all of these technologies conceptualized in a tiny form factor that ALSO solves Vengeance Accommodation Conflict, has reverse passthrough sensors/displays, and just looked really futuristic. One thing that I found fascinating was the lack of discussion on their Multiview Eye Tracking Hologram lenses. And the fact they had cameras in a place most headsets don’t put them (the eye cameras until now have mostly been within the lens housing). So I wanted to take the opportunity not only to talk about how this concept is very similar to the ImagineOptix patent (which is now owned by Meta) but really just explain in a simple way on how it works.
Holograms are honestly a “magic” technology. They feel like the only optical technology that can almost break the laws of physics (except metalenses). They basically can act as a mirror for different wavelengths of light. This includes the IR wavelengths (700 nm – 1,400 nm), visible red (~650 nm), visible green (~550 nm), and visible blue (450 nm). You can also add a “power” in holograms to increase/decrease the size of each wavelength during the redirection. I have even seen use cases for HOEs to improve display color contrast just by tweaking a color wavelength to do more/less.
Since IR is just a wavelength of light that is invisible to the eye, Holograms can be used mirror that one IR to a camera sensor behind you while letting the visible color wavelengths of the display through. This is the basic principle of how the eye tracking works in the Mirror Lake concept. And apparently, doing it this way will allow better eye tracking that is needed for Varifocal. The industry is looking into “event-based sensors” which are low power, high frequency, but seem to be slightly larger than cameras installed in current eye-tracked HMDs. Size is notable because you won’t be able to fit them in a super thin Holocake lens stack.
So as I get close to the end of this deep dive, I wanted to say that the technologies shown in Mirror Lake are magical and theoretically possible. But it is still strange to me why they pushed this concept to begin with. There is a lot of talk in the community about how Meta’s concept also closely resembled a caricature of what Apple’s rumored $3000 dollar Mixed Reality HMD will include. Especially with pass through cameras on the front of the concept that would show a representation of a wearer’s upper face to outside onlookers.
Even though this was a large dump of information about what has been and what is maybe to come… I still think Meta is not showing their true hand. The industry is aware of many technologies that could get us closer to a HMD like the one Mirror Lake showcases. And it does so without expensive lasers that can bring a host of problems. Both Meta and Apple are constantly hiring people straight out of University of Central Florida’s large Optics and Photonics (CREOL) division. Most of these graduates focused heavily on LCHOEs during their study and it’s easy to see why. All the companies know it’s probably the way to go. And I expect seeing some of them very soon in (expensive) products.
Gallery of CREOL showing all the use cases for LCHOEs at SID Displayweek 2022 (sponsored by Meta)
Bye, I’m Brad!
