I’m in the process of working with the U.S. Patent Office on our latest display-related technology, and a really interesting related technology came to light. Microsoft recently filed a patent for a projection-based display technology for laptops. The patent filing describes an interesting coupling of the existing laptop display paradigm with augmented reality. By embedding a projector into the base of the laptop so it can illuminate a wall behind the screen, a number of interesting applications arise. For example, your laptop could act like Google Glass by allowing you to see through the semi-transparent screen to the real world while augmenting with that world with data on the screen itself. By allowing projected light to pass through the screen completely and illuminate a display, your screen real-estate will grow by a large amount. I’ll leave it to my math-fan readers to compute the effective screen size for various distances between the laptop display and a wall (hint – you only need introductory trigonometry to solve).
This isn’t the first time Microsoft has worked on augmented reality and novel displays, and I see hints here of some of the earlier intellectual heritage of projector camera systems in the works. The patent relies on fundamental concepts of projective geometry that researchers in computer graphics and computer vision explored deeply from the mid-90s until around 2002. At the center of this work was this question: Given a viewer’s position in front of an arbitrarily shaped viewing surface, a given position of a projector, and a scene I want the viewer to perceive – what should the projector render so the scene is perceived without distortion when it hits the surface? How do I paint a display surface to fool you into thinking that an object sits between you and that surface? If I can answer that question at 60hz, you may be fooled into thinking you are immersed in a scene that doesn’t truly exist. I worked on this (and so did a lot smarter folks than me) for quite some time, and ultimately it led to things like immersive caves, holographic/auto-stereoscopic displays, and real-time shader lamps.
This patent seems to pull from these results to draw a scene onto a display surface behind the laptop that appears correct when viewed from the user’s position. The patent hints at holographic display materials being used in the laptop screen to redirect the projected image, probably making the problem easier, and preserving brightness by directing the light to a steeper angle onto the rear display surface.
Microsoft has been exploring exciting end-display technologies for a long time with larger scale displays like the Illumiroom project. This invention seems to be focusing on the personal devices we all carry us. Maybe it will be successful with perhaps a gentler introduction to augmented reality devices than competing form factors like Google Glass. Augmented reality from my laptop at least allows me to carry a device that is part of social convention. I won’t have to worry about being labeled as a Google “Glasshole” if I walk into a meeting with a laptop – unless of course my laptop display sprays participants and not a wall.