Reported this week at CHI, Mistform is “a shape changing fog display that can support one or two users interacting with either 2D or 3D content.” (, p. 4383) Cool!
The basic idea of this kind of display is to generate a “fog” of water droplets in front of the person, and project information from the back. With cleaver geometry, the projection is seen by the eye as 3D objects hanging in mid air. The cool thing is that the user can reach into the fog to touch the objects hanging there.
This version from Yutaka Tokuda and colleagues at University of Sussex, adds the wrinkle that the shape of the fog can be manipulated, to create a curved “screen” . This calls for clever squared geometric computations, to account not only for the fog and the eye, but also for the curvature of the fog. The latter is computed from the position of the pipes that generate the mist.
The projection is, in principle, “mere geometry”. Working from the eye position (via head tracking), the color and brightness of each pixel is computed. Working backwards, the pixel is mapped to a region of the go, and then back to the projector. Voila.
Interacting with the display uses hand tracking with a Kinnect. The fog is segmented into regions that can be touched (“actuators”). This is coordinated with the projected objects, so the user can reach into the fog and “touch” an object in a natural motion.
This is a very nice piece of work indeed. The paper  gives lots of details.
This is a great example of the potential of projective interfaces, which will replace the ubiquitous screen in the coming decade or two. (If you have any doubts, take a gander at this wizardry from some Illinois alums.)
Of course, the mountain we have to climb is to make one big enough and clever enough that we can walk into it. This will also combine with haptics so the objects ‘push back’ when you tough them. Not that will be cool.
- Yutaka Tokuda, Mohd Adili Norasikin, Sriram Subramanian, and Diego Martinez Plasencia, MistForm: Adaptive Shape Changing Fog Screens, in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2017, ACM: Denver, Colorado, USA. p. 4383-4395. http://dl.acm.org/citation.cfm?doid=3025453.3025608
- University of Sussex. MistForm: adaptive shape changing fog screens 2017, http://sro.sussex.ac.uk/67147/.
PS. Wouldn’t “Shape Changing Fog Screen” be a great name for a band?
Or how about, “The Fog and the Eye“.