Many of us have been dreaming of tangible interfaces and smart matter for many years now (Ivan Sutherland dreamed of this over 50 years ago ). We are finally beginning to see inklings of real systems. (link link)
This fall Mathieu Le Goc, and colleagues demonstrated their “Zooids: Building Blocks for Swarm User Interfaces”. The technology is a swarm (dozens) of 2.6 cm disks, each of which is a little mobile robot. They are deployed on a table top, where the act as a bunch of tangible pixels, for both input and output. The positions of the swarm are tracked via structured light, and guided by wireless instructions.
It’s pretty cool!
The hardware and software is open source (Prof. McGrath awards extra credit), and includes and interesting editor that is “[i]nspired by traditional stop motion animation tools”. (, p. 101) Cool!
The researchers are interested in design principles for “swarmUIs”. Their paper notes some initial ideas ():
- Number and granularity of particles (Zooids are middling)
- Fixed versus movable elements (and possibly some of each)
- Fixed or variable number of elements
- Interchangeable or identifiable elements
- Direct, indirect, and hybrid manipulation (see video)
- Distinguished roles
- Other visual elements (e.g. projections, etc.)
These are only the beginning, there are lot’s of interesting design questions to explore.
I’d like to build truly interactive tangible interactions, it that makes sense. Try to envision this scenario:
the swarm should dance around, attracting attention and showing me where to “grab” or “push”. When I stick my hand in, elements would swarm around the hand, touching and caressing me, and leading me to one or more possible gestures. (This might well be coordinated with other visual and audio information, of course.) When I make a gesture, the elements can guide the movement, preventing gross error, and tangibly indicating that the system is getting the message.
Another interesting area to explore is multiuser interfaces. Lot’s of interesting things to try here:
- Transmitting messages via “touch” – I push part of he swarm, another part of the swarm pushes on you.
- Cooperative (or competitive) teamwork—complex actions that require the right combination of gestures from two or more people.
- Displays that fade from attention while highlighting attention to the other people
We could also do a lot of interesting things with the swarm integrated into other digital systems, such as Augmented Reality. This would extend the effects of the gestures in many ways.
Finally, I would note that this would make a great game for cats.
Enough for now.
This is a very inspiring project, and I’m so glad that they open sourced it so others can try some of these other crazy idea.
- Mathieu Le, Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Dragicevic, and Sean Follmer, Zooids: Building Blocks for Swarm User Interfaces, in Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 2016, ACM: Tokyo, Japan. p. 97-109. http://dl.acm.org/citation.cfm?doid=2984511.2984547
- Ivan E. Sutherland, The Ultimate Display, in Proceedings of the IFIP Congress. 1965: New York. p. 506–508. http://worrydream.com/refs/Sutherland%20-%20The%20Ultimate%20Display.pdf