This month at the Computer Human Interfaces (CHI) conference, there were a lot of Augmented Reality interfaces demonstrated. (CHI is one of my favorite conferences because it is pretty open to thinking about out-there interactive systems.)
One project caught my eye, “Drill Sergeant” by UC Berkeley students Michelle Nguyen, Eldon Schoop, Mitchell Karchemsky, Valkyrie Savage, Björn Hartmann and Sean Follmer [2], The system is a whole set of augmented hand tools. Ordinary commercial hand tools are “augmented” with sensors, displays (i.e., touchscreens) and wireless networking. The augmentation enables continuous kibitzing by computer systems (or, potentially, the entire internet, right?).
This is a really clever project. But it raises a lot of interesting questions, far beyond what could be covered in one short study.
The augmentation does things like help you drill holes in the right place and right depth, make the right cuts with a saw, and so on. The computer can also help interpret and translate digitized plans, print out paper templates, and potentially help in planning. The system has “macros” of sequences of operations, and has options for “mastery” building repetitive exercises.
This is all reasonably clever, but what is it for? And is it useful?
As far as I can tell, this system is targeted at novices, people who might benefit from a kibitzing expert. As such, it must be compared to other forms of instructions and tutorials. The authors cite these types of materials as the precursors and inspiration. The question is, is this method better, or even as good as, other forms of teaching?
The paper reports some very brief trials with novices. These trials showed little advantage to the augmentation. (It is important to recognize that CHI traditionally requires “user evaluation” for most submissions, though many are the barest bones possible. This study was not a real effort to evaluate the system.)
My own view is that this idea is highly problematic.
First of all, I don’t want to harp on the aesthetics, but it is important to ask whether adding in these displays and “augmentations” to the otherwise simple and hands-on practices of wood working is an asset? Woodworking is fun because you touch the wood, see the work in process, smell the work happening. Adding in a bunch of “augmentations” to suck away attention is, to my mind, detracting from the pleasure of the game.
More seriously, we should consider whether interposing this digital “expertise” between the human and the work is beneficial or harmful. Nguyen and colleagues make a case for this expert help as a benefit for learning how to use the tools, and to make better use of them. Well, maybe. But what do these novices learn? They don’t learn how to use the tools, they learn how to use these augmented tools. Faced with unaugmented tools, how well would they do? My guess is, no better that if they had never had any “help” at all, and probably worse. This is a well know problem with augmentation (e.g., see Carr [1]).
Third, I have to say that the alleged help is questionable. While the implementation is certainly clever—for example, it takes pretty neat programming to monitor and guide the hand drill—the actual benefit to the user is minimal. OK, the computer assistance can help you get very precise results, but so what? It’s just a bird house, for goodness sakes!
Worse, the augmentation is on screens and projections that draw the eye and attention away from the tool and the material. Not only is the computer substituting it’s knowledge for yours, using the augmentation precludes actually performing the work in the same way, or learning the tools and processes themselves.
This same comment applies even more to their higher level assistance, the “macros” for multiple steps of construction. These strategic combinations of “moves” certainly require expertise, but they are also one of the things that people may do their own way. Encapsulating the steps into a one “optimal” recipe eliminates not only the fun of figuring it out, but the variability and individual style of different craftspeople.
Nguyen’s team experimented with “receding feedback” in their “skill building” exercises, gradually reducing the continuous feedback. Their limited tests did not explore how well the training might transfer, how long benefits might last, or indeed, what was actually learned. And they could not really examine the effects of their “macros” in the limited study they were able do.
This is a serious problem, because there is good reason to suspect that novices are learning some complex “knowledge” about the augmented system. What we would consider traditional hand wood working is in that knowledge somewhere, but all mixed up with screens, numbers, and feedback from a very complex software system. There is a lot more to learn about this augmented system!
Remember, these are supposed to be hand tools, used to construct things with your own hands. They are not supposed to be machine tools with a human pilot/passengers. I would say that however successful “Drill Sergeant” might be, it isn’t really the same thing as actually knowing how to use hand tools. And once you really learn the tools and materials, I suspect you might well quickly stop using the expert system.
As I said, this is nice work, and a really clever project. But it raises a lot of interesting questions, far beyond what could be covered in one short study. That makes it an even better project, no?
- Nicholas Car, The Glass Cage: Automation and Us, New York, W. W. Norton’s Company, 2014.
- Michelle Nguyen, Eldon Schoop, Mitchell Karchemsky, Valkyrie Savage, Björn Hartmann, and Sean Follmer, Drill Sergeant: Supporting Physical Construction Projects through an Ecosystem of Augmented Tools, EECS Department, University of California, Berkeley, 2016.
Robert McGrath's Blog 