We told You This Would Happen

Today’s “smart home” technology has been cooking for quite a while.  We were working on it back before the turn of the century. 

There are plenty of challenging problems, but one of the most difficult is what we called “semantic interoperability” [2].

Or, as Julie Jammot puts it, “The oven won’t talk to the fridge” [1]. 

This was a completely predictable problem, and we thought hard about it as we were booting up the first tries at “smart rooms”.  Our academic solution, circa 2003, was deep and elegant: using semantic web technology and AI to reason about “ontologies” describing devices (e.g., [2-4], TLDR).  (FWIW, this is an elegant solution because the reasoning can involve anything, including everything in the environment, including people, tasks, and policies, etc..)

The problem with deep and elegant solutions, especially when they are 10 years ahead of their time, is that no one knows or cares how cool they are.  Eventually, companies deploy their stuff, and pretty soon they rediscover the problems we thought about, and reinvent solutions, usually nowhere near as deep and elegant.

That’s the academic research biz. We’re used to it. It’s called “a purely academic solution”. Sigh.

Anyway, companies have been deploying so-called “smart home” stuff for years now, and—surprise!—users are discovering interoperability problems. 

Solution number one was to try to grab a monopoly.  Everything works great as long as you only use Amazon, or Google, or Apple or Samsun or….  In this case, world domination was not achieved.

Solution number two was to create an interoperability standard so other companies can talk to your hub.  As we joked back in the 80s (and it was an old joke then), “We love standards.  That’s why we have so many of them.”  This is basically an effort for world domination by other means, and it failed. We have many “standards”, and nobody uses them.

So, by now, perhaps in desperation, solution number three is to create a real, neutral standard, and get everybody on board.

The new standard is called “Matter”, and is supported by Connectivity Standards Alliance and has been implemented by all the major companies.  IEEE Spectrum reports that Matter was well represented at CES this year [5].  There has been lots of favorable chatter in the blogosphere.   This could work. 

---

Glancing at what Matter is, we see that it’s not particularly new.  There have been similar standards before.  The big new thing is the buy-in from multiple companies, and the commitment to neutrality.

What it does is pretty much the same as what our stuff did circa 1999.  I mean, this isn’t really that tricky.  Conceptually, there is a flexible notion of common types of device (e.g., “lightbulb” or “thermostat”), with an abstract interface that can wrap specific instances.  This is implemented by a pretty simple “hub” which is basically a translator unit.

How does this compare to our deep and elegant and literally academic ,solution?  Basically, we were talking about one more level of abstraction, working with metadata descriptions of abstract and specific devices.  The API essentially interpreted these descriptions rather than hard coded them.  Using semantic web stuff meant that we could reason about the descriptions, to work out that this thing should be treated like a “thermostat” or a “lightbulb” or whatever, and then load the code for that type of device.

This approach meant that we could load totally new classes of device on the fly, which I’m pretty sure Matter can’t.  We also could deal with messy cases, such as a “smart lightbulb” that is also a “smart thermostat”.

---

So, is Matter really any good?  

It’s not deep and elegant like our stuff was.  But it has the right sociotechnical oomph to become ubiquitous.  And ubiquitous is literally the name of the game (we named it “ubiquitous computing” for a reason).

So, yeah.  Deployment is certainly nine tenths of the law in this case. (Reliability and security are the last tenth, and we’ll have to see how Matter really works.)

If Matter succeeds, we can slip more elegant solutions in the back door.  I can easily imagine a fancy AI system that essentially generates Matter code and protocol as one of its options.  Elegant and deep and ubiquitous.

So—well done, Matter.  Well…done well enough to solve the problem for now.

---

1. Julie Jammot, The oven won’t talk to the fridge: ‘smart’ homes struggle, in Agence France-Presse, January 6, 2023. https://www.yahoo.com/now/oven-wont-talk-fridge-smart-172031064.html
2. Robert E. McGrath.”Semantic Infrastructure for a Ubiquitous Computing Environment.” Ph. D. Dissertation, Computer Science, University of Illinois, Urbana-Champaign, 2005. http://hdl.handle.net/2142/11057
3. Robert E. McGrath, Anand Ranganathan, Roy H. Campbell, and M. Dennis Mickunas, Incorporating “Semantic Discovery” into Ubiquitous Computing Environments, in Ubisys 2003. 2003: Seattle.
4. Robert E. McGrath, Anand Ranganathan, M. Dennis Mickunas, and Roy H. Campbell, Investigations of Semantic Interoperability in Ubiquitous Computing Environments, in International Conference on Parallel and Distributed Computing Systems. 2003: Marina Del Rey.
5. Matthew S. Smith, The Best Tech of CES 2023, in IEEE Spectrum – Consumer Electronics, January 9, 2023. https://spectrum.ieee.org/ces-2023-best-tech

WaggleNet: IoT Sensing for Beehives

Yet more Bee research from the University of Illinois Urbana-Champaign: WaggleNet.

This spring undergraduate students report a neat project, implementing an ad hoc sensor net to measure the conditions in Bee hives [1].

The students pulled contemporary technology; low cost, low power computers, radios, and sensors, to implement an inexpensive package suitable to drop in to beehives in the field. The datastreams ping pong from one node to another until they find a router and finally reach an internet connected data repository.  The data can be analyzed to monitor the environment and other aspects of the bee environment.

The prese release notes that this is an interesting project for several reasons.  The initial idea is driven by a “customer”, a bee keeper who wants to monitor the bees over winter.  The technology requires solving the whole end-to-end problem which includes not only the electronics, packaging, and networking, but also dealing with the real world of bee hives.

This is an undergraduate project, and nicely done.  It’s fine that it isn’t exactly ground-breaking.  But let me drop some links  to other work they may want to look at.

---

This is a great age of sensornets and “smart farming”.

There probably have been many, many beehive sensing projects (not to mention zillions of agricultural sensing designs. I know of at least one project in Utah that is extremely similar to WaggleNet.

The team expresses a desire to make this available to beekeepers everywhere.

I certainly encourage the effort to make an open source version.  I’ll note the “open source hardware” movement as one place to publish it (e.g., see this, this, this, this, this, this, as well as things like Instructables  which has dozens of DiY bee hives and gazillons of DIY sensor projects).  Publishing the whole thing, hardware, software, instructions will take considerable work.  (Contact me if you want some help organizing all this.)

On another front, I’ll point out that if this is to be really used in the world, it wil probably need to be (re-)built with solid security.   if the data is ever to be trusted the system has to be secure.  For that matter, it is important that the sensors and network cannot be hijacked to spy on people or invade other networks.

I know that this product seems harmless and not worth hacking, but unfortunately, that’s just not good enough.  (If the team has any dreams of commercial products, then they really, really need to make things secure from A to Z.)

---

Again, this is a very nice piece of work.  Making a real produce and/or publishing an open source version will require even more work, and collaborations with additional experts.

---

1. Heather Coit, Students Develop Beekeeping IoT for Renowned Research Lab, in Ilinois Enginering – News. 2018. https://engineering.illinois.edu/engage/wagglenet.html

CES 2017: “Social Bots”

Evan Ackerman has and interesting discussion of the “social robots” appearing at CES this year. These bots are intended for home use, and, like the voice controlled assistants all over CES, are designed to interact with the inhabitants in everyday life.

Readers commented that all the social robots look similar, and they all look a lot like Jibo, which was unveiled in 2014.

“White. Curvy and smooth. Big heads on small bodies. An eye or eyes, but no ears or mouth, and no arms.”

Photos: Mayfield Robotics (Kuri), Jibo (Jibo), Bosch (Mykie), LG (Hub)

So, why do they all look so similar? He talked to the designers about these choices.

For home use, the bot needs to match or at least not clash with the décor, so white or black kind of goes with everything. Chrome and gunmetal, not so much. Curvy is sort of organic and therefore friendly, and also safer if there is a collision. And, he says, round heads are easiest to make, mechanically.

The relatively large heads obviously play on the well known human (and other species) perception of “cuteness”, especially of baby animals. Combined with large eyes, the “face” suggests a cute, harmless pet.

Some of the robots have only one (large) eye, while others have two. (Nobody has thought three or more eyes would be a good idea!) Interestingly, none of them have any other facial features.   These choices reflect efforts to navigate the creepy “uncanny valley”, having enough of a “face” to orient and interact with, but not look too close to human.

Aside from the creep-out factor, the designers are working to contain expectations of the humans. If you unconsciously take the bot for more human than it is, then you may expect it to do more than it can. “At this point, robots that try too hard to seem human can only disappoint.”

Given the minimal functions of these robots (they are scarcely more than mobile versions of the voice “hubs” so prominent at CES), I guess it isn’t too surprising that they might resemble each other. After all, how many ways can you make a minimalist head and face?

Obviously, there are other ways to go, besides commodity minimalism. The “Jimmy” bot, goes a more Pinocchio route, with a full body, as well as a complex back story. Johnson’s approach illustrates the potential value of having a personality and personal story. It’s not just “the bot I bought”, it is “Eric, who lives with me now”.

Personally, I’ve always wanted a personal assistant robot that is a raptor, or maybe a whole flock of them. Velociraptors aren’t “cute” or “friendly”, but they would make a sort of “rotweiller” statement. Grr!. Don’t mess with my people.

Size comparison between the early dinosaur “Eoraptor” and a human

One interesting thing to watch this year is the strikingly different interactions designed into the assistants “hubs” (a la Amazon, Google, et al, and appliances so equipped) and these “social bots”. They do many of the same things, but they have different modes of conversation.

In the case of the “hub”, there is no eye contact. Indeed, they are often portrayed as behind and out of sight. But they are listening. So you just talk to the sir, or to your house. It’s always there, always ready.

The bots are designed to face you, and for you to look in the eye (or eyes).   You aren’t speaking to the air, you are speaking to “eric” or whoever you are looking at. “Eric” might be listening all the time, but you don’t necessarily expect it to.

Clearly, talking to a face is more “natural” (at least in the sense of “old fashioned”), but that doesn’t mean people will like it more. To me, issuing commands to the invisible household servant strikes me as awfully aristocratic (which ain’t my style at all). Or it might also resemble not-especially-humble prayers to some kind of minor secular god.

It would be very interesting to do some field studies of actual families with both social bots and “hubs”. I wonder how this kind of augmented household will actually work and evoldve. Which assistants will be liked best? Will something get turned or or ignored after the shine wears off?

Inquiring minds want to know.

 

Robot Wednesday

The Just Things Foundation Wants to Make Responsible IoT

Last year I endorsed and signed on to the IOT Manifesto, which calls for and aims to articulate “Guidelines for responsible design in a connected world”.

This effort has now spawned the Just Things Foundation (which is an exquisite pun in English), which “builds on these principles and aims to transform them into actionable standards for a broader audience to work with.”

They also dream of “tools for professionals”, and I am very curious to know what they mean by that.

JustThings will be presenting an exhibition at Dutch Design Week Eindhoven this week, they call, “An Internet of Things We Can Be Proud of”.  As the poster says, “Why does my refridgerator need to know my birthday?”

I will be interested to see more about this exihibition.

By coincidence, this week saw an appalling cyber attack which is reported to exploit major design flaws in some of the current generation of IoT devices. Absolutely no one is surprised by this event, nor at the irresponsibility of the companies that sold these awful things that are now crippling our DNS. Thanks, a lot, guys.

This incident is a clear reminder of how much we need better design, much, much better design of IoT devices.

I stand with Just Things, and I urge everyone to support their efforts.

Arieff On “The Internet of Way Too Many Things”

I have expressed many concerns about the Internet of Things in the past few years (along with nearly identical complaints about wearables). Apparently there is a growing consensus that Silicon Valley has gone completely off the rails on this one, creating idiotic and harmful products, moreover, products that no one will actually buy.

Allison Arieff provides a nice summary of this case in the NYT this week. The calls it “The Internet of Way Too Many Things” (TIOWTMT – pronounced “tee-ow-temt”?). She is right on target with her comprehensive analysis.

For one thing, she is offended by poor design, which is rife in this area. She cites an example of Leeo, which is supposed to reinvent the hone smoke alarm. But it turns out to be night light gadget that listens for the sound of your smoke alarm, and calls your smart phone to let you know about it. Not only does this not even replace the old smoke detector, but as Arieff says, “to “improve” a $20 smoke alarm, the designer opted to add a $99 night light and a several-hundred-dollar smartphone.”

As she says, “This is not good design.”

She comments on several other such stupid and badly designed products, including notably 94fifty, “The only connected smart basketball for iOS and Android to help improve shooting and ball handling skills – fast.” (This one is on its way to the Inappropriate Touch Screen Files -fast!)

As Arieff points out, the products are worse that stupid, they are bad for customers (or, not to put a fine point on it, “evil” in the Google sense of the word).

“What the products on display have in common is that they don’t solve problems people actually have. Technology is integrated not because it is necessary, but because the technology exists to integrate it — and because it will enable companies to sell you stuff you never knew you were missing.”

Arieff lays out four key topics that need research and dare I say, good design: “may I make a plea for R&D in four major areas? 1) integration of functions 2) usefulness 3) sustainability and 4) privacy/security.”

As a designer, Arieff is rightly critical, and even offended, by the multiplication of devices which do not work together. This is bad for everyone, and simply cannot work for very long. There are many (bad) reasons for these bad designs, but it has to stop.  Standards and interoperation are difficult, but they are critical to good design.

The IOT seems to bring out the worst in designers, who present us with an amazing array of useless products that solve no problems and no one wants. My own view is that our efforts in the 80s and 90s to make the Internet and computing accessible to all have succeeded to the point of diminishing returns. Just because it is possible, even easy, to build a basketball that sense senor reading to and Internet service does not mean that it is a useful thing to do. As Arieff says, there are plenty of real problems to try to solve.

And one of the big problems is sustainability in many forms. Considering the ugly footprint of mobile devices (which have also have a short life and mostly are not recycled), adding a mobile device to any product is a big hit. Adding a bunch of similar devices throughout the home adds up to a huge footprint, one that is not likely to be compensated by the trivial “efficiencies” they might or might not implement.

Finally, Arieff pulls no punches to complain about the fact that the IOT is basically designed to invade privacy, and generally with rudimentary security. This is not only bad design, it is embarrassingly poor engineering. It is increasingly clear that the combination of business interest in collecting your data and the inbred culture of Silicon Valley have created a mindset that this is just the way things work.

Wrong.   This is the way things come crashing down.

We’re trying to warn you guys.

Go back to the drawing board, get your thumb out, and figure out how to build stuff that is both good for people and solves real problems in new ways.

NPR Covers Appliance Hacking

NPR picked up on the points I discussed yesterday with an interview with someone operating in the business I called for.

The NPR piece is pretty clear, though it gives us no idea what we should do.  Can I avoid these devices when I don’t mean to have them, or is there no choice?  How do I disable the features I don’t want?  How do I know what the devices are doing?