Category Archives: Internet of Things

Fribo: culturally specific social robotics?

This spring a research group from Korea report on a home robot that seeks to address social isolation of young adults [2].  Fribo is similar to many other home assistants such as Alexa, but is specifically networked to other Fribos that reside with people in the same social network.  (The network of Fribos overlays the human social network.)

The special feature is that Fribo listens to the activity in the home and certain sounds are transmitted to all the other Fribos.  For example, the sound of the refrigerator door is played to other Fribos, offering a low key cue about the activity of the person.

Actually, it’s a little more elaborate: the Fribo actually narrates the cue.  The sound of the refrigerator is accompanied by a message such as, “Oh, someone just opened the refrigerator door. I wonder which food your friend is going to have”.  ([2], p. 116)

The idea is that, the network of friends– who live alone– gain an awareness of the presence and activity of each other.  It may also encourage more social contact with others.

The “creepy” factor with this product seems obvious to me.  Yoiks. But I know that there is a very dramatic difference in attitudes about creepiness among younger people, so who knows?

There are also significant issues with privacy (how much to you trust the filtering?) and security (if one Fribo is hacked, the whole network is probably exposed).   I wouldn’t touch it with a barge pole, myself.

But the field study reported is very interesting for another reason.  First, the fact that people were even willing to try this device indicates an interest in this kind of social awareness.  In particular, there seems to be an implicit sense of belonging and trust in a group of peers.  Not only that, but the participants seem to share similar concerns about the isolation of living alone, and the idea that these kind of cues are a way of feeling connected.  The study also suggests that being aware of others stimulates more contact, such as phone calls.

I have to say that the reports of the users experiences don’t resonate with my own experience.  Aside from the obvious digital-nativism of the young users, there seems to be a definitely cultural factor, i.e., young adults in Korea.  There is a level of mutual trust and solidarity among the users that I’m not sure is universal.  If so, then Fribo might be a hit in Korea, but a flop in the US, for instance.

By the way, the users refer to how quite their one-person apartment is.  My own experience is that even living alone there is plenty of noise from neighbors, for better or worse.  If anything, there is probably way to much awareness of strangers in most living spaces.  Deliberately adding in awareness of your friends might or might not be an attractive feature, depending on just how much other “awareness” there is.

If my speculation is correct, then this is an interesting example of using ubiquitous digital technology in a culturally specific manner.   As the researchers suggest, it would be very interesting to test this hypothesis by replicating the study in other places in the world.

Finally, I have to point out that if what you want to do is achieve a sense of joint living, it is always possible to live together.

A group house or dormitory could provide awareness of others, as well as even easier opportunities to socialize.  Why not explore alternative living arrangements, rather than install intrusive digital systems in isolated units?  This would make another interesting comparison condition for future studies.


  1. Evan Ackerman, Fribo: A Robot for People Who Live Alone, in IEEE Spectrum – Home Robotics. 2018. https://spectrum.ieee.org/automaton/robotics/home-robots/fribo-a-robot-for-people-who-live-alone
  2. Kwangmin Jeong, Jihyun Sung, Hae-Sung Lee, Aram Kim, Hyemi Kim, Chanmi Park, Yuin Jeong, JeeHang Lee, and iJnwoo Kim, Fribo: A Social Networking Robot for Increasing Social Connectedness through Sharing Daily Home Activities from Living Noise Data, in Tthe 2018 ACM/IEEE International Conference on Human-Robot Interaction. 2018: Chicago. p. 114-122. https://dl.acm.org/citation.cfm?id=3171254

Shock Report: IOT Home Devices Are Not Secure

I’ve complained about the bad design of consumer “IOT”—“let’s slap a touchscreen on ‘X’ and connect it to the Internet”—and noted grievous security and privacy issues.

As far as I’m concerned, these beasts are a Bad Idea ™ even when they are working as intended.

But, of course, the odds on them working correctly is vanishingly small.


This winter, a team of researchers at Ben-Gurion University of the Negev report on studies of consumer IOT devices including baby monitors, door bells, and a thermostat [1]. The paper sketches their cookbook of methods to analyze, break into, and take over these and similar devices.  This is a great tutorial on just how vulnerable your device really is!

They start by opening the cover, which is usually possible without disabling the device.  Visual inspection reveals the components which have easy to read identification printed on them, of course.  The hackers can easily identify the juicy memory modules and gaping access ports to go after.

The crucial step is, as they say, “Extraction of Firmware and Data”.  The paper makes clear just how naked a computer is when the cover is off and it is in the enemy’s hands. Simple techniques sufficed to capture passwords and take control of the system.  Ultimately, one way or another, they were able to extract the firmware for analysis.  The game is pretty much over at this point.

It is interesting that the widespread use of Linux versions in IOT means that a captured firmware image can be analyzed with standard Unix utilities which have been around since I was a young programmer.  For example, IOT devices which use Linux password protection can be conveniently analyzed on any Linux system, because the libraries are the same.  Of course, brute force cracking can be slowed by good passwords, but they report relatively easy success with these IOT devices. Beside breaking passwords, analysis of the firmware can also reveal any number of other vulnerabilities.

Overall, they were able to break passwords in a number of devices (and found one null password—oops!).  In some cases, they found open network access ports (telnet and FTP), or were able to set them up (because they had root access).  Yoiks.  They found WiFi credentials, they found hard coded private keys. Double Yoiks!

They also found strong indications that some of the devices were rebranded, i.e., built on the same components as other devices.  In this case, the shared hardware and software also shared passwords and vulnerabilities as well.  In any case, there are many replicates of each type of IOT device, so breaking in to one potentially compromises whole swathes of the network.

The authors make some completely straightforward recommendations for producers. Disable the portsUse good passwordsEncrypt as much data as feasible!

Users have little they can do except don’t use these things!


The authors note that economic forces press manufacturers to create cheap products, cutting corners on security. It is also true that many designers have inadequate understanding of security issues.

I’ll add that these problems don’t stem from devices that are too dumb.  Quite the contrary. The problem is that very capable processors with a full Linux operating system are cheap and ubiquitous.  The success of these technologies has a perverse effect on design, lifting constraints an allowing complex and insecure systems to be built.  These devices are far too capable for their intended use.

It is also interesting to note that these ubiquitous and mostly open source technologies are so widely used they are becoming a sort of technical monoculture. The same vulnerabilities are found in many different products because they are all built from a very limited “gene pool” of cheap technology.

Open source development is often put forward as a good approach to creating trustworthy and secure software.  The more eyes, the better the software.  And open software can’t have secret back doors or other shenanigans in them.

But, as we see in IOT, when relying on open source software also means that everyone is using the same software, there is the risk that serious bugs will be ubiquitous throughout a software monoculture.

Overall, it’s not a comforting picture.  And there isn’t much a consumer can do, other than Turn It Off.


  1. Omer Shwartz, Yael Mathov, Michael Bohadana, Yuval Elovici, and Yossi Oren. Opening Pandora’s Box: Effective Techniques for Reverse Engineering IoT Devices. In Smart Card Research and Advanced Applications, 2018, 1-21. https://link.springer.com/chapter/10.1007%2F978-3-319-75208-2_1

 

Fu and Xu on the Insecurity of Sensors

In past posts, I have complained about the poor design and insecurity the “Internet of Too Many Things”.

This month Kevin Fu and Wenyuan Xu give me yet more sphincter tightening reasons to worry about these systems: transduction attacks [1].

Much of the IoT is built of small devices equipped with sensors of many kinds.  These clever little devices are transducers, converting physical phenomena (light, vibration, head, motion, etc.) into electrical signals and ultimately digital numbers.  Anyone who has explored sensor technology knows how awesome this magic can be.

Any input device can be attacked with bogus input, and the output of any module can be subverted by intercepting or messing with its output. Conventional security measures are designed to protect the integrity of inputs and outputs (e.g., with cryptography and error detection).

But physical sensors are also face a third kind of attack, a transduction attack which “exploits a vulnerability in the physics of a sensor to manipulate its output or induce intentional errors”. ([1], p. 20) Yoiks!   If I can’t trust my sensors, the whole system is in peril.

F&X say this is a huge problem because most commercial systems to date are not designed with this threat in mind.

Billions of deployed sensors lack designed-in protections against intentional physical manipulation” ([1], p. 20)

For instance, most consumer-grade voice recognition is vulnerable to DolphinAttack: ultrasound signals that fool the system into executing commands unheard by the humans. This is possible because the microphone can detect ultrasound (which it arguably doesn’t need to) and the algorithms can be fooled.

Hacking my phone is annoying enough, but this is far more dangerous when it is an airliner, autonomous car, or a hospital being hacked this way.  Yoiks!

Protecting against transduction attacks is difficult because the consequences arise as software symptoms, but the risks begin in the physics of hardware.” ([1], p. 21)

What can be done?

Obviously, relying on software testing isn’t sufficient, nor is reliance on tests of individual components.  The overall system needs to be designed to be able to check outputs of sensors, with the assumption that they may be untrustworthy. (This will be painful and add cost and complexity.)

F&X note that many of these attacks exploit the resonance of physically connected components.  E.g., attacks on motion sensors via sounds sent to the speaker works because the components are mechanically coupled so the vibration of the speaker can fiddle the accelerometer.

They note a recent CERT advisory that recommended ways to mount the component in order to minimize physical coupling and control vibrations in critical ranges (i.e., the resonant frequencies of nearby components).

F&X also note that this problem calls for broader, interdisciplinary design expertise. Computer Scientists don’t necessarily know the physics and mechanical engineering needed, and vice versa.  This teamwork is particularly important for systems such as autonomous vehicles, which need to be trustworthy.

F&X refer to this as “back to basics” for the Computer Science curriculum. It is so easy to focus on all the power of software, and simply assume that the hardware black-box just “works”.

students may seek comfort hiding behind a beautiful Java facade rather than facing the ugly limitations of computing machinery.” (p. 23)


  1. Kevin Fu and Wenyuan Xu. 2018. “Risks of trusting the physics of sensors.” Communications of the ACM 61 (2):20-23. https://dl.acm.org/citation.cfm?id=3176402

IoT Isn’t Even Close To Trustworthy

I have already beefed about the current version of the Internet of Things, which is of dubious value and badly engineered, to boot. (Here, here, here, here, here, here)

The most visible face of these developments are the network connected home “Assistants”, such as Alexa, Siri, Google Home, and so on. Aside from the extremely questionable rationale (Why do I need a voice interface to my refrigerator? Why do I need my refrigerator to connect to the entire freaking Internet?) there are famous cases that illustrate that these beasts are deeply invasive.

Last fall, Hyunji Chung and colleages at US National Institute for Standards and Technology (NIST) wrote about the trustworthiness of these systems.

[S]uch interactions should be solely between you and the device assisting you. But are they? How do you know for sure?” (p. 100)

These are complicated, network connected systems which are not trivial to understand and evaluate. But they are in our homes, so everyone needs to know just how far to trust them.

The researchers sketch the “ecosystem” of network connected components and services. The very fact that they are complex enough to warrant the term, “ecosystem”, is the fundamental problem.

[W]e performed cloud-native artifact analysis, packet analysis, voice-command tests, application analysis, and firmware analysis” (p. 101)

Uh, oh. Does anyone besides me see a problem with deploying such a system unsupervised in private homes?

The threat envelope is huge. The basic logic of the assitant is implemented mainly in “the cloud”, with components on local devices that communicate with the cloud. Many assistants have third party apps as well. They report that the Alexa “Skill Store” has 10,000 such voice-actuated apps.

The point of the analysis is, of course, risk assessment. They identify many, many risks—basically, everything that might threaten the Internet.

  • Wiretapping
  • Compromises devices
  • Malicious voice commands
  • Eavesdropping

Wireless communication is, of course, a weakness. The researchers report the appalling fact that not all the communications are encrypted. Even when encrypted, traffic sniffing can still reveal considerable information about the devices and users.

Obviously, devices may be hacked. In this case, there is no expert IT department to defend the network, detect intrusions, or patch bugs. One has to think that home devices are relatively defenseless, and certain to be cracked over time.

One reason I don’t like voice commands is that they are hard to secure. Even the best voice recognition systems are vulnerable to mistakes, and low-cost, consumer-maintained systems probably aren’t top of the line. (And who wants your Alexa to reject commands because it isn’t certain that you are really you.)

And, of course, every link is a potential channel for someone to listen in on your life.

This article makes clear that these systems have a lot of potential issues, even if they are configured correctly and work as designed. Unfortunately, personal and home devices are not likely to be carefully configured or monitored. I have a PhD in computer science and have done my share of sysadmin, and I have not the remotest clue how to set up and keep one of these systems.

These researchers carefully don’t answer the question, “can I trust you?” But it is very clear that the answer is “no”.

I’m afraid that people are taking these devices on faith. They are sold as appliances, and the look like appliances, so they must be as safe as a consumer appliance, right?

Well, no.

This is a really great article, and everyone should read it before turning on any cloud service, let alone installing an “assistant” in their home.

And if you don’t understand what this article says, then you definitely shouldn’t install one of these assistants in your home.


  1. Hyunji Chung, Michaela Iorga, Jeffrey Voas, and Sangjin Lee, “Alexa, Can I Trust You?”. Computer, 50 (9):100-104, 2017. https://www.computer.org/csdl/mags/co/2017/09/mco2017090100-abs.html

CertiKOS – This is what the IOT should be

Now and again I have criticized the software architecture of contemporary consumer electronics, including mobile devices, home automation, and the Internet of Things.

In particular, in response to the never-ending stream of security oopsies, I have said that it is a mistake to use a general-purpose operating system on these extremely exposed and unmanaged devices.  Realistically, there is no way to make complex systems such as Linux, Android OS or iOS secure, and once breached, the whole system is exposed (and worse, can be repurposed).

It is easy to say what not to do, Bob, but how do you do it right?


Secure operating systems are very, very hard to make, and every cunning advance in hardware (multicore, multiple busses, complex memory hierarchies, integrated networking, etc.) makes it all that harder.  Furthermore, creating a provably secure OS has to be fast and cost effective, because there are so many devices coming out.

One good example of what I’m talking about is CertiKOS from Yale.  As in any secure system, the logic is formally verified to help assure that it does what it is supposed to and only what it is supposed to.  Importantly, their system is composable, designed to be able to build up a system from components, and keep it kosher [1].

Phew!  This stuff makes by brain hurt to try to follow it.  But this is what needs to be done.

What I would like to see is to use something like this to build “certified” software for each and every IoT device and home automation gadget.  In particular, I want the IoT device to do only the limited things it is supposed to do, and simply not be able to do anything else.

A “smart light bulb” should only do what a SLB should do. Which, honestly, isn’t much.  Hack into my light bulb?  Even if you can, who cares?  It can’t do anything.

OK, I know this isn’t the whole story.  Securing networks of devices is more complicated, since there will still be problems, such as fiddling with traffic and loads.

It is also true that it will be very challenging to create a device that can load endless third-party apps, like your mobile phone can today.  But, that’s probably a good thing to cut back on, anyway.

This kind of modular device can probably help privacy as well.  How many times do we have to learn about some app that is surreptitiously spying on users.  If a device can’t spy on you, then you don’t need to worry so much about who is subverting your pocket or your living room.

Obviously, this kind of privacy is hard.  A “secure” Alexa (Salexa?) would have components to exactly and  do only what is needed. That would require designing the functions in advance, which would require understanding how people live, and designing to meet those needs

This contrasts to current methodology, which release a powerful, general-purpos platform and loads whatever combination of apps the user buys.  And in many cases, the device really knows very little about what should be done, and surveilles the user to “learn” their behavior and preferences.  This approach both invades privacy (by design) and exposes the user to huge risks if the system is infiltrated by enemies (not by design).


I think the bottom line is that  secure kernels are necessary but not sufficient to make network systems more secure.  Security and privacy are end-to-end problems, and only as good as the weakest link. But making components stronger is a good start.

The cool thing about the Yale work is that they are building tools and processes for building and composing components.  This is a crucial technology for building systems, if not the complete solution.

Nice work.


  1. Ronghui Gu, Zhong Shao, Hao Chen, Xiongnan (Newman) Wu, Jieung Kim, Vilhelm Sjöberg, and David Costanzo, CertiKOS: An Extenisble Architecture for Building Certified Concurrent OS Kernels, in USENIX Symposium on Operating Systems Design and Implementation. 2016. p. 653-669. http://flint.cs.yale.edu/flint/publications/certikos.pdf

3D Printed Objects ‘Connect to Wifi’

Researchers at University of Washington report some interesting work creating simple 3D printed objects that “connect” to wifi without complex electronics [1]. The objects can be used a sensor or input device.

The basic idea is that this technology can be low-cost, ubiquitous, and customized. While this system isn’t quite “without electronics” [2], the objects are simple enough to be fabricated with a 3D printer.  The fancy stuff is in a mobile device and the wireless network—which is assumed to be ubiquitous infrastructure.

The researchers made several examples, including an anemometer, a flow meter (water wheel), and a spring scale.  They also created input devices, a push button, a slider, and a dial knob.  These examples are building blocks for user interaction and for environmental or industrial sensing.

The idea is to use radio back scatter as a simple signaling device. Basically, an antenna is embedded in the printed object, with a switch that changes the reflectivity of the antenna. The 3D objects trigger the switch using clockwork or other mechanical action (e.g., a push button or turbine).  This mechanism can represent a (low bit rate) signal as a series of modulated reflections.

The signal is detected by processing the reception from a wifi or other radio receiver. Filtering out the high frequency signals will detect the low frequency data from the sensor.  This can be done with a mobile device such as a phone of tablet.

The coolest part is the 3D printed clockwork mechanisms used to modulate the transmission. Gears! Springs! Buttons!

The technical paper has details about how well it works, as well as interesting examinations of the interference of human movement (which modulates back scatter).  In order to use more than one such device, the “channel” can be frequency multiplexed by printing devices with different gear ratios!  That’s kind of cool.


  1. Vikram Iyer, Justin Chan, and Shyamnath Gollakota, 3D printing wireless connected objects. ACM Transactions on Graphics, 36 (6):1-13, 2017. http://printedwifi.cs.washington.edu/printedwifi.pdf
  2. Jennifer Langston, In first, 3-D printed objects connect to WiFi without electronics in UW News. 2017. http://www.washington.edu/news/2017/12/05/in-first-3-d-printed-objects-connect-to-wifi-without-electronics/

 

Semantic Web and The Internet of Things

I’m not a gigantic fan of the contemporary Internet of Things (AKA The Internet of Way To Many Things), but this is not because I don’t understand the concept. Quite the reverse actually – I was doing this stuff back at the fin de vingt-et-unième siècle.

I must say that I thought most of that early work was out of date, surpassed by big data, machine learning, and billions of dollars.

This month I was shocked to see not one but two articles about using the Semantic Web for the Internet of Things—exactly what I was talking about back in ought-five [4].

Problem:  the world is filled with more and more autonomous devices which need to connect, configure, and cooperate with each other.

In the IoT, there are no system admins, let alone configuration files, and so on.

So how do nodes know who is who, what is what, and how to talk to anything?

This was the motivating problem at the beginning of my own thesis work (circa 1998-9).

Configuring these devices requires something like scripts, rules, and triggers, but simple scripts

can only apply to known places, users, and IoT devices personal needs.

This lack of discovery and adaptation severely restricts the rules’ expressive power.” ([1], p. 18)

Way back in the last century, I started exploring the use of Semantic Web technology to address this challenge (“Semantic Infrastructure for a Ubiquitous Computing Environment” [4])  I thought it was the right idea, but I haven’t seen much development in the ensuing ages.

Fulvio Corno and colleagues at Politecnico di Torino describe a system that follows this very approach.  Key elements are [1]:

  • An abstract desctiption of devices, fucntions, etc.
  • Logical rules for composing and reusing about these descriptions
  • Standard, cross-vender representation

The purpose is to have a way for computers to know when a rule applies to some ‘node’, and how to apply it, even when the node is a complete stranger.

Corno and colleagues are particularly interested in “programming by functionality”, i.e., making the right thing happen without knowing all the details in advance.

Their system includes a “Semantic Reasoning block” which “maps user-defined trigger-action rules to devices and services in the IoT Ecosystem in order to reproduce the desired behaviors.” (p. 21 )  In other words, it takes an abstract description of what is supposed to happen, and figures out a set of concrete devices and actions that will accomplish the intended goal. (This is “semantic” because the computer is deducing what is “meant” to do.)

The project uses semantic web standards because they (a) are universal and portable and (b) they define a formal logical model for the reasoning that is needed.  The former means that the rules can be vendor independent and will work anywhere, and the latter means that you can do the mapping they need to do.

A second article discusses the more general problem of “Machine to Machine” (M2M) interoperability [2]. Even absent the swarm of IoT things, there is a plethora of computers, systems, and data on the network. Getting systems to work together can be a painful process, and in principle, semantic web technology is designed to ease this process.

However, Hodges and colleagues point out that most semantic models are poorly written and unlikely to be useful for interoperability.

many of these ad hoc ontologies are information silos

This group is working to develop better semantic models based on accepted industry standards.  I.e., the semantic models describe the concepts of existing standards, as well as mapping between multiple standards.

These models have obvious advantages.  Based on accepted standards, they represent existing models, with existing audiences, and known domain applicability. In short, given the effort invested in creating, promulgating, and implementing a standard, it is worth the trouble to create a good semantic model for it.

I would also say that a good standard has a rigorous definition of conformity, so it is possible to check that a node actually implements the standard correctly. That means that it is possible to know that the semantic model can map to one or more concrete implementations.

The big news, though, is that the formal logic of the semantic web makes it possible to formally map between multiple standards, making it possible for a computer to translate.

The general idea here is to use these standards as the basis for the abstract description of the decentralized system.  The semantic web makes it possible to automatically combine and translate between descriptions from many sources.

(Another blast from the past: Hodges et al. suggest using PROV-O ontology to model the behavior of a sensor.  PROV-O is a representation of the Provenance Data Model, which I contributed to at its birth.)

It was nice to see these articles confirming that the technology I called out as important might actually be important.  The motives and general approach are pretty much exactly as I was thinking way back when.  It is great to feel like a vindicated pioneer.

On the other hand, these articles were a pleasant surprise because these technologies are not actually in use yet.  Ten years and more, and it’s still a future wonder.  “They hold the promise of interoperability in name only.” ( p. 27)

I still think this is a good approach, not least because of the principle that:

Any problem in computer science can be solved by an extra level of indirection.”  (The original source of this quote is not precisely known. Lampson et al. [3] attribute this phrase to David Wheeler, citing the authority of Roger Needham.)

I also think that the burgeoning Internet of Too Many Things will make this problem more and more pressing.  However, the IoT is currently dominated by centralized architectures managed by large vendors.  These companies are selling silos, and have little reason to provide open solutions.


  1. Fulvio Corno, Luigi De. Russis, and Alberto Monge Roffarello, A Semantic Web Approach to Simplifying Trigger-Action Programming in the IoT. Computer, 50 (11):18-24, 2017. https://www.computer.org/csdl/mags/co/2017/11/mco2017110018-abs.html
  2. Jack Hodges, Kimberly. García, and Steven Ray, Semantic Development and Integration of Standards for Adoption and Interoperability. Computer, 50 (11):26-36, 2017. https://www.computer.org/csdl/mags/co/2017/11/mco2017110026-abs.html
  3. Butler Lampson, Martin Abadi, Michael Burrows, and Edward Wobber, Authentication in Distributed Systems: Theory and Practice. ACM Transactions of Computer Systems, 10 (4):265-310, 1992. https://www.cs.utexas.edu/~shmat/courses/cs380s/lampson92.pdf
  4. Robert E. McGrath, Semantic Infrastructure for a Ubiquitous Computing Environment, in Computer Science. 2005, University of Illinois, Urbana-Champaign: Urbana. http://hdl.handle.net/2142/11057