Augmented Reality Food Explainer App

From the Ideo and MIT CoLab, Pickl, a concept app to improve food shopping, including some Augmented Reality features. (CoLab is really into blockchain things these days, but this particular project doesn’t foreground that technology.) The AR caught my eye, though it turns out to be an extremely insignificant part of the app.

The statement of the problem gives us a good idea of what they are trying to do. “Finding food that is good for you and the planet” is “complicated” and takes hard work if it is even possible. Worse, the “food shopping experience” is highly “manipulated”, and we can’t trust food labels.

The idea is to build an app to help “navigate” this difficult problem as easily as possible.

Pickl, the App

The basic features are a profile of your preferences (which apparently might include stereotyped guesses based on, say, your political party affiliation ?!), and a plethora of data sources about food in the store. The app does various simple filtering and matching, to point you to the choices that fit your inferred preferences.

The AR component is basically overlaying information on the physical store. This might point you to the items on your list, or give you rankings of the items on display.

It turns a quiet walk to the market into an information intensive tooth-grinder, working hard to “optimize” which apple you buy.

Where does the data all come from?

This is a concept app, so it is mainly to teach us what we might want an app to do. While the app itself is not especially brilliant, it does clearly show one of the big pieces missing today: where would you get all that cool information?

First of all, much depends on their user profile. The initial concept is to collect some self report data, and, we assume, use some statistical models to infer food preferences from the personal profile. I note that this could well be extended to use other information, such as social media profiles, and even data from social media. It might even make inferences, e.g., from recent social history, connections to other people, and your ratings of restaurants. It should also know about the household—who is eating and cooking together.  There might be other semantics baked in, such as geographic or cultural specifics.

The filtering and matching algorithms are pretty simple, and there is lots that could be done to make them snazzier—providing we have a rich sea of information at our fingertips, information that is tied to the specific food on offer, and information that is valid and trusted. If we had all that, we could build any number of interesting apps.

But where do we get all that info? This is a big problem, not least because there are competing interests here. Suppliers want to sell as much as possible, and are motivated to “manipulate” buyers to buy. Why would they provide all this information, so you could decide to not buy their products? And why would we trust what they report, anyway?

I note that this is the problem that provenance.org is tackling, and they are using blockchain technology.

Furthermore, some of the fantasies are truly expansive. John Brownlee comments in FastCo,

When scanned by Pickl, you could learn anything you wanted to know about that fruit. Not just its nutrients, its type, or how many calories it is, but how much energy it took to grow it, the path it took to get to your supermarket, how much CO2 it is responsible for, and even what its specialties are: for example, if it’s a better apple for baking than juicing.

Socioeconmic critiques

I found the problem statement at least a little problematic, because it is so clearly tackling rich people’s problems. More precisely, it is tackling the problems of young, trendy design students.

The scenarios involve designing clever menus and then shopping for ingredients. This isn’t the way everyone does it. The examples are people shopping and cooking for themselves, i.e., the app doesn’t have to consider anyone else’s preferences, just the one user.

I think the app could well deal with variations, e.g., a weekly shopping trip, and planning menus for a family. But it is clear that the designers haven’t considered the issue in this first concept.

The whole enterprise assumes that food is abundant and you have a choice of multiple sources. Furthermore, it assumes that you care about meta issues, such as “finding food that is good for you and the planet”.

Not everyone has the luxury of choice, and not everybody thinks so philosophically about their chow.  Some people don’t care about dinner, so long as it is fast and easy.

Weak Political Analysis

The political agenda is clearly stated, if not carefully analyzed. The problems include “manipulation” and trust, and also a preferences for certain sources and practices, e.g., local and sustainable stuff. The app is designed to solve these issues by putting information in the hands of customers, and honoring preferences of the customers (rather than the corporate suppliers).

The hope is that Pickl will put pressure on vendors and supply chains to source foods more responsibly and make ingredients more transparent.

Furthermore, all this information has to be nearly free (what good would it do if there is a steep subscription fee?), but it would certainly cost quite a bit to maintain an up to the minute, extremely accurate database. Actually, probably many different databases.

In other words, I don’t immediately understand the business model here.

How will this happen?

I guess the logic is that if people have apps like this, then vendors will want to be visible in the app. I.e., if the app sees something that has no data available, it may black it out or mark it as undesirable. (I.e., the app will “manipulate” the shopper’s experience.)

Will this work? Can consumers “manipulate” the behavior of the food suppliers, instead of the other way around? Who knows.

I think it is safe to assume that there will be data sources and apps created by suppliers and retailers which will “manipulate” the experience to their own financial advantage. Can a consumer-oriented app compete successfully? Again, no on knows.

A Useful Concept Piece

This app is a useful concept piece, making the ideas concrete enough to easily imagine how you would want it to really work.

Sure, I was able to poke lots of holes in it—that’s kind of what it is for, no?

And most of my criticisms are things that definitely could be handled with some additional design.

The huge, huge missing piece is trustworthy supply chain data. This is why things like provenance.org <<link>> and other projects at MIT and elsewhere are so important. If we get that working better, than we can create as many apps as we want, and let natural selection kick in.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s