Part III. ResortwARe? pARadisewARe?: A Modest Proposal

What started me on this essay was a vacation in Hawaii, in which I realized that everything I’ve been talking about for museums, schools, etc. could also be done in the context of a resort such as the Hilton Waikoloa.  Let me flesh out my thoughts for you.

Why would you even consider it?

But first, I have to consider why you would want to do this. Do you really want to fill up a resort with what might well be considered “reality spam”?  Do you want people paying attention to their mobile device instead of playing in the sun?  Can or will people enjoy these “augmentations”?

Obviously, if you do deploy something like this, I would advise some empirical testing to gauge the reception.

But I think it is worth considering for several reasons.

First of all, let me be clear: above all, we want to augment the experience, not turn it into work, school, or sitting indoors playing video games. Hawaii is great, you should focus on being there.  Put away your mobile device and enjoy it!  But I think AR, by its very nature, is unobtrusive unless you want to see it.

Second, at the resort I noted that essentially everyone is carrying smartphones, and many have tablets in their rooms or with them at the pool.  I saw girls wearing nothing but a tiny bikini—using their smartphone. So, it seems pretty clear not going to disrupt what has become normal behavior, if we were to augment the resort with AR content.  Furthermore, I would expect that many of the guests, who come from around the world, will soon expect AR content.

Third, the resort already has tons of “content”, delivered by signage, TV, recorded sound, print, web, and well-trained staff.  The Dolphin Quest experience features hands on stuff, videos, and other media—not to mention actual marine mammals that you can play with!  Very cool, though out of my usual price range.

What occurred to me was a bunch of ways you could deploy AR around the resort that would unobtrusively augment the experience, and possibly create new experiences which I’m confident the resort can find a way to monetize.

Let me walk through them in words.

Ideas for AR in a Resort


The resort is covered with many signs in several languages (reflecting the wonderful international flavor of the visitors). Besides the usual functional signs, there is a wealth of interpretive material. The resort has a huge art collection throughout the buildings and grounds, and there is even a small ethnographic collection. In fact, the resort refer to a “museum” walk.  Similarly, the grounds contain many interesting plants and animals, including endangered Nene’s and Sea Turtles.

Many of the objects are labeled with signs, which have interpretive material just as in a museum.  From time to time, the resort features tours of the grounds, visiting these collections. In other words, this aspect of the resort is very much the same as many museums.  Clearly, the same concepts we have dreamed up for museums can be applied here.

As in the museum, any and all of this material could be augmented with pop up videos, audios and text.  This digital material would be unobtrusive, it would be visible only if you point your mobile device at the sign or the object itself.

The augmented content could be a simple as text or web pages that pop up on the mobile device, or it could be an audio or video track that plays only when you have the device pointed at the sign or object.  These materials might be very similar to existing signage, though they could be customized for the visitor’s preferred language, or in a assistive format (e.g., audio for those with limited vision, age-appropriate material for younger people, etc.).  Also, as I will discuss below, the material can consider the visitors history and context, can incorporate digital coupons and other extra inputs, and potentially can be stored on the visitor’s device as a souvenir.

The content can also be 3D, as in several of the examples above.  So, a sign explaining an art object can pop up a digital rendition of it, which the person can view from many angles.  Or, a sign pointing to one of the venues could pop up a 3D rendition of the attraction. These 3D scenes can have animation and audio, as well.

It is important to emphasize that these augmentations are unobtrusive.  All the signage works just the same as now if you don’t have a device.  But if you do have a device (most guests do) and choose to use it, you would get additional, cool content, popped up—just for you.

Magic Books

Many of the examples of AR are variations on the concept of the “magic book”—2D texts and images augmented with 3D pop ups, such as the ACCESS magazine discussed above.  The resort has many opportunities to create such augmented texts. Each room has a book of information, and there are numerous flyers and posters that advertise attractions and activities. These texts could be augmented with audio, video, and 3D popups, visible with a mobile device.

For example, when you point your device at a list of restaurants, you could see a 3D view of the interior of the restaurant, and/or items from the menu.  A description of an attraction (e.g., a water slide) could pop up video or 3D on your mobile. The app should recognize what language the document is written in, and provide content in that language.

Again, the augmentation is unobtrusive. These items still work as before, if you do not choose to point your mobile device at them.

Interactive Content

Because the application runs on a mobile device, this technology also enables customized interactive experience, set in the real world of the resort.  There are a number of ways an AR app can be designed to customize the presentation, to track the user’s activity and context, and to detect aspects of the user’s activity. In addition, the app can directly sell products, both through electronic “coupons” and through purchase of the digital content itself. Such an app is much more complex to create, but has the advantage of creating a much more interactive experience that can extend beyond the visitor’s stay.

For example, let’s imagine an app that lets you walk through the resort, pointing your mobile device at various signs or objects. At each location, the guest gets to see special content, only available at that location (i.e., not available on the web or in your room). This content might be customized to the individual, possibly even knowing his or her name, home town, etc.. Furthermore, the app keeps track of his or her path and progress on the walk, and customizes the content.  For example, the app might lead you on your walk, by issuing instructions for where to go next to get to your destination.

But this can do much more.  The app could be made into a game which might deliver  an orientation tour, or it might be a treasure hunt.  Perhaps, when you have “viewed” a certain set of items, the app would deliver something extra, e.g., a cool 3D video or something.  This content is something that you can only get by following the trail, or “collecting” the right set of objects that you have viewed.  The app might award coupons loaded to your mobile when you have complete certain “tasks”. Or, the app might trigger other events, such as a phone call with a personal welcome from live resort staff.

The critical idea here is:

  1. The visitor is invited to move around, observe, and enjoy the resort
  2. The augmented content is customized, to make the experience more enjoyable
  3. At certain points, the visitor receives bonus stuff, only available so someone who has done the “work” to get it.

The general goal is to create enjoyable experiences that also educate, advertise, and otherwise lead to fully using the resort.

There are even more possibilities. Augmented Reality content can be triggered not when a person points at a location, but when he or she presents a token or tokens at that location.  In this case, the person must present a “coupon” at the location, and the app detects it and the signage, and only then delivers the cool content.

The important concept here is that the visitor must bring something, and also do something, in order to discover the secret content.  For instance, the poster about sea turtles might tell a generic story, but if you hold your “turtle lover’s card” up next to the poster, you get a special animated greeting from a 3D cartoon of a sea turtle.

How do you get these tokens?  The token can be something like a trading card, or a sticker, or a flyer.  These might be acquired at the resort, e.g., as part of a purchased package.  Or they might be emailed to the visitor in advance.  In the latter case, the email gives instructions for what to do, but does not provide the special content—you have to wait until you get there!

This concept can be more elaborate.  The app might require more than one token be present, requiring people to “play together”.  For example, the special content might happen only when both you and your baby sister both hold your tokens in the right place.  This concept could make the app more social and sociable.

But wait, there’s more. A “magic mirror” application might be deployed to, say, show the visitor wearing a lei and flowers in her hair.  Or wearing a divemask and snorkel, suggesting an activity.  Or, for that matter, the visitor might see him or herself the head of a turtle. This app could be a diversion, e.g., in a lobby, or could be in a retail setting where item are sold.

Beyond the Resort

As suggested above, AR technology could invite and attract the visitor to learn about, enjoy, and consume the attractions while at the resort. But the AR can also be used to get guests to anticipate the visit in advance, and to take it home with them when they leave.

As discussed above, some of the content can be triggered by tokens, such as coupons or trading cards which fully “work” only when you get to the resort. The tokens could be personalized coupons keyed to purchased packages, and could be attached to instructions for family or group games.

The idea here is to send promotional material that describes the resort, and includes tokens with instructions for how to use them. Parents might read through appropriate materials with their children, so the kids can anticipate the experience. Then, when they arrive, they can try the “game”, to get the special content.

The tokens might be sent on paper, or digitally, e.g., through email.  The tokens can include instructions for how to load and use the AR application—getting the visitor ready for the resort experience.  The tokens can even deliver AR in the visitors home, though it should be limited content designed to whet the appetite, for the coming meal.

Finally, the AR content itself could be a product.  Visitors might be allowed to keep some of the pop up content, or purchase it. For example, the special “happy birthday” from the 3D animated porpoise can be saved to the device, so the birthday kid can show his friends at school.  A visitor might be allowed to purchase a digital version of an art object or rare plant as a souvenir.  You might be able to email a 3D postcard to your friend.

The whole idea here is to have the resort experience reach out to pull in visitors (and guide their introduction), to create immersive social experiences, and to take home additional, special souvenirs to remember and tell about their visit.

How to Do It

Everything I describe here can be done with current technology. In fact, assuming that guests bring their own mobile devices, developing the apps is relative straightforward, and would require no major technical investment. There are several commercial frameworks which could be used, and design studios who would provide the expertise.

The biggest tasks are the creation of the digital content, and the interaction.  The whole idea is to provide content that is fun and closely tied to the actual resort environment and activities. Also, the content should be customizable, so it should work in several languages and for several age groups.

If I were to tackle a project like this, I’d need to set up a small studio that includes a few programmers and digital artists and interaction designers. It would have to be created in close consultation with the resort, to mesh with their own design and marketing.  I imagine a few simple things could be done quickly, and more could be added incrementally.

It’s a big project, but very doable.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

A personal blog.

%d bloggers like this: