In-place User Interfaces

My primary motivation for choosing this project was to explore in-place user interfaces. My experience at the Aware Home had taught me that once a sensor joins your smart home, it is pretty much lost in the sea of devices. God help you if you name your devices 'test' and leave it be, because when their battery runs out, you'll need to check every single appliance in the house to see if it is still alive.

We also anticipate a future where Smart Home service providers like Comcast and Cox will install the devices for you, or your new apartment already comes with sensors and actuators installed. In these cases, you don't really know what "Turn on bedroom light" really does, when you have multiple bedrooms with multiple sources of lights in them. Alexa would just respond with "I'm not sure how to help you with that", or "There are multiple devices that share that name, which one did you want?"

The project also was in line with the ongoing research at the Aware Home for location-aware appliances. We drew some constraints and made some assumptions from some of the location aware equipment available in the home, trying to design an interface that will tie in with the system at large, working across different devices in different locations using different protocols.

We conducted a brief survey to understand what users feel about the installation, configuration and interaction processes in their past experience with smart homes. After disseminating this survey on several smart home forums, we received about 103 responses.

Our key insights from the survey were:

• Each smart home device comes with its own app, and it is hard to work with them simultaneously, both due to functionality and UI inconsistency.
• It takes time to install devices, and most times, there is also a steep learning curve.
• It is hard to connect one component to another, if they are made by different manufacturers.
• In 90% of the cases, users do not do much beyond the most basic interactions after the novelty of the new gadget wears off, like turning something on or off.

To better understand the process (mainly, the surprises and frustrations) people undergo while setting up their shiny brand-new IoT gadget, we tried think-aloud task analysis.

We created a list of tasks our participant would have to perform - right from unboxing to physical installation to downloading and configuring the app. At the end of task analyses were questions about the task, as well as the general cognitive load questionnaire based on NASA TLX.

Our insights from task analysis were:

• Amazon's Alexa setup received a fairly enthusiastic response(possibly because it was the first installation), and the participant said it was quite straightforward.
• For Phillips Hue, the participant expressed frustration with the task flow (although it was not much different from the Amazon Echo installation, the order and instructions were quite different)
• Instructions for downloading and using the app were not very clear in the box.
• If a user buys multiple devices, the optimal order to set it up in is not very obvious.

Along the lines of concerns our users expressed, we decided to dig deeper into a few apps to validate some of their claims. We looked at the task flow, iconography, labels, interactions and so on.

Even though I consider myself fairly comfortable with installing and using smart homes, this deeper dive helped me understand the difficulty in switching these apps and using them one after another.

• Most applications are centered around one ecosystem, and do not consider any bindings with other systems at all.
• Icons, symbols, labels and UI differ significantly from one app to another, making it a inconsistent experience downright unusable to certain users.
• Perhaps justifiably, based on the purpose of the device, the controls on the UI and task flow differ.

We summarized what we had learned from the task analysis and case studies into a document, and for a good measure, added some idea that could help us getting around it.

After that was clearly outlined, the team started sketching design alternatives to best solve the problems at hand.

We discussed and took the best ideas from our sketches and put them into wireframes to get some feedback about our design.

Since the whole team was working on the design, it was necessary to come up with a UI library and a consistent design language.

The final part of UI design was to create the screens and interactions that flow from one screen to another.

Finally, we came up with a video to quickly explain the app without having to completely explain what we did.