social interaction with eye-tracking based entertainment suite for patients with Locked-In Syndrome

Even though accessibility is considered more important now, people with Locked-In Syndrome (LIS) have been rarely considered when designing or building for. Locked-In Syndrome leaves the patient paralyzed except their eyes, but even that comes with limitations depending on the individual. For example, Bob Veilette can only do vertical movements well. A LIS patient usually encounters boredom, and lack of social interactions and bonding with their loved ones. We hope Myeplay can be centered around improving the life of people with LIS, and can alleve at least a small part of an LIS patient's pain points and improve quality of life through social bonding.

Deliverables: Website | GitHub | Final Presentation


Researcher, UX/UI Designer


Alex Yang, Andrew Wang, Danny Zhang, Kane Chong, Luke Kim, Sam Suen, Sean Burke, Pavan Kumar


Locked In Syndrome (LIS)

We were tasked with improving life for Bob Veillette: a man with Locked-in Syndrome (LIS) which leaves him completely paralyzed except for his eyes.

This task includes using an EyeTribe, a cheaper eye tracker to make it more accessible. Also, we needed to tailor the application to Bob, but we had no access to Bob as a user.


Understanding LIS Patients through Other Resources

Since we had no access to Bob, yet had to tailor the application to him, we could only rely on other sources to understand the motivations and struggles of LIS patients. We started by looking at different resources and papers [1] [2] [3] [4] to better understand their lives, and to read on the challenges of using eyes as input.

From different resources, such as watching The Diving Bell and the Butterfly, reading Bob Veillette's previous website, and reading the first paper by Smith and Delargy, I got a glimpse into the daily life of someone with LIS-- constantly bored and obligated to have someone take care of them, which can take an emotional and physical toll on their loved ones. LIS patients are completely cognizant of reality, yet cannot communicate or socially interact in any way possible, and it is extremely difficult for others to interact with them as well.


More Social Interactions for the Cognizant

Based on the challenges they faced, we wanted to help facilitate a better life with more social interaction by creating an entertainment suite for LIS patients. While there are many apps for entertainment for people without disabilities, there are only a few tailored to those with movement disabilities.

LIS patients also have communicated that they feel trapped in a limiting body, because they can still hear and be cognizant of the world around them, and their ability to think has not been damaged. Loved ones usually only have a one-way communication with people with LIS. We wanted Bob's life to be enriched by relieving his boredom and to be enlightened when he is alone, and to socially interact with his loved ones.

design process

While I would generally spend more time in different modes for designing (sketches, then digital wireframes, then prototypes with words, then detailed prototypes) time was of the essence, so I had to communicate my ideas to the engineers as soon as possible. I also wish I were able to spend more time on different ideas, and could test them on Bob and other LIS patients.

design solutions

Self Entertainment

More Self Reliance to Alleve Boredom

While jazz and piano are specific to Bob's interests, other LIS caretakers and their loved ones could choose videos tailored to their patient's interests with a media player.

Multiple Modes of Interaction

Bonding with Loved Ones and Social Interactions

We wanted to focus on the social interaction part, which is one of the biggest difficulties with the patients and their caretakers. We created 2 player games that could be played with the eyes and hands. While this is not exactly a form of communication, this allows for more quality time.

Different Issues across Users

Bob's Difficulty in Looking Horizontally

This is specific to Bob, but it does bring to light different difficulties other LIS patients may face. Even though their eyes may be the only thing that can move, there are still other physical limitations other patients have with regards to their eyes. I wondered about how others struggle with different kinds of eye movements, and how we should allow for customization in this area, based on the user. As for this specific design, I wanted to allow the affordance of looking up and down, and also a menu that repeats.

Creative Solution for Constraints

Cheaper Eyetracker is Inaccurate

One of our requirements/constraints was needing to use the eyetracker that was provided. This is considered a cheaper eye tracker, and therefore more inaccurate. We wanted the user to accurately choose what they wanted, so we chose to make the design so that there was only one option at a time.

final product

What We Encountered-- Compromise

The end result was not exactly what I designed, since time was of the essence. We already had difficulty using the open source plugin to create the circular rolling design, and having multiple steps created more complexity we did not have time for. Since our app was so simple anyways, having one screen for the game and the videos was fine with me.

personal takeaways

Creating New Experiences for People with Disabilities

Having the chance to design for our users has created new possibilities for interfaces, user experiences, interactions, and even new discoveries to what is intuitive specifically to these users. This project opened up new possibilities to rethink already existing functions and purposes of the different parts of the people's bodies , because people with LIS must learn to use their eyes for more purposes than people without LIS usually need to. I started to understand the idea that we have different kinds of users with disabilities, and that we need to start thinking about accessibility, and designing and considering these users. We feel privileged to have designed something to improve and enrich Bob's life, and to have redefined the eye's purpose and function as not just a passive way to indicate attention, but as an active decision maker.


Testing on Bob

I hoped we could really test our product on Bob to see if it was successful, and if Bob could even control it or use it with his eyes. Although we tried to accomodate to a cheaper eye tracker whose eye tracking was not completely accurate, it is still unknown how well Bob's, or any LIS patients', eyes could move anyways. We already improved the accuracy of each option, and allowed for more space to allow for errors, but it is still unknown how well LIS patients can move their eyes.

Measuring Success

I would consider this product successful if it actually affected his life and his loved one's lives by improving their social interactions, and by providing more avenues to interact with one another and spend time with each other. Quantifying it would be difficult, but I would consider it a success if 1) time spent with loved ones increased, 2) the product was used consistently over a long period of time. However, even if time did not increase, it does not mean that their social interactions didn't improve. The LIS patients could have spent more time with their loved ones, but not enjoyed it more. The same is the other way around: just because time spent increased doesnt mean that it was quality social interaction or bonding.

Since we want to somehow quantify an increase in positive emotions while using the product, I would have the participants fill out surveys before and after. LIS patients would need their caretakers to help them take the surveys. They would also fill it out consistently over time, answering questions about if they felt happier, with more time with their loved ones. While this is a qualitative method, we are trying to quantify something that is difficult to measure.