eye-tracking based entertainment suite for patients with Locked-In Syndrome

People with Locked-In Syndrome are rarely considered when designing or building for, therefore lacking tools centered around allowing people with LIS to use them. Locked-In Syndrome leaves the patient paralyzed except their eyes, but even that comes with limitations depending on the individual. For example, Bob Veilette can only do vertical movements well. We hope Myeplay can be centered around people with LIS, can alleve at least a small part of an LIS patient's pain points in life of social bonding.


Website | GitHub | Final Presentation


Researcher, UX/UI Designer


Alex Yang, Andrew Wang, Danny Zhang, Kane Chong, Luke Kim, Sam Suen, Sean Burke, Pavan Kumar


We wanted to improve life for Bob Veillette: a man with Locked-in Syndrome (LIS) which leaves him completely paralyzed except for his eyes.


using EyeTribe, a cheaper eye tracker
tailoring to Bob, but no access to Bob as a user


We first started by looking at different resources and papers [1] [2] [3] [4] to understand the motivations and struggles of LIS patients, and reading on the challenges of using eyes as input.

From these different resources, such as watching The Diving Bell and the Butterfly, reading Bob Veillette's website, and reading the first paper by Smith and Delargy, I got a glimpse into the daily life of someone with LIS-- constantly being bored and having someone take care of them. More social interaction is necessary to help improve their quality of life, but it seems like an impossible task without being able to move. I wish we had more access to Bob, but unfortunately, that was not the situation.

Based on the challenges they faced, we wanted to help facilitate a better life with more social interaction by creating an entertainment suite for LIS patients.

While there are many apps for entertainment for people without disabilities, there are only few tailored to those with movement disabilities. LIS patients also have communicated that they feel trapped in a limiting body, because they can still hear and be cognizant of the world around them, and their ability to think has not been damaged. Loved ones usually only have a one-way communication with people with LIS. We wanted Bob's life to be enriched by relieving his boredom and be enlightened when he is alone, and to be able to socially interacting with his loved ones.

design process


While I would generally spend more time in different modes for wireframes (sketches, then digital, then prototypes with words, then detailed prototypes) time was of the essence, so I had to communicate my ideas to the engineers as soon as possible. I also wish I were able to spend more time on different ideas.

user needs + design solutions



Bob and other LIS patients get bored easily without their caretakers


Media player with Bob's favorite jazz and piano pieces

While jazz and piano are specific to Bob's interests, other LIS caretakers and their loved ones could choose videos tailored to their patient's interests.



Bob and other LIS patients need to bond with their loved ones and have more social interaction


2 player games that can be played with the eyes and hands



Bob has trouble looking horizontally


Continuous vertical menu

This is specific to Bob, but it does bring to light different difficulties other LIS patients may face. Even though their eyes may be the other thing that can move, there are still other physical limitations that other LIS patients struggle with with regards to their eyes. This made me about how others struggle with different kinds of eye movements, so this is something allow customization for, based on the user.



The eyetracker is more inaccurate


Single option at a time

One of our requirements/constraints was needing to use the eyetracker that was provided. This is considered a cheaper eye tracker, and therefore more inaccurate. We wanted the user to accurately choose what they wanted, so we chose to make the design so that there was only one option at a time.

final website


The end result was not exactly what I designed, since time was of the essence. We already had difficulty using the open source plugin to create the circular rolling design, and having multiple steps created more complexity we did not have time for. Since our app was so simple anyways, having one screen for the game and the videos was fine with me

personal takeaways

Rethinking and creating new possibilities and experiences for people with disabilities

Having the chance to design for our users has created new possibilities for interfaces, user experiences, interactions, and even new discoveries to what is intuitive specifically to these users. This project opened up new possibilities to rethink already existing functions and purposes of the different parts of the people's bodies , because people with LIS must learn to use their eyes for more purposes than people without LIS usually need to. I also started to open my eyes to understanding the idea that we have different kinds of users with disabilities, and we need to start thinking about accessibility, and designing and considering these users. We feel privileged to have designed something to improve and enrich Bob's life, and to have redefined the eye's purpose and function as not just a passive way to indicate attention, but as an active decision maker.


Testing on Bob and Measuring Success

I wish we could really test our product on Bob to see if it was successful, and if Bob could even control it or use it with his eyes.

I would consider this product successful if it actually affected his life and his loved one's lives by improving their social interactions, by providing more avenues to interact with one another and spend time with each other. That would be difficult to quantify, but I would consider it a success if 1) time spent with loved ones increased, 2) the product was used consistently over a long period of time. However, just because time did not increase does not mean that their social interactions didn't improve. The LIS patients could have spent more time with their loved ones, but not enjoyed it more. Since we want to somehow quantify an increase in positive emotions while using the product, I would have the participants fill out surveys before and after. They would also fill it out consistently over time, answering questions about if they felt happier, with more time with their loved ones. While this is a qualitative method, we are trying to quantify something that is difficult to measure.


  back to work