The eyes have it as eyegaze tech picks up awards. Carol Allen and Ian Bean explain why it's so important
'Popred': eyegaze art by Sarah EzekielFrom April 15 in Finchley, London, artist Sarah Ezekiel will hold another in her series of art exhibitions created on a computer controlled by her eyes. Sarah, who has motor neurone disease, uses an eyegaze system developed by Tobii to produce stunning images by controlling the art package by looking at the on-screen tools and menus.
Computers controlled with your eyes? Surely that’s science fiction, alongside hover-packs and teleportation? Yet it is here now and in classrooms, and homes, near you.
We have always been interested in our eyes and their direct link to emotions, learning and thought – and for many years the focus (ha!) has been on barriers to learning due to faults in the eye itself or the processing of visual information in the brain. However, work with eyegaze technology has moved on from merely interpreting eye movements to allowing that same technological link to facilitate control of a computer using eye movements alone.
How it works
How does it work? Current systems available for education operate on an optical tracking model where the user does not wear any special equipment. An eye-gaze camera is mounted on the computer monitor which emits an infrared beam that is reflected by the eye back to the sensor. The computer interprets this information, where the eye is looking, focusing, or tracking and translates this into mouse movement. For this to work effectively, a calibration exercise must be carried out to establish repeatable, quantifiable and reliable user data. For some students, for example those experiencing difficulties with head control or variable gaze, this can be problematic.
'See Hope, See Love' by Sarah EzekielCorrect positioning is key to success with eyegaze. Special needs technology suppliers offer a wide range of screen-mounting solutions to accommodate the needs of wheelchair users. Calibration software helps position the device at the correct distance and angle from the users face. Clever settings in the software can then be used to personalise the calibration to best meet the specific needs of the user.
This can involve tracking individual eyes, compensating for spasmodic movement and using your own images or video as the target of the user’s gaze. It is a strength of some systems that one-stop calibration is offered as an option – for those with complex barriers, this is sometimes all that is reliably achievable.
'Dwelling' on a point is how you mouse-click
Eyegaze breaches barriers of keyboard and mouseEyegaze systems map the movements of the eyes to the movement of the mouse pointer. Clicking is handled through a system called “dwelling”, where the user maintains his or her gaze on a target for a fixed period, usually about a second, after which the computer will click the mouse button on the target. This means that any software which can be operated with a mouse can be operated with eyegaze, providing the icons and buttons are large enough for the user to accurately target.
When it emerged, eyegaze technology was primarily considered as a means of capturing the data produced by eye movements, and it has been used for many years, particularly in the marketing industry. By collecting data on eye movements, length of dwell and frequency of fixation on elements of a design, it was possible to assess visual reaction to adverts. Thus valuable feedback became available to clients about the key points of interest in their advertising media.
However, the technology also allows control of computers, and for people experiencing severe physical challenges such as motor neurone disease it has been incredibly successful in facilitating access to computers, mobile telephony services and the internet. Those of us who work with children with physical challenges understand that often the biggest barrier to using the computer has been the mouse and keyboard. Our students need to learn how to use these before they can use the computer to learn. For some of our students, eyegaze has helped breach that barrier.
The user feedback that has proved so useful for marketeers and advertisers is also a very useful feature of educational use. Educators are sometimes wary of ‘losing’ children with communication issues in technology. That's where the children are clearly absorbed by using a computer but without teachers being clear of how or with what they are engaging.
Feedback on what is interesting children
Eyegaze: valuable feedback but don't assume too muchHowever, eyegaze technology gives teachers valuable feedback about precisely where those children are looking and what is attracting their visual interest, even though it might not help with the “Why”. So while that may be helpful to know, it is vital that assumptions are not made as to what the child is thinking, based on where their eyes are reacting to stimuli.
More recently eyegaze suppliers have turned their attention to people with profound and complex intellectual difficulties and a range of developmentally appropriate software is now available to include them. Inclusive Technology’s was recently recognised with an Education Resources Award 2015 (SEN including ICT category) for its Inclusive EyeGaze Foundations offering which is part of a carefully structured range of activities and tools that support students and their educators.
Sensory Software’s Look to Learn software also provides a range of graded activities and assessment tools designed to help a student learn and generalise the skills they will need to operate a computer with their eyes. Both of these companies also provide helpful workbooks to assist in the assessment and teaching process (Inclusive has just released its Eye Gaze in the Classoom publication as a free download).
Students at Trinity Fields School and Resource Centre in Caerphilly have been using eyegaze systems together with specialist software for more than two years now and have achieved some excellent results. Teacher Anthony Rhys publishes a long running eyegaze wiki sharing their experiences and highlighting useful free and paid eyegaze resources appropriate to the needs of this student group.
New approaches for literacy work
The data gathering behind the software is also generating intense interest among those working with literacy as it is hoped that the feedback data will inform educators as to fluidity of visual tracking and pinpointing the places in texts where children are having problems with a great deal of accuracy. In North Tyneside early results are encouraging, showing clear evidence in these areas which we have had no way of gathering feedback before. As with all ‘new’ technology, the engagement and motivation factors of introducing such a new way of interacting with technology are also having a positive influence on reading as an activity.
Eyegaze systems have revolutionised access for people with severe physical challenges, opening up access to a world of communication, learning and leisure through the screen of their computers. Now, with the use of specialist software, eyegaze systems are available to our more complex students with profound and multiple needs. As the price drops with the wider range of products and increased competition, the technology becomes available for home use and parents and carers are already very keen to establish potential applications, for example in environmental control.
To get a measure of just how effective this technology can be, reflect on the experience of this teacher:
“Working with Daniel who has a degenerative condition. Daniel very quickly understood he was activating the programme with his eyes and this was apparent through very obvious eye movements – widening and blinking when he found the object so he could 'pop' it. I can't describe the feeling we all had when we realised he could do it, it was one of those goose-bump moments we all felt at the same time.”
We still have much to learn about the efficacy of eyegaze with this student group. Just because we can see where a student is looking doesn’t necessarily mean that we can see what they are thinking and understanding. In-built eyegaze assessment tools are useful and show us WHAT our students are looking at. An in-depth knowledge of our students and their specific needs is still critical to understanding WHY they are looking in the first place. As usual, the technology may be amazing, but its full potential will only be evident in carefully planned educational provision if it is to be more than just a neat trick!
Ian Bean is an independent ICT consultant and software developer who specialises in the use of ICT to support the communication, learning and leisure needs of children and adults with severe and complex learning difficulties www.ianbean.co.uk/.
Sarah Ezekiel Eye Gaze Art
Eye Gaze SEN WIKI
Look to Learn Software from Sensory Software
Eye Gaze Learning Curve software from Inclusive Technology
Inclusive’s free downloadable Eye Gaze in the Classroom publication
“Artist draws detailed portraits with only his eyes” (Wired Magazine)