2023 Volume 5 Issue 1 Article ID: 2023-0006-RA
Objectives: This scoping review aims to examine the existing use of eye-tracking technologies being applied to measure negative mental health-related outcomes. The review was guided by the following questions: 1) What eye-tracking methods are currently in use?; and 2) What type of negative mental health-related outcomes are these methods being applied to for estimation? Results will be evaluated to determine their prospective implementation in remote work as a mental health indicator. Methods: A scoping review was conducted in order to collect data from a range of sources and evaluate many distinct research methodologies. A scoping review was chosen for this study to widely report on the research currently being conducted, rather than answer a specific question from a focused set of evidence. On May 26, 2022, a systematic search of the scientific literature was conducted to identify any eye-tracking methods that have been used to measure stress and anxiety. Results: Out of an initial 5,356 eligible articles, a total of 14 articles were included in this scoping review. Estimation outcomes also ranged from various mental health-related outcomes with the most common outcome relating to stress and fatigue. Other outcomes included sleepiness, drowsiness, arousal, frustration, hypervigilance, defensive state, peritraumatic dissociation, and anxiety. Conclusions: Preliminary results show a very promising connection between eye metrics and negative mental health-related outcomes, which are very relevant to workplace mental health as well.
In the middle of the novel coronavirus disease 2019 (COVID-19) pandemic, the number of individuals working remotely has surged significantly1). Many countries have also implemented a “Stay at home” policy in an effort to manage and moderate the outbreak2). This would also reduce the burden on national healthcare systems and the economy as a whole3). Since then, working from home has become a standard component of the contemporary workplace culture. On the other hand, there is an ongoing discussion regarding whether remote work has positive or harmful effects on workers. According to the results of a meta-analysis, there is a marginally significant positive correlation between remote work and organizational outcomes, such as enhanced productivity, employee retention, and organizational commitment4). Yet, other research indicates that there is minimal evidence that working remotely increases job satisfaction, and numerous studies suggest that remote workers experience greater isolation and stress compared to on-site workers5,6).
The World Health Organization (WHO) and several previous studies indicate that organizational support from the workplace, peer support by fellow workers and managers, and self-care by individual workers all have a significant effect on mental health outcomes7,8). However, when employees are geographically separated, it is much more challenging to recognize subtle differences in one another, which is a requirement for many conventional mental health safety nets in the workplace. Furthermore, the WHO indicates that home settings often do not meet the same occupational safety and health standards as traditional worksites9), and according to the results of a recent survey, 53 percent of employees believe that their employer makes no effort to mitigate the negative mental health consequences of working from home6). Even though the introduction of new technology has altered work patterns and made it possible for employees to work from anywhere, a new established system for managing the mental health of remote workers is required.
The eye and pupillary response provides vital information regarding an individual’s higher cognitive function and emotional processes10,11). Recent technological and software advancements in eye-tracking have enabled the use of mobile devices in numerous fields, such as psychology and human-computer interactions12). Historically, research on mental state assessment has been based on questionnaires, or more recently neurophysiological data, studying the relationship between disorders affecting the brain and the nervous system function. More recently, physiological measures, such as cortisol levels, have indicated a strong association with cognitive load and stress along with traditional brain activity measurements, such as electroencephalography (EEG)13,14). However, many of these methods are unable to provide realtime feedback due to intrusiveness or signal noise, which can lead to user abandonment issues similar to other wearable devices15). In comparison, eye-tracking technologies are less obtrusive and easier to handle for the user. Most video-based eye trackers utilize a high-frequency camera pointed at the subject’s eye, along with a infrared/near-infrared light to illuminate the pupil and reflect off the cornea16). These eye trackers are capable of tracking a viewer’s line of sight and detailed eye metrics when calibrated properly.
Although there are still very few studies reviewing the effectiveness of eye-tracking technologies at estimating various mental health-related outcomes, results of a previous review suggest that eye metrics, such as fixation, saccades, blinking, and pupil size, are valid indicators of human responses to emotional arousal and cognitive workload11). In addition, there are numerous studies focusing on classification of patients with mental health-related disorders17,18). However, there is also evidence that video-based eye-trackers may measure tiny eye movements inaccurately. In addition to the minuteness of eye movements, there are numerous types of video-based eye-trackers, resulting in a vast array of outcomes even when seemingly measuring the same outcome11). While new camera technologies will allow for more precise tracking as time passes, it is necessary to assess the methodologies employed in current studies.
This scoping review aims to examine the existing use of eye-tracking technologies being applied to measure negative mental health-related outcomes. The review was guided by the following questions: 1) What eye-tracking methods are currently in use?; and 2) What type of negative mental health-related outcomes are these methods being applied to for estimation? Results will be evaluated to determine their prospective implementation in remote work as a mental health-related indicator.
A scoping review was conducted in order to collect data from a range of sources and evaluate many distinct research methodologies. A scoping review was chosen for this study to widely report on the research currently being conducted, rather than answer a specific question from a focused set of evidence.
On May 26, 2022, a systematic search of the scientific literature was conducted to identify any eye-tracking methods that have been used to measure negative mental health-related outcomes. The following search engines were used to locate pertinent information: PubMed, Ovid MEDLINE, Cochrane Central Register of Controlled Trials (CCRCT), and PsycInfo. While eye-tracking is a relatively new field of inquiry, the search terms were left purposefully imprecise to extend the scope of the findings. ((“mental health”[Mesh]) OR (mental health) OR stress* OR (“Stress, Psychological”[Mesh])) AND (“Eye Movements”[Mesh] OR “Eye Movement Measurements”[Mesh] OR (eye AND (track* OR movement*) OR gaze). Slight modifications were made to the search terms to make them compatible with the databases PsycINFO, Ovid MEDLINE, and CCRCT. The following are the requirements for inclusion: 1) study focuses on estimation of a negative mental health-related outcome; 2) study focuses on eye-tracking metrics and methodologies; and 3) participants were healthy individuals over 18 years old. On the other hand, research that met any of the following exclusion criteria were not considered for inclusion: 1) full text was unavailable; 2) they were not published in the English language; 3) they did not describe an intervention or trial; or 4) they focused on patients or individuals with pronounced mental health problems. Gray literature was evaluated for inclusion if the eye-tracking methods and procedure for estimating mental health outcomes were sufficiently described.
Two reviewers participated in conducting the preliminary screening and selecting full-text articles. Any differences that emerged between the reviewers were discussed and resolved individually. If the appropriate contact information was available, attempts were made to contact the authors of those studies for which the full texts were absent from public databases. The following information was extracted from the included articles based on previous studies and the research objective19): 1) the author’s name; 2) the publication year; 3) the country where the research was conducted; 4) the total number of participants; 5) the mean age of participants; 6) the method used to recruit participants; 7) the eye-tracking method; 8) the intervention method; 9) the groups that were compared; 10) the primary outcome measures; and 11) the study’s key findings. If the research country was not disclosed in the text, the country of the corresponding author was included. Results were compared across studies in a descriptive manner to summarize current findings, as well as identify elements potentially applicable to remote work situations.
After duplicates were removed from an initial database search of 12,281 records, 5,356 articles were screened by title and abstract independently by two reviewers. A total of 296 articles were identified as eligible for full-text screening, which were again screened independently by two reviewers (Figure 1). Out of an initial 5,356 eligible articles, a total of 14 articles were included in this scoping review. Table 1 presents a summary of the demographic characteristics of the included studies. All studies were conducted within the past 15 years, illustrating the freshness of the topic and the emergence of new eye-tracking technologies and their applications to new fields. Six (No. 2, 4, 8, 10, 12 13) studies were conducted in North America, one (No. 7) study in Australia, five (No. 3, 5, 6, 9, 11) studies in Europe, and two (No. 1, 14) of the studies were conducted in Asia. Our findings revealed that studies were heavily skewed towards western countries; therefore, cultural differences may influence results, particularly in the field of mental health. Although one (No. 1) study developed eye-tracking equipment for the purpose of the study, every other article utilized commercial options with state-of-the-art technologies. An overwhelming majority (No. 2, 3, 4, 5, 6, 7, 9, 10, 12, 13, 14) also utilized video-based eye trackers, which the basic concept is to illuminate the eye and capture an image of the eye. Study No.1 and No. 11 both employed methods of electrophysiological monitoring, although study No.1 utilized electrooculogram (EOG) signals to identify eye blinks, whereas study No. 11 utilized electroencephalography (EEG) to measure brain waves. Finally, a proximity sensor was utilized in study No. 8 to detect blinking. Although sex, age, and target demographics were somewhat varied, many studies lacked detailed explanations regarding study participants.
| ID Number | Author | Year | Country | Eye-tracking equipment | N (female) | Participants | Mean age, years |
|---|---|---|---|---|---|---|---|
| 119) | Kim, et al | 2008 | South Korea | Helmet system developed for study (Electrooculography (EOG)) | 3 (0) | Healthy individuals with a crew cut hairstyle | — |
| 220) | Kimble, et al | 2013 | USA | EyeLink 2000 (Video-based) | 71 (50) | Undergraduate liberal arts school in the north east | 18.4 |
| 321) | Matthews, et al | 2019 | UK | Tobii X2-60 (Video-based) | 40 (13) | University students | 26.3 |
| 422) | Zheng, et al | 2012 | Canada | PT-Mini (Video-based) | 23 (−) | Surgeons | 34.8 |
| 523) | Vatheuer, et al | 2021 | Germany | SMI eye tracking package (Video-based) | 74 (0) | Healthy individuals | 24.9 |
| 624) | Merscher, et al | 2022 | Germany | EyeLink 1000 Plus (Video-based) | 50 (40) | Sample of convenience recruited through an online platform | 28.0 |
| 725) | Tichon, et al | 2014 | Australia | EyeLink II (Video-based) | 12 (5) | Undergraduate students enrolled in School of Aviation and has flying experience | 26.1 |
| 826) | He, et al | 2017 | USA | Google Glass (proximity sensor) | 23 (13) | Experienced drivers recruited through posters and online advertisements | 25.0 |
| 927) | Danböck, et al | 2021 | Austria | Tobii Pro X3-120 (video-based) | 71 (71) | Healthy individuals | 21.2 |
| 1028) | Nahar, et al | 2011 | USA | ISCAN ETL-400 (video-based) | 35 (−) | Individuals recruited from a university or nearby community | range= 18–30 |
| 1129) | Morales, et al | 2017 | Spain | TGAM headset (Electroencephalography (EEG)) | 15 (5) | Active drivers | 24.3 |
| 1230) | Di Stasi, et al | 2013 | USA | EyeLink 1000 (Video-based) | 12 (2) | Individuals with no prior air traffic control experience | 30.0 |
| 1331) | Ahmadi, et al | 2022 | USA | Tobii Pro Glasses 2 (Video-based) | 21 (17) | Registered nurses recruited from a 40-bed cardiovascular intensive care unit | 34.1 |
| 1432) | Yamada, et al | 2018 | Japan | EMR ACTUS (Video-based) | 20 (8) | Healthy individuals | 47.5 |
Table 2 describes the intervention and outcomes of each study in greater detail. All studies conducted an experiment or task which induced a negative mental health-related response from the participant. None of the studies were randomized controlled trials, with six (No. 1, 2, 3, 6, 7, 10) of the studies using a controlled variable as comparison. Estimation outcomes also included various mental health outcomes, with the most common outcome relating to stress (No. 5, 10, 13) and fatigue (No. 11, 12, 14). Other outcomes included sleepiness (No. 1), drowsiness (No. 8), arousal (No. 3), frustration (No. 4), hypervigilance (No. 2), defensive state (No. 6), peritraumatic dissociation (No. 9), and anxiety (No. 7). Key findings of eye-tracking metrics also varied from blink duration, target fixation, and pupillary response, with many negative mental health outcomes sharing the same metrics.
| ID Number | Intervention | Comparison | Estimation | Key findings |
|---|---|---|---|---|
| 119) | Sitting down whilst wearing the helmet in a room lighted by an incandescent electric lamp and staring at one point on the wall in the afternoon (16:00 hours) | Same experiment in the morning (09:00 hours) | Sleepiness | Longer blink duration |
| 220) | Searching pictures for threatening targets in order to avoid a loud white noise burst | Viewing the same image without instruction | Hypervigilance | More fixation; more visual scanning; larger pupil size |
| 321) | Completing four tasks on website with disruptions including a simulated operating system failure, pop-ups, internet time-out and mouse malfunction | Completing four tasks on website without disruptions | Arousal/frustration | Shorter fixation duration; larger pupillary response |
| 422) | Performing laparoscopic procedure on a virtual reality trainer | — | Frustration | Reduced blink frequency |
| 523) | Completing the Trier Social Stress Test (TSST) with speech task and arithmetic task in front of three judges | — | Stress | Shorter gaze duration looking at judges |
| 624) | Being signalled that they would be shocked, not shocked, or have an avoidable shock if they reacted quickly enough to a task, before proceeding to the shock phase of experiment | Not shocked phases served as comparison | Defensive state | Larger pupil size |
| 725) | Completing a flight scenario in a simulator where they had to conduct climbing and descending turns at 30 degrees, at 80 knots, at 500 feet/min. | A flight scenario in a simulator where participants were required to level off at 1,000 feet. | Anxiety | Larger numbers of saccades; lower fixation duration |
| 826) | Completing a simulated car-following task with the car in front braking at random intervals | — | Drowsiness | Higher blinking frequency |
| 927) | Watching a 7.5-minute montage of sections of the trauma film ‘Irreversible’ | — | Peritraumatic dissociation | Less/smaller eye movements; more/longer fixations |
| 1028) | Reading a text for 30 minutes with induced astigmatic refractive error (1.0 diopter [D]) in front of both eyes | Reading a text for 30 minutes | Visual stress | Reduced aperture size (ie. Squinting) |
| 1129) | Driving in a simulator for two hours | — | Fatigue | Decreased saccadic peak velocity |
| 1230) | Conducting an air traffic control simulation with two levels of task complexity and two different viewing conditions resulting in four sessions | — | Mental fatigue | Decreased saccadic velocity; Increased drift |
| 1331) | Nurses working through a 12-hour nursing shift | — | Stress | Increased number of fixations; Increased entropy; Decreased pupil size; Decreased fixation duration |
| 1432) | Conducting a modified version of the paced auditory serial attention test (mPASAT) twice in between watching five minute video clips | — | Mental fatigue | Decreased pupil size; Increased blink rate/duration |
This study identified that eye-tracking metrics are associated with various mental health outcomes in the workplace. Conditions, such as sleepiness, fatigue, and drowsiness, can lead to lower productivity, and more serious illness both mentally and physically33,34). Meanwhile, hypervigilance, defensive state, and peritraumatic dissociation are all associated with outcomes, such as trauma20,24,27). Furthermore, arousal and frustration were associated with and can serve as a proxy for anxiety, which can have detrimental effects on the body if left untreated21). Lastly, and most notably in the workplace, stress was found to be a wide-ranging variable that can affect mental health and significantly increase the employee’s risk of developing physiological and psychological disorders, which can result in increased absenteeism, organizational dysfunction, and reduced productivity35).
However, a lack of uniform evidence suggests a need for a randomized study design and validated assessment methods and environments in order to properly evaluate the impacts of eye-tracking technologies. Eye metrics displayed various results, from autonomous responses to conscious behavioral changes. Blinking can be an autonomous response or conscious movement, and was associated with sleepiness, frustration, drowsiness, and stress. Larger pupillary response was associated with hypervigilance and defensive state, which are both properties where the participant is trying harder to concentrate on the immediate task. On the other hand, smaller pupillary response was associated with stress and fatigue, where the participant presumably had trouble concentrating. Fixation on target areas of an image were associated with several mental health outcomes, such as stress, frustration, hypervigilance, peritraumatic dissociation, and anxiety. This may be due to the fact that fixation can be a reactionary response as well as a conscious effort by the participant. Especially as seen in stressful situations, a reaction to a stressful social anxiety situation has a very different mental effect compared to a stressful outcome due to heavy mental workload. The studies in this review reflect previous research in cognition, where stress resulted in both shorter and longer fixation responses depending on the situation11). Although a few studies utilized secondary physiological responses, such as salivary cortisol levels, in order to further identify the participant’s state, there is a limitation to the evidence due to each task and environment being vastly different. Furthermore, depending on visual and cognitive factors, such as stimuli size or emotional context, outcomes, such as eye movement, can vary even if the intended intervention is the same36). Therefore, even if experiment designs may differ depending on specific objectives, moving forward it is imperative that studies provide further context of the stimuli to give better insight into the results of the study.
Although the results of this review are far from conclusive, detection of the identified mental health-related outcomes may play a large role in remote workplace mental health efforts. The WHO promotes a multi-faceted approach to preventative mental health interventions in the workplace, ranging from the organizational level to individual self-care37,38). On the organizational level, employers have a role in protecting and promoting safety and mental health in their company, which can be even more difficult when workers are outside the direct control and environment of a single workplace9,39). Organizational interventions are recommended to focus on larger environments and workplace designs that address psychosocial risk factors, and the automatic detection of mental health-related outcomes may help create a comprehensive set of measures and work schedule modifications needed for remote work9,37). Results of this study may also help in manager/peer communications, where workers feel that virtual communication lacks a social aspect and is difficult to convey high-ambiguity information compared to face-to-face interactions40). This physical isolation makes it more difficult to help each other, but an automatic detection of mental health-related outcomes can serve as an additional non-verbal cue. Finally, self-care is the basis of mental health care and prevention in all settings38). While workers can find it difficult to manage work-life balance during remote work, automatic detection for one’s own self can help with self-awareness and management, which are both integral steps of self-care in the prevention of negative mental health conditions38).
However, there are still many limitations to implementing eye-tracking technologies into workplace mental health. Many studies utilized a version of the Eye-link system for tracking eye metrics, which is a full system of high-definition cameras, a workstation, mounts, software systems, and technical support41). Other studies also utilized commercial products utilizing all-encompassing eye-tracking systems, which can be expensive to implement. Although new software solutions to regular web cameras have started to emerge as valid eye metric trackers, there is still very little evidence of use cases in the mental health field42). Furthermore, differences in hardware and software can lead to varying methodologies in signal processing and calibration, which make reproducibility considerably more difficult. Previous eye-tracking studies dictate that there is no universal best method of eye-tracking, and that the temporal and spatial accuracy needs, suitability for operational conditions, invasiveness, and cost should be considered depending on the objectives of each study43). For example, studies measuring pupil size may prioritize high sampling rate and resolution, while other studies focusing on emotion may need a setup allowing for head movement and wide viewing angles. Along with a need for more intervention studies utilizing everyday eye-tracking technologies with consistent methodologies, there is a need for researchers to understand the characteristics of different eye-trackers and the affinity of each design with the study they wish to conduct.
This was a first study in assessing the current evidence of existing eye-tracking technologies and their association with various negative mental health-related outcomes. Preliminary results show very promising connections between eye metrics and negative mental health-related outcomes, which are relevant to workplace mental health as well. However, the lack of empirical evidence suggests a need for a randomized study design and more uniform methodologies in order to properly evaluate its effects, identifying a path for future work.
We greatly appreciate the Toyota Foundation for supporting this research through their Co-Creating New Society with Advanced Technologies 2021 grant.
DY was in charge of supervising the process, and providing his expert opinion on the subject. KO organized the study design. KO, KA, and KM contributed to conducting the screening process. KO wrote the first draft of the manuscript, and KA, DY, KM reviewed the manuscript critically. Finally, all authors approved the final version of the manuscript.
Ao Research Institute, Ltd. and Michele Holdings are not directly related to the topic of current research. The authors also declare that there are no direct conflicts of interest, however, being employed in a company (Ao Research Institute, Ltd. and Michele Holdings) may be a potential conflict of interest.
This work was supported by the Toyota Foundation, Co-Creating New Society with Advanced Technologies 2021, Japan (D21-ST-0010).
