A team of U.S. Army and academic researchers are investigating how eye-pupil size changes can indicate a person’s cognitive state as a means to enable teaming with autonomous agents.
The future Army battlespace will require humans and AI agents to team effectively to accomplish mission-critical goals. Although AI agents can fill gaps in human performance, they are rigid and lack the flexibility inherent to human behavior, which could interfere with teaming.
“Humans’ brains are amazing, adaptable systems that automatically apply the right cognitive processes to complete a task and initiates each process at the right time,” said Dr. Russell Cohen Hoffing, a scientist at the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory. “However, our brain’s resources are limited. Being able to predict a Soldier’s mental status before resources are maxed out is an opportunity for an autonomous agent to deploy capabilities to aid the Soldier. To make progress on enabling this technology, we wanted to better understand how physiological signals, such as pupil size changes, are related to performance and cognitive states.”
A joint effort between researchers from the Army and the Institute of Collaborative Biotechnologies at the University of California, Santa Barbara, the Cognitive Resilience and Sleep History, or CRASH, project seeks to understand how variations in state (as measured by physiological sensors) influence subsequent performance. PLoS One journal published the team’s research on a subset of the data set, “Dissociable mappings of tonic and phasic pupillary features onto cognitive processes involved in mental arithmetic.”
In this research, the team sought to understand the cognitive processes that affect pupil size changes–and the reliability of these relationships–as a foundation to estimate how human cognitive processes and performance can vary in real-world, cognitively challenging tasks, Cohen Hoffing said. The pupil is a unique data source, as it is the only internal organ of the body that brain networks directly modulate and is visible to the outside world.
“The potential of this research is exciting because eye-tracking technologies are becoming universal in both commercial and military contexts,” Cohen Hoffing said. “Inherent to eye-tracking algorithms, pupil size is estimated but rarely used for analytics. Our research program aims to generate knowledge products that enhance usability of this type of data to have greater insight into cognitive processes such as attention and decision making.”
The researchers collected repeated measurements from participants on eight separate occasions over four months. The data provided insights into the consistency of the pupil response and relationships to behavior both within and between individuals–with a unique glimpse into cognitive processes over time rather than in single-session studies, Cohen Hoffing said.