Driving seems so natural that many people are hardly aware of the enormous amount of information a driver has to process whilst behind the wheel. The display shows figures such as speed, revs and fuel level; the sat-nav displays the route; and drivers must keep an eye on traffic lights, road signs and other road users. On top of that, there may be a conversation with the passenger or news broadcasts on the car radio. It can become dangerous when the driver reaches the limits of their cognitive capacity – and then has to make the right decision in a split second in a critical traffic situation. Researchers at the Fraunhofer Institute for Integrated Circuits IIS in Erlangen are working on a system that supports people in complex tasks across various applications or situations, whilst taking their cognitive limits into account.
Recognising cognitive strain whilst driving
Researchers at the Fraunhofer Institute for Integrated Circuits IIS are working specifically on an intelligent support system along these lines. They are combining a mobile sensor network with AI-based algorithms. The system recognises individual stress limits based on various biosignals. These include, for example, heart rate, respiratory rate and movements. Should people reach their personal stress limit, the technology will be able to provide support in future. Vehicles could then automatically take over certain tasks. The system could also recommend taking a break at the next service station.
Mobile sensor network for biosignals
The maphera sensor network forms a key component of the research project. It enables the mobile measurement of various biosignals outside the laboratory. The system has a modular design. Depending on the application, different sensors can be combined with one another.
The technology can, for example, be integrated into textiles. Wristbands and so-called smart patches are also possible. Smart patches are small sensors designed for use on the skin. They can continuously record and transmit biosignals. Data is transmitted via Bluetooth Low Energy, making the system suitable for mobile applications.
Different microcontrollers present a particular challenge. Some of these components operate at different clock frequencies. For short measurements, this variation is hardly noticeable. Over several hours, however, significant time shifts can occur. As a result, biosignals can no longer be precisely assigned to a specific point in time. This can make subsequent analysis considerably more difficult. The Fraunhofer researchers have therefore taken these time drifts into account. Their system synchronises the sensor data with a tolerance of just 30 microseconds. The project name ‘maphera’ also refers to this simultaneous transmission.
Measuring cognitive load in computer games
Another key area of research examines the link between biosignals and cognitive load. To this end, participants carry out various tasks. The tests take place in a special exposure chamber, where external influences can be largely controlled. These include, for example, noise, draughts and changes in temperature. This enables the researchers to keep the experimental conditions as constant as possible. The tasks become progressively more difficult. Computer games are also used.
One example is a virtual sushi restaurant. The participants take orders and prepare sushi dishes. As the game progresses, the number of orders increases. At the same time, the dishes become increasingly complex. Meanwhile, sensors record various physiological signals from the body. The researchers then combine this data with a so-called n‑back test.
This standard psychological test assesses, amongst other things, attention and memory. In addition, the researchers conduct face-to-face interviews with the participants. All the information is then incorporated into a joint analysis. To do this, the researchers use AI algorithms they have developed themselves.
This results in an individual profile of cognitive load. This profile is intended to reflect individual stress thresholds as reliably as possible.
AI recognises individual stress limits
The combination of sensor technology and artificial intelligence opens up new possibilities. This is because it allows stress levels to be monitored continuously and on an individual basis. The system does not simply look at individual readings; instead, it analyses various biosignals together. This enables it to recognise typical patterns as stress levels rise. The AI also takes into account individual differences between people.
It is precisely this individualisation that is particularly important for practical applications. This is because people react differently to the same tasks and stressful situations. The method could therefore go beyond traditional monitoring systems. In future, it could enable situation-specific support.
Driver monitoring systems for new cars
Modern vehicles represent an important area of application. In such vehicles, sensors and cameras could continuously monitor the driver. If the cognitive load reaches a critical level, the vehicle could react. For example, certain functions could be taken over automatically. A recommendation to take a break would also be conceivable. In this way, the system could enhance safety on longer journeys. Car manufacturers are already working intensively on driver monitoring systems. These systems are designed to detect whether drivers remain alert.
New requirements apply in this regard under the EU General Safety Regulation. As a result, camera-based driver monitoring systems are becoming increasingly relevant for new cars. This also applies to vehicles with automated driving functions, as the driver must still be able to intervene if necessary.
Targeted reduction of cognitive load whilst driving
Future vehicle systems could therefore do more than simply detect distraction. They could also take an individual’s cognitive load into account. The vehicle could adapt its functions accordingly. This might make it possible to reduce the complexity of a driving situation in a targeted manner. This opens up a new approach to intelligent driver assistance systems. The technology responds not only to the vehicle, but also to the driver.
