Designing a technology that detects falls in nursing homes and the homes of the elderly

Collaborator:

Viktor Nyitray

Interim CEO, Attentify

Over 25% of individuals aged 65+ fall each year, but fewer than half report it to their doctor, increasing the risk of recurring falls. These falls often result in injuries, hospitalisations, and emotional distress. Attentify, part of Biometric Ventures, tackles this issue by aiding caregivers in fall detection at home or in professional settings.

Attentify, a startup company owned by Innovatrics, wants to develop algorithms that assist non-professional caregivers, including family and friends, in providing home care for the elderly. This involves monitoring their health conditions and alerting caregivers to any potential issues.

“In order to provide help to the elderly in their own homes, we first set out to design a product for nursing homes that will help professional caregivers detect falls and prevent situations where the elderly are left without help and assistance,” says Nyitray. The goal was to create an inclusive design and adaptability product that caters to both professional caregivers and, eventually, friends and family.

AI-powered fall detection

“The entire design process began with a scientific approach,” explains the interim CEO, “our initial hypotheses were that falls occur often and the elderly fall more during the day, as they spend more time standing or moving around.”

Attentify set out to develop an algorithm using state-of-the-art AI for precise body and movement detection. This algorithm essentially performs three key functions: it maps the body’s points, assesses the body’s position, and determines whether the change in position constitutes a fall or only a sudden movement.

The different shades of falls

Early on, the first big obstacle emerged. Nursing-home rooms are small, and typically furnished with a bed, table, armchair and other furniture. Most of the time, people fall behind a piece of furniture. When the AI doesn’t see all the body points needed for recognition, it can sometimes mistake a fallen person for a poster or a photograph or evaluate that the person has left the room. To address this, the company had to create an entirely new algorithm and teach AI what falls in nursing homes really look like.

“We had to create a whole new database of videos of falls for AI to learn from. Falls from different angles, in different light conditions, falls behind tables, chairs, beds, etc. Initially, we used stuntmen and databases from the internet, but both fell short of capturing what happens in real life,” says Nyitray. 

Stuntmen know very well how to fall and not hurt themselves. Unfortunately, the elderly don’t. On the other hand, videos from the internet were primarily shot in laboratory conditions – an empty room with a mattress in the middle – that definitely didn’t resemble a nursing home environment. “That left us with only one choice. To fall ourselves,” he adds.

Training AI to recognise falls

To record falls, the company rented rooms in nursing homes and in the homes of elderly people. They used three to four cameras and recorded both daylight and nighttime incidents.

This comprehensive method covered falls from chairs or beds, stumbles over carpets or furniture, and falls occurring beyond the camera’s view, all filmed from different angles. By the time they had finished, they had recorded thousands of falls for the AI to train from, so it could learn to recognise all manner of falls, including those such as falling from sitting or when tying shoelaces.

After every set of videos, the company evaluated what they needed to do more of. For example, if fall detection accuracy was only 50% when the fall occurred to the left of the screen and partially out of the camera’s view, they concentrated on creating a new set of videos to educate AI about such scenarios.

“It took us a couple of hundred falls to realise that we needed to buy some protective gear.”

– Viktor Nyitray, Interim CEO, Attentify

As for the original hypotheses, both proved wrong. “After some collaboration with nursing homes, it turned out that falls are not as frequent as we had thought, but they do tend to be serious when they happen. As for the timing of falls, most of them happen during the night. During the day, the staff often visit rooms, so residents are rarely alone. It’s different at night, when the elderly tend to be alone and become disoriented if they wake up, which can result in them falling,” explains Nyitray.

“Attentify is like insurance. Most of the time, you don’t need it. But when you do, then you are really glad you have it.”

– Viktor Nyitray, Interim CEO, Attentify

Data security

The AI itself might seem quite difficult to understand, especially for older people without a lot of experience with technology. In order to get some insight into how the elderly would react to the product, Attentify talked to residents in the nursing homes.

“A lot of people didn’t understand the technology and many associated cameras and AI with face identification, which is not the case at all,” explains Nyitray, adding that making people comfortable with sensors and movement evaluation is a key priority for the company.

Data security is a crucial element in every step of the design process, so the video recording can be anonymised. By mapping out and colour-coding elements on the screen according to depth, the technology only sees the movement, but not the person.

In order to determine AI’s success, Attentify assesses two key metrics: precision and recall. Precision indicates that the algorithm accurately detects falls, while recall signifies that AI correctly evaluates them as a “yes”. Currently, the success rate is at approximately 70%.

A user interface that suits everyone

The initial product version is currently undergoing real-life testing, starting at a nursing home in Brno, Czech Republic. The company is now developing a user interface that suits everyone, emphasising the importance of adaptability.

“The goal is to give caregivers a choice of the device that suits them best. Do they prefer smartphones to get information about the elderly? Are smartwatches better for them? Or is there another device they might choose?” explains Nyitray. 

The same goes for notifications. A caregiver can choose whether a simple fall notification will suffice, or if it is more beneficial to show the last 10 seconds of the recording so that the caregiver can deduce the severity of the fall and whether it needs medical attention.

Bold future

According to the OECD, Organisation for Economic Co-operation and Development, countries will need 60% more skilled long-term care workers (around 13.5 million) by 2040 due to the growing elderly population. Technology and digitalisation could help alleviate this pressure, and support comfortable and safe homestays for the elderly.

That is why Attentify aims to become a central healthcare monitoring hub for both professional and non-professional caregivers. By collaborating with other smart devices like smart bottles and pill cases, Attentify’s goal is to provide comprehensive health information about the elderly and send alerts to caregivers during emergencies. 

While they’re working towards becoming the main source of elderly health information in the future, Attentify believes AI will play a key role in making people healthier and more active in old age.