The AI tool, when developed, will collect and analyse voice recordings to assess seniors’ mental health, with their consent.

Singapore, 16 October 2024 – As part of a three-year research study and pilot programme named SoundKeepers, seven partners from the healthcare and social sectors will come together to create an AI tool using voice biomarkers to detect early signs of depression in seniors and a community intervention programme to manage the condition outside healthcare settings.

Just as blood and stool samples are collected to provide doctors with insights into a patient’s physical health, a voice biomarker tool collects voice samples to provide clues into the patient’s mental health, with consent. When completed and if deemed successful, it is envisioned that the tool and programme will provide professionals and patients with an objective measurement to normalise conversations about a topic that is often hidden, hard to define, and often difficult to talk about.

The seven partners include Nanyang Technological University, Singapore’s (NTU Singapore) Lee Kong Chian School of Medicine (LKCMedicine) and College of Computing and Data Science (CCDS); two healthcare institutions of the National Healthcare Group, National Healthcare Group Polyclinics and Institute of Mental Health; two Social Service Agencies, Fei Yue Community Services and Club HEAL; and philanthropic house Lien Foundation. 20 GP clinics in Hougang and Woodlands will also play a supporting role.

Involving over 600 seniors, SoundKeepers aims to help seniors aged 55 years old and above suffering from subsyndromal depression (SSD) – a stage where depressive symptoms start to emerge but are not yet severe enough to warrant a diagnosis. Among seniors, it is a largely unaddressed health risk.

Those with SSD are five times more likely to develop depression within a year and have a 12 times higher risk of dementia. It is one of the most common mental health conditions to affect seniors in Singapore, with 13.4% of community-dwelling seniors older than 60 affected by it, a likely under-estimation as it is based on self-reported assessments. Seniors with SSD incur higher healthcare costs than those without SSD and have comparable use of outpatient services to those with depression. Chronic diseases and disability, conditions that commonly plague seniors, exacerbate SSD.

Compared with depression, SSD is three times more prevalent and has an incidence rate five times greater, i.e., at any point in time, there are three times more people with SSD compared to those with depression and over a one-year period, five times more people develop SSD compared to those who develop depression. Recovery and mortality account for the difference between the prevalence and incidence rates.

“We need new ways to listen to our seniors. While they may not express their worries through words, we can now try to hear it through their voices,” said Mr Lee Poh Wah, CEO, Lien Foundation.

“Currently SSD is not actively diagnosed or treated. However, with the focus on early detection and treatment emphasised by both HealthierSG and the National Mental Health and WellBeing Strategy, this project becomes extremely relevant as it can facilitate the early detection and diagnosis of SSD with a tool that can be easily used in the community setting,” said Dr Mythily Subramaniam, Assistant Chairman, Medical Board (Research), IMH and Co-Principal Investigator of SoundKeepers.

How the pilot will work

Recruitment of participants will be at Hougang and Woodlands Polyclinics. There will be two components to the programme structure: The collection of voices at the polyclinics and IMH with the development of the voice biomarker tool at CCDS, and a referral process for identified seniors from the polyclinics to the SSAs where they will attend a 24-week community-based early intervention programme.

In the first component, researchers will collect a voice sample of several minutes long from each patient with his or her consent via a casual conversation or passage reading. Many such voices will be used to build the algorithm for detecting SSD using a designated smartphone. Once developed, the voice biomarker will provide an indication of the state of a patient’s mental health of depression or subsyndromal depression in an objective manner, akin in some ways to how doctors currently take readings of a patient's temperature or blood pressure.

Deterioration in mental health often leads to physiological changes in the muscles used in voice production. For example, stress can cause muscle tension in the throat, neck, and jaw, affecting the vocal cords and therefore, the pitch and tone of the voice generated. Strong emotions may lead to physiological arousal such as faster speech or a trembling voice. Shallow or irregular breathing, commonly seen in patients with anxiety, can affect vocal projection and clarity. Insomnia can lead to difficulty in concentrating, affecting speech rate, fluency, and articulation. Fatigue can affect vocal stamina and volume.

AI data models can pick these up via analysis of the acoustic properties in a voice sample, which range from commonly-known ones such as pitch and volume, to lesser-known ones such as timbre, rhythm, shimmer, jitter, Harmonics-to-Noise Ratio, and others. An AI model will be built with a representative sample of the population that comprises the well, patients with subsyndromal depression and patients with depression, as well as in the same patient across time.

“When we use our voice, we are activating and coordinating more than a hundred different muscles and neurobiological processes. A change in speech acoustic features can reveal abnormalities in these neurobiological processes. With the AI voice tool, primary care physicians will be able to identify seniors with SSD faster, without subjecting them to a battery of questions that may be distressing to seniors. This is a new area of research in the field of mental health that is being led by LKCMedicine. The potential is immense and represents a paradigm shift in the way we screen and diagnose depression,” said Assistant Professor (Clinical Practice) Dr Lee Eng Sing, Clinical Director of Primary Care and Family Medicine Research Programme at Nanyang Technological University, Singapore (NTU Singapore) Lee Kong Chian School of Medicine (LKCMedicine) and the Co-Principal Investigator of SoundKeepers.

“The SoundKeepers project aims to leverage advanced AI technologies to identify individuals with SSD using their natural voices. This represents a novel area of research, as while previous studies have demonstrated the feasibility of detecting depression through vocal analysis, little work has been done on the more subtle SSD. The project focuses on developing more advanced technologies to address this challenge. The advantage of using voice as a biomarker for depression, including SSD, lies in its non-intrusive nature and broad application,” said Professor Guan Cuntai, President’s Chair Professor in Computer Science and Engineering, Deputy Dean of NTU CCDS and Co-Principal Investigator of SoundKeepers.

Voice biomarkers can be built upon acoustic properties, linguistic properties, or both. Linguistic properties include coherence, repetition and use of specific words. In SoundKeepers, acoustic properties will be mainly used for efficiency and effectiveness, due to the large linguistic diversity in Singapore’s population, particularly seniors.

Currently, a patient’s mental health is mainly assessed via self-reported data, in the form of questionnaires provided by healthcare and social service professionals to be completed by patients. Accuracy often hinges on the ability of patients to recall accurately, and each patient’s subjective assessment. Those who prefer to keep their mental condition under wraps can do so easily by providing inaccurate answers. As a result, healthcare providers sometimes struggle with reconciling questionnaire results with their intuition. This challenge is often accentuated when working with seniors who are reluctant to share their mental health issues.

“Sometimes, clients provide general or 'textbook' answers to the questionnaire we use to screen for mental health conditions, even though we believe they need help based on information they have verbally shared with us previously. When this happens, it becomes challenging for us to follow up and provide the help they need," said Danny Loke, Head of Community Mental Health Department, Fei Yue Community Services.

The goal of SoundKeepers is not to replace existing screenings, for the rapport forged through the conversations and manual screenings is vital. Rather, the voice biomarker tool will complement by helping professionals make their assessments with greater confidence and consistency by providing another reference point.

In the past decade, American, Canadian and Chinese startups have been developing voice biomarker technology. Developing a native technology for Singapore, while requiring more effort, will have advantages, making it easier to manage in accordance with national standards of healthcare data protection. All voice samples will be anonymised, stored in a secure central storage terminal and not on the cloud. In time, it is hoped that the tool will be part of the set of instruments used routinely in a regular doctor’s consultation.

“Primary care is often the first point of contact for individuals experiencing mental health issues, even though they may not be aware of their underlying mental health struggles. This tool can help primary care physicians in polyclinics identify these mental health issues early and accurately, enabling a holistic and timely management of both physical and mental health conditions in a non-stigmatising environment. This collaboration also exemplifies National Healthcare Group’s ongoing efforts towards making mental health more accessible for Singapore’s population.” shared Dr Eugene Chua, Family Physician, Associate Consultant and NHGP Site Investigator of the SoundKeepers programme.

The second component of the pilot is a 24-week community-based early intervention programme where the polyclinics will refer patients to the SSAs. Co-developed by IMH, NHGP and the SSAs, it will comprise psychoeducation, community activities, and befriending. Psychoeducation will come in the form of 7 modules that will equip individuals with a range of strategies and techniques to combat SSD. Some of these strategies include making lifestyle changes such as exercising more frequently and eating well, as well as learning ways to establish achievable goals and identifying a trusted confidant who can help them tide through emotionally challenging periods.

Seniors will also be encouraged to participate in community activities that improve sociability and mood, such as exercise, outings, skills learning, recreational activities, and assistance with job search. The simplicity of these activities is intentional: They are selected precisely because they are low-intensity, cost-efficient and short-term, which will aid adoption and scalability of the programme. Staff from the SSAs will check in with the seniors every fortnight in the first 12 weeks and monthly for the next 12 weeks.

“Mental health conditions do not always require therapy and medication. Psychoeducation which can guide people to practice mindfulness, and teach them how to solve their problems proactively as well as social prescription that can lead to a flourishing social circle and active lifestyle may be enough to treat milder conditions and what is pathbreaking is that they can be done in the community,” said Dr Mythily Subramaniam, who is also an Associate Professor at LKCMedicine.

As part of the pilot, IMH will run the community intervention programme, assess its feasibility and effectiveness, as well as train the SSAs to deliver the modules. Evaluation will involve comparing levels of loneliness, anxiety, well-being and depression literacy among seniors with SSD before and after the intervention.

“In mental health, it is important to intervene early before a condition escalates. The challenge is that there are not many effective ways to do so, which is what we hope SoundKeepers will change. Through it, seniors can potentially benefit from a seamless continuum of care – from early detection at the polyclinics to comprehensive, community-based interventions,” said Rabi’atull Syerida Juma’at, Manager of the Recovery Wellness Programme (RWP) of Club HEAL.

Randomised Controlled Trial

NHGP will helm the randomised controlled trial and recruit 300 seniors with SSD from polyclinics and GP clinics in Hougang and Woodlands. At the end of the intervention programme, the research team will study and evaluate the effectiveness of the interventions. The evaluation and findings will include comparing levels of loneliness, anxiety, well-being and depression literacy among seniors with SSD before and after the intervention as compared to those from the control group.

“The goal is for SoundKeepers, if successful, to eventually become part of the national mental health ecosystem, perhaps in the form of a ‘CREST Plus’, slotting in between CREST and COMIT,” said Mr Lee Poh Wah.

CREST and COMIT are two key government-funded community mental health programmes that serve the general public who are at risk of mental illness (in the case of CREST) and those with mental health needs (in the case of COMIT). SoundKeepers may raise the efficacy of CREST in this manner: Many who attend the mental health awareness activities organised under CREST will presumably suffer from SSD, given its high prevalence and incidence rates. Some will indeed step forward to seek help, but it is likely a large proportion will remain hidden due to the lack of tools and workflows that allow professionals to address SSD in a structured manner. With a voice biomarker tool, however, CREST professionals will be able to identify seniors with SSD more confidently and persuade them to seek help, since having SSD, by definition, does not equate diagnosis of a mental illness.

Mr Lee Poh Wah added: “It is possible that in time, voice biomarkers can democratise mental health assessment and monitoring. These could be for stressed parents of children with special needs, patients newly-diagnosed with serious illnesses, etc. Mental health is deeply interconnected with physical health, and should be integrated across the prevention, diagnosis, and treatment of every disease.”

The project will receive $5.6m in funding from the Lien Foundation, as well as in-kind support from the partner organisations. After completion, it is hoped that SoundKeepers will be expanded to more polyclinics, GP clinics, and patient profiles.

Download PDF