Meta is introducing a safety update for its AI chatbots on Instagram that notifies parents when a teenager discusses suicide or self-harm. The system utilizes a specialized AI to identify sensitive conversations, which then triggers a manual review process before an alert is dispatched to parents through Instagram’s supervision tools. Meta stated it prefers to trigger false alarms rather than miss potential risks. The feature is currently live for parents using supervised accounts in the US, UK, Australia, and Canada, with a global rollout planned by the end of the year.

This safety initiative was developed in consultation with more than 75 clinicians and advisory groups, including the AI Wellbeing Expert Council. The system is designed to provide resources to both teens and parents, with the AI guiding at-risk users toward crisis helplines. Meta is also developing plans to contact emergency services for users at imminent risk. This expands upon Meta's existing suicide prevention efforts, which have resulted in over 19,000 referrals to first responders to date. These measures complement broader safety settings for Teen Accounts, including restrictions on sexual or alcoholic content, and coincide with upcoming UK government mandates for social media usage among those under 16.

Meta is introducing a safety update for its AI chatbots on Instagram that notifies parents when a teenager discusses suicide or self-harm. The system utilizes a specialized AI to identify sensitive conversations, which then triggers a manual review process before an alert is dispatched to parents through Instagram’s supervision tools. Meta stated it prefers to trigger false alarms rather than miss potential risks. The feature is currently live for parents using supervised accounts in the US, UK, Australia, and Canada, with a global rollout planned by the end of the year.

This safety initiative was developed in consultation with more than 75 clinicians and advisory groups, including the AI Wellbeing Expert Council. The system is designed to provide resources to both teens and parents, with the AI guiding at-risk users toward crisis helplines. Meta is also developing plans to contact emergency services for users at imminent risk. This expands upon Meta's existing suicide prevention efforts, which have resulted in over 19,000 referrals to first responders to date. These measures complement broader safety settings for Teen Accounts, including restrictions on sexual or alcoholic content, and coincide with upcoming UK government mandates for social media usage among those under 16.