By Alexandra Evans, Head of Safety Public Policy, Europe

We love seeing the joy, connection, and inspiration that TikTok brings to millions around the world. Whether starting a new trend or using Duet to collaborate, the creativity of our community shines through. Fostering an environment where this creative expression thrives requires that we also prioritize safety for our community, and especially our younger community members. We share the interests of parents and caregivers in this mission, and we're committed to listening and working with external experts to inform our work.

A few months ago, we launched a global project to better understand young people's engagement with potentially harmful challenges and hoaxes. While not unique to any one platform, the effects and concerns are felt by all – and we wanted to learn how we might develop even more effective responses as we work to better support teens, parents, and educators. We also hope to contribute to a wider understanding of this area, so today we want to share more on our research and some of the steps we are taking in response.

During this project, we:

  • Surveyed more than 10,000 teens, parents, and teachers from Argentina, Australia, Brazil, Germany, Italy, Indonesia, Mexico, UK, US, and Vietnam.
  • Commissioned Praesidio Safeguarding, an independent safeguarding agency, to write a report to capture key findings and recommendations. The report, written by Dr. Zoe Hilton, Director and Founder of Praesidio Safeguarding can be read here.
  • Convened a panel of 12 leading youth safety experts from around the world to review and provide input into Dr. Hilton's report.
  • Partnered with Dr. Richard Graham, a clinical child psychiatrist specialising in healthy adolescent development, and Dr. Gretchen Brion-Meisels, a behavioural scientist specialising in risk prevention in adolescence to guide and advise us.

What teens told us about how they assess risks

Most challenges are fun and safe: the ice bucket challenge from 2014 helped to promote awareness of amyotrophic lateral sclerosis (ALS), while #BlindingLightsChallenge, dubbed the "family-bonding craze", took off on TikTok and helped bring families closer together. We heard from both Dr. Graham and Dr. Brion-Meisels that adolescence is a period that has always been associated with heightened risk taking. We wanted to understand how this propensity for risk taking motivates teens to participate in challenges to then enable us to work with experts to identify opportunities to better support them in making safe choices.

In the study, teens were asked to describe the risk level of a recent online challenge they had seen. Almost half (48%) believed they were safe, categorizing them as fun or light-hearted. 32% included some risk but were still safe, 14% were described as risky and dangerous, while 3% of online challenges were described as very dangerous. Just 0.3% of teens said they had taken part in a challenge they categorized as very dangerous.

The research also found that teenagers use a range of methods to understand the risks that may be involved in online challenges before they participate. This included watching videos of other people taking part in challenges, reading comments, and speaking to friends. Empowering teens by providing them with guidance on how to assess potential risks was identified as one of the most important measures that can help to keep them safe. Close to half (46%) of teens stated they wanted "good information on risks more widely" and "information on what is too far."

Listening to concerns from parents on the impact of hoaxes

Suicide and self-harm hoaxes attempt to make people believe something frightening that isn’t true. Hoaxes like these often have similar characteristics, and in previous cases, false warnings have circulated suggesting that children were being encouraged to take part in "games" which resulted in self-harm. Once planted, these hoaxes largely spread through warning messages encouraging others to alert as many people as possible to avoid perceived negative consequences. While the forward sharing of such warnings might seem harmless, the research found that 31% of teens exposed to these hoaxes had experienced a negative impact. Of those, 63% said the negative impact was on their mental health.

We heard too how caregivers aren't sure how to talk to teens about these hoaxes. They worry that if they mention the name of the hoax, their teen could become aware of potentially harmful behaviour they previously had no knowledge of. More than half (56%) of parents said they wouldn't mention a self-harm hoax unless a teen had mentioned it first, and 37% of parents felt hoaxes are difficult to talk about without prompting interest in them.

Strengthening our protection efforts

We have used the findings from Dr. Hilton's report to inform a review of our policies and processes, and we're making a number of improvements to build on our existing safeguards.

The research showed how warnings about self-harm hoaxes - even if shared with the best of intentions - can impact the well-being of teens. While we already remove and take action to limit the spread of hoaxes of this nature, to further protect our community we will start to remove alarmist warnings about them as they could cause harm by treating the self-harm hoax as real. We will continue to allow conversations to take place that seek to dispel panic and promote accurate information.

On one hand, it's encouraging that just 0.3% of teens say they have taken part in challenges that are very dangerous, but it's important that this isn't viewed as "job done." Having strong policies is an important part of our work to protect our community, and it's essential that these policies are coupled with strong detection and enforcement measures. We created technology that alerts our safety teams to sudden increases in violating content linked to hashtags, and we have now expanded this to also capture potentially dangerous behaviour. For example, a hashtag such as #FoodChallenge is commonly used to share food recipes and cooking inspiration, so if we were to notice a spike in content tied to that hashtag that violated our policies, our team would be alerted to look for the causes of this and be better equipped to take steps to guard against potentially harmful trends or behaviour.

New resources to support community members

One of the main findings from the report is that teens, parents, and educators need better information about challenges and hoaxes. We have worked with Dr. Graham, Dr. Brion-Meisels, and Anne Collier (Founder and Executive Director of The Net Safety Collaborative) to develop a new resource for our Safety Centre dedicated to challenges and hoaxes. This includes advice for caregivers that we hope can address the uncertainty they expressed about discussing this topic with their teens.

Finally, we have worked with Dr. Graham and Dr. Brion-Meisels to improve the language used in our warning labels that would appear to people who attempt to search our platform for content related to harmful challenges or hoaxes. A new prompt will encourage community members to visit our Safety Centre to learn more, and should people search for hoaxes linked to suicide or self-harm, we will now display additional resources in search.

As we started our initial conversations about this project, we talked with online safety experts and we kept hearing how they felt caregivers, teachers, and NGOs weren't sure how to intervene effectively. We hope the work we've undertaken with these world-leading experts can help make a thoughtful contribution to this topic that others can draw insights and opportunities from as well. For our part, we know the actions we're taking now are just some of the important work that needs to be done across our industry, and we will continue to explore and implement additional measures on behalf of our community.