The arrest of an American man who influenced the Wieambilla shooters with his online content is another example of popular platforms like Facebook, YouTube and Twitter becoming breeding grounds for the radicalisation of Australians.

The FBI this week arrested Donald Day Jr, a 58-year-old American, partly because of his online correspondence with Nathaniel, Gareth and Stacey Train, the perpetrators of a religiously motivated killing of two police officers and a neighbour in December 2022.

Day Jr shared videos, comments and messages with the trio on YouTube before the shooting, which resulted in the death of six people, and praised their actions in the aftermath.

Two police constables, a neighbour, and the three perpetrators died during the religiously motivated attack. Photo: AAP

Dr Naomi Smith, a digital sociologist at the University of the Sunshine Coast, said mainstream platforms like YouTube often have “pockets of extremist ideology”.

“Once people engage with that material, it takes them off not only deeper into the material on the platform itself, but also into other associated communities,” she said.

“The horse is out of the barn so to speak, and the people who are interested in that content have already found their way to alternative platforms.”

YouTube removed the accounts and content posted by Day Jr and the Trains after the shooting, but it can still be found on alternative video hosting sites.

The attention trap

Dr Joanne Gray, lecturer of digital cultures at the University of Sydney, said while YouTube has rules against certain harmful content and advertisers do not want their ads running alongside it, it is a “difficult moderation problem”.

“The algorithm is automated and responds to user engagement, freshness and different variables that go into prioritising what comes next,” she said.

“We know that these algorithms tend to share radicalising information.”

Although YouTube has now memory-holed Day Jr’s account and content, he is just one of many using the platform to promote and recruit to their cause.

Gray said content, like what was being posted by the Trains and Day Jr, regularly breaks YouTube’s terms of service, but is often left unchecked in the comments section.

“It’s a difficult space for them to regulate where people can communicate, talk with each other and talk about and engage with content,” she said.

“It becomes a social space where you can see communities connecting and that is good in some contexts, bad in others.”

The Trains mixed sovereign citizen-style anti-government rhetoric with Christian-fundamentalist doomsday beliefs fostered online with violent results.

Rabbit hole

Smith said digital platforms need to be accountable for the content spread online.

“It’s an expensive thing that isn’t attractive from a business perspective,” she said.

“It’s a decision about where they are spending their money, and that decision is causing harm.”

Sovereign citizen-style pseudo law has been on the rise since the COVID-19 pandemic, which supercharged communities engaging in potentially violent movements and rhetoric.

Gray said it is important to balance individual rights when trying to regulate online spaces.

“We don’t have or need a right to global public dissemination of our speech,” she said.

“If I stand in my street and say something racist or potentially radicalising, I’m only going to have a very small reach. But if you say it online, it can potentially influence millions of people.”

The perpetrator of the Christchurch mosque shootings in 2019, like the Trains, was a prolific poster and engager with extremist material online.

New Zealand’s worst mass shooting was committed by a homegrown Australian terrorist, influenced by online content. Photo: AAP

Smith said if someone in your life is heading down the conspiracy rabbit hole, it can be tempting to withdraw from the relationship.

“That often only reinforces this tendency to go further into these communities,” she said.

“As much as it is possible for people, it’s really good to try and still be present in those people’s lives.”

She said doing so can provide a connection to broader social values and norms.

“If you cut off that friendship, they won’t have access to that any more and they will only become further embedded in a community,” Smith said.

“That may be reinforcing and encouraging their worst impulses.”