The dark web has made headlines as thousands of Australians have their private lives exposed on blogs and forums by the hackers of Medibank and Optus.

Hundreds more Medibank customers are bracing for their medical histories to be published there later this week after hackers promised to leak more information on their blog.

What is the dark web? Why do authorities struggle to track criminals using it?

A single internet

The dark web is just the internet with extra steps to make it more anonymous, veteran cybersecurity consultant Troy Hunt says.

In other words, it is the same internet we all use but some people use  different technologies to mask their identities and locations.

“It is very much the web, a portion of the web which has many similarities, including HTML addresses and HTTPS,” Mr Hunt said.

“But TOR, is by design, an anonymity-first network.”

What’s often called the dark web usually refers to a subsection of the web accessed by people using The Onion Routing (TOR) network, originally developed for the US Navy in the 1990s and eventually refined into the TOR browser.

The purpose, as stated on the TOR foundation’s website, was to layer encrypted network traffic to make it very difficult for anyone to work out from where – or from whom – web data came.

It was a remarkable success. TOR is now used around the world for all sorts of purposes, from just browsing the web anonymously to shopping on marketplaces selling illegal merchandise.

Mohiuddin Ahmed, a senior lecturer in cyber security at Edith Cowan University, said the system works by bouncing encrypted traffic between servers, making it very difficult for anyone to pin down a source location.

“The servers are owned by individuals and groups who want to be anonymous, so it’s really hard to track down where the data is coming from,” he said.

“The TOR network hops between many networks before traffic finally reaches its destination.”

Morally neutral

Mr Hunt said one of the big misconceptions about TOR is that it’s somehow inherently immoral, largely because it has been so closely associated with selling illegal things such as personal data.

Instead, he says, it’s more useful to think about TOR as a morally neutral technology that’s used by people around the world for various reasons, some innocuous and others clearly not OK.

“Encryption is very handy if you want to send your credit card over the internet,” Mr Hunt said.

“There are always these use cases where we might not have things we want to hide, but we may have things we don’t want to share. Understanding that moral neutrality is important.”

Dr Ahmed warned anyone trying to use TOR they could be confronted with some grisly material, or even increase their risk of being hacked.

“If you do anything without maintaining cyber hygiene you might be the next victim,” he said.

“Cyber criminals are not only opening up traps for victims in the regular surface web, but also in the dark web.”

Authorities struggle

Authorities have difficulties tracking down criminals who use the technology. In the case of the Medibank hackers, a simple blog is being used to commit one of the most serious personal data crimes in Australian history.

It’s difficult in practice for authorities to track these people down because TOR effectively hides where their connections, and in some cases their web servers, are coming from in the first place.

And even when they can determine a rough location, it’s often in countries like Russia where hopes of conducting further investigations with support from local authorities is, in practice, remote.

But as Mr Hunt said, that doesn’t mean it’s impossible. Authorities in the US, Europe and Australia have had success tracking down dark web marketplaces and shutting them down.

For example, the infamous arrest of Silk Road founder Ross Ulbricht came about not because investigators cracked the dark web, but because his username was linked to a separate forum where he had posted his full name and email address years earlier.