Demolition crews are continuing the difficult task of dismantling a century-old Sydney hat factory gutted in a devastating fire, before the search for any human remains can start.

The seven-storey, 111-year-old building was destroyed by fire last week in inner-city Surry Hills.

The building’s roof and two of its street-facing walls partially collapsed during the blaze, which lit up the city skyline on Thursday night.

On Tuesday, a 60-tonne long-armed excavator hammered at the remaining unsupported walls of the taller building, sending bricks crashing into the street as crowds watched from outside an exclusion zone.

A neighbouring three-storey building was also destroyed. Fire crews will begin safety operations on its remains after demolition works are complete.

NSW Fire and Rescue Superintendent Adam Dewberry said the timeline of the demolition works would be reworked if crews encountered problems during the precarious operation.

The site sits above gas, electricity and water mains, which all needed to be considered as the building is pulled apart.

The derelict factory was known to regularly house rough sleepers and up to 15 people were in the building the night before the fire. Two of them are still unaccounted for.

Once the area is safe, police will take possession of the site and cadaver-detection dogs will comb the area for evidence.

Arson squad detectives investigating the cause of the fire spoke with a fourth teenage boy about the incident on Tuesday.

Three other teenage boys handed themselves in to police over the blaze last week.

No charges have been laid.

– AAP