A Sydney man charged with murdering five members of his extended family will face a fourth trial next year after the jury in his third trial failed to reach a verdict.

Robert Xie stood trial in the New South Wales Supreme Court for nearly 10 months, accused of using a hammer to bludgeon the victims to death.

The bodies of newsagent Min Lin and his wife Lily Lin, their children Henry, 12, and Terry, 9, and Mrs Lin’s sister Irene were found inside their North Epping home in July 2009.

• Economists eye growth rebound
• Four teens cut from car crash
• Families arrive in Mexico

Mr Lin was Xie’s brother-in-law.

Xie was interviewed extensively by police not long after the deaths, and again the following year.

He was arrested and charged with the murders in May 2011.

Xie first stood trial in May 2014, but the jury was discharged for legal reasons.

A new jury in a second trial was also discharged, this time due to the ill health of the trial judge, Peter Johnson.

The current trial began in February, before Justice Elizabeth Fullerton.

Jurors retired to consider their verdict on November 12 and were soon sequestered at a Sydney hotel to help them focus on their task.

They first indicated they were unable to reach a unanimous verdict on Friday.

On Monday, they restated that position and were granted permission to hand down verdicts of guilty or not guilty, by a majority of 11 to one.

“The circumstances have arisen in this trial in which I may take a majority verdict,” said Justice Fullerton at the time.

“No other number but a majority of 11 will suffice,” she said.

Robert Xie’s wife Kathy Lin (centre) leaves the NSW Supreme Court after the jury was dismissed in her husband’s murder trial. Photo: AAP

However, on Tuesday the jury again stated it could not reach a verdict.

It prompted Justice Fullerton to discharge the jury.

“I am absolutely confident you have each given your all to what has been a lengthy trial,” Justice Fullerton said to the jury.

“The law should not be something you are fearful of.

“You know what your job was … that you could not return verdicts is not something that you should not take as a personal failure or any of you as a collective failure.”

Justice Fullerton said the trial was an example of the “criminal justice system in operation” and said to the jury “on a personal note, I thank you”.

Xie appeared composed when the foreman of the jury revealed it could not reach a verdict.

He dabbed his eyes with tissues on several occasions but did not appear to cry as he listened intently.

His wife Kathy Lin cried in court and was comforted by a Salvation Army chaplain.

Xie was on the verge of tears as he collected his files and was taken into custody.

Outside court, Ms Lin said “my husband is innocent.”

“We never ever give up,” she said.

Xie now faces another trial next year.

– ABC