The United Nations refugee agency has suggested Australia refrained from returning asylum seekers back to detention on Nauru, and to think of the welfare of the children.

The comments came as the High Court of Australia on Wednesday threw out a challenge to the Australian Government’s immigration detention centre on Nauru.

The case was launched by a Bangladeshi detainee on Nauru who was brought to Australia for treatment and later gave birth to her daughter in Brisbane.

• Offshore detention: possibly legal, absolutely immoral
• Churches offer sanctuary to asylum seekers
• PM praises High Court decision on Nauru

UN high commissioner for refugees spokesman Rupert Colville said the group of 260 asylum seekers – including 37 babies and 54 children – were in a fragile way and may be suffering from mental health issues.

“Of course being sent back to this situation may just make them worse,” he told ABC radio.

Mr Colville told the station that if they were returned to Nauru, they could breach the convention against torture which included degrading, cruel or inhumane treatment.

Immigration Minister Peter Dutton told ABC radio on Thursday that the cases of 267 people impacted by a High Court’s validation of Australia’s offshore processing regime would be individually considered on medical advice.

Mr Dutton said his intention was not to put any children in harm’s way.

“We have to be compassionate on one hand but we have to be realistic about the threat from people smugglers,” he said.

“We’re acting in the best interests not only for these children but children that would follow them.”

Greens MP Adam Bandt told reporters in Canberra that Prime Minister Malcolm Turnbull faced a test of humanity in deciding whether to send children back to the “mental illness factory” on Nauru.

Meanwhile, Anglican churches across Australia were opening their doors to asylum seekers facing removal back to offshore detention centres, the ABC reported.