Prime Minister Anthony Albanese has accused Peter Dutton of “weaponising anti-Semitism” in a furious spray in Question Time.

Albanese unleashed on Wednesday after the Opposition Leader ramped up attacks on the government by apparently linking increasing anti-Jewish and anti-Palestine tensions to last week’s High Court decision on indefinite detention of asylum seekers.

He also accused Albanese of “flying off again overseas”, with plans to attend the Asia-Pacific Economic Cooperation in San Francisco.

“I never thought that I would see in my lifetime a repeat of the horrific scenes that we saw and that we have read about during the course of the Second World War repeated in our lifetimes. But to see people of Jewish faith cowering in their homes, being dragged from cupboards out into the street, when children are still abducted and still held hostage, this Prime Minister needs to stand up and to be united with the community – and he’s not,” Dutton said, before pivoting.

“What compounds it is the fact that in the last 48 hours, we learn that the hapless Minister for Immigration has taken a decision … to release 83 hardened criminals, rapists, paedophiles, murderers – the rest of the detail he won’t disclose – released those 83 people to live at large in the community with the Australian public and he’s done nothing about it.”

He was referring to last week’s landmark High Court decision that holding refugees in indefinite detention was illegal.

It forced the government to release into the community more than 80 immigrants, including murderers and sex offenders. They face strict monitoring conditions, including reporting to police frequently.

On Wednesday, questions emerged about the visa status of some of the freed detainees.

Dutton’s words prompted a furious response from Albanese, his strongest yet on the ongoing conflict in the Middle East. He said there was “no issue too big for [Dutton] to show how small” he was.

“The weaponisation [of], or attempt to weaponise, anti-Semitism in this chamber and make it a partisan issue is frankly beyond contempt,” he said.

“I make no apologies for standing up against anti-Semitism and I will do it unequivocally … I have a track record on this and I’m proud of it. But I also have a track record of standing up for the rights and for justice of Palestinian people.”

Albanese said political leaders also had a role in bringing communities together.

“Jewish-Australians are fearful at the moment. The sort of activity that is occurring is scaring them and I stand with them. No one should threaten people because of their religion or their race in this country,” he said.

“But it is also the case that Arab-Australians and Islamic-Australians, and women wearing hijabs in the streets of Sydney and Melbourne, are being threatened – and I stand against that as well.”

He said Dutton, a former home affairs minister, had previously refused to comment on the High Court case because he had been responsible for the issue of asylum seekers’ detention.

“Justifying the no vote [on the Indigenous Voice], day after day, week after week … [he said] … Parliament cannot out-legislate the constitution,” Albanese said.

“But when it’s convenient, we’ll just chuck it out and pretend it doesn’t exist … The legislation that’s been thrown out by the High Court was Coalition legislation. And then they say there’s no responsibility.

“The opportunism of the Leader of the Opposition has been exposed for all to see here today. But I didn’t think that he could go this low as to link these two issues.”

Wednesday’s uproar came as federal electoral offices across the country were targeted in pro-Palestinian protests involving bloodied replicas of Gazan corpses.

Those targeted included Deputy Prime Minister Richard Marles and fellow Labor MPs Ged Kearney and Peter Khalil.

Dutton said earlier his key concern was the rise of anti-Semitism in the community. He has written to Albanese proposing a meeting with premiers and chief ministers to ensure greater social cohesion and crackdown on anti-Jewish behaviour.

“I fear that there will be a significant act within our country which will cause harm to people in the Jewish community, or the community more broadly, at a time when temperatures are rising,” he said.

A YouGov poll released in early November found a majority of Australians backed a ceasefire in Gaza.

-with AAP