Prime Minister Tony Abbott’s terrorism “clamp down” has been criticised across the political divide, with even conservatives worried about over-reach.

On Monday, Mr Abbott delivered a speech on national security at the Australian Federal Police headquarters in Canberra, in which he foreshadowed a tipping of the balance between safety and personal freedom, and called on the muslim community to do more to counteract the rising threat.

A Liberal Party-linked think tank has hit back at the PM’s plan for “stronger prohibitions” on free speech.

• PM pushes terror laws further
• National security vs job security — the equation
• Schoolgirls turn to Islamic State

Those who vilify, intimidate or incite hatred should face harsher penalties, Mr Abbott said, which the Institute of Public Affairs (IPA) has opposed.

“The federal government must rule out further restrictions on freedom of speech,” said Simon Breheny, director of the Legal Rights Project at the Institute of Public Affairs (IPA).

Freedom of speech is needed to fight against terrorist threats, Mr Breheny said.

“Australians must be free to engage in a vital debate about the threats we face without laws restricting freedom of speech,” he said.

It was Mr Abbott himself who called on Muslim leaders to speak out against extremism, although his wording has drawn criticism from the community.

“Everybody, including muslim community leaders needs to speak up clearly, because no matter what the grievance, violence against innocents must surely be a blasphemy against all religion,” Mr Abbott said.

His comments ‘scape-goated’ Muslims, said a spokesperson for The Islamic Council of Victoria, who called for more inclusive language.

“We continue to encourage the PM to reflect the set of values he espouses by utilising a repertoire of inclusive rather than divisive language,” ICV spokesman Kuranda Seyit said.

“The PM’s lack of acknowledgement or appreciation of Muslim community leaders and members who volunteer their time to consult with all levels of government and enforcement agencies misleads and incites ill-feeling from the broader community,” Mr Seyit said.

The Prime Minister’s mentor, John Howard, would have delivered a far better message to muslim leaders.

That’s the view of Professor Andrew O’Neil of Griffith University, who said Mr Howard always spoke in a “very temperate” and “very balanced” way.

“[Howard] realised the Australian government and police and intelligence agencies right across the spectrum were very reliant on the support of moderate muslims right across Australia,” Professor O’Neil said.

“ISIS is a law unto itself, everyone knows that,” the Professor said. “I don’t see what more muslim leaders can do domestically here to condemn terrorism. I think they’ve done as much as they can.”

On the political cross-bench, NSW Senator David Leyonhjelm said: “I’m just a bit worried that our national security and Tony Abbott’s job security are, kind of, linked a little bit too closely,” the ABC reported.