Anthony Albanese has stared down a challenge from Labor MPs to oppose new legislation to stop animal rights activists targeting farms – warning “vegan terrorists” were not a Labor constituency.

The Labor leader issued the edict on Tuesday during a 22-minute address to ALP caucus over the party’s direction under his leadership.

Mr Albanese said taking the same policies to the next election will guarantee another loss.

“We are not the Liberal Party and we are not the Green party. There are two major pieces of legislation in industrial relations. Trade unions are part of our constituency. Vegan terrorists are not,” Mr Albanese said.

Several Labor MPs including former frontbencher Kim Carr, Ged Kearney and Libby Coker raised concerns the ALP was “capitulating” – a claim rejected by Penny Wong.

Shadow treasurer Jim Chalmers and Labor leader Anthony Albanese.

Mr Albanese said he was prepared to take a stand on important issues.

“We are out campaigning on Newstart. We are arguing the case about why drug testing people on welfare is a bad approach. We are campaigning strongly for the [Indigenous] Voice to Parliament,” he said.

Mr Albanese also said the ALP had “voted the right way” when it backed Prime Minister Scott Morrison’s $158 billion tax cuts.

“On tax cuts, there is nothing Labor about voting against putting more money in the pockets of cleaners. We voted the right way,” he said.

A split has emerged in Labor ranks over whether the election platform should be retained or trashed, with ALP president Wayne Swan arguing the party should be proud of the policy agenda and not “resile” from it.

This was firmly rejected by frontbencher Mark Butler, who said the ALP needed to reflect that it just lost to a “muppet show” of a government.

“When you get your backsides handed to you by Fozzie Bear and Kermit the Frog, it’s time for some serious reflection,” Mr Butler said.

Bill Shorten backed Mr Albanese’s call for an overhaul of ALP policy as “inevitable” after the election loss amid warnings the status quo will guarantee another election loss.

The former Labor leader said the party needed to show voters it had learned from defeat, but did not stipulate whether he believed franking credits – the so-called retirees tax – or negative gearing needed to be dumped.

“We didn’t get enough votes at the last election, so some change is inevitable,” Mr Shorten said.

“And I think if Labor didn’t change some of its points of view, then that would be showing we hadn’t learned the lessons from the election.”

In Parliament, the PM also tried to drag Mr Albanese into the corruption scandal engulfing the NSW Labor Party.

Mr Morrison said he would not be lectured on integrity by the Labor Party, referencing the fact Mr Albanese worked there more than 20 years ago.

“He had a desk in the office, Mr Speaker, in the Sussex Street headquarters of a party that stinks with corruption, where they get money in plastic bags and count it out on the table,” Mr Morrison said.

“With the number of Labor Party members from the NSW division who used to serve in Senator Kristina Keneally’s former government that are in jail, you could establish a branch at the Silverwater prison,” he said.