Anthony Albanese has returned fire over union leader John Setka’s threat to withdraw ALP funding, warning Mr Setka is wasting his money on a legal challenge and has shamed the Labor Party.

The Labor leader said he won’t back down from his plan to expel the CFMMEU leader from the party.

“I think that John Setka, over a long period of time, has brought the party into disrepute,” Mr Albanese told Sky News on Monday.

“There’s just a range of public commentary going back for a few years now that aren’t consistent with Labor views and aren’t consistent with the views we want to put out there about representing all Australians.

“He used his kids to send a very frank message to the ABCC (the Australian Building and Construction Commission), he gave a speech at a rally in Melbourne where he spoke about knowing where people live and people won’t be able to go to their local sporting clubs and their local activities. It essentially was an attempt at intimidation that can only be viewed in that way.”

The New Daily has been told the Federal Police investigated Mr Setka’s Melbourne speech over the intimidation claims, and informed him that no crime was committed.

Mr Albanese also rejected Mr Sekta’s claims that his reflection on Rosie Batty’s work and its impact on the legal system were not disparaging.

“It was very clear that was the context. He says he was misinterpreted by I certainly can’t place any other interpretation,” Mr Albanese said.

“The person sent out to defend him on those issues, of course, said that he never mentioned Rosie Batty – so he obviously wasn’t paying a great deal of attention.”

The Victorian branch of the CFMMEU, which Mr Setka leads, would have conservatively donated $4 million to the ALP’s and the ACTU’s election coffers in the past three years.

“What I said was no more money to the ALP. We are freezing everything. Not one more cent,” Mr Setka has told The New Daily.

“The $12 million the ACTU spent, they might as well have gone down the racetrack and gone to the Crown casino and got a better return. It’s pretty bad.

Mr Setka has also said he will take on Labor in court if Mr Albanese tries to expel him from the party without due process when the ALP executive meets in July.

“I will challenge it. It’s up to the members because it would be a costly exercise,” Mr Setka said.

“I am a little bit old school, I actually think you should have actually done something before you get cooked on a spit about it.

“Good luck to them. It’s going to be a long, drawn-out thing.”

Mr Albanese said on Monday a legal challenge would a waste of union members’ money.

“Mr Setka is entitled to defend his position … but the Labor Party is entitled to determine who joins us,” he said.

“That’s been dealt with in the courts before, so I’d say that anyone who wishes to waste money on legal cases, particularly using other people’s money – that is, the members of the union’s money – to fight legal cases which have no prospect of being successful.”

Mr Albanese said Labor sought legal advice before his announcement of the plan to expel Mr Setka.

“That’s why we did the suspension and then a process for the expulsion,” he said.

“It’s well documented what we needed to do and we are doing that on July the 5th. I’m very confident that the national executive will support the motion to expel John Setka.”

Mr Setka separately faces criminal charges for using a carriage service to harass a woman. He has previously indicated he will plead guilty to sending the text messages, which included calling the woman “a dog” and a drunk.