The Australian Council of Trade Unions (ACTU) has decided against taking further legal action against the head of the trade union royal commission, Dyson Heydon.

Two weeks ago Justice Heydon ruled he was not biased and should continue to lead the commission, despite accepting an invitation to deliver a legal lecture at a Liberal Party fundraiser.

ACTU secretary David Oliver said in a statement “this is a political process, not a legal process and we will treat it as such”.

• Labor steps up Heydon fight
• Ten tips for Dyson Heydon on how to use email
• Labor to escalate Heydon fight to governor-general

“It remains our view that the only course of action available to the Prime Minister is to end [the royal commission] and we call on him to do so,” he said.

The ABC understands the ACTU and other unions have decided a court appeal would take up too much time and money, with a decision unlikely before the royal commission is due to finish at the end of this year.

Last week the Federal Opposition failed in a bid to ask Governor-General Sir Peter Cosgrove to intervene and revoke Justice Heydon’s commission.

The vote was tied at 34 to 34, with Palmer United’s Dio Wang and Liberal Democrat David Leyonhjelm abstaining.

The bid came after unions argued Justice Heydon’s decision to speak at a Liberal Party fundraiser created a perception of bias, even though he withdrew from the event.

In his published reasons, Justice Heydon emphasised he intended to give a legal address, not a political one.

He also determined it was not the case that a “fair-minded lay observer” would conclude that he would be incapable of bringing an “impartial mind” to the work of the commission.