Prime Minister Anthony Albanese has refused to commit to any reconsideration of the controversial stage three tax cuts, which will deliver hefty refunds to wealthy Australians.

Despite growing political opposition to the cuts, legislated by the Morrison government in 2019, Mr Albanese said Labor’s view had not changed.

“The Parliament made a decision to legislate those tax cuts, and we made a decision that we would stand by that legislation rather than re-litigate it, and we haven’t changed our opinion,” he said.

The stage three tax cuts, to implemented in 2024, will create a flat tax rate of 30 cents in the dollar for anyone earning between $40,000 and $200,000. It is estimated the change will cost the Australian economy more than $180 billion by 2031-2032.

Labor had backed stages one and two of the Coalition’s tax cuts but opposed stage three in Parliament. A year ago, Mr Albanese committed Labor to supporting the package if it won government.

“I said at the time that it wasn’t wise to consider that you knew in 2019 exactly what the economy would look like in 2024-25. But the Parliament made a decision,” he said in an address to the National Press Club in Canberra on Monday, to mark 100 days since winning the May federal election.

“We were in a situation of all-or-nothing at the time. And we voted for tax cuts because to vote against the package would have been voting against tax cuts including for people who desperately needed it at the time.”

There is growing opposition to the cuts, as Australia emerges from the pandemic with a trillion-dollar debt.

At the weekend, 18 crossbench MPs and senators revealed they were opposed to them. They include senators David Pocock, Jacqui Lambie and Tammy Tyrrell – giving the government the numbers in the upper house to overturn the package, if it wanted to do so.

Senator Pocock said the government should reconsider Jobseeker rates, rather than tax cuts for the wealthy.

“We have to be looking after people who need that support to actually be able to get back into the workforce. It doesn’t make sense to have Australians living in poverty,” he told ABC radio on Monday.

“We saw during COVID when the rate was raised, poverty came down across the country.

“There’s no reason why we can’t do that [now], particularly when we’re busy talking about giving wealthier Australians $243 billion worth of tax cuts over the next decade.”

Employment Minister Tony Burke said Australia’s debt burden made it difficult to change the Jobseeker payment, currently $46 a day.

Mr Burke said with the unemployment rate at a historic low of 3.4 per cent, there was an opportunity to reform government services.

“Because there’s fewer people in the system … there should be an opportunity to wrap as many services around people as possible to be able to help them … so I’m still very mindful of what we can do,” he told ABC radio on Monday.

But the government may not be able to lift the Jobseeker rate as part of its first budget, Mr Burke says.

“There will be things that we would want to do, that people would like us to do, that aren’t going to be possible, he said.

“It’s a decision that gets made in the budget, but it’s hard.”

Jobseeker payments will be discussed at the federal government’s jobs and skills summit later this week, with a peak welfare group calling for the rate to be lifted to $70 a day.

This would help lift people out of poverty while they searched for employment, the Australian Council of Social Service said.

“We’ve still got 935,000 people who are trying to survive on unemployment payments … who are currently being overlooked by employers and locked out of paid work,” ACOSS acting chief Edwina McDonald said.

ACOSS also wants the government to seize the “historic” opportunity of low unemployment to secure jobs and life wages.

Australia’s low unemployment rate was unlikely to remain unless the government acted, and a commitment to full employment must be at the heart of any reform, Ms McDonald said.

The council is proposing the government implement a jobs and training guarantee for people who experience long-term unemployment.

This would replace the work for the dole and employability skills training schemes which the council argues are ineffective and built to humiliate people.

A paper released by ACOSS ahead of the summit outlined subsidised employment, paid work trials and vocational training as alternatives to existing schemes to help people overcome barriers to employment.

-with AAP