Prime Minister Anthony Albanese says he does not think Opposition Leader Peter Dutton’s decision to actively campaign against the Indigenous Voice will sink the prospects of the referendum.

“Peter Dutton has underestimated the number of Liberal and National party voters who will show generosity and goodwill and who will vote yes in this referendum,” Mr Albanese said.

The PM said he did not think that an old rule that a referendum without bipartisan support would fail would still hold.

Party loyalty has declined substantially over the past decades, making the role of political parties in this referendum the least consequential for many years, experts have said.

“I think Australia’s political system has changed substantially,” Mr Albanese said.

“Just a couple of weeks ago, in the Aston by-election, history was created with something that hadn’t happened in over 100 years, the government winning a seat from the Opposition in a by-election.”

Some members of Mr Dutton’s frontbench are understood to be opposed to aspects of his position on the referendum, especially to campaigning actively against it.

Others not bound by the parliamentary party’s decision will lend their voices to the ‘yes’ case. They include former NSW premier Dominic Perrottet and the most senior elected Liberal in Australia, Tasmanian Premier Jeremy Rockliff.

“The Premier of Tasmania is a strong advocate for ‘yes’, and in a bit over a week’s time I’ll be with him in Tasmania with Pat Farmer, a former Liberal member in the House of Representatives who will be conducting a run around Australia to support the yes campaign,” Mr Albanese told Sky News on Sunday.

“We have trade unions. We have sporting organisations. Non-government organisations are all out there campaigning for yes.

“This has never been about the politicians; this is about the people of Australia having a say.”

Mr Albanese declined the invitation to reform the wording of the proposed amendment to be put to the Australian people in the second half of this year to try to make it more acceptable to Liberal opponents.

The Voice will advise “executive government” and not just the legislature, something that critics have seized upon but which Mr Albanese notes has been signed off on by eminent jurists.

“[Nationals leader] David Littleproud declared the position of the National Party last year. It is very hard for them to argue they’re participating in this committee process in good faith,” he said.

Former deputy prime minister Barnaby Joyce, who has been active in the ‘no’ campaign, spoke on Sunday against what he said was an unproven model for recognition.

“You’ve got to prove it works … Commissions, committees, so many [Indigenous bodies],” Mr Joyce said.

“But the thing about them is if it didn’t work, we had the capacity to get rid of it and to get something better.”

Mr Albanese said that such logic had swayed people against change before but cautioned against it.

“People voted on a republic in Australia last century, at the end of the 1990s. And some of the people during that campaign … ‘Oh, well, I support the republic but I don’t support this particular model, I want to wait for something better’,” he said.

“Well, guess what? Something better hasn’t come around in their view.

“People have an opportunity here to seize this moment.”

The referendum on the Indigenous Voice to Parliament is expected between October and December.