Anthony Albanese has posed a simple question to opponents of an Indigenous voice to parliament as he calls for the nation to come together.

While thousands of people took part in invasion and survival day rallies across the country – many of them campaigning against the voice – the prime minister sharpened his pitch.

“It is something missing from our nation’s birth certificate, it’s missing from our constitution,” Mr Albanese said on Thursday.

“And I say this – if not now, when? If not now, when will this change occur? And if not the people of Australia this year, who will make this change?”

Mr Albanese said he would not engage in partisan politics on the matter, including the position taken by Greens senator Lidia Thorpe.

Senator Thorpe said she would not support the voice unless she was satisfied it guaranteed Indigenous sovereignty would not be ceded.

At an Invasion Day rally in Melbourne on Thursday, she said Aboriginal and Torres Strait Islander people deserved more than an advisory body and called for 10 seats in the senate to be allocated to Indigenous representatives.

“We have to rid racism and heal this country, bring everyone together through a sovereign treaty,” she said.

Speaking at a rally in Canberra, Ngambri and Ngunnawal woman Leah House labelled the proposed voice to parliament as “crumbs” and a distraction from the goal of First Nations sovereignty.

In Sydney, Wiradjuri woman and Greens candidate for the NSW upper house Lynda-June Coe called the push for a voice and constitutional recognition a fallacy.

Indigenous academic Tom Calma, the newly appointed Senior Australian of the Year and a campaigner for the voice referendum, said the issues facing Indigenous people and the need to move towards reconciliation could co-exist.

“It’s important that we as Aboriginal people have an opportunity to be able to contribute to policies that impact on us and programs and legislation and that’s the first step,” he told ABC Radio.

“If you support the principle of Aboriginal and Torres Strait Islander people having an opportunity to have an input into public policy and into legislation, then that should be what you support.”

At the official Australia Day celebration in Canberra, Ngunnawal elder Aunty Violet Sheridan called for national unity on the issue, urging people to vote “yes”.

“I am a proud Ngunnawal Aboriginal woman but I am also a proud Australian, so I want us to come together,” she said.

The National Party announced it would not support the voice last year, a decision which led to backbencher Andrew Gee resigning from the party to sit on the crossbench.

The Liberal Party has yet to adopt a formal position but Opposition Leader Peter Dutton continues to call for more details on the proposal.

Mr Albanese said the voice was a very clear proposition that would constitutionally recognise Aboriginal and Torres Strait Islander people as Australia’s first peoples.

– AAP