Liberal MPs have been called back to Canberra this week for a surprise meeting to determine the party room’s official position on the Indigenous Voice to Parliament.

The meeting will come only days after the party was handed a bruising defeat at the weekend’s Aston by-election.

The National Party came out early against the government’s referendum proposal to amend the constitution to enshrine recognition of Indigenous Australians and create an advisory body to the Parliament.

Opposition Leader Peter Dutton has repeatedly criticised the government’s plan, which he says lacks details. But the Liberals are yet to adopt an official position on this year’s referendum.

Shadow attorney-general Julian Leeser told the National Press Club on Monday that the Liberals’ genuinely held concerns about the referendum’s legal impact had been wrongly represented as being oppositional.

“We’ve been genuine from day one,” said Mr Leeser, a long-standing advocate of constitutional recognition.

Among the issues to be discussed on Wednesday is whether the Liberals will allow MPs a conscience vote or a free vote on the referendum question before the Parliament, or whether they will adopt an official position.

Mr Leeser said in Canberra that no decisions had been taken, amid speculation the party was likely to arrive at a compromise position against the referendum but advocate an alternative, legislated form of recognition.

Mr Leeser said the conduct of the referendum had been rushed unnecessarily, raising the spectre of future legal challenges and complex constitutional issues.

“I think the government needs to ask themselves whether there is enough support at the moment to put the referendum,” he said.

“At this point in the cycle, the [1999] republic [referendum] was tracking at 70 or 80 per cent.

“[The Voice is currently] bouncing around with support in the 50s. That is not a good sign. We have always taken the view that there’s no the worst thing that could happen is to put a referendum fail, particularly given the sensitivity of this subject.

“When I go around my community and I asked people: What do they think of the Voice? The thing people most often say to me is I love the TV program.”

Details of the referendum question to be put to the Australian people were released last week.

A date is yet to be set for the referendum yet, but will be in the second half of last year.

Mr Leeser said the Liberal party room meeting would not necessarily result in a definitive position on the issue, which is also subject to review by a parliamentary committee.

He denied that Saturday’s by-election result in Aston should prompt the Liberals to change tack on their approach to the issue; Prime Minister Anthony Albanese has criticised as disingenuous Mr Dutton’s requests for more details on the Voice.

Mr Leeser put the historic by-election loss down to voters weary of going to the polls (less than a year after the federal election and only months after a Victorian election) and the impact of retiring MP Alan Tudge’s personal vote. But he said there were also lessons in the surprise 6.5 point swing to Labor.

“We need to redouble our efforts to ensure that we as a party to meet the aspirations of Australians,” he said.

Mr Leeser said the Voice had been presented as an urgent matter by Mr Albanese for political reasons but said its constitutional impact could have lasting consequences.

“The unintended consequences that can cascade through our system of government,” he said.

“To argue for changes to the government’s amendment does not mean that you oppose the Voice

“It means you want to ensure it doesn’t detract from a system of government.”

Mr Albanese told reporters in Adelaide after the speech that Mr Leeser’s philosophy of constitutional conservatism meant his long commitment to constitutional recognition had not delivered results when he was in government.

“That is why we need to get this done,” he said. “To not put it to the Australian people is to not advance it, is to, by definition – if you don’t run on the field you cannot win. You cannot succeed.”