Linda Burney has slapped down alarmist critics of an Indigenous Voice to Parliament, saying the proposal wasn’t pulled out of thin air.

Ms Burney continued her campaign for a Voice alongside community leaders including former minister Ken Wyatt, who sensationally quit the Liberal Party over its opposition to the advisory body.

She said enabling the Voice to counsel the public service and politicians would ensure policies and programs were implemented properly.

“Many decisions that are made in government are made by senior bureaucrats, which is why the executive government was so important to be included,” Ms Burney said.

“We have taken advice from an expert legal group … the words that are there now are words that are not going to attract some of the alarmist statements we’ve heard from constitutional conservatives.”

Opponents of the Voice have taken aim at the advisory body having the authority to advise executive government, warning it would be too powerful.

But the Indigenous Australians Minister said the constitutional changes had been carefully worked through.

“The Prime Minister has made it very clear that he is open to better suggestions and other suggestions but the wording hasn’t come out of thin air,” she said.

“The wording that is in front of the Parliament, that is the question and three amendments to the constitution, have been worked through for years.”

Opposition Leader Peter Dutton visited Alice Springs to convince people it would take more than a voice to improve Indigenous outcomes and community safety.

“If you aren’t out there and you aren’t listening to what people are saying, then it’s very hard to find the solutions,” he said.

“I don’t believe a Canberra voice of 24 people who predominantly come from capital cities is going to be the solution to the problems here on the ground. If it did, I’d embrace it straight away.”

Shadow attorney-general and Indigenous Australians spokesman Julian Leeser quit the frontbench this week to campaign for the ‘yes’ vote.

Mr Leeser plans to work with the government to bring more conservatives on side, with moderate Liberals hoping his resignation will help wavering Australians get behind the Voice.

Liberal frontbencher Simon Birmingham said he would not resign from the shadow cabinet, despite indicating he would not campaign for the ‘no’ vote.

“My intention is to respect the Australian people who will go about this referendum applying their judgment to the issues they have before them,” he said.

-AAP