The coalition’s control over parliament will be tested next week when Labor and the Greens attempt to move a motion of no-confidence in Home Affairs Minister Peter Dutton over the au pair saga.

Pressure is mounting on the failed Liberal leadership challenger over his intervention in two visa cases to save two European au pairs from deportation in 2015.

Despite his denial of misleading parliament about his use of ministerial discretion, Greens MP Adam Bandt said the minister has a case to answer.

Fellow crossbencher Andrew Wilkie and Labor are set to support him.

The vote is set to be tight with the government’s numbers in the lower house diminished by former prime minister Malcolm Turnbull’s resignation and Kevin Hogan’s move to the cross bench.

“Peter Dutton has failed to explain why he misled parliament and now he must go,” Mr Bandt said on Monday.

“Peter Dutton told parliament he had no personal connection with the au pair’s potential employer, but then he went on radio and said the man was a former police colleague.”

Dutton told Parl he had no personal connection with someone he helped with an au pair.

Then said on radio the person was a former police colleague.

Today he doubled down, denied misleading Parl &stood “100% behind his statements”.

You’ve been caught out, Dutton.

Time to go.

— Adam Bandt (@AdamBandt) September 1, 2018

“Peter Dutton has since doubled down, confirming that he knew the man he helped while maintaining there was nothing wrong with his statement to parliament.”

The cases relate to an Italian au pair who was linked to Mr Dutton’s former Queensland Police colleague, and a French woman who had worked for a relative of AFL boss Gillon McLachlan.

Mr Bandt says Mr Dutton misled parliament by saying he had no personal connection to the cases.

“Peter Dutton has been caught out, he has no explanation and if he won’t resign, parliament should make it clear it has no confidence in him,” the Greens MP said.

But Mr Dutton has strenuously denied any misconduct, saying he’s kept a list of Labor MPs who have come to him with “quirky” visa cases.

“Labor can ask me 10 questions every day when we go back if that’s what they want to do, but they’ll get a whack back,” the minister told reporters in Brisbane.

“To say I had some personal link or that I was acting on behalf of, you know, somebody that I was personally associated with is complete nonsense.”

Labor leader Bill Shorten said the home affairs minister was under the pump because of internal pressure relating to his role in dumping Mr Turnbull.

“It’s not what you know it’s who you know under this Liberal government. That’s not the way to run an immigration policy,” Mr Shorten told reporters in Brisbane.

Meanwhile, Opposition spokesman for infrastructure Anthony Albanese made light of Mr Dutton’s challenge, telling ABC’s Q&A: “Anyone who thought Peter Dutton was the answer must have been asking themselves a pretty weird question.”

“Peter Dutton hasn’t polled in double figures as preferred leader ever,” he said.

-with AAP