The morning of 11 November dawned crisp and blue. At nine o’clock, Fraser, Phillip Lynch and Doug Anthony went to Whitlam’s office. Whitlam offered again the option of a late Senate election; Fraser rejected the offer. Fraser again offered his compromise — elections for both houses in May; Whitlam refused. Fraser looked at Whitlam, and sounded the warning that the Governor-General, Sir John Kerr, had not given him.

“You know, Prime Minister, the Governor-General can make up his own mind what to do. You can’t necessarily assume he will do just as you advise.’

Whitlam dismissed the suggestion: “Nonsense,” he said.

• In pictures: Malcolm Fraser’s life and career
• ‘Voice for human rights’: tributes flow for Malcolm Fraser
• Lib elders slam child detention
• Vale Gough Whitlam: political titan farewelled

Australia has lost a political giant and “true gentleman”. Photo: AAP

Fraser left and was preparing for a joint party meeting when the telephone rang. It was 9.55 a.m. On the other end was Sir John. “He opened by saying that this would have to remain confidential,” recalls Fraser. What followed was one of the most momentous phone calls in the history of Australian politics.

Fraser turned over the agenda for the joint party meeting that he had in his hand and took notes on the back. That piece of paper remains in Fraser’s possession today. It records, in Fraser’s unmistakeable scrawl, the substance of the undertakings that Kerr sought from him.

If Whitlam was dismissed and Fraser was made Prime Minister, would he agree to call a double dissolution election? Would he agree to run a caretaker administration, making no policy changes? Would he obtain supply straightaway? Would he advise a dissolution on that very day? There was a further condition—less expected, but highly significant. Kerr asked Fraser to guarantee that no action would be taken against the ministers of the Whitlam government over the loans affair, and that there would be no royal commission.

Fraser understood this commitment not to pursue Whitlam and his ministers to be a permanent undertaking, applying not only for the period of the caretaker government, but for good. He was glad to give the commitment. Although only months before he had called for a royal commission into the loans affair, that had been before he knew there was to be an election.

Fraser held firm to this promise, despite the urgings of his colleagues and claims that there was evidence that pointed to more than just incompetence in the loans affair. In the months following, keeping this promise and refusing to even consider investigating allegations against Jim Cairns, Whitlam and Lionel Murphy cost Fraser one of his most valuable ministers, and a man he still respects—his first Attorney-General, Robert Ellicott.

While Kerr extracted the commitment, it accorded with Fraser’s convictions. Fraser had accused Whitlam and his ministers of corruption, but he had meant political corruption. If there was to be an election, then Whitlam and his ministers would be judged by the people, and pay a political price. That was appropriate, Fraser believed. There should be no other cost. Here, too, there was a principle at stake. “Governments pursuing previous governments through the courts is the kind of thing that happens in a dictatorship,” he says today.

The phone call with Kerr ended. Fraser wrote the time and date on his note of the conversation, and signed it. He knew now that today was the day that Kerr intended to act. He still wasn’t certain that, once faced with an ultimatum, Whitlam would not back down and choose to go to an election as Prime Minister.

There has been some controversy about this telephone conversation. Kerr later denied that it ever took place. There can be little doubt that it did. Fraser’s Principal Private Secretary at the time, Dale Budd, remembers seeing the signed note of the conversation on Fraser’s desk later on 11 November. National Country Party member of parliament Peter Nixon was in the room at the time of the call and remembers it, although he heard only one side —Fraser’s monosyllabic responses.

So why did Kerr deny that the call took place? Fraser believes that this was a measure of the pressure Kerr was under in the wake of the dismissal, with conspiracy theories being floated. There was nothing improper in the call, Fraser thought. Kerr was simply doing the sensible thing: making sure that if he sacked Whitlam and called on Fraser, he knew how Fraser would act. Fraser says, “Look, denying it was a sign of weakness. There are many signs of weakness in his character, and that is probably true of most of us. It was an error of judgement and it was a weakness not to explain it how I’ve explained it.”

After the telephone call, Fraser walked into the joint party meeting, which started late, at 10.30 a.m. The minutes show that at eleven o’clock, the members stood for two minutes’ silence to mark Armistice Day. Lynch briefed the room on the meetings that had been held between banks and senior public servants.

Anthony told the room that ‘things are building to a crisis in the next twenty-four hours and to stay firm and resolute’. Fraser asked his colleagues not to press him on current events. Matters would soon be resolved.

The events of the rest of that day have been recounted many times. Parliament sat at 11.45 a.m. and Fraser moved a motion of censure against the government for planning to continue without supply. At 12.34 p.m. Whitlam delivered his last speech as Prime Minister, moving an amendment censuring Fraser. Whitlam had announced his intention to call a half-Senate election, and was planning to call on Kerr to arrange it. Kerr, meanwhile, was preparing for the dismissal and had arranged for Fraser to go to see him immediately after Whitlam. There was a mix-up. Fraser arrived first and was kept waiting.

It was always my understanding that Whitlam had a choice, that he could, at that very last minute, have agreed to call an election, and he would then have contested that election as Prime Minister.

Kerr and Whitlam have given differing versions of what occurred between them. According to Kerr’s version, he warned Whitlam, saying that he would be dismissed if he continued in his determination to govern without supply. In this version, Whitlam had a few moments to reconsider. Although the documents dismissing him were ready and waiting, he was not actually dismissed until they were handed to him. He could, had he been able to think very quickly indeed, have gone to the election as Prime Minister. According to Kerr, Whitlam responded by looking at the telephone and saying, ‘I must get in touch with the Palace at once’, thus confirming that, if he was given the chance, he would advise the Queen to dismiss Kerr. It was after this that Kerr handed him the documents, and formally dismissed him. According to Whitlam, there was no chance to reconsider, and no mention of telephoning the Palace: Kerr dismissed him without warning. Fraser, who walked into the room minutes later, says today that he believes Kerr’s version. “It was always my understanding that Whitlam had a choice, that he could, at that very last minute, have agreed to call an election, and he would then have contested that election as Prime Minister.”

How would we remember the events of 11 November 1975 if this had happened? Would Whitlam still be a Labor hero? Would Fraser have been a better loved Prime Minister? Would Kerr have been so reviled? Whatever happened in those few moments, Whitlam seems to have been stunned. He did not return to parliament to speak to his advisers; he left and went to the Lodge, and rang his wife, his departmental head and office chief and invited them to join him. When they arrived they found him eating his lunch. It was steak.

Meanwhile, Fraser was shown in to Kerr’s office. Kerr again asked him for the commitments agreed to earlier that day by telephone. Fraser gave the same answers and was sworn in as Prime Minister, returning immediately to Parliament House to organise the passing of supply. In an atmosphere of deceptive quiet, the word was passed among the coalition senators: “Do not allow your expression to change, but Whitlam has been sacked. Malcolm is Prime Minister and we will get the budget as quickly as we can”. Labor’s senators were not briefed: they didn’t know the government had changed. As planned previously, Labor moved that the budget Bills be declared urgent and put immediately, expecting the opposition to block as usual. Reg Withers, on behalf of the coalition, agreed to what Labor proposed and the budget was quietly passed at 2.24 p.m. Too late, the Labor senators realised what had happened: the opposition was no longer the opposition but a caretaker government, and it had just been given supply.

Crowds were gathering on the lawns in front of Parliament House as Fraser left at 3.15 p.m. to advise Kerr to call a double dissolution election. Protesters gave him fascist salutes and pounded on the roof of his car. Kerr dissolved parliament and David Smith read the proclamation on the steps, followed by Whitlam’s famous speech: “Maintain your rage”. Fraser’s office—still that of the Leader of the Opposition—was a flurry of senior bureaucrats, many struggling with the fact that when they were told the Prime Minister wanted to see them it meant Fraser, not Whitlam. In the halls of Parliament House, Labor ministers were evacuating their offices, loading boxes with papers. There was confusion and panic.

I don’t remember the emotions much at all.

How was Fraser feeling? Fraser remembers his dominant emotion as relief. ‘But, really, I don’t remember the emotions much at all.”

That night Fraser, Tony Staley and Fraser’s personal staff dined in the Lobby restaurant. The mood was subdued—relief and weariness in equal measure. Tamie Fraser arrived at Canberra airport and was met at the stairs of the plane by the manager of the airport, who steered her away from an angry press pack and into a Commonwealth car.

On the way back to their hotel later that evening, Tamie noticed a flag on the front of their car and realised they were riding in the prime ministerial vehicle. She told Fraser she didn’t think it was right; after all, he was only the caretaker Prime Minister. He hadn’t been elected. Fraser told her it was necessary: it was the position that was important; the position had to be protected. They arrived at the Rex Hotel, where they were staying, and security people shepherded them through a kitchen door, down a long passageway then into the service lift. When, finally, they closed the door to their room and were left alone, Tamie turned to Fraser and said, ‘I can’t believe it. It’s hideous. Is this the freedom we’ve been fighting for?’ Then she realised. She knew. Their lives were no longer their own.

This is an edited extract from Malcolm Fraser: The Political Memoirs by Malcolm Fraser and Margaret Simons, published by MUP