Malcolm Fraser strode out of the lift, his voice booming through the timber-panelled foyer of this swank ‘men-only’ institution.

“You’ll have me thrown out of the Australian Club.”

It was just after 7pm in late May 2014 and I’d arranged to meet Fraser, who was in Sydney to promote his book Dangerous Allies at the city’s Writers Festival.

In the process I’d committed the unpardonable sin of answering my iPhone as I waited for the former Prime Minister.

• Former PM Malcolm Fraser dead at 84
• In pictures: Malcolm Fraser’s life and career
• ‘Voice for human rights’: tributes flow for Fraser

Following this awkward introduction, I spent an hour or so with Fraser.

Dressed impeccably in a sombre-coloured suit and orange tie, Australia’s 22nd Prime Minister was in fine form as he spoke about his book.

In later life, Fraser became celebrated by the ‘latte set’. Photo: Getty

Co-written with Cain Roberts, it cemented Fraser’s reputation as an unlikely radical, a man who’d distanced himself from the ultra-conservative foreign policies that he’d embraced as national leader from 1975 to 1983.

In Dangerous Allies, which Robert Manne described as the ‘most radical book ever to have been written by a former Australian Prime Minister’, Fraser argues that Australia should adopt a more independent foreign policy stance.

He calls for an Australian foreign policy that is much less reliant on the Stars and Stripes.

The man, who was Defence Minister during the Vietnam War, argues that Australia’s close relationship with the United States is highly questionable in the post Cold War era.

He challenges the status quo by arguing that forging alliances with ‘great and powerful friends’ has actually placed Australia in a more insecure position.

Fraser’s shift in foreign policy only helped to cement his popularity on the progressive side of Australian politics. In fact the man who was once viewed as the Antichrist, the Liberal hit-man who had toppled Gough Whitlam in the most controversial of circumstances in November 1975, is now more in tune wit the policies of the Greens than the party he once led.

But while Fraser became the darling of the latte set in his latter years, he was at one point one of the most divisive figures this country has seen.

‘Friendless’ after leaving office, the former PM drifted to the left in recent years. Photo: Getty

Even though he went on to win the most emphatic of election victories in December 1975 – less than a month after the November 11 ‘Dismissal’ – he was despised for years by many in the Canberra Press Gallery who never quite accepted the legitimacy of his leadership.

In researching the history of the National Press Club, I came across several anecdotes that underscored the amount of scorn once directed at Fraser.

There was the delicious anecdote of former Gallery journalists Anne Summers and Mungo MacCallum who, in May 1980, secretly signed in half-a-dozen protesters to the Press Club where Fraser was opening an exhibition of press photography.

The protesters confronted Fraser as he opened the Rothmans-sponsored exhibition (it was, incidentally, his 50th birthday), causing a near meltdown.

In the ensuing ruckus and post-mortem, the NPC board decided to suspend Summers and MacCallum under rule 35(1) of the Club’s constitution.

MacCallum challenged the ruling on the basis that he had been denied natural justice, and the Club eventually paid out several thousand dollars in legal fees.

The protest reflected the amount of derision the Press Gallery felt towards Fraser during much of his leadership.

Indeed when he lost in 1983 to Labor’s Bob Hawke, Fraser was said to have been near friendless on both the Right and the Left of the Australian political landscape.

Still he went about resurrecting his image and reputation over the years, teaming up with his former foe Whitlam in 1991 to jointly denounce the prospect of the Fairfax newspaper group being sold to foreign interests.

It was perhaps the clearest demonstration that the two men at the epicentre of the most controversial period in Australian political history had reconciled, their friendship only growing stronger as they entered their twilight years and reflected on a very different political era.

Vale John Malcolm Fraser.