McIlroy birdied the 18th hole to close with a final-round six-under-par 66 to steal the Stonehaven Cup from Scott, who bogeyed the last after leading the tournament from the first morning on Thursday.
Scott had started the day with a four-stroke buffer and never trailed until making a meal of the 72nd hole of the championship.
After driving into the middle of the fairway, he overcooked his approach then blasted his recovery shot long and was unable to save par.
McIlroy, meanwhile, drained his 15-foot birdie attempt to end a 12-month winless streak for the relieved one-time world No.1.
McIlroy, a two-time major champion, finished at 18-under-par 270, one ahead of Scott, who closed with a one-under 71.
“I wanted to get a win by the end of the season and I’ve been able to get one,” McIlroy said.
“But more satisfying than that is that I’ve been able to take on one of the best players in the world down the stretch and come out on top.
“Adam’s a phenomenal player, a great competitor and probably an even better guy.
“I feel a bit sorry that I was the one that sort of ruined the Triple Crown for him, but I’m very happy and Adam should be very proud.
“He’s a credit to the game and he’s also a credit to this country.”
Scott, who won the Open in 2009, said he was shattered to have let a national title slip.
“I’m gutted,” he said.
“I felt I’d never have a better chance to win the Aussie Open. It was tight the whole back nine.”
Scott had been hoping to join Robert Allenby as the only the second player to complete the Triple Crown in the same season after also winning the Australian Masters and Australia PGA titles last month.
But Australia’s first US Masters champion couldn’t buy a putt when he had countless opportunities to slam the door shut on McIlroy.
All up, Scott lipped out on five occasions on Sunday.
“The putter didn’t behave itself,” he said. “That’s the way golf is.”
Former champion John Senden closed with a 66 to claim outright third – and automatic qualification for next year’s British Open – at 11-under.
Rhein Gibson and Brydan Macpherson also clinched Open berths by carding 69s on Sunday to share fourth spot at nine-under.
In an enthralling final-round shootout, McIlroy wiped away Scott’s four-stroke lead by the eighth hole with a birdie, eagle, birdie run.
Scott regained his outright lead with a birdie on nine before the back nine developed into a gripping matchplay struggle between Australia’s world No.2 and the world No.6 from Northern Ireland.
Scott wasted golden opportunities to double or even triple his advantage when he missed six and four-foot birdie attempts on the 10th and 11th holes as the pressure intensified.
It became a putting duel as both rifled long irons to within close range to set up eagle chances on the par-5 13th.
McIlroy missed his putt and Scott, with the chance to go two up, left his on the lip as the pair settled for birdies.
Clinging to a one-shot lead, Scott had another big chance to clinch it on the par-5 16th only to three-putt for par when a birdie or eagle would have left his challenger two or three behind with two holes remaining.
McIlroy dodged another bullet when he got up and down for a par three on the difficult 17th hole and Scott missed yet another eight-footer for birdie.
The succession of missed opportunities came back to haunt Scott on the final hole of the tournament.