The founding chairman of depression charity Beyond Blue says gardening was often a sanctuary when he was driving a controversial reform agenda as Victoria’s leader from 1992 to 1999.

Kennett and his wife, Felicity Kellar, have lived in their house in the eastern Melbourne suburb of Surrey Hills for 36 years, continually rejuvenating the property, which includes a putting green, a giant chessboard and a playground for the grandchildren.

“Gardening is the key to my sanity. Even when I was premier, if I could only get 30 minutes in the garden I always say it was the time where I had my best thoughts because there was no pressure,” Kennett says.

“If I was under stress or tension, getting my hands back in the soil was an antidote.”

Kennett says he and his wife first tried to buy the property at auction in the 1970s when he was running an advertising firm. “I couldn’t afford it. It was $91,500 and all I could afford was $90,000,” he says. Five years later the couple successfully bought it.

He says they had to undo the work of the previous owner, who tried to replace the English garden with native plants.

The home covers two blocks in the affluent area, about 11km from the Melbourne CBD. Last year, Kennett knocked down the house adjacent, which he owned, and replaced it with an orchard. Juvenile fruit trees planted throughout the yard will take two or three years to fully develop.

“But we had one almond last year, some figs, some passionfruit, a lot of strawberries and couple of mulberries,” he says.

Adding a smidgen of old to the new part of the home is a piece of campaign paraphernalia hammered to the fence of the orchard, with his younger-looking photo accompanying the slogan: “Keeping Victoria On the Move.” The poster is a flashback to his transformative yet controversial tenure as Victoria’s premier, a period known for privatisations and public sector job cuts as the government sought to lower debt.

Kennett says the 1999 election loss was a shock but denies his perceived abrasive personal style contributed to the defeat.

“When people disagree with you they will often slide back into using the term ‘he’s arrogant’, because they don’t agree with you,” Kennett says.

“You’ve got to understand, we were living in a different time. Victoria was stuffed: we had a massive debt, no cranes, high unemployment, and we needed a government and a bureaucracy that was prepared to govern.

“And this is the big thing with Australia right now and what we haven’t had for a long time, which is a government that governs. We’ve got enormous challenges, but everyone is trying to please everyone else. That’s not why you are elected. So I don’t care what people call me and I can’t change the past so I don’t look back.”

In June next year Kennett will abdicate chairmanship at Beyond Blue — which he says is the most important work of his life — and concentrate on the charity Torch, which encourages the creative endeavours of incarcerated indigenous Victorians in a bid to give their lives purpose and prevent them from reoffending.

Inside the house, Kennett points with pride to an Aboriginal artwork that was painted by a formerly jailed indigenous man.

“Just a stunning, stunning piece of work, so I’ve got another one of his in my office, but that’s the calibre. This guy is now out (of jail), he’s continued to do his work exceedingly well,” he says.

“That is my future,” Kennett says, adding his goal is to expand the charity outside of Victoria.

He proclaims with pride that he has taken up the piano in recent weeks, and tentatively plays two verses of Jingle Bells.

“I’ve just taken it up. I’ve got three grandchildren that play and I’ve had three lessons,” he says.

“You might think that is funny but I am learning to play … (It’s) part of my thing of living a full life.”