Prime Minister Malcolm Turnbull said the alleged terrorists planned to "explode improvised explosive devices" in the city's business district.
He called it "one of the most substantial terrorist plots that have been disrupted over the last several years."
There is no ongoing threat, police said.
"We are approaching the Christmas season - a time when we come together in peace and love with our families. We congregate in public places for Christmas, for New Year's Eve," Turnbull said.
"It is a time of happiness and joy. These terrorists sought to disrupt it. They have been thwarted. They are in custody. They are no longer a threat to Australia's security."
One man and the woman were released without charge, federal police said in a news release.
Victoria police chief commissioner Graham Ashton said of the five suspects still in custody, four are Australian-born and the other is Egyptian-born with Australian citizenship. He said the suspects were self-radicalized.
Victoria Police have said the attack was inspired by ISIS and was aimed for Federation Square, Flinders Street Station and Saint Paul's Cathedral.