"Both Papua New Guinea and Australia are in agreement that the center is to be closed," Papua New Guinea Prime Minister Peter O'Neill said on Facebook.
The detention center has been housing asylum seekers since 2012.
Speaking from inside the detention center on Manus, Lebanon refugee Ahmad Trad told CNN he hoped the closure would be good for him and his fellow inmates on the island.
"I don't have anything to lose ... we suffer enough, really really, these two years, they've been very horrible for me," he said.
Following the announcement of Manus Islands' closure, Australian Immigration Minister Peter Dutton said in a statement those refugees still in the facility would never be settled in Australia.
"It has been the longstanding position of this Government to work with PNG to close Manus and support those people as they transition into PNG or return to their country of origin," he said.
When asked about the minister's comments, Trad said he was destroying souls. "No future here, no life, no job, no society, no safety, nothing at all," he told CNN.
Former employees called for closure
This latest announcement comes less than a day after more than a hundred former and current staff members at Australia's offshore detention camps have called for all refugees to be moved immediately, saying children's lives are being "destroyed."
In a public statement detailing stories of abuse and neglect on Nauru and Manus Island, 103 detention camp staff said the government must bring the refugees and asylum seekers back to Australia.
"It's critical, it's beyond that now, we don't need another inquiry that will prolong any solution. We need them to be immediately brought to Australia and the world needs to hold Australia accountable," said Eliza Seaborn, a former senior child and youth recreation officer at Nauru with Save the Children Australia.
No emotion
Katie Price, a former child and youth recreation officer on Nauru said children had ended up devoid of emotion. "I watched these children's lives being destroyed by these camps," she said.
Australian Immigration Minister Peter Dutton said in a statement he understood that people wanted every refugee released from detention.
"The fact is I have to make decisions to remove people as quickly as possible as we have done with every child in detention, but it must be done in a way that we don't see boats and deaths at sea recommence," he said.
In a series of tweets on Tuesday, the official Nauru government Twitter account said the reports of abuse were fabricated.
Former carers: Children don't dream anymore
Speaking to CNN, Seaborn said the allegations in the report were the "tip" of the what was going on at Australia's detention centers.
"The government is accusing us of fabricating a lot of these incidents. I've been there, I've seen it and it's true what happened," she said.
Seaborn said when she'd first gone to Nauru in 2013 she had met normal, friendly children who had dreams of a career or a new life in Australia.
"When we left, you wouldn't hear dreams or hopes anymore. They were very withdrawn, they were being verbally abused by security guards on a daily basis," she said.
Whenever her or her fellow workers would report sexual abuse or harassment, Seaborn said there was little follow up or any noticeable changes to the running of the camp.
"I've seen a child under 10 years old just fall down on the ground and scream and squirm and scratch her face until it was bleeding. Behavior like that is a level of anguish that is very hard to describe," she said.
Stopping the boats
Since 2012, refugees arriving in Australia by boat have been transferred to offshore centers in small pacific nations such as Nauru and Papua New Guinea's Manus Island to make it clear they would not be settling in Australia.
Successive Australian leaders have defended the policy as humanitarian, to stop people drowning at sea, despite multiple reports of abuses at the country's offshore camps.
Between 2007 and 2013, the Australian government says at least 1,200 people lost their lives trying to make the journey over water.