The two-month-old baby died a short time later in Wagga Wagga Referral Hospital and his father Mr Crichton, 32, was detained at the home in Kooringal and taken to Wagga Wagga Police Station for questioning.

He was held for hours but not charged, released in the early afternoon as police formed special Strike Force Banco to investigate the child’s death.

The baby’s heartbroken mother Millie Butler, 21, also spoke to police

Detectives and forensics staff remained at the scene on Henwood Ave, Kooringal late into this evening and were expected to continue a painstaking examination of the house tomorrow.

Neighbours said they woke before 5am to chilling screams as the man lent against the front of his home, then was led into a police wagon.

“There was this guy just screaming, saying “I’m sorry, I’m sorry”,” Charli Gowland said.

“Even in the back of the paddy wagon he was screaming and banging.

“It was really awful.”

Police said Archie was “seriously unwell” by the time paramedics arrived but would not speculate on the cause of death.

Detective Inspector Bob Noble said they were treating the incident with a “degree of suspicion” and the man was incredibly distressed.

“It’s sort of unclear at this stage ... We can’t definitively say whether it was caused by an illness or an injury,” Det Insp Noble said.

Mr Crichton is understood to have lived in the rural Wagga Wagga community for years, while Ms Butler attended Kooringal High School.

“He’s such a tiny, young child and it does make it quite profoundly upsetting,” Det Insp Noble said.

“We are trying to treat everyone involved with as much dignity and care as we can because there’s a long way to go for these people, it’s a terrible incident.

He said detectives were still hopping to get more information from neighbours.

“If anyone in the community does have any information they should ring us here at the station immediately.”

Neighbour Simone Gowland heard the baby’s father screaming and crying as police arrived at the scene in the early hours of Monday morning.

“He didn’t put up any fight — he just sobbed as police loaded him into the wagon,” she said.

Police will wait for a post-mortem examination to determine the exact cause of the baby’s death.