A man has been charged with the murder of his five-year-old son after the boy was allegedly stabbed at a Sydney home.

The boy was allegedly attacked with a knife by a man living at the same address during a domestic argument in Paul Place, Carlingford.

The 36-year-old father is in custody at Castle Hill and will face Parramatta Bail Court today.

In what has been described by police as a "heartbreaking incident that would affect the community", officers were called to Moseley Street at 7am.

The boy's grandmother had got him out of the house, cradling him into a car and driving away from the scene, before pulling over and calling triple-0.

He was treated by paramedics and taken under police escort to Westmead Children's Hospital, but could not be saved and died an hour after arriving, at 8.02am.

The grandmother, 60, was treated for shock.

Superintendent Rob Critchlow, Hills Area commander, yesterday described the level of violence meted out to the boy as "horrendous".

"The level of violence is horrendous ... it's probably about as bad it gets, really," he said

"To have a young child murdered in his own home brutally is as bad as it gets."

He confirmed the family was Australian and said the grandmother's actions had been "heroic and caring".

The child's mother went to hospital, but it's not known where she was at the time of the stabbing.

Supt Critchlow said the man in custody might be known to police and the weapon, believed to be a knife, had been secured.

As well as affecting a tight-knit community, he said the killing would have a "serious impact" on emergency service workers.

Witnesses have spoken of their shock.

One neighbour said: "The poor grandmother who was looking after him... can't imagine how she'll be coping".

© Nine Digital Pty Ltd 2018