Western Australia's Nationals want Barnaby Joyce to quit as leader, while other state branches want nothing to do with his scandal.

The deputy prime minister, who's taken personal leave after his affair with former staffer Vikki Campion was made public, played down the significance of a nationwide phone hook-up between Nationals officials on Monday afternoon.

But on Tuesday the WA Nationals became the first state to officially withdraw support for the federal leader.

WA Nationals state leader Mia Davies issued a statement saying she told Mr Joyce he is now a distraction.

"Mr Joyce's actions have caused pain for his family but it is the ongoing damage Mr Joyce is causing the Nationals organisation that is of greatest concern to me as WA leader," Ms Davies said.

The WA branch of the party does not have any federal MPs and will play no role in any potential vote on the leadership, but it was second only to the NSW branch in political donations last financial year, receiving $1.785 million.

Mr Joyce shot back in a statement, pointing out that WA didn't have any federal MPs and that the eastern states, which had more "skin in the game", supported him.

"I find it surprising that a federal issue has so much momentum in the west, when people in the east in the National Party have, in the majority, a different view," Mr Joyce said.

Victorian Nationals leader Peter Walsh said the federal leadership was a matter for federal MPs, which the NSW Nationals agreed with.

"Further commentary on this issue in the public domain is unhelpful," the NSW branch said in a statement.

A spokeswoman for NSW Nationals leader John Barilaro, who was strongly rebuked by Mr Joyce last year after he called for Prime Minister Malcolm Turnbull's resignation, said he supported his federal counterpart.

Nationals senator Matt Canavan said there was a "level of disappointment" about the situation, with party supporters split "50/50" on the issue.

Asked about Ms Davies' statement, Senator Canavan said: "It's a sentiment I've heard from lots of people ... (but) it's obviously not 100 per cent in one direction."

Mr Joyce's NSW Nationals colleague Michael McCormack, who's been touted as a potential replacement, refused six times to explicitly back Mr Joyce's leadership on Monday.

Former prime minister Tony Abbott told Sky News the government would be weaker without Mr Joyce, who he described as a very strong and well-known retail politician.