TREASURER Tim Pallas says building the estimated $9 billion project will be hard given the federal government refuses to help fund commuter rail, but he says Labor won't give up on it.

"I want to be frank with people, it will challenge the government, particularly in the current economic environment," Mr Pallas told The Age on Wednesday. Labor has pledged $300 million in its first budget for Metro Rail to fund planning, design and early works. It would fund a third of construction costs, with the federal government and the private sector hoped to equally fund the remaining two thirds. But Prime Minister Tony Abbott has repeatedly said $3 billion of commonwealth money set aside for the East West Link - which Labor promises to dump - is not transferable to other Victorian projects. "I've got to say the few discussions I've had with the federal government, they've been pretty adamant about this," Mr Pallas said. "Quite frankly it is disappointing for rational transport infrastructure delivery if the federal government distorts its investment into only one form of infrastructure such as roads." Labor hasn't said when it will build Metro Rail, which will double the city loop in size and add five new underground stations, transporting an extra 20,000 people during the peak. Opposition Leader Matthew Guy says Labor is backtracking on one of its own transport policies after scrapping the coalition's East West Link. "For the new government to scrap the biggest road project in Australia and now put in doubt the biggest rail project in Australia, shows the Andrews Labor government is all talk and limited action," he told reporters. He said federal governments had always made it clear they wouldn't fund suburban rail projects and Labor could either blow the state's budget to build the metro rail link, go for the coalition's cheaper Melbourne rail link proposal or build neither. Victorian Greens leader Greg Barber said the government had billions in projected surpluses over coming years and could build metro rail if it wanted to.