All city tram routes have been affected. "The streets affected... are all major tram corridors in the city. We advise passengers to allow extra time or use the City Loop," a spokesman said.

The crowd stopped just after 11am outside the Melbourne Magistrates court where CFMEU secretary John Setka and his deputy, Shaun Reardon, are appearing for a committal hearing after allegedly blackmailing two Boral executives.

Speakers at the rally urged those gathered to show support for the two high-profile union members.

“They care that all of us come home from work. Isn’t that important comrades?” said Mr Hilakari.

“Do we support Shaun and John?” he asked to loud applause and cheers.

The crowd then looped back moved toward Lonsdale Street before arriving at its final destination next to Federation Square and Flinders station before midday.

Those attending the rally were from a broad range of backgrounds and professions, with some bringing their young children along to watch the protest.

Selene Sharp and her four-year-old son Mitchell marched towards the front of the huge crowd.

“I’m here to support all the workers and the rules they want changed,” she said. “Flexibility at work, good childcare... all the things that are important and are important for my son.”

The Melbourne accountant is a member of Financial Services Union and said she had attended a rally for penalty rates recently but it was nowhere near as big.

“(The rally) shows people from all different backgrounds fighting,'' she said. "It’s not just about the CFMEU and fluro jumpers, it’s bigger than that.”

Mich-Elle Myers, a member of the Maritime Union of Australia, was a wharfie for nine years and said she was marching after seeing too many friends lose jobs unfairly over the years.

“The war on workers is on,” she said.

Victorian Trades Hall Council Secretary Luke Hilakari told the crowd that the system was "fundamentally broken for working people.

"It's been broken for far too long," he said. "Wages are at record lows yet company profits right now are at record highs.

"Last night's budget did nothing for working people. Malcolm Turnbull threw crumbs to Australian workers. Crumbs are no longer good enough for working people - we want the whole cake. We want our penalty rates back, we want equal pay for women.

"We want to make sure our jobs are safe," he said.

"That's why 60,000 to 80,000 of us are on the streets today. This is a big effort for the union movement. This is going to be the biggest campaign we've seen in more than a decade. Things need to changes comrades and it starts with us."

Melbourne University staff will also strike for four hours on Wednesday as part of their long campaign for a new enterprise agreement, and Melbourne Port workers have reportedly walked off the job.

Earlier on Wednesday, Victoria police were bracing themselves for a crowd of 50,000 to 60,000 people.

“Victoria Police is providing a significant police presence and is working closely with the City of Melbourne and Trades Hall Council to provide a safe environment,” a spokesperson said.

“We respect the right of the community to express their views peacefully and lawfully, but we will not tolerate people breaking the law.

“Police will be present to monitor the rally, manage traffic disruptions and ensure public safety.”