Mr Warren died nine days ago, aged 61, after battling lung cancer for two years.
Hundreds of soccer fans listened to the funeral broadcast outside the cathedral, many wearing soccer shirts and scarves.
His nephew and godson, Jamie Warren, was among the mourners paying tribute to him.
“Our family is exceedingly proud of John, of what he achieved, certainly, but more so because of the type of person that he was,” said Jamie Warren.
“John was so firmly committed to everyone in our family, concerned for our welfare, putting our own problems and worries in front of his life threatening condition.”
Johnny Warren was also well-known for being a soccer commentator on SBS television.
SBS presenter Les Murray said the death of his friend is an enormous loss for Australian soccer.
“What do we do now? How do we do what still has to be done without you?,” said Mr Murray.
“You told us to believe and to have faith, and we do. But please John this is not easy for us. We believe and we do have faith, but it’s hard if you’re not here to guide us.”
His emotional but exuberant state send-off was highlighted by a samba-style Brazilian parade.
Attendees included billionaire Frank Lowy, politicians including Neville Wran and Olympic boss John Coates.
Mr Warren is credited with popularising the game of soccer, or football, within Australia.
“We all thought football and the passing of a footballer would never have such an impact in your country. But we were wrong, and you told us,” said Les Murray.
The former Socceroos captain played 42 times for his country over his 10-year international career, which included Australia’s only World Cup finals appearance in 1974.
He was awarded a FIFA Centennial Medal of Honour this year, joining the ranks of such international stars as Brazil’s Pele.