He also said he aimed to support the music industry “which has been largely destroyed, by featuring local musicians at future events.”
Along with the drummers and rapper, the gathering also featured a singer performing a John Farnham cover you are the voice.
Protesters also held up signs with messages such as “Freedom doesn’t need a passport”, “Hands off our children” and “Vax free are the new 1940s Jews”.
Several marchers also held up orange signs with the words “Protect freedom of choice” and the One Nation logo.
A protester carried a globe with needles stuck in it and a skeletal face towering over two syringes.
In Hyde Park, crowds gathered outside a small stage where two long-haired drummers played before giving way to a rapper whose crude performance seemed ill-suited to rally organizers’ attempts to portray the event as family-friendly .
A One Nation candidate spoke before Mr Kelly took the stage to protest the decision to cancel Djokovic’s visa on Friday.
“We also heard last night at this late-night hearing that the reason for Mr. Djokovic’s visa being canceled was not because he did anything wrong,” Mr. Kelly.
“It was not because he had filled in a form incorrectly, but because, in the opinion of the Minister for Migration, that somehow his presence in the country would inflame what they call the so-called anti-vax sentiment.
“There’s no anti-vax sentiment,” Kelly said. “The sentiment is against taking away your freedom and forcing you to inject yourself with a substance against your free will.”
Mr Kelly concluded his remarks by saying he hoped Prime Minister and Labor leader Anthony Albanese had taken note of the rally.
“Because I tell you what,” he said. “We just started fighting. We are going to bring them every day until this next election and, as we say, send them all back.