Conservative leader Michael Howard will be reported to the police for "inciting racial hatred", the chairman of the Gypsy Council said.
Charles Smith said he planned to report Mr Howard to Scotland Yard some time next week.
He was speaking at a meeting in London as gipsy groups unveiled a seven-point charter calling for better rights for travellers.
Mr Smith said: "I think it's despicable what Michael Howard is doing. I think Michael Howard is inciting racial hatred."
Last month, Mr Howard pledged to stop travellers "bending" planning laws and using their right to family life under the Human Rights Act to build and remain on unlawful settlements.
Trespass by travellers would be made a criminal offence under a Tory Government, as part of a package of measures designed to crack down on illegal encampments.
