Offenders are being housed at public expense in a Birmingham block of flats which is required by law to accommodate students.
Police have been called out on numerous occasions to the block, on Waterworks Road, in Edgbaston, following complaints by neighbours about its residents.
An emergency meeting has been called by local people today as part of an attempt to tackle what they described as anti-social behaviour.
Planning permission was granted for the flats in 2000 on the condition that only students live there.
However, Crossmatch Solutions Ltd, an agency contracted by the council to house offenders, is using the block to accommodate more than 25 people. An official at the Harborne- based agency yesterday refused to confirm who it had been contracted by to carry out the project.
Councillor Deidre Alden (Con Edgbaston) said she was outraged the council was funding an organisation which was flouting its planning rules.
"I do not think this sort of thing helps when you are in the process of upgrading a neighbourhood."
The Waterworks Road area has recently been identified for extra funding in order for its historical sites to become more of a focal point for tourists.
The block of flats at the centre of the row is across the road from Perrot's Folly and yards from the Oratory, which are both closely associated to JR Tolkien's time living in Edgbaston.
Crossmatch Solutions, based on Harborne Road, help people, who are often exoffenders, rebuild their lives if they have drugs or alcohol issues.
A spokeswoman for Birmingham City Council said housing officers had launched an investigation into the project.
Simon Friend, from Crossmatch, said the agency was involved with helping train people who had not been successful in mainstream education.
He said he had been asked to "answer a number of questions" by the council.
"I am fully confident that when we answer their questions we will be able to come to an agreement."
