More than 1,000 foreign prisoners, including three murderers and nine rapists, who should have been considered for deportation at the end of their sentences were freed with no action being taken, Home Secretary Charles Clarke admitted.
Mr Clarke apologised for the failure and conceded that some of the public would be angered by the oversight.
He said: "To the best of my knowledge between February 1999 and March 2006, 1,023 foreign national criminals who should have been considered for deportation or removal completed their prison sentences and were released without any consideration of deportation or removal action."
Asked if he believed all of the 1,023 prisoners would be recaptured for possible deportation, Mr Clarke said: "There are a large number of people involved. I can't say hand on heart that we will identify where each one of those is, but we are working on that very energetically."
The situation only came to light after the Commons all-party Public Accounts Committee asked questions about released foreign prisoners in a hearing last October.
Mr Clarke said: "We take it extremely seriously in every respect. The concern, possibly anger, that people will feel, I think, is entirely understandable.
"But I think it's better to acknowledge and admit it, and deal with it in that way."
The error took place because the Prison Service was not focused on the nationality of its prisoners while the Immigration and Nationality Directorate was preoccupied with other matters, he explained. At the same time, the number of prisoners in England and Wales who were born over-seas was increasing rapidly from 4,300 in 1996 to more than 10,000 at the end of February this year, he added.
"We simply didn't make the proper arrangements for identifying and considering removal in line with the growth of numbers that were t here," said the Home Secretary.
"That is a failure of the Home Office and its agencies for which I take responsibility."
Mr Clarke insisted he did not want to apportion blame between different parts of the Home Office for the error.
"I'm not going down the blame game here," he said. "Both the Prison Service and the IND failed to carry out their responsibilities in the way they ought to have done.
"They have both taken steps to lead me to be confident that it is now being done properly."
In a letter to PAC chairman Edward Leigh, Mr Clarke said: "During 2004 and 2005 we considered approximately 5,500 foreign national prisoners on release and deported approximately 3,000."
Assistant general secretary of probation union Napo Harry Fletcher said the chances of tracking down large numbers of the released prisoners was "remote".
He added: "The Home Office is facing crisis - the budget is frozen for the next three years. What this shows is the urgent need for invest-ment in multi-agency liaison so that agencies can talk to each other."
Liberal Democrat leader Sir Menzies Campbell said the Home Office's bungle was "extraordinary".
"All the Government's tough talk on crime counts for nothing in the face of this incompetence," he said. Charles Clarke apologised and conceded that some of the public would be angered by the oversight