Home Secretary John Reid insisted yesterday that an additional 8,000 prison places are needed to help reassure the public that prisoners are released early for legitimate reasons.
Mr Reid added that the Government needed to be "smarter and more efficient" in how it protected the public.
He was speaking during a wide-ranging interview about his plans to reform the Home Office, "rebalance" the criminal justice system and get a grip on immigration and failed asylum seekers.
In the past fortnight Mr Reid has outlined proposals to increase the minimum terms served by dangerous offenders sentenced to life, create more prison places and reform the Immigration and Nationality Directorate.
Other measures include embarkation controls and uniformed border patrols.
Mr Reid said: "The public has to believe that those decisions (to release prisoners early) have been made not because we have run out of prison places but because they are legitimate in themselves and that the serious, violent and dangerous sexual offenders will be put away for the protection of the public for as long as is necessary and places provided."
He added: "That is why I wanted extra prison (places). It is the sine qua non, not only for protecting the public from the dangerous offenders but for legitimatising taking out of prison and putting into community service or fines those who ought not to be in prison in the first place."
Mr Reid also said he did "not for one moment" regret calling the sentence of paedophile Craig Sweeney "unduly lenient" - comments which reportedly angered Attorney General Lord Goldsmith.
He predicted: "I think the way that we are looking at the sentencing now will make sure that in future cases there will be no restrictions on anyone involved which cause unduly lenient sentences to be granted.
"I am very pleased with that because the public believe that the present method of issuing sentences where you appear to take a number, then halve it, then take a bit off it, then add back the number you first thought of, is at best totally bewildering to the public and at worst gives the impression of undue leniency."