Unions have urged Gordon Brown to sack Lord Jones of Birmingham, after he criticised Government plans for a crackdown on non-domiciled foreigners living in the UK.

His claim that the proposals would threaten London's role as a world finance centre have re-opened the controversy over Gordon Brown's decision to appoint him as a Trade Minister.

Digby Jones, the former Director General of the CBI, is not a member of the Labour Party and refuses to reveal whether he even votes Labour.

But he is one of a number of independent figures appointed by Mr Brown in an effort to create a "Government of all the talents" - or GOATs, as Ministers such as Lord Jones are known in Whitehall.

Last night, union leaders accused Lord Jones of being "a business lobbyist" and called for him to be sacked.

But the Minister was unrepentant, insisting that he was doing his job by talking about the concerns of the business community.

And he was backed by Birmingham MP Richard Burden (Lab Northfield), who said: "This is the whole point of appointing someone like Lord Jones to the job."

The row focused on Government proposals to charge non-domiciled foreigners living in Britain a fee of £30,000 a year, in return for continuing to enjoy tax breaks.

Lord Jones, whose role as a Trade Minister is to promote British business across the globe, said he had frequently been asked about the tax changes on trips to India and the Gulf.

He said: "I can give you five reasons as to why you should invest in Britain before you go and invest anywhere else in Europe. But maybe there were seven and now there are five."

The proposed change meant the UK would lose its "badge as the place to come and bring your skill and work hard in the developed world," he said.

"I don't want to be in the position where one morning we wake up and people are saying, 'Digby, no matter how good you are at doing what you do, the product isn't as good as it was'."

The comment provoked an angry reaction from trade unions - who called on Gordon Brown to sack him.

Paul Kenny, general secretary of the GMB union, said: "It does not come as a big surprise that Digby Jones would embarrass the Government by failing to go along with the collective responsibility for a policy that actually originated from the Tories."

TUC general secretary Brendan Barber said: "He must make up his mind whether he is a member of the Government or a business lobbyist."