Tory leader David Cameron's attempts to reform his party suffered a setback after two West Midland councillors defected to Labour.
Walsall councillors Coun Aqeel Aslam (Pleck) and Coun Haqnawaz Khan (St Matthews) accused Mr Cameron of "PR stunts" and claimed their local party was secretive and undemocratic.
Mr Cameron has attempted to improve Conservative fortunes by appointing shadow ministers for major cities, including Sutton Coldfield MP Andrew Mitchell in Birmingham.
But Coun Aslam claimed the move would do nothing to tackle the problems faced by urban areas.
The pair were criticised by Coun Tom Ansell, the council leader and head of the Conservative group, who said they were joining Labour while it was collapsing.
The Conservatives will continue to control Walsall Council, with 32 councillors. Labour, the second largest party, will now have 21.
Coun Khan said: "We have found ourselves increasingly isolated in the Walsall Tory group. Our attempts to focus the minds of the Conservative Party in Walsall on the issues that matter to local people have proved impossible.
"If the Tory cabinet was run better, less secretively, more democratically and was more willing to listen to the people it could achieve the better services that we all want and therefore improve the lives of people."
Coun Aslam said: "David Cameron talks the talk about wanting to make lives better for people in our urban areas. But what have the Conservatives done to help people here in Walsall? "We have given Cameron a chance. But nothing that Cameron has said has made me feel that the Tories have anything to offer.
"His PR stunt, appointing shadow ministers for the cities that do nothing, has hardened my opinion that my values are not those of the Conservative Party.
Coun Ansell said: "We have more than 30 councillors and only eight places on the Cabinet, so it's not possible for everyone to be a member.
"This was the system created by the Labour Government which councils have to follow.
"We had no indication that Coun Khan was unhappy. Just this weekend, he was out campaigning for the party. He has never come to me with concerns or problems.
"Coun Aslam on the other hand has always been a suspect as to whether or not he was in the right party."