Labour’s commitment to high speed rail was thrown into doubt during the party’s Brighton conference – as Shadow Chancellor Ed Balls defied Shadow Cabinet colleagues and spoke out against the project.

His unexpected suggestion that the £50 billion cost of the High Speed Two (HS2) rail line might be better spent on house-building was contradicted less than 24 hours later by Maria Eagle, the Shadow Transport Secretary, who insisted Labour still backed the scheme.

But it provoked an angry response from a senior manager at Birmingham Airport – who told Labour councillors and activists that voters in the West Midlands would not forgive the party that “blinked first” on HS2.

Labour had previously warned that it would end its support plans for a new high speed rail line linking Birmingham, London, Leeds and Manchester if the cost exceeded £50 billion.

The party had also been at pains to stress that it believed HS2 could be delivered within budget, particularly as the £50 billion figure included a £14.4 contingency fund, and promised to manage the project better than the current government.

Ed Balls struck a different tone in his speech to Labour’s conference in Brighton, as he suggested for the first time that a future Labour government might not build the line at all – even if the costs didn’t rise any further.

He told delegates: “Let me be clear, in tough times – when there is less money around and a big deficit to get down – there will be no blank cheque from me as a Labour Chancellor for this project or for any project.

“Because the question is – not just whether a new high speed line is a good idea or a bad idea, but whether it is the best way to spend £50 billion for the future of our country.”

Officials close to Mr Balls confirmed after the speech that this meant other options for expanding capacity on the rail network would be reconsidered.

At the same time, Ed Miliband’s press team were insisting there was no change in Labour’s commitment to HS2, which was seen as a sign that Mr Balls has gone further than his party leader expected.

But Mr Balls confirmed that HS2 was at risk at a fringe event following his speech, telling an audience of Labour activists and journalists: “To use an economics word, money is fungible. What that means is every billion pound we spend on HS2 is a billion pounds we won’t spend on roads or cross-country train links or building new homes or new schools or hospitals.

“I think a better approach for a Labour Shadow Chancellor is to say ‘we need to know the costs are properly under control, that the benefits are really there, that this would be the best way to spend £50 billion which could be used on other transport projects or housing’.”

The apparent dithering over HS2 went down badly at a fringe event organised by Centro, the West Midlands passenger transport authority, attended by councillors, MEPs and business leader, as well as Lilian Greenwood, Labour’s Shadow Rail Minister.

John Morris, the Public Affairs Director at Birmingham Airport, told the meeting: “Whatever doesn’t get spent on HS2 will not end up being spent on other schemes in the West Midlands.

“It is about time we had a bit of a chip on our shoulder. It’s about time we said the rest of the UK deserves the same level of investment of the south east.”

He added: “It means politicians on all sides have the will to carry through the cross party consensus and not to play short term political games that could stymie the project.

“Because frankly the next election will be fought in the Midlands and my personal belief is that the electorate would not forgive whoever blinks first on HS2.”

Centro Chief Executive Geoff Inskip warned that massive and expensive transport projects in London and the South East did not receive the same level of scrutiny.

He said: “While it is right to examine infrastructure schemes thoroughly we must be fair and consistent and this should not mean that the regions lose out.”

He described suggestions that work should be carried out on the existing network instead of HS2 as a ‘sticking plaster’ approach.

Birmingham City Council also made its views known privately to Ed Miliband’s office – and received assurances that the party still supported HS2.

Labour’s Shadow Transport Secretary Maria Eagle had the difficult task of clarifying Labour’s policy in a speech the next day.

A draft of her speech sent to the media revealed she would say: “Let’s tackle overcrowding on our railways that can make the journey to work such a misery.

“So let’s free up space for new commuter services by moving the growth in longer journeys onto a new north-south rail line. Reducing journey times. Getting more freight off our roads.”

But when she actually spoke, she added the words: “That’s why we support high speed two” – making it clear that Labour still remained committed to the project.

However, Ed Miliband made no mention of high speed rail in his speech to the conference.

As the conference came to a close, both supporters and opponents of HS2 were claiming victory.

Stop HS2 Campaign Manager Joe Rukin said: “We are ecstatic that the shadow chancellor is looking serious at the cost of HS2 and whether the project is worth it, opposed to the coalition who seem to be saying, no matter what the cost that we can’t afford not to build it.”

This is not the only apparent disagreement between Mr Balls and Mr Miliband over transport projects. The Shadow Chancellor backs a third runway at Heathrow which the Labour leader is opposes.

Conservative MP Andrew Mitchell (Con Sutton Coldfield) said Labour’s dithering was putting HS2 at risk.

He said: “It is disgraceful that The Labour Party is playing political games with HS2 – one of the most important infrastructure projects in Britain for decades. By trying to move away from the building of the line, after supporting it previously, the weakness of Ed Miliband is clear for all to see.

“HS2 is essential for both Birmingham and the country. It is needed for growth, capacity and infrastructure reasons and can actually boost the local economy here in Birmingham by up to £3 billion per year.

• The HS2 project is to get a new chairman at an annual salary of £591,000.

The new boss of HS2 Ltd is Network Rail’s current chief executive Sir David Higgins, who was also former chief executive of the UK’s Olympic Delivery Authority.