Labour needs “a big discussion” about immigration that could last for years, according to Shadow Brexit Secretary Keir Starmer.

But one thing he’s certain of is that freedom of movement between the UK and EU must come to an end.

That’s not to say workers won’t migrate from one country to another.

But speaking to regional reporters at Labour’s annual conference in Brighton, Sir Keir said there must be “a balance” between the benefits migration brings to the economy and businesses, and the impact it has on communities.

He said current rules, allowing unfettered freedom of movement, “haven’t worked well everywhere”.

It’s one of the few things Labour does have a clear view on when it comes to Brexit. Sir Keir stressed that Labour was committed to getting a deal for the UK which retained “the benefits of the Single Market and the customs union”, but there’s currently no policy on whether the country should actually remain a permanent member of the Single Market after Brexit or not.

Instead, he insists: “I’ve always said it’s the outcome that matters. And we shouldn’t get hung on the model.”

Speaking to Sir Keir, you can understand how he achieved the difficult task of uniting the Shadow Cabinet around some sort of Brexit policy. He chooses his words carefully and seems like he’d make a good diplomat (or even, dare one say, the type of person we want representing us in Brexit negotiations with the EU).

And having sorted out Labour’s divisions, he has an even more ambitious goal - uniting the country.

“Whatever we do, we’ve got to find a way of bringing the 48 per cent and the 52 per cent together,” he said.

When it comes to freedom of movement, his position is clear. The 2016 referendum, when the country voted for Brexit, means that it has to end.

Sir Keir said: “I’ve always said the referendum result and freedom of movement are bound up, and therefore freedom of movement will end when we leave the EU. That is a statement of fact.

“What we therefore need to do is to begin the discussion, both in our party and across the country, as to what does a changed immigration policy looks like.

“And that’s a discussion that has to take place over the next few months and years, across all political parties.”

He added: “We need something that works for our economy and businesses, and we need something that works for our communities. How we strike that balance is a matter we need to discuss going forward.

“The current rules have worked well in some respects but they haven’t worked well everywhere and we have to acknowledge that.

“But moving from the position we are in to the position we will need to be in is obviously a big discussion for us as a party, and a big discussion for the Government.”

It’s unclear whether the UK could remain in the Single Market while abandoning freedom of movement. However, Sir Keir insists there is currently no policy on whether the UK should stay in (after a proposed transition period comes to an end), or try to negotiate a new trade deal with the EU instead.

“I’ve always said it’s the outcome that matters. And we shouldn’t get hung on the model.”

Highlighting his speech to the party conference, he said: “But I’ve also said in my speech we shouldn’t sweep options off the table.

“That includes, at least as an option, staying in a customs union and, as an option, a new relationship with the Single Market. And whether we do it that way or do it as a bespoke trade deal is secondary to the outcome.”

More than 30 Labour figures sign a letter over the weekend arguing it makes no sense for Labour to call itself an anti-austerity party if it backs leaving the Single Market, on the grounds that this would cause enormous damage to the economy.

However, party leader Jeremy Corbyn made it clear he is deeply sceptical about the Single Market in an interview with the BBC’s Andrew Marr Show.

Mr Corbyn suggested that membership could prevent a future Labour government carrying out its policies, saying: “I would also say that we need to look very carefully at the terms of any trade relationship ‘cause at the moment we’re part of the single market, obviously. That has within it restrictions in state aid and state spending.”

I put it to Sir Keir that Jeremy Corbyn seemed to want the UK to quit the Single Market.

He insisted that his leader had said no such thing. Sir Keir said: “You can understand the concern of all Labour party Members of Parliament and party members to make sure we can implement our manifesto.”

Sir Keir added: “I don’t think there’s anything in the current [Single Market] rules that we have that does stop us implementing our manifesto, but obviously we are negotiating to a different position.”

There’s a strange moment during the interview when he refuses to rule out holding a second referendum on whether to go ahead with Brexit. It seems clear from everything he has said that a Labour government would go ahead with leaving the EU, and there are no plans to hold a second vote.

Still, when asked directly whether Labour could possibly ask the country to think again, he declines to give a straight answer - four times.

First, he said: “We have never called for another referendum”.

This is clearly true, but he wasn’t asked to confirm what the party has said so far. The question he was asked was whether Labour might consider holding a second referendum in the future, if it forms a government. So, for a second time, what’s the answer?

He replies: “We have been 15 months since the first referendum and we’ve not called for a referendum so it’s as simple as that.”

But the question isn’t about what Labour has done so far, it’s about what the party might do in the future. Could Labour hold a second referendum?

“We have never called for a second referendum and that’s our position.”

That does seem a bit clearer. Given a fourth opportunity to give a clear and unambiguous reply, he points out that the Liberal Democrats backed a second referendum and failed to make a breakthrough in June’s general election.

“This is the idea that the Liberal Democrats put to the country in this general election as their flagship policy.

“But whatever we do, we’ve got to find a way of bringing the 48 per cent and the 52 per cent together.

“And I know that sounds vague but we it is very important that we understand why those who voted Leave voted Leave, but we also understand that those who voted Remain do not want to be written out of their own future.

“And we have therefore to find something that truly reflects the vote but also brings the country back together. We can’t ignore one side or the other in this.”