Midland MP Chris Pincher has "voluntarily stood down" from the Whips' Office and referred himself to the Tory party's complaints procedure and the police following allegations over his behaviour, a Downing Street spokesman said.
The move came after the Mail on Sunday reported that Tamworth MP Mr Pincher had been accused of making an unwanted pass at former Olympic rower and Conservative activist Alex Story.
Mr Pincher told the newspaper: "If Mr Story has ever felt offended by anything I said then I can only apologise to him."
The development followed a turbulent day for the Tories which saw three other MPs having allegations against them looked into.
Former Cabinet minister Stephen Crabb was the most prominent MP being referred for investigation under the party's new code of conduct.
And former health minister Dan Poulter was also referred to the party's disciplinary committee over allegations about his behaviour, along with fellow MP Daniel Kawczynski.
As the wave of sleaze allegations engulfing Westminster showed no sign of easing, Home Secretary Amber Rudd signalled that MPs found guilty of sexual harassment could be kicked out of the Commons under a tough new crackdown.
The Home Secretary said she wanted the sanction of sacking MPs to be considered as part of a major overhaul of anti-harassment procedures at Westminster.
She told Sky News: "I think that that is one of the things that I would encourage the review to look at. It may be the case, it may not.
"It is wrong for us to have a knee-jerk reaction based on the past week. I think what we need to do is look at the whole issue. There needs to be a procedure put in place as soon as possible."
The Home Secretary said Westminster was undergoing a "watershed moment", and insisted the end result of the spate of claims about inappropriate behaviour that has rocked politics will be positive after a "clear out".
Ms Rudd described as "disgusting" an incident in which former defence secretary Sir Michael Fallon is alleged to have lunged at a journalist and tried to kiss her on the lips.
The comments came after Jane Merrick alleged in The Observer that the incident took place after a 2003 lunch when she was a 29-year-old junior political reporter.
Asked if such behaviour was disgusting, Ms Rudd told Sky News: "Completely disgusting. Absolutely wrong. And it was right that he has stepped down."
Ms Merrick contacted Downing Street about her claims just hours before Sir Michael's shock resignation.
Labour was also embroiled in controversy as Jeremy Corbyn defended appointing an MP to the shadow cabinet who he knew had been reprimanded after claims of inappropriate behaviour.
Mr Corbyn made Kelvin Hopkins shadow culture secretary in 2016 after complaints about the Luton North MP had been made by activist Ava Etemadzadeh.
It is understood that Mr Hopkins was spoken to about why his behaviour was inappropriate and reprimanded by then-chief whip Dame Rosie Winterton when Ms Etemadzadeh made her initial complaint after dealings with the MP in 2014 and 2015.
Mr Hopkins was suspended from Labour last week and an investigation launched after it is understood Ms Etemadzadeh contacted the party again with fresh information.
The MP "categorically denies" any inappropriate conduct in relation to the activist.
As the swirl of Westminster allegations continued to spiral, Conservative former minister Anna Soubry said the Prime Minister should call in Tory whips and find out what information they have on MPs.
"Allegations of a criminal nature must go straight to the police," she told the BBC.
The Home Secretary denied that Tory whips keep a "black book" detailing the wrongdoings of MPs.
Ms Rudd said: "Of course the whips office should share the information with the Prime Minister, but I'm also saying there isn't the, sort of, black book in operation that is sometimes suggested."
Mrs May is to meet other party leaders on Monday to try to agree a way to tackle the culture of harassment at Westminster.
