The prime minister believes expletive-laden text messages sent by Sir Gavin Williamson to a colleague are "unacceptable", No 10 has said.
A series of messages sent by Sir Gavin to ex-chief whip Wendy Morton were published on Sunday.
Downing Street confirmed the prime minister knew about a complaint against Sir Gavin before appointing him to cabinet, but had not seen the messages.
The Liberal Democrats have called for Mr Williamson to be sacked.
The Sunday Times published texts said to have been sent by Sir Gavin to Ms Morton in which he accused her of excluding some MPs from the Queen's funeral.
Ms Morton lodged a complaint against Sir Gavin, which is currently being investigated by Tory HQ.
Cabinet minister Oliver Dowden, a key ally of Prime Minister Rishi Sunak, said Sir Gavin sent texts to Ms Morton "in the heat of the moment at a very difficult time".
Mr Dowden told the BBC's Laura Kuenssberg Sir Gavin was "frustrated" at the then government, but accepts that he "shouldn't have sent them and he regrets it".
It was "no secret" that Sir Gavin and other backbench MPs did not enjoy "a good relationship" with Ms Morton.
But he added the messages were "not acceptable" and "should not have been sent".
Former Conservative party chairman Sir Jake Berry said he had told the prime minister of Ms Morton's complaint.
Rossendale and Darwen MP Sir Jake says he told Mr Sunak about the complaint accusing Sir Gavin of "bullying and intimidation" on 24 October.
Mr Sunak succeeded Liz Truss as prime minister on 25 October and appointed Sir Gavin a minister of state at the Cabinet Office.
Ms Morton is yet to hear how her complaint will be acted upon
Sir Gavin is reported to have sent her the texts in the run-up to the Queen's funeral in September.
In them, he appears to have complained that MPs who were not "favoured" by Ms Truss were being excluded from the ceremony at Westminster Abbey.
He accused Ms Morton of "rigging" the ticket allocation to punish people - including himself - who were not supportive enough of the then prime minister.
Sir Gavin reportedly warned Ms Morton "not to push him about" and that "there is a price for everything".
Sir Gavin is quoted in the Sunday Times as saying: "I of course regret getting frustrated about the way colleagues and I felt we were being treated.
"I am happy to speak with Wendy and I hope to work positively with her in the future as I have in the past."
Downing Street, which is handling the complaint, said it would not comment while the formal process was under way.
A Conservative Party spokesman said the party had "a robust complaints process" and complainants could come forward in confidence.
Both Ms Morton and Sir Jake lost their jobs in the reshuffle when Mr Sunak came to power, while Sir Gavin - a former chief whip and a key member of Mr Sunak's leadership campaign - returned to government.
Sir Gavin was defence secretary in Theresa May's government but was sacked in 2019 over claims - denied by him - that he had leaked details of a national security council meeting. He then became education secretary in Boris Johnson's cabinet but was replaced after two years.
Labour deputy leader Angela Rayner said Mr Sunak's decision to bring Sir Gavin back into government saw him put "party management before the country" and his judgement was "in question".
She said: "Rishi Sunak's pledge to restore integrity, professionalism and accountability has been exposed as nothing more than hollow words."
Liberal Democrat Deputy Leader Daisy Cooper MP said: "These shocking revelations raise yet more serious questions about Rishi Sunak's judgement...
"If the prime minister was serious about restoring integrity, he would sack Gavin Williamson."
Questioning judgement
Mr Sunak is also facing questions over his decision to reappoint Suella Braverman as home secretary after she admitted to repeatedly using private email to conduct government business.
Ms Braverman was reappointed to her role six days after she was forced to resigned over two breaches of the ministerial code.
Then prime minister Liz Truss asked Ms Braverman to step down after discovering she had sent confidential material from her private email address to Conservative MP Sir John Hayes, a long-time ally.
She has since admitted using her personal email for official business six times so she could read the documents while taking work video calls.
In a letter explaining her actions, Ms Braverman insisted there was no top secret or market-sensitive information.
Labour's shadow home secretary, Yvette Cooper, also said her party "will use every parliamentary mechanism open to force government to come clean over her reappointment, to get answers and to require detailed documents to be released to the Intelligence and Security Committee".
Watch: PM not aware of 'specific messages' from minister - Dowden