Boris Johnson has a "lot of work to do" to win over Tory MPs after 41% voted to oust him on Monday night, Welsh Secretary Simon Hart has said.
The senior Welsh Tory said the vote was "decisive" in backing the prime minister, but he admitted Mr Johnson has "lots to prove".
While 211 Conservative MPs supported the prime minister in the secret ballot, 148 voted against him.
One Welsh Tory MP told BBC Wales: "This is far from over, sadly."
The MP said the vote leave rebels "enthused", while loyalists asking Conservatives to move on will annoy others.
"That's in the realms of 'the worst possible result'," they said.
Fearing defeat to Labour at the next election, the MP added: "He has to resign. If not, the 2024 election will look like Blair's second coming."
Another Welsh Conservative MP, asked if the prime minister was safe, said: "With this PM I don't know. Under normal circumstances someone would consider their position."
But backing the prime minister, Mr Hart said it was a "very clear result".
"Of course he has lots to prove," he told the BBC News Channel.
"It's been a difficult choppy chapter. Lots to prove to voters, lots to prove to colleagues, but tonight was pretty decisive."
He added Mr Johnson would be the "first to say: 'I realise that I have a lot of work to do to be able to restore the confidence of my colleagues, and some of the 148, as well as other people too'".
"I would like to think that we can now take a deep breath and get on with the job that we were elected to do."
Simon Hart is a member of Boris Johnson's cabinet
Six Conservative MPs in Wales said they would back the PM ahead of the vote.
None of the 13 Welsh Tories in the Commons have publicly stated they would oppose Mr Johnson, although one has privately said they would do so.
The leader of the Welsh Conservatives in the Senedd, Andrew RT Davies, urged Tory MPs to "draw a line" and move on with tackling the cost of living crisis.
"While people are struggling, today the Conservative party has sadly spent all day talking to itself," he said.
Mr Davies said it was time to "start the job of rebuilding trust and confidence in the government's ability to improve people's lives wherever they live".
Labour's Shadow Welsh Secretary Jo Stevens called Mr Johnson a "lame duck PM", while Plaid Cymru's Parliamentary leader Liz Saville-Roberts said Mr Johnson's authority had been "shredded".
She added: "It speaks volumes that not one Welsh Tory MP openly called for Boris Johnson to resign. They must reflect tonight on whether it is respect for their constituents or fear of the Tory whips that drives them."