Rishi Sunak defended his decision to launch an ethics inquiry into Nadhim Zahawi rather than sacking him, at Prime Minister's Questions.
Sir Keir Starmer said the PM was "hopelessly weak" for not firing the minister for "seeking to avoid tax".
"Is he starting to wonder if this job is just too big for him?" the Labour leader asked.
Mr Sunak said it was Sir Keir who was weak because "he has no principles just petty politics".
Downing Street initially said Mr Sunak's tax arrangements were "confidential" when asked by Labour if he had ever paid a penalty to the UK tax authorities, like Mr Zahawi.
But the PM's spokesperson later confirmed that he had not, saying: "The prime minister has never paid a penalty to HMRC."
The PM will publish his tax returns "in due course", Downing Street has said.
In the House of Commons, Mr Sunak said it would have been "politically expedient" to sack Mr Zahawi as a minister before PMQs got under way at noon but he believed in "proper due process".
That was why, he said, he had asked ask his ethics adviser to investigate whether the Conservative Party chairman had broken ministerial rules.
It will be up to the PM to decide whether to sack Mr Zahawi if his ethics adviser says he has broken the ministerial code.
Mr Zahawi was chancellor at the time the estimated £4.8m settlement was agreed with HMRC.
Sir Keir asked Mr Sunak why he had said at last week's PMQs that Mr Zahawi had "addressed this matter in full".
"Since I commented on this matter last week more information, including a statement from the minister without portfolio [Mr Zahawi], has entered the public domain which is why it's right that we do establish the facts," the prime minister said.
He accused Sir Keir of "simple political opportunism" for urging him to appoint an ethics adviser then wanting a decision before they had investigated the case.
And he claimed "the difference between him [Sir Keir] and me is I stand by my values and my principles even when it is difficult," accusing the Labour leader of indulging in "petty politics".
Sir Keir said the PM's "failure" to sack Mr Zahawi showed "how hopelessly weak he is - a prime minister overseeing chaos, overwhelmed at every turn".
"He can't say when ambulances will get to heart attack victims again. He can't say when the prisons system will keep streets safe again. He can't even deal with tax avoiders in his own cabinet," said the Labour leader.
"Is he starting to wonder if this job is just too big for him?"
Conservative MP Nigel Mills told BBC Radio 4's The World at One that Mr Zahawi must explain why he had to pay a penalty to the tax authorities.
"I just don't see how an investigation into the ministerial code resolves this because if he is cleared by that, that won't stop people asking questions about what on earth happened."
He added: "I think the only way to resolve this is to make clear what the situation was that gave rise to a significant penalty.
"If that can be explained we can all move on. If it can't, then clearly his position won't be tenable."
Mr Zahawi confirmed on Saturday that he had made a payment to settle a dispute with HM Revenue & Customs (HMRC), adding that the tax authority accepted the error was "careless and not deliberate".
The BBC understands the dispute was resolved between July and September last year, when Mr Zahawi was chancellor under Boris Johnson, and that the total amount paid is in the region of about £5m, including a penalty.
The tax was related to a shareholding in YouGov, the polling company he co-founded in 2000 before he became an MP.
Mr Zahawi has not confirmed how much his penalty amounted to, nor the total value of the final settlement with HMRC.
Watch: Is this job too big for Rishi Sunak? - asks Sir Keir Starmer
WATCH PMQs in 70 secs: Sunak insists he believes in "due process", as Starmer turns up the heat over Zahawi investigation.
Watch: SNP Westminster leader asks PM for 'financial advice'