Nadhim Zahawi should release "the absolute facts" on his tax affairs after paying a settlement with HMRC, former Conservative Party leader Sir Iain Duncan Smith has said.
Tory party chairman Mr Zahawi has faced mounting pressure over claims he tried to avoid tax and had to pay it back.
On Saturday the ex-chancellor said he had paid a settlement and HMRC had accepted it was a "careless" error.
Labour said the matter had been "corrosive to public trust".
Asked on Sunday with Laura Kuenssberg if it would be better for Mr Zahawi to publish everything that had happened relating to his tax affairs, Sir Iain said "the sooner you can get the absolute facts out the better", rather than it "coming out in phases."
He said he would say to Mr Zahawi to "get it all out now whatever you have to do and clear it up".
But he defended Mr Zahawi, saying he "genuinely" did not believe he was "deceitful".
Foreign Secretary James Cleverly said it was for Mr Zahawi to decide "how much detail to put in the public domain".
According to the Guardian, Mr Zahawi had to pay back tax he owed with a 30% penalty, the total amounting to £4.8m.
The BBC has been unable to verify that figure, but when the paper asked about the penalty, Mr Zahawi's spokesperson did not deny one had been paid.
This weekend Mr Zahawi issued a statement saying he wanted to address "confusion about my finances".
He did not make clear whether or not he had paid a penalty as part of his settlement, nor did he say how much he had paid to HMRC.
Mr Zahawi said when he was being appointed chancellor, questions were being raised about his tax affairs and he had discussed it with the Cabinet Office at the time.
An aide to Mr Zahawi confirmed to the BBC that the tax matter was resolved while he was serving as chancellor.
On Sunday, the foreign secretary was asked about what he knew about the circumstances surrounding Mr Zahawi's tax affairs.
He said he did not know if Mr Zahawi had sorted out his settlement with HMRC while he was chancellor, nor if he had paid a penalty as part of a settlement, nor if this was all discussed with the prime minister.
Labour's deputy leader Angela Rayner called on Rishi Sunak to "come clean on what he knew and when" about Mr Zahawi's tax affairs.
The case "is corrosive to public trust, with a chancellor in charge of the nation's finances asking the public to pay their taxes while apparently having failed to do so himself," Ms Rayner said.
She added it was the prime minister's responsibility to "decontaminate his scandal-ridden cabinet by setting out clearly what steps he has taken to ensure that all ministers' tax affairs are in order".
Speaking to the BBC, Labour's shadow chancellor Rachel Reeves said she "would be happy" to publish her own tax returns "if that was the thing that was necessary".
Liberal Democrat Deputy Leader Daisy Cooper said allegations against Mr Zahawi were "becoming impossible for Rishi Sunak to ignore".
"The public deserves to know the substance behind these allegations about Nadhim Zahawi's tax affairs, how much he paid, when and why," Ms Cooper said.
'Right thing to do'
In the statement issued on Saturday afternoon, Mr Zahawi said: "As a senior politician I know that scrutiny and propriety are important parts of public life.
"Twenty-two years ago I co-founded a company called YouGov. I'm incredibly proud of what we achieved. It is an amazing business that has employed thousands of people and provides a world-beating service.
"When we set it up, I didn't have the money or the expertise to go it alone. So I asked my father to help. In the process, he took founder shares in the business in exchange for some capital and his invaluable guidance.
"Twenty one years later, when I was being appointed chancellor of the exchequer, questions were being raised about my tax affairs. I discussed this with the Cabinet Office at the time.
"Following discussions with HMRC, they agreed that my father was entitled to founder shares in YouGov, though they disagreed about the exact allocation. They concluded that this was a 'careless and not deliberate' error.
"So that I could focus on my life as a public servant, I chose to settle the matter and pay what they said was due, which was the right thing to do."
Watch: Cleverly defends Zahawi over tax avoidance claims