Chancellor Rishi Sunak has said he "is preparing a Budget that provides support for people" as Covid lockdown rules are eased.
At the earliest restrictions in England are set to be fully lifted by 21 June.
Mr Sunak said he would provide help during that period but added that he wanted to "level with people" about the "shock to the economy" caused by Covid.
He dismissed claims he told Tory MPs he wanted to raise taxes now so he could cut them ahead of the next election.
This comes as the government announced £5bn to help high street businesses.
Meanwhile Labour's shadow chancellor, Anneliese Dodds, accused the chancellor of being "focused on politics" over protecting the economy.
Mr Sunak told the BBC's Andrew Marr the government had gone "big and early" when providing support to those hit by Covid at the beginning of the pandemic - and added "there's more to come".
The furlough scheme - which supports around 4.7 million people - is due to finish at the end of April along with the £20 weekly increase in Universal Credit.
Mr Sunak said he was focused on "preparing a budget that provides support for people and businesses and families through the remaining stages of this crisis" and in line with the easing of restrictions as set out by Prime Minister Boris Johnson.
However he also said he wanted to "be honest" with the public about the pandemic's impact on the economy and "clear about what our plan to address that is".
He warned high levels of borrowing had meant Britain was "more sensitive to interest rate changes" and that debt could "rise indefinitely" if borrowing continued after the recovery.
He said making public finances sustainable "isn't going to happen overnight" but would not confirm newspaper reports he was planning to freeze income tax thresholds or raise corporation tax in a bid to lower debt.
Freezing thresholds would mean more people paying more tax as wages rose.
Businesses are watching the Budget closely for information about the future of key support schemes, especially furlough - the job retention scheme - which is due to end in April.
Since it started in March 2020, it has been key to preventing spiking unemployment, but no-one expected the government to still be paying workers' wages a year on. Let alone 4.7 million of them.
With so many dependent on it, and having pledged "support for people" it's difficult to see how the chancellor can end furlough while some businesses are still closed.
But the scheme is not cheap. And with the country's debt topping £2tn, the same value as the UK economy, the chancellor could start to spell out plans on how he plans to pay at least some of the bill.
The UK's low corporation tax looks a likely contender for rises - although businesses that have struggled through the pandemic argue this is hitting them while they're down.
Asked whether he had privately told Conservative MPs he wanted to raise taxes now so he could cut them ahead of the next general election Mr Sunak replied: "I don't recognise that."
Speaking to Sky News he said he "would like to be able to keep taxes low for people in general" but added that he wanted to "be responsible" with people's money.
Ms Dodds said the chancellor should "follow what is in the interest of our country" not "party politics".
Speaking to Sophy Ridge, she said Labour would "look carefully" at proposals on corporation tax but added: "What we don't want to see right now is what the chancellor is doing, particularly in relation to council tax, the imposition of additional costs at a point when our country is experiencing the worst economic crisis of any major economy."
She also called on the chancellor to confirm that the furlough scheme and £20 increase to Universal Credit would be kept for as long as Covid-based restrictions were in place.
During his interview, Mr Sunak also defended his Eat Out to Help Out scheme, arguing that it protected jobs, but wouldn't say if he would revive the scheme when hospitality venues were allowed to re-open.
Mr Sunak will lay out the government's tax and spending plans in his Budget on Wednesday, at a time when public borrowing has led to the highest national debt level since 1963.
Conservative former chancellor Lord Clarke has argued Mr Sunak should consider raising VAT, national insurance and income taxes in order to repair the public finances.
However some Conservative MPs - including ex-Brexit Secretary David Davis - have warned against such rises and potential Tory rebels have been told they risk being kicked out of the parliamentary party if they vote against the Budget.
Martin Beck, senior economic adviser to economic forecasting group EY ITEM Club, said repairing the public finances could wait and warned that "premature fiscal tightening could undermine recovery".