The chancellor has vowed to "always balance the books", despite increased spending in the wake of the Covid-19 pandemic.
In a speech to party members, Rishi Sunak said the Conservatives had a "sacred duty" to "leave the public finances strong".
He vowed the use the "overwhelming might of the British state" to help people find new work.
But he said debt and spending needed controlling "over the medium term".
In an online speech during the Conservatives' annual party conference, he said: "I won't stop trying to find ways to support people and businesses."
However, he added the party could not argue there was "no limit on what we can spend", nor that "we can simply borrow our way out of any hole".
Mr Sunak cited the furlough scheme and its successor, the jobs support scheme, as examples of government action to support employment during the crisis.
He said though that, although the government would "keep striving to be creative" on employment support, he would also have to be "pragmatic".
He told members that "no chancellor" would be able to save every job or business, adding changes to the economy due to Covid-19 "can't be ignored".
Official figures published in September show government borrowed £35.9bn in the previous month, its highest amount for August since records began in 1993.
Borrowing between April and August totalled £173.7bn, as ministers spent billions on coronavirus-related schemes to support the economy.
Sunak keen to shrug off big-spending reputation
Government borrowing is at stratospheric levels because of the pandemic.
It is not clear precisely what the chancellor means, promising to get it under control in the "medium term".
Nor was there even a whisper of how that could be done.
Treasury sources suggested it's unlikely to happen by the time of the next election, likely to be in 2024.
But while the chancellor's first few months in the job have been characterised by enormous crisis-level spending, that is a characteristic that he is keen to shrug off.
In an interview after his speech, the chancellor said government debt - which passed £2 trillion for the first time in history in August - was vulnerable to increases in borrowing costs.
"Now that we have so much debt, it doesn't take a lot for suddenly 'yikes' - we have to come up with X billion pounds a year to pay for higher interest," he said.
Mr Sunak - who has been touted as a potential future Tory leader - also said he did not want to become PM, and described his "close personal friendship" with Boris Johnson.
Asked if he eventually wants to replace Mr Johnson, he replied: "No. Definitely not seeing what the prime minister has to deal with, this is a job hard enough for me to do."
BBC economics editor Faisal Islam said: "The chancellor knows that just days after his Winter Economic Plan, with unemployment set to go above 5%, and social restrictions intensifying, not loosening, there is now further backroom pressure to increase the generosity of his worker subsidy schemes.
"This is a continuation of the pattern we have seen in the past few weeks since the cancellation of the Budget. There'll be more support for the economy, but with the really tough decisions - for example, on tax, - put off.
In response to his speech, shadow chancellor Anneliese Dodds said Mr Sunak had "nothing to say" to millions of people whose jobs were at risk.
She told reporters more "targeted support" was required for sectors of the economy that have been hardest hit by restrictions during the pandemic.
"Sadly there was nothing from the chancellor today to suggest that he grasped the magnitude of the jobs crisis we're facing," she added.
Dame Carolyn Fairbairn, director-general of business lobby group the CBI, said the best way to balance the books was by "protecting our economy's ability to recover".
Adding that the costs of the pandemic had fallen "deeply and unevenly," she said it was vital to protect at-risk sectors such as aviation, manufacturing, and hospitality.
Do Conservative governments balance the books?
Chancellor Rishi Sunak told the virtual Conservative Party conference: "This Conservative government will always balance the books."
If it does, that would be an unusual achievement.
Very few governments balance the books
Public sector net borrowing as % of GDP
Balancing the books usually means that a government has repaid more than it has borrowed in a year - ie it's in surplus.
The government can still have debt overall, but the debt hasn't risen during the year.
The last time that a government balanced the books was under the Labour government in 2000-01, and for the two years before that.
The most recent Conservative government to achieve that was under Margaret Thatcher in 1988-89 and 1989-90.
Another measure that recent governments have liked to talk about is whether the economy is growing faster than the debt. If it is, the government can say that debt is falling as a proportion of GDP (which is the value of everything produced by the economy in a year).
That happened in both 2017-18 and 2018-19.