He added that Ukraine can still win the war if given the right support and praised the efforts of its military.
"We want to see Russia weakened to the degree that it can't do the kinds of things that it has done in invading Ukraine," the US military chief said.
Mr Austin was speaking after meeting President Volodymyr Zelensky in Kyiv.
Accompanied by US Secretary of State Antony Blinken, the visit marked the highest level trip to Ukraine by US officials since the invasion began over two months ago.
The meeting between the US and Ukrainian parties, which ran for over three hours, comes as Russia escalates its military campaign in the south and east of the country.
Mr Austin's comments calling for a weakened Russia were unusually strong for a US defence secretary. It is one thing to help Ukraine resist Russian aggression, it is quite another to speak of weakening Russia's capabilities.
At a news conference in Poland after the visit, the US defence secretary told reporters that US officials still believe Ukraine can win the conflict "if they have the right equipment" and the "right support".
The 68-year-old announced that the US will allocate an additional $713m (£559m) of military aid to the Ukrainian government and 15 other allied European governments fearful of Russian aggression.
It brings the total US security assistance provided to Ukraine since the invasion began to more than $3.7bn (£2.9bn).
Mr Zelensky has been pleading with Western leaders to increase the flow of military equipment for weeks, vowing that his forces overcome Russia's military if provided with fighter jets and other vehicles.
Last week the US confirmed that it has supplied Ukrainian troops with howitzer artillery cannons and anti-artillery radars for the first time.
Russia's ambassador in Washington said Moscow had sent a diplomatic note demanding an end to the US supply of weapons to Ukraine.
Meanwhile, Mr Blinken announced that some US diplomatic staff will start to return to Ukraine from next week. They are expected to be based in Lviv in western Ukraine initially, with a longer term plan to re-open the US embassy in the capital, Kyiv.
It comes as the White House announced that President Joe Biden intends to nominate Bridget Brink, a career diplomat, as US ambassador to Ukraine - a post that has remained vacant for more than two years.
Mr Blinken also defended America's diplomatic approach, telling reporters that it has put pressure on President Vladimir Putin's regime.
"The strategy that we've put in place, massive support for Ukraine, massive pressure against Russia, solidarity with more than 30 countries engaged in these efforts, is having real results," Mr Blinken said. "And we're seeing that when it comes to Russia's war aims, Russia is failing, Ukraine is succeeding."
"A sovereign, independent Ukraine will be around a lot longer than Vladimir Putin's on the scene," Me Blinken added.
Speaking following the meeting, Mr Zelensky said his government appreciated "the unprecedented assistance" from the US and added that he "would like to thank President Biden personally and on behalf of the entire Ukrainian people for his leadership in supporting Ukraine".
In recent days Russia has refocussed its advance on Ukraine's eastern Donbas region and US sources believe that Moscow has committed over 76 battalion tactical groups inside Ukraine.
On Monday, the UK's ministry of defence said in an update that Moscow has made minor advances in the region but has "yet to achieve a breakthrough" due to logistical issues.
The update claimed that "Ukraine's defence of Mariupol has also exhausted many Russian units and reduced their combat effectiveness".
The World Health Organization says nearly 200 cases of monkeypox have been reported in more than 20 countries not usually known...