No 10 initially said they would not have to isolate, as they were taking part in a pilot scheme that involves daily testing instead.
But opposition parties said it suggested there was "one rule for them and another for the rest of us".
The PM later said they had "briefly" considered taking part in the scheme.
It comes ahead of the lifting of lockdown rules in England from Monday.
Almost all legal restrictions are to end, including limits on how many people can meet, and nightclubs will reopen, but self-isolation rules will remain.
'Stick with the programme'
Following news that Mr Javid had tested positive and been in contact with Boris Johnson and Rishi Sunak, No 10 said a workplace pilot scheme would allow them to keep working from Downing Street.
But the announcement met with an angry response, with Labour's Jonathan Ashworth saying it was unfair politicians appeared to have access to "VIP testing" to avoid self-isolation, while Liberal Democrat Leader Ed Davey asked if it was only available to the "privileged few".
Speaking in a video on Twitter after the U-turn, Mr Johnson said: "We did look briefly at the idea of us taking part in the pilot scheme, which allows people to test daily, but I think it's far more important that everybody sticks to the same rules and that's why I'm going to be self-isolating until Monday 26 July.
"I know how frustrating it all is, but I really do urge everybody to stick with the programme and take the appropriate course of action when you're asked to do so by NHS Test and Trace."
The prime minister also defended the lifting of restrictions - amid ongoing rises in cases, deaths and hospitalisations - but urged people to be cautious.
He said: "We've got to remember that this virus is sadly still out there. Cases are rising. We can see the extreme contagiousness of the Delta variant, but we have this immense consolation and satisfaction that there is no doubt at all that the vaccine programme has very severely weakened the link between infection and hospitalisation and between infection and serious illness and death."
Mr Johnson will now conduct meetings remotely from Chequers, the prime minister's countryside retreat, where he was when the health secretary received his positive test result.
Mr Sunak said on Twitter: "Whilst the test and trace pilot is fairly restrictive, allowing only essential government business, I recognise that even the sense that the rules aren't the same for everyone is wrong."
Labour leader Sir Keir Starmer has written to the prime minister, demanding he answer how he and the chancellor were included in the workplace programme.
Among 10 questions, he also asked when the PM travelled to his country residence of Chequers from Downing Street, how many other ministers were covered by the workplace pilot and for a figure on how many members of the public have now deleted the NHS app.
Sir Keir wrote: "Today's latest fiasco of yourself and the chancellor being magically selected for a testing trial so you could avoid isolation like the rest of us is just the latest example of the Conservatives fixing the rules to benefit themselves, and only backtracking when they were found out."
The managing director of the Iceland supermarket chain, Richard Walker, also criticised the proposal for Mr Johnson and Mr Sunak to avoid self-isolation.
"Shame the hundreds of Iceland staff who've been pinged can't avoid self-isolation. We can all do a daily lateral flow test," he said.
Mr Javid tested positive on Saturday morning after a meeting at Downing Street the day before, and both Mr Johnson and Mr Sunak were contacted by NHS Test and Trace.
'One of the fastest government U-turns ever'
This must be one of the fastest government U-turns ever: 157 minutes after saying the PM and chancellor wouldn't be isolating, Downing Street decided they would.
The reason is set out by Rishi Sunak on Twitter - ministers can't be seen to be following different rules to everyone else.
But questions will be asked about why No 10 initially said they would be escaping isolation. Who made the decision? Who signed it off? The prime minister and chancellor must have known about the plan.
There's also the fact that the PM and chancellor had the choice over whether to isolate - which most people don't at the moment.
The political symbolism is also highly significant. Three senior government ministers will be in isolation on the day almost all legal restrictions are lifted in England.
The government said there were 20 public and private sector organisations involved in the pilot scheme, including Network Rail, Transport for London (TfL), Heathrow Airport and Border Force.
Heathrow said the programme had been an "important tool" that allowed it to keep the airport open, and was now calling on ministers to extend it beyond the end of July, when it is due to end.
But TfL said it was "still waiting for formal notification that we are part of this trial". One London Underground line was forced to close on Saturday because too many staff had to self-isolate.
Housing Secretary Robert Jenrick had defended the programme on the Andrew Marr Show, describing it as "well known" and "long standing".
Former Prime Minister Tony Blair added it would have been reasonable for Mr Johnson not to self-isolate, given he has been vaccinated twice, was previously infected, and is being tested regularly.
He said: "I don't want the prime minister of the country to be in isolation at the moment. I need him, you know, at his desk doing his job."
The UK recorded a further 48,161 cases and 25 deaths within 28 days of a positive coronavirus test on Sunday. On both Friday and Saturday daily cases exceeded 50,000 - the first time this level has been reached since mid-January.
More than half a million alerts telling people to self-isolate were sent by the NHS Covid-19 app in England and Wales in the first week of July, with supermarkets saying they may have to cut opening hours or shut shops due to a lack of staff.
Absences from school have also surged, with 840,000 pupils self-isolating in the first week of July.
Mr Jenrick said self-isolation was "an important part of keeping the virus under control", adding that a study had showed the NHS Covid app had prevented 600,000 infections and 8,000 deaths.
However, he said the government was ending self-isolation in England for entire school bubbles on Monday and from 16 August fully vaccinated people will not have to self-isolate if a close contact tests positive.
Mike Cherry, national chairman of The Federation of Small Businesses, said the government should now urgently review whether it should bring forward the changes to self-isolation.
"It could expand the pilot scheme that ministers currently enjoy to those at the front line of the economy," he said.
Mr Javid is the latest in a series of government figures to test positive during the pandemic, including his predecessor Matt Hancock and Mr Johnson.
The health secretary said on Twitter that he had felt "a bit groggy" on Friday and took a rapid lateral flow test the following morning, which was later confirmed with a PCR test.
In addition to the meeting at Downing Street, Mr Javid had been working from his office at the Department of Health and Social Care in Whitehall last week and was in the Commons chamber three times.
He is also known to have visited a care home in south London on Tuesday. Aashna House in Streatham has a 100% vaccination rate among staff and residents, however.