Sajid Javid told MPs that he expected to set a deadline for the beginning of April to give 103,000 unvaccinated workers time to get both jabs.
He said the move would help protect patients and the NHS as a whole.
But concerns have been raised that it could lead some workers to leave, adding to healthcare staffing issues.
More than 93% of NHS frontline staff have had their first dose and 90% are fully vaccinated, Javid said. That is higher than the general working-age population, where about 81% have had both doses.
The government's decision follows a consultation which considered whether both the Covid and flu jabs should be compulsory.
Mr Javid said the flu vaccine would not be made mandatory.
Those with a medical reason not to have the Covid jab would be exempt, he said, as would those who do not have face-to-face contact with patients.
In a Commons statement, Mr Javid said compulsory vaccination would "protect patients in the NHS, protect colleagues in the NHS and, of course, protect the NHS itself".
He said the requirement would be enforced 12 weeks after parliamentary approval - likely to be from April.
No unvaccinated worker should be "scapegoated or shamed", said Mr Javid, and should instead be supported to make "a positive choice".
Thursday is the deadline for care home workers in England to get vaccinated.
Each of the four UK nations makes its own decisions on the issue.
Scotland and Wales have not made any proposals to make Covid jabs compulsory for NHS workers or care home staff, while in Northern Ireland there is to be a public consultation.
Chris Hopson, chief executive of NHS Providers which represents England's NHS trusts, said: "We understand why people are vaccine-hesitant. We need to win the argument with them rather than beat them around the head."
The possibility of losing staff was a "real problem" as the NHS runs on fine margins and already relies on staff to work extra shifts, he added.
Labour's shadow health secretary Jonathan Ashworth urged Mr Javid to proceed with caution - pointing to waiting lists "close to six million" and more than 90,000 job vacancies across the NHS.
There will be anxiety that chronic understaffing problems could be exacerbated, he said. "We simply cannot afford to lose thousands of NHS staff overnight."
Will staff vacancies now balloon?
The logic behind this move is obvious - the virus presents a real risk to vulnerable people in hospital and the vaccine reduces the chances of infection and transmission.
So it's not hard to understand why insisting on jabs for staff who have contact with patients will, in theory, help protect patients.
But if it causes more staffing shortages that simply increases risk in another way.
There are already more than 90,000 vacancies in the NHS in England. It is a number that has actually fallen during the pandemic after staff put off retiring to help out.
If those staff take up their option to retire and mandatory vaccination forces others out, the NHS could find itself severely short of staff come the spring.
Mandatory vaccination comes into force for the care sector this week. Since it was announced, only two-thirds of the unvaccinated have come forward.
If a similar trend is seen for the NHS, vacancies would likely balloon to unprecedented levels.
Unison head of health Sara Gorton said she feared the move might knock staff morale further and prompt workers to leave - or lose their job.
The government should consider alternatives like daily testing, she said, and not risk making the same mistake made with mandatory jabs in social care - which she said had led to an unprecedented staffing crisis.
From Thursday, care homes will be required to refuse entry to workers who have not been fully vaccinated, unless they have a medical exemption or there is an emergency.
Mr Javid said that since announcing the requirement for care staff, the number of unvaccinated workers in the sector has fallen from 88,000 to 32,000 in the last month.
'You've got to do the right thing'
Douglas Ferguson, a surgeon at Exeter Hospital, said it was the government's responsibility to protect the population, describing the policy as "quite sensible".
"I think, to be honest, if people who don't want to be vaccinated feel that an injection, which is very well scientifically researched, is something that they decide they're going to change their entire career for, it seems unwise, I would say."
Overall, Mr Ferguson thought the move was likely to be welcomed by staff.
"If all the staff know everybody else is vaccinated - I mean the majority are already anyway - I think it would give further confidence in you doing your job and potentially from those members of staff also."
He added that he only knew of one colleague who had chosen not to be vaccinated.
NHS figures show vaccination rates vary hugely: among hospitals, Dorset County hospital has the highest at 94.6%, while Barts Health NHS Trust has the lowest rate of fully vaccinated staff, at 79.7%.
At Southampton General Hospital, frontline workers were told by letter in September they would face redeployment if they did not get vaccinated.
One worker, who has been redeployed after refusing to get the vaccine, said it was "disappointing" but not unexpected that jabs would be made compulsory for NHS workers.
She said that she felt it was against her human rights to be forced to take the vaccination and said it should be her decision.
"I'm angry," she said. "Probably I will just go and get it, what other options have I got?"
Some countries have already introduced compulsory vaccination - including France, which had to suspend 3,000 unvaccinated healthcare workers in September.
More than 45.8 million people in the UK have had two doses of a Covid-19 vaccine, meaning 79.8% of people aged over 12 are fully vaccinated, while 87.5% have had a first dose.
Vaccinated people have a lower risk of catching Covid and are less likely to become seriously ill or need to go to hospital, but there is still a chance they can transmit the virus.
The number of daily confirmed cases of Covid-19 remains high but Tuesday was the sixth day in a row that the figure came in under 40,000, with 33,559 cases reported. A further 262 deaths within 28 days of a positive test were reported, down from 293 for the same day last week.