From April, front-line NHS staff in England will need to have the Covid jab - or will be moved to another role.
King's College Hospital chief Clive Kay said his job was to encourage workers to get jabbed - and 10% of his 14,000 staff were still unvaccinated.
The government's Nadhim Zahawi defended the policy as "the right thing" to do.
From the start of April, under law NHS staff in England who have direct face-to-face contact with patients will need to have been vaccinated against Covid.
A similar policy has already been brought in for staff working in social care in England - with care homes warning it has worsened the staffing crisis. The NHS is already facing a severe lack of staff and is short of about 93,000 workers.
Mr Kay was speaking after one of his own doctors challenged Health Secretary Sajid Javid over the issue, saying he was "not happy" that he faced being dismissed for not getting the jab.
The doctor, Steve James, an ICU consultant, argued that he believed he already had antibodies after catching Covid - and has since told the BBC he believes vaccination should be a matter of personal choice.
Hospital boss Mr Kay refused to say whether he thought the new rule was fair - although argued that staff were "not being forced" to have the jab, but instead "being encouraged".
But pressed on the rules, he acknowledged it could mean people leaving their current job: "There's a possibility if they choose not to be vaccinated they could be redeployed. And if we can't find that opportunity to redeploy them then the consequence is that they will [not have a job]."
"We will not make individuals be vaccinated," Mr Kay told the BBC's Sophie Raworth on the Sunday Morning programme. "We must treat them with kindness, with compassion, we must give them every single opportunity to talk through if they don't want the vaccine... but ultimately it is their choice."
Asked how many front-line staff he could lose under the law change, Mr Kay said: "I am confident that we are already seeing a number of staff choosing to be vaccinated. I don't want at this stage to predict or give any numbers."
But he said he was "of course" worried about staffing, adding: "My job is to worry about everything in relation to whether or not we have enough staff here to provide care for patients safely."
According to Mr Javid in November, more than 93% of NHS frontline staff had had their first dose and 90% were fully vaccinated. This is higher than the wider population, where 90% have had a first dose and 83% have had both.
Speaking to the BBC's Sophie Raworth on Sunday morning, Education Secretary Nadhim Zahawi defended mandatory vaccines for NHS staff in England, saying: "We have a duty of care when looking after the most vulnerable to protect ourselves and to protect them."
Dr James told the BBC's Broadcasting House programme on Sunday: "My view is that we should not mandate vaccination."
"Normally you go through this process of informed consent, where you weigh up as a doctor with the patient, the risks the benefits and that person's personal preference."
He said he has been written to by "hundreds of people" since he spoke to Mr Javid, who have told him they were thankful that he has spoken up about how he felt.
"I've been written to by a group of midwives who say that, I think 41 or 45 of them in that unit will all not be vaccinated. That means the entire midwife staff from one hospital are going to leave."
Each of the four UK nations makes its own decisions on the issue.
Scotland and Wales have not made any proposals to make Covid jabs compulsory for NHS workers or care home staff, while in Northern Ireland there is to be a public consultation.
Scotland's Health Secretary Humza Yousaf said they were not going down the same route as England "partly because the uptake from NHS and social care staff is so incredibly high". "I don't think making vaccines mandatory is the way to go," he said.
It comes as the UK recorded 141,472 new cases, as well as 97 deaths within 28 days of a positive test - although the number of deaths recorded over weekends tends to be lower because of reporting delays.
The total number of cases over the past seven days is up 6.6% on the previous week, while the figure for deaths is up 30.9%.