And they have the potential to discriminate against the young, pregnant or those who can't have the jab for medical reasons.
This must be factored into any such scheme before launching, according to a Royal Society report.
It sets out 12 tests which should be met by any vaccine passport plan.
The criteria include having a way of accommodating "changes in vaccine efficacy against emerging variants".
And a vaccine passport would need to meet ethical and legal standards including around data protection, human rights and equality and discrimination laws.
It is also vital to be clear about what the passports would be used for, whether that's international travel or greater domestic freedoms, the report outlined.
The main scientific issues are around how long immunity lasts and whether vaccines will protect against new variants.
It's difficult to say how long a vaccine passport should be valid for if it's not known how long the protection conferred by the jabs hang around for.
And it will need to be clear whether all the vaccines in use protect against every new variant - especially if people are travelling to parts of the world where the dominant strain is different from the one in their home country.
The vaccine is proving to be very good at protecting people from becoming ill with coronavirus.
But there are still questions around how much it stops people carrying the virus and transmitting it to others.
Understanding this will be important if vaccination status becomes a pass to mix more freely with others, Prof Christopher Dye, an epidemiologist at the University of Oxford who contributed to the report, explained.
He said the only vaccine currently mandated for travel around the world is yellow fever - and there is only one type of vaccine available, it's highly effective, and confers lifelong protection.
The Covid-19 vaccines are not currently in such a "comfortable" spot, he said.
Legal and ethical issues
As well as scientific concerns, there are also legal and ethical considerations, according to the report's lead author Prof Melinda Mills.
"What would it be used for - getting a job or attending a football match or buying milk?" she questioned.
"What if we start barring people from essential goods and services?"
"There is a risk of unjustly discriminating in hiring, attending events, insurance, housing applications, you can think of many examples," Prof Mills explained.
If a passport scheme was rolled out before everyone in the country has been offered a jab, this could unfairly disadvantage young people or people living in certain parts of the country.
Even then, some people are unable to or are not recommended to take the vaccine, for medical reasons or during pregnancy unless they are at particular risk, for example.
And, "people of different ethnicities have different levels of vaccine hesitancy," Prof Mills said, which opens up the potential for discrimination against people's "religious or political beliefs" - or where this hesitancy is linked to a long history of racism and marginalisation.
'No jab, no job'
If employers implement "no jab, no job" policies - where the requirement to be vaccinated is written into workers' contracts - equality laws would need to be balanced with legal duties of care, Prof Mills said.
For example, the right of individuals working in care homes to choose not to be vaccinated would have to be balanced against the imperative to protect the vulnerable in their care.
Some countries have already developed passport schemes, including Estonia, Denmark and Israel.
* be internationally standardised
* have verifiable credentials (you can prove someone has been vaccinated)
* have defined uses
* be based on a platform of interoperable technologies (ie on different operating systems like Android or Apple, and on different devices like phones, tablets and offline)
* be secure for personal data
* be portable
* be affordable to individuals and governments
* meet legal standards
* meet ethical standards
* have conditions of use that are understood and accepted by the passport holders