The House of Lords has heard a call for the Duke of York and the Duke of Sussex to be removed from being stand-ins for King Charles III - and for the Princess Royal and Earl of Wessex to be added.
Peers have debated plans for two extra "counsellors of state".
But a Labour peer said Prince Andrew and Prince Harry, as non-working royals, should be taken off the list.
Lord Berkeley said the question of who was eligible to act for the King needed greater "transparency".
Peers on Monday were debating the Counsellors of State Bill, which would widen the pool of royals who could carry out official duties if the King was overseas or ill.
Lord True, the Lord Privy Seal, said the proposal was a "practical solution" necessary for the "machinery of government" - and said that the Royal Household had confirmed that in practice only working royals would be called upon to act as counsellors.
This legislation would add two more, rather than remove any of the current five stand-ins - Camilla, the Queen Consort, the Prince of Wales, Prince Harry, Prince Andrew and his daughter Princess Beatrice.
Lord Berkeley supported the addition of Princess Anne and Prince Edward but he put forward an amendment that would exclude Prince Andrew and Prince Harry and any other non-working royals.
Counsellors of state should not include anyone who had not "undertaken royal duties on a regular basis" for the preceding two years, he proposed.
Prince Andrew withdrew from royal duties in the wake of his association with US sex offender Jeffrey Epstein.
Prince Harry, who lives with his family in the United States, has stepped back from being a "working royal".
'Quick fix'
Conservative peer Lord Balfe also argued Prince Harry's status as a counsellor needed to be more clearly resolved.
The legislation, launched in response to a request from the King, is being fast-tracked through Parliament.
It is expected there will be overseas trips next year for the King, Queen Consort, Prince William and Catherine, Princess of Wales - and without a change, this could mean a lack of available stand-ins to act on behalf of the monarch.
Labour peer Viscount Stansgate welcomed the increase in counsellors but described the addition of two specific royals as a "quick fix" and suggested that a longer-term view was needed for deciding who should be appointed.
Crossbencher Lord Janvrin, who had been a private secretary to the late Queen Elizabeth II, used a football analogy to say the two extra substitutes for the monarch would "give much needed strength and depth to the bench".
But Baroness Jones, of the Green Party, said the whole issue was "inconsequential to the lives of people who are struggling".
Counsellors of state can carry out constitutional duties such as:
* the State Opening of Parliament
* signing official documents
* holding Privy Council meetings
There is already a legal requirement for a counsellor to be "domiciled in some part of the United Kingdom" - but this is being interpreted as including a "domicile of origin", which would allow Prince Harry to continue.
An 18th Century law still prevents a Catholic from being a counsellor - and the Cabinet Office would not comment on how this could be compatible with other equality legislation.
The legislation will move on to its next stages later this week - with the government not expected to accept the proposal to exclude Prince Andrew and Prince Harry.