Senior members of London's Royal Thames Yacht Club reportedly discussed letting the Duke go in a secret meeting on Monday, over his links to Jeffrey Epstein.
The club is keen to replace Andrew with Kate, who is a keen sailor, the Daily Mail reports.
The club has had discussions with Buckingham Palace about the switch, leaked minutes reportedly show.
They say: "Contacts with the commodore and his office regarding patronage are current in the light of the changing developments within the royal household."
It is thought this refers to Prince Andrew's stepping down from public duties amid the Epstein scandal.
The Duke has been keeping a low profile at his Windsor home after he was unceremoniously forced to step back from public life in November over his links to the convicted paedophile.
The disgraced financier killed himself in jail in New York while awaiting trial on further charges.
The Duke vehemently denies allegations he slept with Epstein victim Virginia Giuffre - who claims she was forced to have sex with him when she was 17.
A source said: "There is a strong feeling that Andrew is not a suitable commodore. It is a family-orientated club and Andrew is not in keeping with that.
"There is a strong desire to replace him with Kate. She is an accomplished sailor and would be a huge asset as the club's first female commodore.
"The Palace is digging its heels in because they want to avoid further embarrassment for Andrew. If the RTYC removes him other similar organisations will follow.
"It is complicated by the club's royal status – the role of commodore is effectively in the gift of the Queen."
The Duke of Edinburgh is the club's patron, and Lord Mountbatten was also commodore. It has had links to the royals since 1775.
Chief executive Christian Horvath said: "We have always had royal involvement. During his 34 years as commodore Prince Andrew has promoted and supported sailing activities.
"Following a webinar meeting... the vice commodore wrote clarifying some issues raised by a small number of members. We have no further comment."
A source close to Prince Andrew told Sun Online: “The duke has been on the receiving end of false accusations for years and understands why some organisations might want to keep their distance.
“All patronages are on hold whilst he has temporarily stepped aside from public duties to clear his name.
“He is thinking about how to best serve his country and will work with his family, advisers and stakeholders on what that will look like.”
The Royal Thames Yacht Club in Knightsbridge
Andrew is also a Vice Admiral of the Royal Navy
The Duke sailing in a yacht at Cowes in the Isle of Wight