the sound is too associated with the wine for me, although I suppose Brandy and Sherry haven’t bothered people in the past. I think there’s also trouble in the spelling though—in most American settings (not sure where you are), the accent would be dropped, and I think the overwhelming majority would think Rosee is just an alternative spelling of Rosie
as a French speaker, I would imagine that it would get strange looks in France. then again, the French aren’t as creative with word names as English speakers tend to be
Rosamée is a nice option for something similar!