I think you’ve got 4 issues to deal with: your attitude toward original names, whether the name is too “out there” since it comes from fantasy, whether it’s too masculine for daughter, and whether it’s feminine enough for a daughter. In the end, there are no right or wrong answers. Instead you have to decide what you are comfortable with and (importantly) what you think a daughter will be comfortable with, both as a child and as an adult.
Basically, I think there are two kinds of people in the world–those who embrace made-up/unique/original names and those who don’t. It sounds like you and your wife have no problem with such names, even though you did choose an established name for your son. The problem that a child with such a name will face is frequently having to explain his/her name–there will be no immediate recognition and acceptance of the name.
The name isn’t extreme–it fits in with the sound of lots of modern names. I don’t think that GoT will have the extended popularity of other fictions, so the name won’t have an immediate association like, say, Legolas or Han would.
Because I don’t think you’ll run into wide association with the novels, I don’t think it sounds particularly masculine, although to my ear, it does sound like [name]Bentley[/name], which seems male.
On the other hand, I don’t think it sounds particularly feminine. Renly/[name]Finley[/name] and the like could go either way. If you have [name]Connor[/name] and [name]Arya[/name], people will assume you have a son and daughter. With [name]Connor[/name] and Renly, you’ll have to explain. You have to decide how you feel about that.
[name]One[/name] last word–a caution, I guess. I didn’t, but it would have been completely possible for me to name my child after something I’m a fan of. My love of the Beatles and LotR could have inspired a [name]Lucy[/name] or a [name]Sam[/name], or a [name]Ringo[/name] or a [name]Frodo[/name] (or more reasonably, a [name]Maxwell[/name] or an [name]Arwen[/name]). Now that I’m a mom of a teen, I’m painfully aware that a child doesn’t necessarily love what his parents do. Given that, I think that a parent would get a pass on a mainstream name inspired by a passion. A more unusual name, though, might become a source of friction.