Not too odd but I wouldn’t do it myself - it’s quite intense to have both your names tied to one person - and with the last name presumably his too, that’s three names (and maybe too patriarchal?). It could be a lot to put on a kid!
Is there a middle name that could derive from your own last name, or one of your partners (your dad’s name or your mother’s name in masculine form)?
Or given James is already an honour name, then a name of his own? Maybe from a book or movie you both like?
I like: James Aurelius! It’s smart! James Aurelian or James Aurelio are other options - both cool imo, and flow well.
Middle ground names (between Ryan & Aurelius)
Marcus
Lucas
Dominic
Ronan
Rowan
Vincent
Benedict
Marshall
Roman
Rafferty
Wildcard: Rylan (or as @ashthedreamer reminded me, Orion) for a first and a different middle!