If you played well enough to reach elite/Olympic level, but then were suddently told you could not because you were "too good" you might consider that unfair.

I think of Caster Semenya again (not transgender). What was done to her at times was inconsistent and unfair.

Sporting rules should be largely consistent across ages, gender, and gender identity. Where it makes reasonable sense to do so.

There is no solution here that isn't at some level "unfair" to someone. Pretending that everything can be hunky dory to everyone is, I think, not a clear-eyed view of the issue.