Xio Sei is very stubborn, it's true. But Xio Sei is raising his kids the way he was raised, undoubtedly. It's what he knows, so to him it's being the best parent they know how to be by setting the most stick-up-his-ass example possible. Xio Voe is his crowning achievement and his greatest contribution to the legacy of his nation. His crown jewel suddenly wanting to be reset in a new crown and change everything - it goes against everything he knows.
Anorians have also been their empire's ONLY rival for ages. The only real threat to their world domination. It is, essentially, his son and heir and pride and joy marrying his worst enemy, even if they haven't been at war with the Anorians. If it ever came to war, the only real weapon they have against the Anorians and Anosukonim is Xio Voe and his national treasure of a brain. And now Voe wants to marry into that. To Xio Sei, I'm sure it looks like abandonment.
As for his siblings - Xio Voe had no idea what he was missing until Kudorin showed him. He was an emotionless void. I doubt he was that was as a child, and I'm sure his brothers and sister aren't quite there - though his sister seems close - but like Xio Sei, they have no idea that another way exists and can be right. Their way and ideals are what they strive for. So I doubt they'd MISS warmth the same way that someone raised to know what it was like to have it would.
That said, I think spending some time with

would be good for them, too.
And yes, Xio Sei is a stubborn bugger, since he's being given a lot of persuasive arguments. But i understand his hesitance I think, at least in part.
Vade wandering around looking for a lay is funny as hell. And I like how Xio Voe respects Vade and Bade, and values talking to them.