Bringing back a traitor to the kingdom

Hi all, so I am playing as vassal of the Southern Empire (married Ira) and I figured that if I want it to survive against the vladians in the future we need to conquer the Asari and Kuwait first, to have two secure edges of the map. I am trying to that end that every time we go to war against them that we always take some land, even if it is a single castle or city.

Things were going well, regarding the Asari I managed to secure for us Quyaz, Sanala and the castles near them. I did not want to own them myself (I am sure there is a limit of fiefdoms, and also I don't have yet enough clan members or companions to govern more than my 4 cities), so Rhagea gave all of those to a single other clan. Cool. Now that clan has defected to the Asari and all my efforts have been for nothing.

The easiest way to revert this would be to have the guy to defect back to us. I captured him, locked him in my castle prison, I already have like 100 friendship with him, but I when I talk to him I have no option to make him my clanmember or suggest that he defects back to us. Do I have to reconquer everything or is there any other option I am missing?

If it helps, I have the better garrisons and diplomacy mods on.