Your allegiance is with your spouse; you cannot break that by showing allegiance to your ex-spouse.