One of the reasons people voted to leave the E.U. is so that we could have the freedom to strike trade deals with countries outside the E.U. Staying locked into the customs union prevents this.