It was a Labour prime minister who appointed the first black male and female cabinet ministers.