When I was an MP, John Prescott barracked me in the House of Commons, shouting: 'Woolly jumper! Woolly jumper!'