If you lose your job because there has been an influx of labour from another country, that is a legitimate cause for concern.