In prosperous times, the marginal workers get by. But in tough times, they get the shaft.