Prisoners do matter when analyzing the severity of racial inequality in the U.S. Yet because they are out of sight and out of mind, it is easy to imagine that we are making far more racial progress than we actually are.