Only by developing a deeper understanding of AI systems as they act in the world can we ensure that this new infrastructure never turns toxic.