Newt Gingrich has certainly seen his own empire rise - and fall.