Athens will again be the host of the Games in 2004, but there are rumors that they might be switched if the Greeks don't begin to construct the sites faster.