If you're looking directly into the eye of the black hole now, it means that was the position 657 million years ago. Gamma-ray speed is the same as visible light, which mean they arrive at the same time.

However, GRB's spread out over 2°-20°, severely reducing its effect over distance.

Which means it has to be pretty close to do any damage. (It would have to be in our part of the Milkyway.)