Its happened to me before, I do the following:
1- Reboot my modem
2- Stop playing if it persists
Usually thats not a good sign, and if I get into a game with that happening I see too much latency.
Good to see it's not just me. Been trying to get a game for the last half hour or so, finally just turned the game off. It was starting to damage my calm.