More than likely you have "Strict" NAT settings in your router. There's threads that explain how to fix this issue, just do a search. If not, post again and I'll post some links.
By chance, are you playing on a wifi connection? If so, that is part of the problem.