You'll have to provide us with more details of your issue to better assist you such as the platform you are on. Also specify the game mode or playlist you have experienced this issue.
Additionally, you may want to try establishing a static IP address and port forward. Here are the port numbers you will need.
Xbox 360: Activision Support
PS3: Activision Support