What type of connection are you using? Is it wired or wireless? Also, where are you located? I see here that you tagged PC. Try to check on your connectivity info. To check your connectivity in game, go to Multiplayer, select Options and press the letter 'S' on your keyboard. You can also check out this link: http://support.activision.com/articles/en_US/FAQ/How-does-matchmaking-work-in-Pu blic-Match/
It's a big help for your concern. Cheers!