Matchmaking works by searching for players that have similar map packs as you. There are way less players with the DLC than those without, therefore you will likely end up playing the same players over and over again. Also you have a reduced pool of players and that adds to the difficulty of matchmaking.
DocChoi is 100% correct about the DLC.
One more thing to concider is how matchmaking is currently working when playing in a party. From what I've seen, when you are playing solo, or in a party with all members located close to you, games are found more quickly, and there is relitively little lag. However, if a single member in your party is located far away or in a different region, matchmaking takes much longer, and you will have much greater lag, as well as getting matched with the same players over and over. If you've played online games before, this makes sense, and is what you would expect for Peer to Peer connections. That said, ghosts only started behaving this way over the last month. I believe the cause is that the Dedicated servers are no longer used for Ghosts, meaning it is now running P2P.
edit: I cant say all of Ghosts's dedicated servers have been taken offline, but I'm relatively certain there are far less now than a few months ago.