I had that problem happen to me once in BO2. What did I do? I accepted the invite, because I COULDN'T FIND A FREAKING MATCH. Anyways, I was in a party with fissure, and miraculously, we both ended up in this dude's lobby. We pwned, as usual, and the guy was never heard from again.
So TL;DR the answer to all your problems is Fissure.