When this happened to me recently I just had to reset my console. Turned out a friend had signed in before me on a different name and it wasn't recognizing me as in my clan or as being able to do just about anything. Give it a try and see if it works. Good luck!
Can you please let us know what console you are playing on? What does your NAT type show up as ingame? Do you have a static ip address set up on your console? Have you configured your ports? Are they inviting you before searching for a game or while they are in a lobby? The issue could be you are trying to join them when they have a full lobby, are they letting you know when the lobby has space? Let us know!