First, I am always playing in an Xbox Live Party. I have tried to send an invite to the party from in the game by hitting Y, going to the right to the Party area, and then hitting X to invite the party. If they don't respond to the invite, they won't get any other invites I send them as it appears to be sitting in their notifications or messages. It should send them another one as that is the easiest way to accept it. You may still be able to send one but you have to go into their specific name and send them an invite one by one.
Secondly, you can't seem to invite someone to the game that is appearing offline. I believe they may get the invite but can't accept it because they are offline. The minute you appear online, then you can accept the invites. Very odd, but sometimes I forget that I'm appearing offline (it remembers from the day before as it only goes into a low powered mode unlike the 360). I believe the error message you get is that the user's session is not valid or something along those lines. I'll try it again later and update with the proper name once I can try again.