Well first they are not "servers", just FYI. When you play against someone you connect to them AFTER you join a lobby on the MS server.
Have you been banned from any game ?
Have you tried OTHER than a COD game to join ?
Do you only have an xbox Live silver account ?
Have you tried testing the connection BEFORE playing any game ?
Why are you hitting the middle button ? assuming you mean the large X on your controller.
Have you been banned of of Xbox Live ?
Have you rebooted you modem or router ?
And when you are testing the connection do you get ALL of the connections saying you are connected ?
Have you modded any of those games ?
When did this start ?
So far without enough details that is the best I can offer right now. If you answer each question below the space I gave you someone else might be able to pitch in as well.
PS Send me a Private Message here with your Gamertag. So that way you can keep your privacy.