When you match with a party the lobby will try and consist of players who are in between where you and your friends are located. However, if the population isn't that high and there's not really anyone to match you with that's in the middle the lobby will expand it's search. This could mean that the lobby is filled with players who are located on the other side of your friends and further away from you.
It really all comes down to match making and right now during this DLC transition period match making will be tricky.
The freezing was pretty frustrating, but I've noticed lately that there have been host migrations instead of freezing. I'm going to go ahead and knock on wood and hope the freezing issues is over and done with.