So my hillbilly work-around payed off. Which is dumb.
But I can play with my friends if I hotspot off my phone and play using the internet through my phone provider.
Now. With a full part of 3 other people, it's lagged at all get out for me and I rubberband a lot. As expected. But it's playable.
That makes me wonder if something through the game is trying to access my internet, through Xbox, when I try to connect to friends normally and my firewall blocks out this action making my friends game to seize up and crash.
Tomorrow I'll re-encrypt my firewall and watch to see if anything pops up looking like a back door attempt while I try to connect to a friend on the Xbox.