Diconnection from friends game [ Edited ]

[Keystinisbest]

So, I'm on the Xbox One. I'm trying to do the endless gnome challenge with my buddy... We cannot do it, however, because every single time we try to do it together, one of us ends up getting disconnected. It's a random wave we get disconnected on, and I'm getting very frusterated.

Whether I host or he does, it doesn't matter. Oh, and it occasionally happens on garden (etc.) ops, as well.