wonders under the sun

I hosted all the games this has happened in, the games load normally, there is no sign of desynch on both ends and only the host seems to get vill swapped, since no one of my coop partners got this problem playing with me - the first 5 villagers that start next to your tc seem to be the only...