It's to the point that I feel like, if I wanna move into a village, the first thing I need to do is surround the whole thing in a fence so they don't wander off... but so much worse, because unlike Minecraft villagers, I can't catch them in a boat or cart and bring them back when they escape!