Yeah, here’s another way to think about it. Imagine constructing an ancestor tree going back 40 generations, which is roughly 1000 years. The top line (listing all your great^38 grandparents) would literally have over a trillion entires (2^40). Obviously a trillion people weren’t alive then or ever, so what you’d see in practice is the same men and women appearing over and over in that tree. But the same men reappear in that tree at a much higher frequency than the women do.