I do not think your zoysia seed was contaminated with weed seeds. Sedges are very efficient at exploiting a freshly prepped seed bed. All those nutlets and seed sitting dormant in the soil are now happy. Throw in some GA summer heat and even better some rain, and the sedges are going to take off. This applies to many other weeds as well, both grassy and broadleaf.
I have never grown zoysia from seed, but most of the warm season turf grass seed has a long germination and development period. Much longer than most weeds. I would not give up on the work you have done, but I would get your weeds under control. That may be as simple as cutting them back twice a week or more. But if they are bad you should be exploring your spraying options.