There should be no wear in that area. Once the hub is assembled, the bearing preload set, and locked, that part of the bearing should not move.
It does seem odd that several different bearings aren't fitting. If you have a caliper measure the bearings and compare the size differences.
Get some emery cloth (also called crocus cloth, any good auto supply store should carry it). It comes in a roll, but some stores will sell it by the foot. I get the roll and beat inflation a bit. Tear off a strip about 12 to 14 inches long and sand briskly, on a clean and dry surface, moving around so that you sand all the way around. Wipe it off with some solvent on a rag, followed with a dry rag, and inspect it visually with a bright light, and feel it. It should be smooth as a baby's butt., and shiny.
Before sliding the bearing on to check the fit, wipe it clean with solvent on a rag, followed by a wipe with a clean dry rag.
Neatness counts here, the tolerances are close and anything that shouldn't be there may keep things from fitting together, and after all, wheel bearings are very important. Have a clean work area, lay freshly cleaned bearings, nuts, washers, seals, etc. on a clean dry shop towel. A roll of those blue things they sell at the chain parts places are suitable, and disposable so your wife won't shoot you for washing the cloth variety out in her washing machine.
Also, I looked at the FSM procedure for setting the bearing preload.
For reference, here it is (I'm skipping the first two steps):
Tighten the wheel bearing locknut to 18 to 22 ft lbs.
Rotate wheel hub a few turns in both directions to seat wheel correctly. Then, retighten spindle nut to the above torque. (this means to losen and then reset the nut torque)
Loosen the wheel bearing lock nut 60 degrees. Install adjusting cap and align groove of nut with hole in spindle. If grove does not align with hole, relocate adjusting cap. If hole and groove still do not come into alignment, loosen wheel bearing lock nut as much as 15 degrees more.
Again, spin wheel hub several times in both directions to see if it rotates freely. Then measure bearing preload as follows: (the diagram shows the mechanic using a small scale to pull and check the rotating force required to begin rotating the hub) The starting torque is 3.5 to 7.4 inch pounds.
Having done a zillion wheel bearings I have developed a feel for the task, so I don't bother with a torque wrench or a scale. I can say that if I did, I would stick to the lower side of the range for the nut tightening. The FSM stresses that care be taken to not overtighten the bearings, and I think even 18 ft lbs is too tight. I also spin the hub as I snug the nut down, something the FSM fails to require.
When you finally get this resolved, choose the correct length cotter key, so the ends can be neatly bent tightly around the spindle and not rub on the dust cap. And when installing the dust cap, don't bash on it with a hammer, tap it in gently on the lip around the edges with a flat tip screwdriver (about 5/16" wide). it should slide in snugly, but easily. If you bend the dome it will rub against the spindle.
Good luck, this really sounds like a trying endeavor for you. I hope this information helps you get your Z back on the road!