the expander ball,I hate even saying these two words) doesn't touch the neck until the case is being removed from the die.
au contraire!!!
The expander ball works the neck on both directions of the ram stroke. This has given many people the idea that it overworks the brass, and pulls the neck out of alignment with the case body. In many cases, using standard,cheaper) FL dies, this is true. However, Forster's benchrest dies have a different arrangement of expander to work with the neck sizing portion of the FL die which actually aids concentricity of neck and body. Others may have copied this design by now, I don't know.
Neck dies, which are falling out of favor with many benchrest and other precision shooters, resize only the outer neck diameter of the case, and so do nothing to set shoulders, or ensure body/neck concentricity. More shooters are coming to understand the importance of these two variables and their control does much to improve accuracy.
Certainly, when we are discussing accuracy as applied to today's factory rifles, neck sizing dies are largely the best way to go. IMHO, Lee makes the best one on the market. Using bushing dies to load ammo for 99% of factory rifles is like putting mag wheels on a Ford escort. Looks neat but does nothing for performance and makes a lot of onlookers chuckle. Factory chambers are much too loose to bother with running down the OD of your case neck, and 99% of reloaders don't ream the insides of their necks, so what good are you doing? Well, you're helping Redding and Forster to meet the payroll................