Well, if dry lube, whether graphite or mica could hurt a barrel mine would be long burned up by now. I have used the dry stuff and it never hurt anything. In fact, I couldn't tell any difference from rifles I fired with it in the cases and those I did not.
This is gonna make some of you guys schidtt down both legs, but I have been using Imperial sizing wax. Just a tiny bit on my fingers when I handle the bullet to seat it works fine. The bullets seat easy, they don't stick or lock in the case and they shoot no different, zero. Cant see it on a chronograph and believe it or not the world hasn't come to a complete or partial stop. No misfires either.
All this said, yeah, I keep the brass annealed and I don't sts clean it every time I load. I guess I better add that I typically store some of my ammo for up to 6 months and still not a problem with the little bit of wax I use. I have pulled a few after that length of time and they came out as easy as they went in......