Without pre-boring I’d say you have a bore that’s wandering off center. The deeper you go, the more binding you have between the pilot and the ‘line’ the reamer wants to travel. I’ve noticed this on longer chambers occasionally where I left more material for the reamer to cut at the front. If you drill most of the chamber and indicate at the throat/pilot area, and then bore the drilled portion, there’s a greater likelihood of keeping things straight. I think your reamer is just trying to follow the bore and now it’s a few thou off the centerline of the reamer.