I honestly think you are going to far too fast. Pick your load with the speed you like. Work on that powder charge with that Bullet and test seating depths. I've found most hunting bullets of boat tail design like to be between -.012 to .030 off of touching. Another thing, are you shooting with chronograph strapped on the barrel? How many shots between cooling? I've had a few magnum rifles, with MAGNUM charges comes Magnum heat. Most factory barreled magnums I've played with walked with each shot after 3. Barrel heat tosses groups. Chronograph on barrels walks groups. Slow down and let the rifle tell you what it wants. Sounds like it wants to shoot. Watch barrel temps. Test your moa load you've posted. Take that bullet and powder at that charge weight and that primer. Load all the same except for c.o.a.l. Then in groups of 3 load at -.005 to -.035 off your touch from the lands. While doing this make sure the barrel isn't hot. I suggest 3 because it's a magnum with a sporter weight barrel and they get hot fast. I think you will find it has a happy place. Don't chase your tail without doing seating tests. That will tighten groups way before any primer switch. Heat watch your heat. Sounds like your one load is close. Test it and smile when that bullet finally hits its happy place. Hopefully this helps. Be patient with testing. Heat isn't good for tight groups.Ok, I bedded the rifle and shot a couple of 5 shot groups today. 2 powders, 2 seating depths, but the results were the same. Shooting goups of 1.2-1.7 moa at 100 yards. The factory remington loads had the best groups.
So, now what? Punt to a new rifle?
Upgrade to a new scope?
Hire someone else to pull the trigger?