You have a lot of cards stacked against you and a 1 piece base aint the answer. A model 70, featherweight, shooting groups with a 308, that stock setup shooting from a bench, and all that time between shots, youre doing surprisingly well for what you have. You can take any load on that page, even mixing them up, and accomplish the mission that gun was designed to do. Sometimes you gotta back up and look at the whole forest.
It's not the M-70 action that is the limitation. It's the barrel.
I have had some M-70 FWTs and lightweight M-70s that would shoot 1/2" groups on occasion. My current one in 300 WSM will do 1/4" on occasion. Day in and day out they were/are .6 to .8 MOA rifles, and I was/am very happy with that level of accuracy. Especially considering I had/have an extremely durable and reliable action.
I made a 435 yd shot on a small Blacktail buck with my 300 WSM, a clean one shot kill. This rifle has a Benchmark M-70 WSM contour, which is about a #2. It's in a McMillan Hunters Edge stock. While it is a joy to carry, It was REALLY hard to get stable with that rifle for the 435 yd shot.
I did have a CRF M-70 that was a legitimate 1/4 MOA 3-shot rifle, at least at 100 yds. It was about a 1/2 MOA rifle at 550. It weighed 12 lbs and was in 338 Edge.
I currently have a 33-28 Nosler on a CRF M-70. It has a #4 contour Benchmark barrel and weighs 8.75 lbs with the scope. It will shoot .5 to .7 with controlled expansion bullets, depending on the load. Not bad for almost 50 ft lbs of recoil.
For me, a reliable action and a reliable bullet are my top criteria in a light to standard weight hunting rifle. So a CRF M-70 shooting well-sorted Accubonds or TTSXs are the magic for me.
For a long range hunting rifle I want at least a #5 contour barrel, and I do expect consistent 1/2 MOA 3-shot groups. However, this is at least a 9.5 lb rifle/scope combination.