You can certainly make a factory gun shoot, if it will let you. You wont find an argument from me on that. My first year back into it I shot a savage 6br factory gun in short range ground hog matches, (200-500) and I did everything I could to make it shoot better. I was on the heels of the full blown custom guys for the aggregate of the season, and some of them are very good shooters. I root for the underdogs and people that find the ways to make things work better. And maybe you don't even want to shoot competitively, maybe you just want a very accurate rifle. There is absolutely nothing wrong with that.
Depending on the distance you shoot, the discipline you're shooting, the goals you have, and how competitive you want to be, will decide how much you need to do. Can you afford to have one shot (or more) come out of your group at 1,000 and open it up to a 10"er? (And in conditions that you should have shot a 4-5"er) Not if you're going for the aggregate. But then again, maybe you're comfortable with having that happen.
I dont mean any of that directly towards "you" and only used that as person I am asking those hypothetical questions to. We all have our limitations, expectations and desires in this game. Thats what brings most of us together, and even makes us butt heads from time to time.