Just nerding out a bit. Don't read if that's not your thing.
I've been looking through Volume 2 of the Bryan Litz book on shooting and find a table where he lists how much your group size will vary depending on how many shots fired. But then he goes on to discuss statistical analysis. Here's where I disagree a bit with his presentation.
Strictly speaking, I would prefer to say that if your gun can shoot a group, or more accurately, shoots POI within x inches of the POA, there is a 95% chance it would land within A, then that is how accurately your rifle/ammo shoots under ideal conditions. If we use this standard there is really no difference due to group size. That is, if you have shot (to take an extreme case) 1000 shots with all within 1 inch of POA, then 2 follow-up shots 2 inches off are not significant enough to change that assessment.
Conversely, the fewer the shots/groups you've fired, the less likely it is that a flyer is not part of your 95% confidence interval.
Of course, this also means that too many shots may also exaggerate your true 95% MOA. Why? Because a large group assumes ALL variables confounding variables are unchanged. That includes weather, pressure, temperature of the barrel, consistency of ammo, and in the case of a non-fixed rest shooting arrangement, shooter error. But when you're testing for the quality of your gun/ammo, you don't want to include shooter error into the calculation of group size. So the larger your group is, the larger the chance that your aim wandered, the weather changed, your barrel heated up, or that you got a randomly off piece of ammo.