Target Statistics For Shooters

[CharlieNC]

How many shots are required to statistically show loads are different? It's actually straight forward, and is a bit of a different than you have been led to believe. I discuss this and include an excel program for your use. Caution.... it's nerdy.

 

[gunsandgunsmithing]

Simply put, I look for predictable and repeatable loads or tuner setting vs a big number. Big numbers may be best for telling you what a gun it capable of doing all the time but I have seen group shapes change with poi predictably, repeat too many times to be an accident. Agreed, a single small group means nothing but if I can predict that your best groups will happen at top or bottom of a sine test AND THE SHAPES OF THE NEXT FEW GROUPS AS IT PROGRESSES AWAY FROM TUNE, WE ALL HAVE TO ADMIT HOW REMARKABLE THAT IS! Oops, sorry for the caps but it does deserve emphasis. Not only that, but that sine test also shows how many marks from in tune to completely out, top vs bottom of bbl swing and witch way and how far to move the tuner when you see very specific group shapes!

Sorry again. I know this isn't a tuner thread but my goodness. If people saw what I have seen a couple of thousand times, they'd be just as excited as me about it.

To the subject...what is the statistical significance of me telling anyone, BEFORE they shoot my test, what groups shapes will be following a sweet spot as well as poi when in tune and how many marks between sweet spots. I refrain from using the word "nodes" for a specific reason. And what if I said I had seen those same things with BOTH powder charge and seating depth changes BEFORE I ever used a tuner?

Back to statistical significance...I think there are things that have always been there, that we have often simply over looked for various reasons, both with and without regard for the T word. Point is...there is commonality. When you can make things happen, specific things..repeatably and predictably, the number of rounds involved is far less to come to a solid conclusion.

 

[CharlieNC]

Mike your point about a tuner providing a consistent, repeatable response is the same as my charge weight/seating depth example using 3-shot groups. It is not necessary to prove the groups are statistically different, only that the knobs we are turning have a defined effect vs the noise.

 

[K22]

This is a technically interesting topic. When I took statistics in college, if I remember correctly, 30 was considered to be a statistically acceptable sample size.

However, I have never applied that to load testing for several reasons. First, costs. Second, there are so many variables involved that I think it is impractical to apply statistics to load development.

In my experience, shooting at the level of precision I need for my purposes, usually 3 and / or 5 shot groups over a range of 3 or 4 test powder charges usually reveals a serviceable load worth pursuing to validate with follow up validation tests.

I found that you can minimize the effort by selecting a powder that has a long and proven history with that caliber and bullet weight. Also, I have found that most of the time, the critically significant component is the bullet selected. If my rifles like a particular bullet, they will shoot it well with a range of powders suitable for that caliber.

 

[gunsandgunsmithing]

Mike your point about a tuner providing a consistent, repeatable response is the same as my charge weight/seating depth example using 3-shot groups. It is not necessary to prove the groups are statistically different, only that the knobs we are turning have a defined effect vs the noise.

Same with me. I' didn't mean to make this a tuner thread so I mentioned the same as I think you allude to as well regarding repeatable group shapes etc, regardless of tuner or not. They happen either/both ways. The trick either way is establishing increment values on the target, be they tuner marks, seating depth or powder charge. The sine test I use so much is freaking great but it didn't happen by accident. It took testing to know those values but the same test can be used for exactly that...learning those values, for whatever you are changing. It doesn't have to be a tuner. IOW, do you need to change in .1gr increments or 1lb increments? .001 of seating depth increments or .100? One mark tuner increment or do we do things like turning in full revolutions or adding big weights? Same exact thing IMHO. Step number one is establishing the value of whatever changes you make. That's all. I'll step aside and let you all talk stats. Sorry to derail a bit.

 

[Krogen]

How many shots are required to statistically show loads are different? It's actually straight forward, and is a bit of a different than you have been led to believe. I discuss this and include an excel program for your use. Caution.... it's nerdy.

Showing that loads are different is one thing. To my thinking, 3 rounds can often tell me a load has no future. It will take more shots to tell me if there's any hope.

By the way, I like nerdy articles. Deep down, I am a nerd and proud of it!

 

[David101]

When I was calibrating objects in the lab. 21 measurements were needed to produce a statistically meaningful result with a high enough level of confidence to rely on the result

 

[CharlieNC]

JFrank there is certainly a lot to digest, and not too many are interested in that degree of detail either. The following graph summarizes a key finding. When comparing two loads for either group size or mean radius, the number of shots required to achieve statistical significance depends upon the degree of difference you would like to detect. For example if you would like to detect a 20% (or greater) improvement in group size then 15 shots per load are required, and if the difference is at least 20% then it is significant ( at 90% confidence ). In this example meaning detecting a 0.8 vs a 1.0moa group. As you can see, to detect smaller differences requires very large sample sizes and vice versa. This assumes you are shooting both loads under comparable conditions, head to head, etc.

 

[CharlieNC]

I’m guessing my one or two shot per charge ladders don’t quite cut the rug.

Depends. If you go into the guts of the paper you will see my 3-shots across a range of charge weight and seating depth exhibit clear correlations with horz spread, vert spread, and group size. Statistically significant. Another major point is the questions, objectives, and factors are different and there is not a single one size fits all approach.

 

[chkunz]

Charlie, thank you for this post. I actually read it all and understood most of it. I have a technical background and my vocation involved testing and evaluation of data for the performance of rocket propulsion systems for many years. I have continued to use statistical analysis to evaluate data related to my hobby of competitive shooting.

We see many examples of posts on the shooting forums where decisions in load development are made based on insufficient data. If you make comments pointing that out it is like calling someone's baby ugly and it is not received well. There is a general disparaging of statistical analysis by those who do not understand the mythology.

There is one point that you make that I think most people can understand and that is the value of mean radius over extreme spread in the evaluation of test groups. Also, you make a great point about the value of the shape of well-formed test groups.

Thanks again for the post.
Best wishes.
Clyde

 

[Jayplace]

So, using statics, how would 4 shot in one very small group and 1 shot noticeably separate and low to the right be calculated. Most groups are not in the form of a clover leaf.

 

[chkunz]

So, using statics, how would 4 shot in one very small group and 1 shot noticeably separate and low to the right be calculated. Most groups are not in the form of a clover leaf.

You simply have one five shot group. Measure the mean radius of the group.

 

[Jayplace]

I guess that I'm missing something.
Since I didn't know what a Mean Radius was, I had to look it up. The definition I got is, "

• Mean Radius: The average of all radiuses in a group, or average distance by which shots miss the mean POI."

In real life, don't you want the smallest possible group including flyers. In group match competition, 4 shots into one hole and another .500 away would give a respectable average using the mean radius method, but in competition, probably not.

 

[CharlieNC]

I guess that I'm missing something.
Since I didn't know what a Mean Radius was, I had to look it up. The definition I got is, "

• Mean Radius: The average of all radiuses in a group, or average distance by which shots miss the mean POI."

In real life, don't you want the smallest possible group including flyers. In group match competition, 4 shots into one hole and another .500 away would give a respectable average using the mean radius method, but in competition, probably not.

That is true. The objective of my writeup is to describe the options to describe shot dispersion and the methods to evaluate those results, primarily as related to load optimization. Flyers are a reality and should be evaluated as such. There simply is not a single measurement that gives the total story.

 

[chkunz]

I guess that I'm missing something.
Since I didn't know what a Mean Radius was, I had to look it up. The definition I got is, "

• Mean Radius: The average of all radiuses in a group, or average distance by which shots miss the mean POI."

In real life, don't you want the smallest possible group including flyers. In group match competition, 4 shots into one hole and another .500 away would give a respectable average using the mean radius method, but in competition, probably not.

Let me complement you on looking up the definition of mean radius and trying to learn from what some of us are saying. There are several ways to measure groups, the most common way is extreme spread which most people use, it is easy to understand but it only values the two most extreme shots where mean radius values all shots in the group equally. From a statical analysis consideration mean radius is a superior measurement of group size than extreme spread. Other group measurements include the circular probable error, probable error, extreme horizontal dispersion, the extreme vertical dispersion, radial standard deviation, radius of the covering circle, etc. Each has its advantages and disadvantages. The various competition disciplines relate to these definitions in different ways. This is a learning journey, and you can get as deep into the science as you choose. This is a part of our sport that some of us enjoy more than others do.

 

[Jayplace]

I do try to understand, it's part of my nature
I also understand that Mean Radius, as well as other ways have their merits and uses.
Sometimes I wish some of the different competitions used that method. I'd probably do better, but they don't.

I'd do great if it was the best 4 out of 5.

 

[wkdickinson]

How many shots are required to statistically show loads are different? It's actually straight forward, and is a bit of a different than you have been led to believe. I discuss this and include an excel program for your use. Caution.... it's nerdy.

Thanks Charlie!

 

[dave@aDave]

If I remember correctly, the NRA practice of 5 X 5-shot groups (25 shots) is statistically significant.

 

[HappyHellfire]

How many shots are required to statistically show loads are different? It's actually straight forward, and is a bit of a different than you have been led to believe. I discuss this and include an excel program for your use. Caution.... it's nerdy.

Ok, this is right up my wheelhouse. I did the at Dahlgren all the time.

You will need a model of the dispersion (grouping) of the impacts. For most of y'all you use a simple circular model or "grouping size" model. The only unknown measured is the diameter of the impact pattern. We often used an ellipse model that separated the range error from the cross range error.

Next you will need a measure of each "sample" in the group. Not just the group size but also where each round hit in the group. This data is needed to see how the pattern is dispersed and calculate something called the standard deviation.

Once you have the data for the two groups, you will use the number of shots fired and the standard deviation to calculate the "confidence interval". This is how big the grouping size would have to be to include whatever percentage of shots you want to be confident in.

For example, you could collect 10 shots of load A and measure a grouping of 1" at 100 yards and 12 shots of load B and measure a grouping of 1.1". If you want to be 95% confident that the loads are different, you would calculate a confidence interval for each load and get something like {Load A 1" =/- 0.01" @95%} and {Load B 1.1" =/- 0.1" at 95%}. Because the two intervals overlap 0.99"<A<1.01" and 1"<B<1.2" you are not 95% confident that they are not the same.

You then go back and shoot 20 more of each load and include that data and get 0.99"<A<1.01" and 1.08"<B<1.12". You are now 95% confident that the two are different! Nothing changed but how many shots you took.

For smaller groups you can you something called a "student-t" distribution in stead of a normal distribution for the calculations.

 

[CharlieNC]

Ok, this is right up my wheelhouse. I did the at Dahlgren all the time.

You will need a model of the dispersion (grouping) of the impacts. For most of y'all you use a simple circular model or "grouping size" model. The only unknown measured is the diameter of the impact pattern. We often used an ellipse model that separated the range error from the cross range error.

Next you will need a measure of each "sample" in the group. Not just the group size but also where each round hit in the group. This data is needed to see how the pattern is dispersed and calculate something called the standard deviation.

Once you have the data for the two groups, you will use the number of shots fired and the standard deviation to calculate the "confidence interval". This is how big the grouping size would have to be to include whatever percentage of shots you want to be confident in.

For example, you could collect 10 shots of load A and measure a grouping of 1" at 100 yards and 12 shots of load B and measure a grouping of 1.1". If you want to be 95% confident that the loads are different, you would calculate a confidence interval for each load and get something like {Load A 1" =/- 0.01" @95%} and {Load B 1.1" =/- 0.1" at 95%}. Because the two intervals overlap 0.99"<A<1.01" and 1"<B<1.2" you are not 95% confident that they are not the same.

You then go back and shoot 20 more of each load and include that data and get 0.99"<A<1.01" and 1.08"<B<1.12". You are now 95% confident that the two are different! Nothing changed but how many shots you took.

For smaller groups you can you something called a "student-t" distribution in stead of a normal distribution for the calculations.

I discussed that among other aspects, and would appreciate your feedback.