Do your choose load B over load A?

[6BRinNZ]

Boyd, he's not asking how to work up loads. He's asking what it takes to tell whether two loads group differently, with 95% confidence in the ranking.

Two loads, A and B. Clearly, 1 shot with each load isn't enough, because 1 shot produces no measure of dispersion. So, how many shots is enough?

Another way to think about this is whether you would be confident ranking the two loads after 2 shots. How about after 3? 4?

This is an inherently statistical question, and you can't answer it without some reference to sampling distributions.

Isn't the question rooted in how the statistic was produced as much as the sample size? I'm not a statistician so the OP can answer for himself but there appears to be a big questions as to relevance even if the sample size is large.

 

[dmoran]

Isn't the question rooted in how the statistic was produced as much as the sample size? I'm not a statistician so the OP can answer for himself but there appears to be a big questions as to relevance even if the sample size is large.

+1.... good reply



tobybradshaw -
If there is more vertical to one then the other after a couple shots of each, or a higher ES, or a angular cluster verses a round nut, how many more rounds need wasted to gain a +95% confident mathematical statistic to tell you the exact same thing?

I will take practical accuracy over statistical accuracy, two-fold...

 

[Toby Bradshaw]

Isn't the question rooted in how the statistic was produced as much as the sample size? I'm not a statistician so the OP can answer for himself but there appears to be a big questions as to relevance even if the sample size is large.

The sampling distribution has to be generated (or approximated) in some way, yes. From the Wheeler paper both normal (Gaussian) and lognormal distributions give good fits to the data.

We've known about the radial dispersion of small arms shots for more than a century, because this kind of thing matters to the military. I can't help it if shooters don't read the literature.

This reminds me of the incredibly involved discussions that always ensue re: wind deflection. Hardly any shooters understand the physics -- most believing that wind deflection is proportional to time of flight rather than lag time, or that the wind blows on the side of the bullet -- but you don't need to understand the physics to read wind flags and hold off according to your experience and/or information from sighters. Luckily, bullet and cartridge designers do understand the physics.

Similarly, many (most?) shooters believe that that they can identify the "best" loads/charges/seating depths/etc. with a few shots, even though they can only detect quite large differences this way.

For those that don't like "hypotheticals," here's a real-world example:

I give you 5 different lots of .22 rimfire match ammo to test in your particular rifle, to be used in BR50 competition. How many shots does it take you to find the "best" lot? No identifiers on any of the boxes of ammo.

What sort of data would you collect, how would you analyze them, and how would you assign confidence levels to your ranking?

And, to make it more interesting, consider what would happen if 2 of the 5 lots are actually the same.

 

[Toby Bradshaw]

tobybradshaw -
If there is more vertical to one then the other after a couple shots of each, or a higher ES, or a angular cluster verses a round nut, how many more rounds need wasted to gain a +95% confident mathematical statistic to tell you the exact same thing?

How much vertical is "more"? 1"? 0.1"? 0.001"? 0.0001"?

You can always generate a higher ES by increasing sample size, which is why it is a rather useless statistic. Given the velocity distribution you can predict the ES for any sample size.

 

[BoydAllen]

Toby,
Most if not all of the distribution information was gathered by testing for military purposes, which means that in terms of really fine accuracy, the results are generally useless because none of us shoot rifles that are that inaccurate, or one size must fit all military loads, in chambers with military clearances. On a day with excellent conditions my results on target, track pretty well with the changes that I make testing loads. Let me give an example. One time I was being less than rigorous about coming up with a starting load for a new powder, and the conditions were good, and I had about a bullet hole of paper between the two bullet holes. Believing that for the caliber, components and rifle type that I was shooting, that nodes have about a 1.2 grain spacing, and noting that the two shots were about as bad as I normally see under those conditions, I guessed that I was about half way between nodes, knew that going up .6 gr would result in excessive pressure, so I set my measure to throw .6 less, and the bullets overlapped by more than half of their diameter, with the offset being all horizontal, where the previous two had been on a diagonal line in the same conditions. From there I began more extensive testing, and the results were consistent with the differences seen in the flags. Personally I think that that demonstrated more than a crude estimation. It was calculated based on best information, and it worked. This is not to say that that sort of change always works. On another day, I had shot a group with a load that came from my notes, based on what had worked well in similar ambient conditions, and all it gave me was a blob. No paper between shots, but about as large a group as can be had without any. I tested .3 up, and .3 down and the results were no better. At that point, having faith in the rifle, and needing to try a different approach I dropped my load to the next lower node, and it came smartly together, and shot that way for the rest of the afternoon. For me is is not about the sample size but the measuring instrument, and the consistency with which what is being measured is produced, both the ammunition and the shooting.

One reason that I have no desire to shoot rimfire benchrest is that it would require a lot more rounds down range to come to a reliable judgment, because I would be at the mercy of ammunition manufacturers. I have a friend who has a number of perfect score targets to his credit, and so I am aware of what it takes to excel in that sport. He has put a large amount of time and resources into getting to his current level of expertise, and even so, continues to experiment.

 

[dmoran]

tobybradshaw -

I am 100% confident it would take me far less rounds then you to be confident.
Now its back to tested and proven simplified accuracy for me.... have a nice day

 

[BradY]

It might only take two shots to tell you its not working. Two shots strung vertically or horizontally compared to a known "good" load will be enough to say no thanks. It may take many practice and match sessions to prove it if its something to really try. If your a varmint shooter then its probably not worth it. If your a benchrest, f class or long range shooter then I would be following the advice of starting low and working up to that load to see if its any good. No two guns are the same- similar maybe, but not the same. You have to ask yourself, is it worth trying to improve something your already happy with? Sometimes its easier to just shoot what you know and be happy with it rather than waste time, money and in this day valuable components chasing something that may not be worth chasing...

 

[Toby Bradshaw]

Toby,
Most if not all of the distribution information was gathered by testing for military purposes, which means that in terms of really fine accuracy, the results are generally useless because none of us shoot rifles that are that inaccurate ...

This simply isn't true. The relative precision (not accuracy) of two loads is measured as the ratio of their dispersion, however you prefer to measure dispersion. The absolute values of dispersion are, of course, lower for more precise rifles.

Do you have to understand statistics to shoot well competitively? No. Nearly every shooter does use statistics (ES, SD, mean, aggregate, etc.), but the vast majority misuse them because of misunderstanding.

 

[Toby Bradshaw]

I fully understood the question.

And yet you didn't answer it.

 

[BoydAllen]

That is my precise point. In the case stated by the OP, I believe that statistics are being misapplied, because the question ignores other important factors. If I were to test loads A and B in a tunnel, using state of the art equipment, the number of shots that it would take to determine which was the best would be a lot less than shooting outdoors with less ideal equipment, so without the other information, no answer is possible. Beyond that, there is the skill of the shooter. A poor shooter with good equipment will make errors that tend to obscure fine distinctions. One cannot answer the question from a statistics text. The number of shots that it would take would DEPEND on other factors that the fellow who wrote the book probably had no knowledge of. Also, it would depend on how close in accuracy the two loads were. If one was really bad, in a particular rifle, and the other particularly good, the number of shots required to determine that might be very small.

Returning to small bore for a moment, my friend has tested lots (as in production lots) of ammunition at both the Ely and Lapua facilities in this country, in Texas, and Arizona. In both cases their standard procedure is to clamp rifles in a fixture for testing purposes. The method of support has been shown to influence the results, and while this does not preclude the usefulness of that kind of testing, it does limit its value because, in competition, and practice, the rifles are not clamped into a fixture, and this difference influences how they shoot, probably because of the difference in vibration patterns.

Added a bit later: I did answer the question, but it was not the answer that was wanted. The answer is that it depends. Evidently he thinks that independent of the factors that I have mentioned that there is some magic number, that can be determined by the application of statistical theory. MY answer is that there is not. That the number of shots required depends on factors that he would exclude from the discussion.

 

[Toby Bradshaw]

The OP asked a question that can be answered quantitatively and correctly. I answered it (or, Wheeler did). The practical value of the question and answer can be debated endlessly, without changing anyone's mind.

 

[artbosco]

Suppose you are using load A and getting good results. A friend recommends load B. You decide to test load A against load B to see if you should change loads. You know the test should only involve one variable, load A and load B and everything else should remain constant as much as possible. How many groups do you need to shoot to make your decision and how many shots do you fire in each group? For the sake of discussion assume you will require a ninety five percent confidence level that B is better than A before you change loads (or you are willing to accept one chance in twenty of being wrong).

3 to 4 Five-shot groups.

Source: Harold R. Vaughn, Rifle Accuracy Facts, Chapter 4: Barrel Vibration

 

[BoydAllen]

If two loads are an eighth of an inch apart in their grouping potential, and the test rifle is not the best, and the test is done outdoors, with no flags, off of a poor rest, by an average shooter, it is highly likely that the difference will be lost in the noise. With those same conditions, if one load has two inch potential in a given rifle, and the other three quarter inch. The difference may become apparent after shooting one group with each. Generalizations are of limited usefulness when it comes to specific cases, as is this sort of bench racing, and conjecture.

 

[savagedasher]

Two to three 2 shot targets should tell you if the change you made is going in the right direction. If load A shoots .250 Load B shoots .500 why would you shoot more then 3 two shots.. I never have seen a 3 or a 5 shot group get any better then the second shot. Larry

 

[chkunz]

Suppose you are using load A and getting good results. A friend recommends load B. You decide to test load A against load B to see if you should change loads. You know the test should only involve one variable, load A and load B and everything else should remain constant as much as possible. How many groups do you need to shoot to make your decision and how many shots do you fire in each group? For the sake of discussion assume you will require a ninety five percent confidence level that B is better than A before you change loads (or you are willing to accept one chance in twenty of being wrong).

3 to 4 Five-shot groups.

Source: Harold R. Vaughn, Rifle Accuracy Facts, Chapter 4: Barrel Vibration

Outdoorsman I took a look at the reference you cite and in Chapter 4, Table 4 he presents data for an accuracy test and says the results are a summary of twenty 5 shot groups. I did not find a reference to "3 to 4 Five-shot groups" that you quote. Is it someplace else in the book?

 

[chkunz]

TRex - I've been involve in statistics enough to know you can prove or disprove anything with enough numbers. I've ran studies where we could show a statistical difference between a 1.1 avg and a 1.2 avg. With that said, most people load modifications with at least 20 shots each. I know a few who shoot F class that will shoot one load per 20 shot string to fine tune loads. They mainly measure the vertical since horizontal can be due to atmospheric conditions. I like to test at least 4 groups of 5 rds or 2 groups of 10 rds when testing an A vs B load or one reloading step modification vs my standard reloading sequence. I always alternate between the two loads to negate any outside influences from atmospheric conditions or barrel fouling.

My 2 cents,
BC

That seems reasonable (four 5 shot groups or two 10 shot groups) but does not answer my question of what it takes to get a ninety five percent confidence level. If we knew we might find that your test is adequate or we might decide that it is too difficult to get there or that it was not worth it to us. I see folks making decisions on a lot less data and have to wonder if their test is good enough. At least I would like to know what the science tells us is required.

 

[chkunz]

We do not shoot hypotheticals; we shoot rifles. To answer your question, if we are talking about my best bench rifle, on a day when the wind is easy, shooting from a bench that does not move at all, with my flags out, only a couple of groups will be required, possibly only one, or part of one. If some statistician disagrees, I would tell him that while he may know his field, he has a lot to learn about shooting, and the costs attendant to doing testing in an manner that would satisfy him as to the sample size, and me as to the conditions of the test. I have a friend with a state of the art 100 yard tunnel, and I can tell you that shooting outdoors yields significantly less reliable results. Talking about having only one variable testing loads is like discussions about what bullets would do in a vacuum.

This is not meant to be personal and I have not questioned how you are evaluating your equipment, but is a simple question relative to the science of statistics and how we could apply that science to rifle accuracy if we choose to do so. I realize that not everyone has an interest and that is ok but I think it is basic to what we do. We work hard to achieve accuracy and it seems like we should do our best to objectively evaluate the results as scientific as possible so we can improve. It sounds like you do not have respect for statistics but the mythology was developed by the gambling industry and it is difficult to beat the house with them having it on their side. Industry makes business decision using this science. I do not think we have anything to loose understanding the science and having the wisdom of how to apply it. But of course I could be wrong. Take care, Clyde.

 

[chkunz]

T-REX, what you are going to hear from most shooters is analogous to Supreme Court Justice Potter Stewart's famous definition of pornography -- "I know it when I see it."

Shooters don't know (in the statistical sense) whether they have the best load, because it isn't practical to know even within 10-20% (ratio 1.1-1.2). In short range benchrest the loads are tuned match-to-match (at least I had to tune that way when I was shooting competitively).

I have shot groups in the zeros in registered benchrest (got the little wood plaques to prove it!), but I very much doubt that I can really hold within the 0.054" c-t-c dimension that my smallest official group measured. Non-statisticians seem to believe that screamer groups are the result of great shooting and great loads, as opposed to the left tail of a distribution.

Excellent points. I consider all groups test groups weather we are shooting test groups or competition groups and we have the opportunity to evaluate all groups we shoot.

 

[savagedasher]

20 - 5 shot 10 - 10 shot or 50- 2 shot should give 95% Larry

 

[chkunz]

20 - 5 shot 10 - 10 shot or 50- 2 shot should give 95% Larry

Simple question, simple answer. Looks like we are done here. Thanks Larry.