Suppose you are using load A and getting good results. A friend recommends load B. You decide to test load A against load B to see if you should change loads. You know the test should only involve one variable, load A and load B and everything else should remain constant as much as possible. How many groups do you need to shoot to make your decision and how many shots do you fire in each group? For the sake of discussion assume you will require a ninety five percent confidence level that B is better than A before you change loads (or you are willing to accept one chance in twenty of being wrong).
Do your choose load B over load A?
- Thread starter chkunz
- Start date
TRex - I've been involve in statistics enough to know you can prove or disprove anything with enough numbers. I've ran studies where we could show a statistical difference between a 1.1 avg and a 1.2 avg. With that said, most people load modifications with at least 20 shots each. I know a few who shoot F class that will shoot one load per 20 shot string to fine tune loads. They mainly measure the vertical since horizontal can be due to atmospheric conditions. I like to test at least 4 groups of 5 rds or 2 groups of 10 rds when testing an A vs B load or one reloading step modification vs my standard reloading sequence. I always alternate between the two loads to negate any outside influences from atmospheric conditions or barrel fouling.
My 2 cents,
BC
My 2 cents,
BC
3 firings with 5 shots during each firing should tell you all you need to know
The most important thing to your having confidence in the result is that you have control of all of the variables that you can. The only shooters that I see with decent bag and rest setups and wind flags out are those that use them in competition. Also, really bad benches are common. If two loads are only slightly apart in accuracy, all of this works against being able to make a decision between them. On the other hand, with a vary good rifle, designed to be shot from rests, a good bench, good rest setup, flags, and favorable winds, it doesn't take many shots for me to come to a conclusion. When I do a load workup, with a particular bullet and powder, I feel that my procedure is pretty good for finding the best load. For that reason, unless I am changing bullet, or powder, I don't waste barrel steel.
It sounds like you really want to understand the statistical power issues surrounding comparing group sizes with different loads. Read "Statistical notes on rifle group patterns" by Robert E. Wheeler, particularly section 2.4 and Fig. 2. I couldn't attach it to this post (file is too large), but PM me with your email address and I'll send it to you if you can't find it.
In brief (and given the assumptions of the analysis), it takes 27 5-shot groups with each load to detect a 1.25-fold difference in extreme spread with 95% confidence. I.e., 270 shots total to tell the difference between a 0.75 MOA load and 0.60 MOA load.
Shooters are, in general, terrible statisticians, and base their decisions on really poor information (from a stats POV). The use of extreme spread for group sizes and chronograph data is the worst possible use of the data, for instance.
In brief (and given the assumptions of the analysis), it takes 27 5-shot groups with each load to detect a 1.25-fold difference in extreme spread with 95% confidence. I.e., 270 shots total to tell the difference between a 0.75 MOA load and 0.60 MOA load.
Shooters are, in general, terrible statisticians, and base their decisions on really poor information (from a stats POV). The use of extreme spread for group sizes and chronograph data is the worst possible use of the data, for instance.
BoydAllen said:
All very good points and I agree completely but my question was "... You know the test should only involve one variable, load A and load B and everything else should remain constant ... How many groups do you need to shoot to make your decision and how many shots do you fire in each group? ...assume you will require a ninety five percent confidence level that B is better than A before you change loads (or you are willing to accept one chance in twenty of being wrong)." thanks for your response, Clyde.
tobybradshaw said:
Excellent, thanks for the reference I will follow up. You obviously understand my question and how difficult it is to get an answer that even has a one in twenty chance of being wrong. This Forum is focused on accuracy but I see post where folks are making decisions on accuracy that have no sound scientific basis. I am not being critical of anybody but I think that folks that have the knowledge and experience in the scientific method and an understanding of statistics have an opportunity to help others learn. Take care, Clyde.
We do not shoot hypotheticals; we shoot rifles. To answer your question, if we are talking about my best bench rifle, on a day when the wind is easy, shooting from a bench that does not move at all, with my flags out, only a couple of groups will be required, possibly only one, or part of one. If some statistician disagrees, I would tell him that while he may know his field, he has a lot to learn about shooting, and the costs attendant to doing testing in an manner that would satisfy him as to the sample size, and me as to the conditions of the test. I have a friend with a state of the art 100 yard tunnel, and I can tell you that shooting outdoors yields significantly less reliable results. Talking about having only one variable testing loads is like discussions about what bullets would do in a vacuum.
dmoran
Donovan Moran
T-REX said:
Most thrive from experience.....
If you don't have experience, then maybe you need a statistical figure to answer your question.
Where all I need is very few rounds of each, with assessments to the vertical dispersion, shape of the group, chronograph data, and the roundness/size of the holes (stability). Knowledge from these things come from experience.
Long strings like your insinuating you need, will create all kinds of additional problems and issues.
BoydAllen said:
To determine what level of difference between the loads (expressed as a ratio of group sizes), and with what confidence (alpha = 0.05 in T-REX's example)? That's what T-REX is asking.
The extreme spread information in 2 5-shot groups is not even sufficient to determine a 2-fold difference in grouping with 95% statistical confidence. Your personal confidence level may be different.
Of course, we all use information besides extreme spread (group shape, factored by conditions, etc.), but almost nobody except professionals (e.g., ballisticians) use statistically powerful measures of dispersion.
To save barrels we all take shortcuts in load development, but it's statistically delusional to believe that you can tell a 10% (or 20%, or 30%) difference in loads with a couple of 5-shot groups. It would be like finding a "lucky penny" by trying different pennies until you find one that produces 5 heads in a row.
BoydAllen said:
Thanks for the response but it does not answer my question,..how many groups..how many shots,,,ninty five percent confidence level, etc.. The question has a scientific answer given the assumptions. The real world only makes the answer more difficult to achieve but that does not mean that the question and the answer does not have value. The hypothetical gives us a starting point. With all due respects, thanks and take care, Clyde
To me its not about statistics but the process used to arrive at a good load that will hold over conditions and seasons.
For instance shooting many 5 shot groups to produce a good statistical sample (IMO) doesn't hold a lot of relevance to the rifle and loads ability to hold vertical over an F-class string, where I might be shooting very fast, slow or simply have a large wait due to a cross fire or a spotter blow up....I have had some conditions where I have fired all but 1 round very quickly for score and then waited 10 minutes for conditions to return before sending the final shot. I wouldn't want to fire enough test shots to produce a statistical sample that was meaningful under this condition.
No disagreeing with the use of statistics though - I use the magneto chrony over the course of the season to build data and as a final method to validate my load choices over seasonal temperature changes, component lot changes etc etc.
For instance shooting many 5 shot groups to produce a good statistical sample (IMO) doesn't hold a lot of relevance to the rifle and loads ability to hold vertical over an F-class string, where I might be shooting very fast, slow or simply have a large wait due to a cross fire or a spotter blow up....I have had some conditions where I have fired all but 1 round very quickly for score and then waited 10 minutes for conditions to return before sending the final shot. I wouldn't want to fire enough test shots to produce a statistical sample that was meaningful under this condition.
No disagreeing with the use of statistics though - I use the magneto chrony over the course of the season to build data and as a final method to validate my load choices over seasonal temperature changes, component lot changes etc etc.
T-REX, what you are going to hear from most shooters is analogous to Supreme Court Justice Potter Stewart's famous definition of pornography -- "I know it when I see it."
Shooters don't know (in the statistical sense) whether they have the best load, because it isn't practical to know even within 10-20% (ratio 1.1-1.2). In short range benchrest the loads are tuned match-to-match (at least I had to tune that way when I was shooting competitively).
I have shot groups in the zeros in registered benchrest (got the little wood plaques to prove it!), but I very much doubt that I can really hold within the 0.054" c-t-c dimension that my smallest official group measured. Non-statisticians seem to believe that screamer groups are the result of great shooting and great loads, as opposed to the left tail of a distribution.
Shooters don't know (in the statistical sense) whether they have the best load, because it isn't practical to know even within 10-20% (ratio 1.1-1.2). In short range benchrest the loads are tuned match-to-match (at least I had to tune that way when I was shooting competitively).
I have shot groups in the zeros in registered benchrest (got the little wood plaques to prove it!), but I very much doubt that I can really hold within the 0.054" c-t-c dimension that my smallest official group measured. Non-statisticians seem to believe that screamer groups are the result of great shooting and great loads, as opposed to the left tail of a distribution.
We have a place to start. Try reading Tony Boyer's book. The testing that he describes in it would not satisfy any statistician, but it is how it is done in the real world for very good reasons. Asking a statistician about load testing is like asking someone who does not drive about car testing. Statistics is a tool, and are only useful to the extent that it produces useful results. Discussing one variable comparisons, and dismissing the other variables with the wave of a hand, serves no useful purpose.
When I go looking for a powder charge node, testing a new one, for my 6PPC my initial test is one shot per load, with, .3 grain intervals, shot over flags, on a day when the wind is easy to read, from an excellent bench. I run the loads from a point that is lower than I am likely to use, to the point where I get pressure signs that tell me to stop. My seating depth remains constant, and is based on my experience. I use a bullet that has performed well with other powders. All shots are fired on a single target, and as I shoot them I take note of how bullet holes cluster and string. After that part of the test, I look for a tight cluster that includes perhaps three different charge weights, and load to the middle weight. Then I take particular care and shoot two shots as rapidly as I can, taking perhaps 5-6 seconds, in a condition that I believe will hold for longer than that, if it does, I believe the results are representative of that load, and if they are excellent, I test with groups of a larger number of shots. I do this loading at the range, so that I can change the load as new information becomes available, on the spot. Generally, back in the days when I was shooting registered competition, and I had a decent day, my 100 yard aggregates (consisting of the average of five, five shot groups) were under a quarter inch, with the usual result of being mid to top third pack.
How do you work up loads?
When I go looking for a powder charge node, testing a new one, for my 6PPC my initial test is one shot per load, with, .3 grain intervals, shot over flags, on a day when the wind is easy to read, from an excellent bench. I run the loads from a point that is lower than I am likely to use, to the point where I get pressure signs that tell me to stop. My seating depth remains constant, and is based on my experience. I use a bullet that has performed well with other powders. All shots are fired on a single target, and as I shoot them I take note of how bullet holes cluster and string. After that part of the test, I look for a tight cluster that includes perhaps three different charge weights, and load to the middle weight. Then I take particular care and shoot two shots as rapidly as I can, taking perhaps 5-6 seconds, in a condition that I believe will hold for longer than that, if it does, I believe the results are representative of that load, and if they are excellent, I test with groups of a larger number of shots. I do this loading at the range, so that I can change the load as new information becomes available, on the spot. Generally, back in the days when I was shooting registered competition, and I had a decent day, my 100 yard aggregates (consisting of the average of five, five shot groups) were under a quarter inch, with the usual result of being mid to top third pack.
How do you work up loads?
Boyd, he's not asking how to work up loads. He's asking what it takes to tell whether two loads group differently, with 95% confidence in the ranking.
Two loads, A and B. Clearly, 1 shot with each load isn't enough, because 1 shot produces no measure of dispersion. So, how many shots is enough?
Another way to think about this is whether you would be confident ranking the two loads after 2 shots. How about after 3? 4?
This is an inherently statistical question, and you can't answer it without some reference to sampling distributions.
Two loads, A and B. Clearly, 1 shot with each load isn't enough, because 1 shot produces no measure of dispersion. So, how many shots is enough?
Another way to think about this is whether you would be confident ranking the two loads after 2 shots. How about after 3? 4?
This is an inherently statistical question, and you can't answer it without some reference to sampling distributions.
I have had the class, and was able to do the math. What I am saying is that the tools need to be sensibly applied. If I test a load in my PPC under perfect conditions, and there is paper between two bullet holes at 100 yards, that is all that I require to move on. The application of statical tools without regard to the realities of the situation to which they are being applied, waiving off important variables " for purposes of discussion" is as near to useless as anything that I can think of. I fully understood the question.
Similar threads
Upgrades & Donations
This Forum's expenses are primarily paid by member contributions. You can upgrade your Forum membership in seconds. Gold and Silver members get unlimited FREE classifieds for one year. Gold members can upload custom avatars.
Click Upgrade Membership Button ABOVE to get Gold or Silver Status.
You can also donate any amount, large or small, with the button below. Include your Forum Name in the PayPal Notes field.
To DONATE by CHECK, or make a recurring donation, CLICK HERE to learn how.
Click Upgrade Membership Button ABOVE to get Gold or Silver Status.
You can also donate any amount, large or small, with the button below. Include your Forum Name in the PayPal Notes field.
To DONATE by CHECK, or make a recurring donation, CLICK HERE to learn how.