Statistics

[Alex Wheeler]

I want to stir the pot a little on this discussion. To ad another point of view to the discussion around statistics. Lately theres been a lot of talk about what is statistically relevant when it comes to testing things or tuning a rifle. Most of the stuff I have seen comes from people with a scientific background. I wanted to ad a point of view from a background of building rifles that have to win matches and shoot the smallest aggs possible.
In my opinion, we want to tune a barrel in the least statistically relevant way possible SO LONG as it produces results. If I could tune a barrel in 5 shots and go break a record and win matches consistently with that method it would be the best tuning method there is. But also the least likely to stand up to scientific scrutiny.
The reality is that none of my tuning methods, or any of the methods used by my customers to win or break records would be considered statistically relevant. It blows my mind when I read some of this stuff. We have to test powders, primers, neck tension, seating depth, tuner test, ext. There are ways to do this with only a small sampling. When I am starting with a new rifle and I want to try 3 or 4 different powders, it only takes about 10 shots to know if that barrel likes that powder or not. Same goes for primers. Group size is another thing. 3-5 shot groups are not relevant? I dont have enough fingers and toes to count the records that say they are. What are we trying to do? Tune a rifle or create data?
What I am trying to stop is the guy that calls and says he is struggling to tune his rifle, hes shot a ton of groups. Piled up plenty of statistically relevant data, but has only tried one powder, one bullet, and one primer, and now has his barrel 1/3 shot out because he would not make decisions based on a small sampling.

 

[Jager]

I think there's absolutely a case to be made for statistical relevance for most rifles. We've all known the guy who runs a ladder, shoots a beautiful, tiny 3-shot group... and nods his head happily as he cuts out that target for his wallet. He's entirely convinced both that he has a fabulous rifle; and that he has found the load that that rifle likes.

He's genuinely shocked when, somewhere down the line, that rifle, shooting that same load, opens up. No worries, though, he rationalizes to himself. "Must've pulled it."

A mild dose of statistics might have disabused him of a bit of his earlier exuberance.

But... there are rifles. And then there are rifles.

Once one enters the exquisite, arcane world of super accuracy, things are different. Finding a load and keeping a load seems more the rule than the exception. It's like the normal rules of statistics are bent until they're turned on their head.

Might be that when we spend the time and effort and coin on a custom action and a custom barrel, put together by a man who knows what he's doing, and then grace it all with a great optic... the inherent benefit of all those things conspires to make a little magic.

 

[DLT]

How do you when to change powder or a bullet. How many groups does it take to say hey this rifle doesn’t like this bullet or this powder ? I struggle with just that part of the process. I have shot many many 1 hole 3-5 shots groups but hardly ever do they repeat either that session or the next.

 

[okie]

I want to stir the pot a little on this discussion. To ad another point of view to the discussion around statistics. Lately theres been a lot of talk about what is statistically relevant when it comes to testing things or tuning a rifle. Most of the stuff I have seen comes from people with a scientific background. I wanted to ad a point of view from a background of building rifles that have to win matches and shoot the smallest aggs possible.
In my opinion, we want to tune a barrel in the least statistically relevant way possible SO LONG as it produces results. If I could tune a barrel in 5 shots and go break a record and win matches consistently with that method it would be the best tuning method there is. But also the least likely to stand up to scientific scrutiny.
The reality is that none of my tuning methods, or any of the methods used by my customers to win or break records would be considered statistically relevant. It blows my mind when I read some of this stuff. We have to test powders, primers, neck tension, seating depth, tuner test, ext. There are ways to do this with only a small sampling. When I am starting with a new rifle and I want to try 3 or 4 different powders, it only takes about 10 shots to know if that barrel likes that powder or not. Same goes for primers. Group size is another thing. 3-5 shot groups are not relevant? I dont have enough fingers and toes to count the records that say they are. What are we trying to do? Tune a rifle or create data?
What I am trying to stop is the guy that calls and says he is struggling to tune his rifle, hes shot a ton of groups. Piled up plenty of statistically relevant data, but has only tried one powder, one bullet, and one primer, and now has his barrel 1/3 shot out because he would not make decisions based on a small sampling.

I think you have done enough tuning to know what to hold as a constant and what to use as a variable. Some have been successful enough to develop controls. These may be in your mind and not actually used to compare each time you tune. (not very scientific--but we know what happens/and what will not happen) If you held every variable seperate, we both know you are going to wear the barrel out before you are finished---thus making a constant a variable. That being said barrels change....How do you account for this??? Doesnt matter. The experience is what makes it work...and stumbling upon a hummer doesnt hurt either! this isnt a scientific method. its more of an art.
Flame on ...marc
In laymens terms: you KNOW what works and what doesnt. There is no substitute for experience.
Only experience allows this...

 

[Doom]

I want to stir the pot a little on this discussion. To ad another point of view to the discussion around statistics. Lately theres been a lot of talk about what is statistically relevant when it comes to testing things or tuning a rifle. Most of the stuff I have seen comes from people with a scientific background. I wanted to ad a point of view from a background of building rifles that have to win matches and shoot the smallest aggs possible.
In my opinion, we want to tune a barrel in the least statistically relevant way possible SO LONG as it produces results. If I could tune a barrel in 5 shots and go break a record and win matches consistently with that method it would be the best tuning method there is. But also the least likely to stand up to scientific scrutiny.
The reality is that none of my tuning methods, or any of the methods used by my customers to win or break records would be considered statistically relevant. It blows my mind when I read some of this stuff. We have to test powders, primers, neck tension, seating depth, tuner test, ext. There are ways to do this with only a small sampling. When I am starting with a new rifle and I want to try 3 or 4 different powders, it only takes about 10 shots to know if that barrel likes that powder or not. Same goes for primers. Group size is another thing. 3-5 shot groups are not relevant? I dont have enough fingers and toes to count the records that say they are. What are we trying to do? Tune a rifle or create data?
What I am trying to stop is the guy that calls and says he is struggling to tune his rifle, hes shot a ton of groups. Piled up plenty of statistically relevant data, but has only tried one powder, one bullet, and one primer, and now has his barrel 1/3 shot out because he would not make decisions based on a small sampling.

I am one of those people that you describe with a scientific/engineering background and have posted many times on the subject. Your question(s)/observations are spot on. People have developed loads for years without detailed statistical analysis. From an analysis standpoint let me "analyze" one of your statements.

You take four powders and test them. I assume you are shooting groups. Those four powders were not just randomly selected because you have a lot of background information on what powders work for a given cartridge. You also know that a given bullet has a preferred seating depth (at least sometimes). You probably have a good idea of charge weight based on experience. Now you load and shoot your four powders. Two of the four show great groups but the other two show a flier (a shout outside the group). So you pick the powder. It's possible your 10 shots might have produced a flier on one of the good groups if one more shot was taken. The point here is there was a lot of data that got you to this point. We tend to call this anecdotal evidence and it certainly works. You did pick a good powder, it's possible you may have rejected a good powder.

Now let's take a look at a shooter that is looking to reload to a standard deviation of 10 fps. He loads three or five rounds and shoots them over his chronograph. He sees one case where he has a SD of 15 and another SD of 8 with a different primer. He sees this a proof that there is proof that the primer for the 8SD is the way to go. If he shot five shots his 95% confidence interval is 9.0, to 43.1. for the 15 SD data and 4.8 to 23 for the 8 SD data. If you analyze the SD data (F Test) it says there is not enough data to say there is a difference in the two SDs or that the 8sd is actually less than the 15sd. This is a case where it's entirely possible to make the wrong decision based on the very limited amount of data. He thinks he has a single digit SD but probably not and he may have the wrong primer. He may well be looking at the SD only and ignoring what the target looks like.

 

[FeMan]

Its tough for shooters that want precision and accuracy but dont have the experience. When I started at this 40 years ago I did a LOT of testing. As I gained more experience and compiled a larger physical and mental data set, I have been able to short cut a lot of the testing as my experience let me know where that bunny trail led.

In addition, sites like Accurate Shooter have given me access to a ton of valuable information from other shooters. I don't buy into everything I read here but I have learned a lot from others that I can quickly validate and add to my trusted data base.

So Alex, you are at the pinnacle of knowledge and experience. Have patience with those at the bottom of the mountain. I am not sure why, but most people would rather climb that mountain hand-over-fist than take the shortcuts offered by others.

 

[Acyr]

Alex, I think it boils down to experience and ability. We all started out knowing nothing. I still don't know, what I don't know.
As a new shooter I shot lots of groups, did lots of testing and burned up a lot of components. Today it takes me no time to develop a load because experience has taught me how to do it quicker with less shots down range. Statistics mean nothing to me today.
How many shooters on the line have the best equipment and great shooting rifles but don't have the ability to set National records or win big matches. That is where ability comes in.
In my working life I swore at engineers on a daily basis. All of their research, data and statistics told them it was going to work great. But out in the field it was a different story.
Getting frustrated at guys who maybe don't have the experience or ability that you have is a normal reaction because you are always willing to help people. I spent a large part of my career mentoring young technicians. I understand how you feel.

 

[okie]

Its tough for shooters that want precision and accuracy but dont have the experience. When I started at this 40 years ago I did a LOT of testing. As I gained more experience and compiled a larger physical and mental data set, I have been able to short cut a lot of the testing as my experience let me know where that bunny trail led.

In addition, sites like Accurate Shooter have given me access to a ton of valuable information from other shooters. I don't buy into everything I read here but I have learned a lot from others that I can quickly validate and add to my trusted data base.

So Alex, you are at the pinnacle of knowledge and experience. Have patience with those at the bottom of the mountain. I am not sure why, but most people would rather climb that mountain hand-over-fist than take the shortcuts offered by others.

yep

 

[Intheshop]

Going to be tough getting in the fast lane WRT the time required to tune,loading anywhere other than at the range....

I'll take a 3 shot group fired on 3 consecutive days,rain or shine,no foulers/sighters over almost any protracted stats... admittedly more of a hunting vibe but it does cut to the core.

 

[linebaugh]

In my experience I see and hear the SD and low ES pushed as gospel for what it takes to make a gun shoot bugholes. It seems many will ignore a target over and over just to come back to low SD. The problem with that is it is typically a very small snapshot of where the rifle is performing at that point in its life and is derived from an even smaller snapshot of data points. Basically I see the set it and forget it mentality based on one outing and 10 rounds.
Again in my experience I have seen a lot of rifles shoot incredibly well with less than stellar SD numbers. In this case I ignore all of that data and let the targets dictate where my loading ends up. I have always done this personally.
I have also seen many a guy burn out a barrel by not listening or "expecting" a rifle and load to produce "X" group size. By gosh if it wont shoot in the .0's we need to tune everything to infinity to get there only to find out 500 rounds later that it's just not going to happen.
I have also seen people buy a custom everything and expect .0's that also decide when they are short on ammo that whatever walmart or the local shop has will work because they were either to lazy or did not plan ahead far enough to supply themselves with the components they needed to keep that rifle running.
How do you wrap your head around all of this? I just try to ignore it personally and do me. At the end of the day if someone is happy or not so happy chasing rabbit holes then they need to learn just like the rest of us had to learn.

 

[K22]

The kind of matches I shoot are typically one shot, the target is not stationary, the distances to the target vary greatly, the environmental conditions are widely variable, the target profile is not consistent, often I'm on vary ground conditions requiring an uphill / downhill etc. shot. The 10 ring is small and it's either scored as a hit or miss, you don't get any points for missing the 10 ring. There isn't much margin for error, and I don't have the luxury of a bench with cover blocking the sun. I am a varmint hunter.

I have been educated in statistical analysis as a quality assurance engineer. And while it certainly has its applications for certain processes which I used extensively in my career, I don't think it works too well with the shooting sports because of all the variables involved.

Component lot to lot variation notwithstanding, wind, mirage, varying light and temperature, at least for me, are the most difficult elements to manage successfully, not load development. Then there is the big one, the ability of the shooter to consistently apply the fundamentals, shot after shot. This is especially true in any practical shooting discipline which is why I like practical shooting; it tests the shooter to the maximum.

My approach is the Mr. Cleckner approach, I establish an accuracy standard that will produce the results I need. For me, it's the 1/2 to 5/8" aggregate MOA. Once I achieve that standard, I focus on marksmanship skills because for me, that is the most significant variable in my game.

 

[CharlieNC]

Most of those quoting statistical relevance don't know what it means. I have installed a number of prefit barrels on actions that shoot well, and it's like comparing my knowledge and skills to being a bona fide gunsmith such as you Alex. Must have 95% confidence, why? Wouldn't you be happy with 75% odds at Vegas.

The simple question is to be able to separate the signal from the noise. If you load one bullet 0.5gr heavier charge and shoot over a chrono and it's 100fps faster are you confident that is different? Well you also know or have a "feel" that the typical variability is around 25fps ES, so of course it's different. But not statistically significant? Knowing the noise does can come from numerous aspects ( including experience), and being able to distinguish a large difference is not difficult. On the other hand it can be nearly impossible to have an adequate sample size to distinguish a minor difference at 99% confidence.

Much of the misperception comes from trying to say my system shoots x MOA all day long, and use that as the statistical benchmark as some have proposed. Must shoot 100 groups over a month to have a baseline for judgement! If this were the case there would be little to no improvement in industry and other fields, but those perspectives and approaches are beyond the time frame for this forum.

 

[BoydAllen]

What I would really like to see (but never will) is a questionnaire that was required to be filled out as a part of the process of coming onto a forum and asking a question relating to accuracy. Yes I know that this will never happen. It would have boxes to check for things like "I have a set of wind flags and use them regularly." ; My rifle's action has been either glued in or pillar bedded and in that case the bedding has been properly checked using a dial indicator.", and so forth. So often I see fellows who choose to ignore important factors while then attempt to do the whole thing with powder charge and or seating depth. They often start off talking about SD even though they have yet to shoot a satisfactory group at 100 yards, where SD is not really relevant. Certainly I believe that the internet can provide a wealth of useful information, but it also poses the problem of the inexperienced not knowing the difference between things that are absolute requirements and those that are only important if everything else is correct. For example, I find it common for guys that have yet to shoot well at short range to think that they can fix that by doing things like weighing to two places. Typically they tell me about their SD as if it was a primary consideration, something that should be sought before better groups. I tell people of national records shot with thrown charges (short range) and it goes in one ear and out the other. Bottom line they seem to think that they can choose what variables to pay attention to, and what they can ignore, without reference to what their rifle is desperately trying to tell them. Shooting a lot of groups to create a decent sample size will do nothing to fix any of these things. There is some comedy in all of this, like when someone refers to the SD of a three or five shot sample. It seems to me that it comes down to one central fact. Some people are just better problem solvers than others, and from my observations, this does not correlate to formal education. Statistics is a tool, but it is often not the right tool for the job because we do not have the luxury of unlimited time, materials, or budgets, and lacking tunnels, most of us do not have the luxury of living where good test conditions are frequent.

 

[sdbob]

If you want to cut down on what it takes to tune a new barrel. Own a reamer, learn what it takes to make it preform and repeat.

 

[INTJ]

I agree with @Alex Wheeler 1000%. I have been saying this for a while.

This approach isn't really a matter of experience and knowing what loads to pick. It's a matter of knowing HOW to tune a rifle for your discipline. I am still learning for 1K BR.

I used to shoot lots of five-shot groups. I would easily have 400 rounds through a barrel before being happy with a load. A load that still had to be checked before every match. 400 rounds is very often 1/3 used up for a LRBR barrel.

Then I starting tuning the way Alex (and @tom and @GlennK and a bunch of others) tune. My groups shrank and I put a lot less rounds though my barrels.

The issue is each time we pull the trigger the barrel changes. There aren't really any constants, so it makes it hard to apply the typical straight-up statistical analysis meaningfully.

That said, I am sure there is a way to use statistics that fits LRBR, but it will be based on match results, not raw data analysis.

 

[Bill Norris]

Well, reading thru threads like these sometimes makes me realize one common denominator does and always will hold true to form. No matter how much statistical evidence is ever produced by any shooter nobody will ever be perfectly on the money. WHY ? Simply because no matter how much testing is done at any point, time, and place, with any rifle or any shooter the atmospheric conditions will always change and be a factor that even John Myers has not over come and never will. Yes, he and some others have a good grip on how to tune but there are times and conditions that dictate their theory don't work. If you ask either of them and they are truthful they will tell you the same. SO, I totally agree the shooting field is no place to try to enter statistics. Statistics are only as good as for how long your present barrel and tune will last under the conditions in which you are shooting.

 

[Bottom Fodder]

The tuning job is never done and the weak link is the shooter
just my $0.02

 

[Bill Norris]

The tuning job is never done and the weak link is the shooter
just my $0.02

True, Because even with the best of bench manners the shooter is not smart enough to over come and tune for every atmospheric change.

 

[DLT]

I agree with @Alex Wheeler 1000%. I have been saying this for a while.

This approach isn't really a matter of experience and knowing what loads to pick. It's a matter of knowing HOW to tune a rifle for your discipline. I am still learning for 1K BR.

I used to shoot lots of five-shot groups. I would easily have 400 rounds through a barrel before being happy with a load. A load that still had to be checked before every match. 400 rounds is very often 1/3 used up for a LRBR barrel.

Then I starting tuning the way Alex (and @tom and @GlennK and a bunch of others) tune. My groups shrank and I put a lot less rounds though my barrels.

The issue is each time we pull the trigger the barrel changes. There aren't really any constants, so it makes it hard to apply the typical straight-up statistical analysis meaningfully.

That said, I am sure there is a way to use statistics that fits LRBR, but it will be based on match results, not raw data analysis.

Maybe you could share that insight for probably the 100th time. For those of us who has no clue or are late to the party

 

[RegionRat]

What I am trying to stop is the guy that calls and says he is struggling to tune his rifle, hes shot a ton of groups. Piled up plenty of statistically relevant data, but has only tried one powder, one bullet, and one primer, and now has his barrel 1/3 shot out because he would not make decisions based on a small sampling.

This guy wasn't using statistics or anecdotal experience the right way, so he doesn't represent either side of the arguments well.

He is what we call a victim of paralysis by analysis.

Some people are just better problem solvers than others, and from my observations, this does not correlate to formal education.

This^
Large defense/aerospace companies have thousands of engineers and scientists who are examples of folks who cannot engineer their way out of a wet paper bag.
They can do some things, but they cannot be used to lead a team or be counted on to make decisions by themselves in the real world.
It isn't because they faked their diploma, it is because mother nature is cruel and some folks are just not good at managing chaotic topics. Many can do the math and pass academic tests, but it doesn't mean they can manage risk.

Even if we limit the discussion to LRBR, there are folks who you can try to help that will just never win a match or even place well against their peers. Just be polite and help when and if you can, but also learn to recognize what we call time bandits or they will suck you dry and waste your time.

If they don't have the shooting background and don't spend their time on the range in weather, they will never be able to use the advice they are given. Let them be.

My advice is... Don't get to focused on bashing math. The math and statistics are not to blame. If we put the wrong driver in a capable race car and they wreck it, that isn't always the fault of the car.

Any fool with a computer can run statistics, but it doesn't mean they understand how to drive down risk or make a decision. The F500 companies went bonkers pushing concepts like 6-Sigma and produced thousands of so called "experts", and the vast majority of them can't manage to make a decision to save their own necks.

If you have the talent to drive the rifle and read wind, you can probably learn to tune a group without a formal credential in math or science. At the same time, beware the folks who can also do the math cause they are dangerous.