Statistics for Handloaders
- Thread starter chkunz
- Start date
memilanuk
Gold $$ Contributor
To my mind at least, there is a difference between 'exploratory data analysis' aka 'EDA', which is really what most people's load development amounts to, and actually trying to prove that load A is better than load B with some degree of confidence. Running stupidly high shot counts on each level of the former just wastes time and materials.
CharlieNC
Gold $$ Contributor
I agree, especially when much of the variability is due to the shooter. To stay in practice for Fclass I try to take 10 shots every week at our 300yd range. Keeping track of score, group size, etc the target results of one batch of optimized loads is not identical every week and reflects the combined effect of load + my shooting capability. After the initial brief load development , the bottom line is I am always addressing my new faults!
CharlieNC
Gold $$ Contributor
Well I took a look at the Lyman pages. Bad news is this is a mis-application, t-test is not properly used to compare variability. Good news is that there is a simple way to test for significance using any number of samples to determine if there is a significant difference in shot dispersion. Group size, like chrono es, is not the best to use. For the five shot examples instead of measuring the group size for the two loads, use the individual shot data. For example On Target gives you the distance of each shot from the center of the group as shown below. The proper test to compare whether the variability is different is the F-test. In Excel you can use the F.test function to do this; the result of this below indicates there is a .37 probability that the variability is the same. Or said another way that there is a 63% chance that the loads resulted in differences in variability. So there is a 63% probability that Load A group size is better than Load B.
There is no magic number of samples that give significant results; it depends on how fine you want to cut the hairs and how much variability obscures your vision. Probably more than you wanted to know.
There is no magic number of samples that give significant results; it depends on how fine you want to cut the hairs and how much variability obscures your vision. Probably more than you wanted to know.
CHKunz, I hope you didn't take my post as negative in any way. The statistics are great for mathematicians, which I am not. If I wanted to do a math project this would be great but I want to make a small group. I was following the math of all this load data that looks at velocity and was watching ES. I tinkered with the load on my 7RUM and got it in to single digits, I was super happy. At distance it was the worst load I had tried so far. I think that rifle accuracy has toooo many variables to allow math to guide you to the goal. Math is exact, rifle accuracy is not, there are no rules. Which bullet is best, surely some of the very best custom bullets will shoot the best in all rifles. NOT.
ebb, I do not take the criticism personal. Most folks agree with you but a small number may choose to consider using statistical analysis to help improve their performance. Mathematics is already a factor in much that we do and statistical analysis is just one more tool that we can use in some of what we do. It takes some effort to learn to use the tool and even more effort to learn how and when to use the tool. Did you read the paper? If not I would recommend that you at least read the introduction. Best regards, Clyde
memilanuk
Gold $$ Contributor
Just to continue kicking this can down the road...
@CharlieNC you mention using the F-test to compare the variability between two samples. The subject came up in discussion with someone a little while back, about whether it was possible to compare things such as standard deviation straight up across the board i.e. to say whether an SD of 5 was really that much better than an SD of 7. One of the things that I came across when looking into it further was that many (most) variance tests are heavily reliant on an assumption of normality in the data. If that assumption is incorrect, then the whole test becomes suspect.
@CharlieNC you mention using the F-test to compare the variability between two samples. The subject came up in discussion with someone a little while back, about whether it was possible to compare things such as standard deviation straight up across the board i.e. to say whether an SD of 5 was really that much better than an SD of 7. One of the things that I came across when looking into it further was that many (most) variance tests are heavily reliant on an assumption of normality in the data. If that assumption is incorrect, then the whole test becomes suspect.
CharlieNC
Gold $$ Contributor
Yes. If you are looking at extremely small target groups as in short range BR then the individual shots are not normally distributed. I can tell you my shots in Fclass are normal, and would be shocked if most are not. Measure a few of your targets to see. You can also pool ("average") sd to increase the sample size, to achieve a more robust analysis. For example the chrono sd using the same load but shot on different days, or the individual shot variability on those days.
memilanuk
Gold $$ Contributor
What would be the best way to go about assessing normality on a target? Plot the X & Y coordinates separately? How do you separate out the variability due to wind (primarily on the Y-axis, but also on the X-axis to a degree) and other effects (i.e. optical distortion due to mirage, etc.)?
Please send a pdf file to oneutdon@gmail.com. Thanks, Don
CharlieNC
Gold $$ Contributor
Monte a lot of wind does require more analysis. Histograms on x, y, and radial distance will tell the entire story regarding normality. Regarding how to quantify the various factors you mentioned I do not have a magic "bullet" to suggest, just brute force logging of the factors vs the performance parameters for shooting sessions. A control chart of those parameters will quickly highlight when one changes. I'm not that industrious and just track score, vertical spread, and horizontal spread for my shooting sessions. This year my sub par performance is all me!
memilanuk
Gold $$ Contributor
@CharlieNC are you doing everything in Excel, or are you using more specialized software like this?
CharlieNC
Gold $$ Contributor
You can do quite a bit with excel and customize it to suit your objectives; I'm partially on this path now. For work I have used Minitab stat package for years, so I use it the most.
dave@aDave
Gold $$ Contributor
Mean radius with a minimum of 20 shots will get you started on how a load will shoot.
I use mean radius and shoot 10 shot groups and multiple ten shot groups to confirm a "pet" load. It is best to shoot the groups on different days. But the question we were discussing is how to evaluate a change in a component to see if there is a significant probability that the change has improved the load or if we are just seeing the normal variability in the data. The reference I gave uses a simplified version of the statistical "T" test at a 95 percent probability and is easy to use, however CharlieNC says the F test is the proper statistical analysis.
Charlie, I have been looking in several references to learn when the F test is more appropriate in data analysis than the T test. Can you provide some guidance on this question?
memilanuk
Gold $$ Contributor
...and that right there is part of the problem why this doesn't gain more traction. Most people don't have a $1600 software package available to them unless someone else (business/employer) is paying for it, and don't have the time to figure out how to bludgeon Excel into doing the work (square peg / round hole), nor the underlying stats knowledge to get something like R or Python to do the work. There are a few web apps like this that help, but more is needed.
memilanuk
Gold $$ Contributor
The t-test is for comparing the mean of two samples (or one sample against a standard). The f-test is for comparing the *variance* of two (or more) samples.
Last edited:
That sounds like the application of the T test in the Lyman article was appropriate, however, CharlieNC thinks not, hopefully he will comment. I am working on a shooting project and would like to do some basic statistical analysis and want to get it right.
XTR
F-TR obssessed shooting junkie
Stats are cool, and sometimes they can be very helpful in ruling out the significance of an anomaly, and they can be infuriating when you find out you have to revisit a sight and collect more data because for whatever reason the confidence level isn't what was called for. (I deal with them in my work)
On another website a stats guy challenged everyone to flip a coin 50 times, record the results( 1= heads, 2=tails), or just make up 50 results and post them and he would tell you whether or not you really did it.
After amazing a few people with his accuracy he gave away one of the "secrets" as it were. He pointed out that in a data set of 2 possible results there will almost always be a string of 7 in a row in there somewhere, and nobody making it up will put that many in a row. Made it easy to spot the fakes.
My point is this, one or two 3 or 5 shot groups can certainly tell you if a load is bad, but they can't tell you if it is good. You need more data for that, and you can't "ignore the flyer" or you are wasting your time.
On another website a stats guy challenged everyone to flip a coin 50 times, record the results( 1= heads, 2=tails), or just make up 50 results and post them and he would tell you whether or not you really did it.
After amazing a few people with his accuracy he gave away one of the "secrets" as it were. He pointed out that in a data set of 2 possible results there will almost always be a string of 7 in a row in there somewhere, and nobody making it up will put that many in a row. Made it easy to spot the fakes.
My point is this, one or two 3 or 5 shot groups can certainly tell you if a load is bad, but they can't tell you if it is good. You need more data for that, and you can't "ignore the flyer" or you are wasting your time.
Upgrades & Donations
This Forum's expenses are primarily paid by member contributions. You can upgrade your Forum membership in seconds. Gold and Silver members get unlimited FREE classifieds for one year. Gold members can upload custom avatars.
Click Upgrade Membership Button ABOVE to get Gold or Silver Status.
You can also donate any amount, large or small, with the button below. Include your Forum Name in the PayPal Notes field.
To DONATE by CHECK, or make a recurring donation, CLICK HERE to learn how.
Click Upgrade Membership Button ABOVE to get Gold or Silver Status.
You can also donate any amount, large or small, with the button below. Include your Forum Name in the PayPal Notes field.
To DONATE by CHECK, or make a recurring donation, CLICK HERE to learn how.