Does “testing” repeat itself?

[rhovee]

I have done testing of different primers and neck tension on multiple rifles. Every time there is a primer or specific neck tension that stands out more than others. Recently talking to a friend about this, he has some hesitation if a one day primer test is sufficient to say it’s better than others. We have all seen our rifles shoot good one day or evening, and be disappointed the next. I am sure at least one of you has tested primers or next tension on more than one occasion. Curious if they are a fixed variable once determined.

 

[fkimble@charter.net]

If you can repeat it with same results great. If different results you need to try something else.

Frank

 

[Straightshooter1]

I have done testing of different primers and neck tension on multiple rifles. Every time there is a primer or specific neck tension that stands out more than others. Recently talking to a friend about this, he has some hesitation if a one day primer test is sufficient to say it’s better than others. We have all seen our rifles shoot good one day or evening, and be disappointed the next. I am sure at least one of you has tested primers or next tension on more than one occasion. Curious if they are a fixed variable once determined.

It's not good to rely on anecdotal evidence, even if it's your own. If one is going to follow the scientific method, one tests more than once and if the results are different, then one needs to find out what the cause is. If one tests more than once, keeping all the variables "the same" (for lack of a better way to put it), the results will be the same no matter when the test is conducted. The trick is . . . keeping all those variables the same.

 

[mikecr]

If it actually is a fixed variable, as in -not varying, and it tests best, it is best (until you change something). So what you're talking about IS change (like temperature).
And the question would be whether other attributes can affect primer or neck tension gains.

One 'day' of the year in itself means nothing to another, so what change are you considering?

 

[rhovee]

I am curious if someone has tested neck tension or primers on multiple days and arrived at same conclusion.

 

[darkangel_r2]

If the result is valid the test will repeat. Just make sure you are only varying one thing at a time.

 

[jelenko]

It's not good to rely on anecdotal evidence, even if it's your own. If one is going to follow the scientific method, one tests more than once and if the results area different, then one needs to find out what the cause is. If one tests more than once, keeping all the variables "the same" (for lack of a better way to put it), the results will be the same no matter when the test is conducted. The trick is . . . keeping all those variable the same.

And, knowing what all the variable are.

 

[Hammer]

It's more fun to change 30 or 40 things at one time...

 

[RegionRat]

It is easy to under sample and get lulled into assuming you now know something that you can extrapolate into the future, and then get disappointed when that performance doesn't repeat the next day. It happens all the time and most folks don't talk about those test groups, only their good ones.

One of the roles I played before retiring was to review "research and design" work, either risky jobs or work that was already in trouble. I also reviewed testing specification standards for things like medical tests and material tests, and testing things like ammunition and ordnance.

It was often the case that the expense of the testing needed versus the actual budgets didn't line up so folks said they "did the best they could" and tried to go with too few tests or the wrong tests. My job was then to evaluate the risks that what they were saying would work or fail, but I had to answer to politicians and lawyers, so there were many things to consider.

Engineers, doctors, scientists, etc., want to test forever, politicians and managers don't want to spend a penny. What testing and how much testing is the right amount takes a study and folks have to set a level of risk or probability we are right or wrong.

With some topics, there are contractual standards for the reliability. Words about reliability like P=0.9995 meant that there could no more than 5 failures in 10,000 trials, for example. So that would be a very high standard for real world specifications. Other times, you had terms like L10 of 1000 hours, which wasn't all that great for things where you are risking your life.... one in ten could fail and that was "okay"!?!

With side by side testing for things like primer changes or neck tension, it takes a little study to know how much testing is enough testing to judge the risk you actually are right or wrong. If it is okay to be wrong 5 times in 10, then it isn't very difficult and doesn't take many trials. If you have to promise somebody the answer is one they can bet their life on, then it takes more samples and tests.

Here is a short paper that I think does a decent job of explaining a simple question without giving the no-maths a headache. If you get through the whole thing you can see how easy it is to find a group size can double and yet still be normal within a recipe.

It is very easy to be misled by small samples of things that are known to have what we call a dispersive distribution behavior. Many times on the forum, the concepts of sample size versus, SD, ES, etc., get muddled. Just because we can take a few numbers and calculate an SD/ES, doesn't mean we know anything about the next day.

Many folks will tell you to test just one thing at a time and that will keep you safe... that only works with very simple relationships like the ones you can describe with three term equations. Very few would claim there are only two other things at work when it comes to shooting performance, so be careful.

The second paper shows some of the dangers of OFAT (One Factor At A Time) testing. Again, when it comes to optimizing a load recipe, one change like a primer might require a rebalancing of other factors to show the real value, or it may just fail no matter what. It takes some luck to get an easy obvious answer, but just remember that it is risky to judge things like primers without a full look.

If you don't have a background in research or testing, just pinch your nose and power through that paper. You will still get the point about how difficult it can be to play with a single variable when things are complicated. They use tomatoes but it only takes a little imagination to see it as shooting accuracy. Till I find a better paper this one will have to do.

I am curious if someone has tested neck tension or primers on multiple days and arrived at same conclusion.

Yes, you just have to be carful not to misjudge turning a single dial on something that might require adjusting the other ones to see the complete answer.

The best primer tests are the ones where the outcome is unaffected by a swap. However, it is important to know that if we get a change, that might mean you don't know the whole answer unless you tune to the new primer. Sometimes, you still get "no" for an answer and your old primers were better.

As a beginner, there is nothing wrong with starting by testing one factor at a time, like a primer change or neck tension change. Just be aware it may take more samples and multiple trials to get a definitive answer, and that answer may just mean that you need to look deeper to know if the new primer is really better or worse.

Curious if they are a fixed variable once determined.

For some weapon systems, take an M-16 for example, there is an ammo specification standard for the "neck tension" because it contains concepts like sealant and crimping. The low ends were for safety in automatics, and the high ends were for performance. Accuracy testing was certainly affected by those concepts so you should set yourself up with a little test some day and go high, medium, low to see if your system has a preference.

Cartridge brass has a yield point that can be hit within the bullet seating and neck sizing process, so each diameter and neck thickness has a zone where it will be more or less difficult to control in the "reloading" context. Be careful not to over work your brass with extremes or when experimenting with very light neck tension. YMMV

 

[XTR]

I am curious if someone has tested neck tension or primers on multiple days and arrived at same conclusion.

Yes to both.
If it won’t repeat it was a data scatter or it was an anomaly. Thats just simple data validation.

I’m not saying I’ve never taken a data point and run with it but I try to go back and validate trends or data points that seem to be significant

 

[JFrank]

I am curious if someone has tested neck tension or primers on multiple days and arrived at same conclusion.

I have done testing of different primers and neck tension on multiple rifles. Every time there is a primer or specific neck tension that stands out more than others. Recently talking to a friend about this, he has some hesitation if a one day primer test is sufficient to say it’s better than others. We have all seen our rifles shoot good one day or evening, and be disappointed the next. I am sure at least one of you has tested primers or next tension on more than one occasion. Curious if they are a fixed variable once determined.

Ryan, I did primers similar to what you're asking and ended up chasing my tail a bit so now I try to follow the clues and read the target hopefully one time through towards the basic tune then continue to fine tune as required.
J

 

[Alex Wheeler]

I am curious if someone has tested neck tension or primers on multiple days and arrived at same conclusion.

Yes. But the test has to be a good one. Neck tension is a pretty easy one. I have done that one many times in a BR rifle at 1k and it repeats. I shot 3, 3 shot groups with different bushings like I would do for primers. The problem is when you dont shoot enough groups or enough shots. Also the tune has got to be good to start with. The number one reason for a load to not repeat in my experiance, is that the barrel does not like the powder your burning.

 

[WyleWD]

I have done testing of different primers and neck tension on multiple rifles. Every time there is a primer or specific neck tension that stands out more than others. Recently talking to a friend about this, he has some hesitation if a one day primer test is sufficient to say it’s better than others. We have all seen our rifles shoot good one day or evening, and be disappointed the next. I am sure at least one of you has tested primers or next tension on more than one occasion. Curious if they are a fixed variable once determined.

I'd agree with your friend that a one day good result on a primer or neck tension test is enough to say that it is "better".
If it shows the same results in subsequent tests, say 3 or 4 times tested and always shows better results, then I'd say that it is a tendency to show better results in your equipment, your materials, and skill level. It might not work for everyone else. JME. WD

 

[243BR]

IMHO this is the great challenge for improving/maintaining accuracy. There are SO many variables that we are dealing with, some we simply can't control (again IMHO). Even if we could shoot enough groups for a test to be statistically significant the barrel would be "shot out" long before it was complete.

 

[Orr89rocz]

Some days i think i myself as the shooter isnt as consistent as on another day and that may explain some of my groups lol.

 

[CharlieNC]

To a great extent the ability to repeat a difference is simply dependent on the magnitude of the difference vs the typical variability observed. For example if you typically observe 1.5moa groups +/- 0.25, change something and observe 0.5moa then that improvement will likely repeat. On the other hand if the improvement yields 1.2moa then it will require repeating to confirm. Statistically the ability to distinguish a minor difference requires many samples and there is not a fixed number, whereby major differences only require few samples to differentiate. There are simple formulas which provide this guidance, but all statistical methods have underlying assumptions to be applied properly. Generally replication is required for validation given the other unknowns beyond those being tested which influence the results.

 

[Lefty Trigger]

I can't even get my key to the range lock to work consistent! I found that with primers I test each one after the load is found and pick the one that gives me the best ES/SD numbers and run with it. I'm turning necks now to clean them up to give better consistent neck tension. My issue is the weather changing things and I have found that the time of day has some weird affect on the optic? POI shifts from morning sun to afternoon sun??

 

[BoydAllen]

If you are going to test something, you need a reliable tool. I have one rifle that I can depend on to give me back what I give it. I have excellent bags and rest that fit it, that have performed will over the years, and I use flags, and I don't test when the wind is challenging. When I do testing with that rifle, I have found that I can depend on the results, but many shooters do not own reliable test instruments.

The mistakes that I most commonly see are in how the test is designed. Fellows will test primers by only changing them and nothing else when they are trying to determine which gives the best accuracy. IMO to do that you need to treat each primer as a separate situation and do the best that you can to make it shoot, altering other variables just as you would when working up a load with a single primer.

Guys who are statistics geeks are the worst, they get so deep in their formulas that they loose sight of test design. Close behind them are those that claim superior knowledge of scientific method.

Another issue that I see all the time is fellows referring to the SD of a small sample. Give me a break.

Bottom line, you need to consider the resolving power of your test system.

The truth is that most of what we learn in life is base on anecdotal evidence, and there is nothing wrong with that. In the case of shooting, we have real limits imposed by barrel life, money, and available time.

A good friend used to carry on about the need for elaborate testing to be done about things he found interesting. My retort was always, "No one is stopping you." That usually put an end to it.

I really appreciate it when people share their results, even if their methods are not perfect,

 

[CharlieNC]

The mistakes that I most commonly see are in how the test is designed. Fellows will test primers by only changing them and nothing else when they are trying to determine which gives the best accuracy. IMO to do that you need to treat each primer as a separate situation and do the best that you can to make it shoot, altering other variables just as you would when working up a load with a single primer.

Boyd this exactly describes the situation where common sense statistics and engineering excels in getting the most information from the least effort. For example Tony Boyer reports fine tuning using a grid of 5 charge weights x 5 seating depths = 25 total trials with 2 shots each. This is a classical full factorial design. Unless there is a reason that only 1 of those 25 grids has the correct anwer, there are several statistical designs which can answer this using only 9 trials (plus a couple more to replicate a given trial a couple of times); and in the process seek to understand Mr Boyer's methods of reading the tea leaves to incorporate into the analysis. I'm in the process of reoptimizing a load now because the throat erosion got away from me and my 9 loads are ready to shoot in order to find the proper charge/seating combination. Whether its a load development technician, or a PhD engineer / statistician, common sense is the predominant trait needed. Evaluations that to on ad nausea are not an inherient trait of the person's background, but the management culture. PS - and question those who say anything less than 20 samples is statistically meaningless.

 

[Alex Wheeler]

If you are going to test something, you need a reliable tool. I have one rifle that I can depend on to give me back what I give it. I have excellent bags and rest that fit it, that have performed will over the years, and I use flags, and I don't test when the wind is challenging. When I do testing with that rifle, I have found that I can depend on the results, but many shooters do not own reliable test instruments.

The mistakes that I most commonly see are in how the test is designed. Fellows will test primers by only changing them and nothing else when they are trying to determine which gives the best accuracy. IMO to do that you need to treat each primer as a separate situation and do the best that you can to make it shoot, altering other variables just as you would when working up a load with a single primer.

Guys who are statistics geeks are the worst, they get so deep in their formulas that they loose sight of test design. Close behind them are those that claim superior knowledge of scientific method.

Another issue that I see all the time is fellows referring to the SD of a small sample. Give me a break.

Bottom line, you need to consider the resolving power of your test system.

The truth is that most of what we learn in life is base on anecdotal evidence, and there is nothing wrong with that. In the case of shooting, we have real limits imposed by barrel life, money, and available time.

A good friend used to carry on about the need for elaborate testing to be done about things he found interesting. My retort was always, "No one is stopping you." That usually put an end to it.

I really appreciate it when people share their results, even if their methods are not perfect,

Good points here. You know if your results are reliable, if your not sure then they are not. It all comes with time working with rifles. Boyd is right about the guys that think you need to test something 20 times to get a reliable amount of data. If thats the case your rifle or you are not up to par. We have to test, powders, powder charge, primers, bullets, neck tension, seating depth, AND still have enough barrel life to go shoot some matches. Some of the science and data guys would burn out the barrels by the time they had enough data to believe it. Good equipment is a must. A good test in the right conditions is a must. Knowing when not to shoot because the conditions wont give good data is important as well.