Efficient Load Development Using Statistical Design Options

[CharlieNC]

Two of the primary variables for developing and optimizing a load are powder charge weight and bullet jump distance to the lands. Traditionally these are evaluated using the “scientific method” whereby each is changed one variable at a time across several iterations; e.g. find an optimum charge weight, then an optimum jump, then refine the charge at that jump. In science and industry this approach has been long abandoned in favor of statistical experimental designs (DOE) where all parameters are varied simultaneously in a systematic manner, such that fewer trials are necessary and the ability to detect and quantify interactions is gained. In the case of loading an interaction could be that the optimum bullet jump depends upon the charge weight. Numerous DOE options are available, and this is just one example of the application. The attached pdf provides a brief overview of this approach which is both faster and more thorough than the one-at-a time methodology.

 

[Ned Ludd]

One big difference between industrial application and individuals shooters is the amount of money they have to put into solving a problem, which often means a significant amount of automation in order to test the the large number of variables and parameters simultaneously. Individuals shooters may not have the same ability in terms of how many things they can realistically test at one time. However, I find the idea intriguing. Have you tried some variation of this approach yourself as yet?

 

[CharlieNC]

One big difference between industrial application and individuals shooters is the amount of money they have to put into solving a problem, which often means a significant amount of automation in order to test the the large number of variables and parameters simultaneously. Individuals shooters may not have the same ability in terms of how many things they can realistically test at one time. However, I find the idea intriguing. Have you tried some variation of this approach yourself as yet?

The attached pdf file reports one example involving two variables, charge weight and jump. It would be straight forward to expand the approach to many more variables. In terms of resources, its all about getting more from less.

 

[ireload2]

It is not going to be useful for a large percentage of the shooting public because they do not have the math background or a good way to get it so they can understand the language much less the methodology.

The guy running the DOE is often a PHD engineer or scientist and he has access to a PHD mathematician if he gets into a jam.

There are additional variables such as alternate primers and bullets that might need to be evaluated. For each item you add the matrix gets larger.

There are variables like the wind that you have to deal with that can spoil the data.

Have you ever ran a DOE?

 

[Ned Ludd]

Very interesting and a very well-written report. Thanks for posting your work.

One point I'm still not quite clear on is the assertion that there is a node at approximately .025" jump and 38.6 gr. It appears that this region actually corresponds to a peak in your G2 surface plot. Wouldn't there be two better nodes (solely in terms of group spread) located at approximately .033" off/38.2 gr, and 0.021" off/39.0 gr? Those are the approximate values that correspond two strong G2 troughs on the surface plot, if I'm reading it correctly. Just curious about your interpretation regarding the .025" jump and 38.6 gr node.

In any event, this represents 18 rounds total, if I understood you correctly (not including any sighters/foulers). The limiting factor in that case might be statistical accuracy/precision of the velocity measurement with only a two shot sample size. Even a single "errant" velocity value could significantly change the output. However, the good news is that even if you increased the number to 5, it would only be 45 shots total, still far fewer rounds than many will use during typical load development. Anyhow, thanks for posting.

 

[Zero333]

This is not the easiest thing to wrap your head around compared to load + shoot + measure groups = smile or frown.

But I like it nevertheless.

 

[Nature Boy]

I'm very familiar with DOEs and did them when I was a young staff engineer many moons ago as a means of validating manufacturing processes. I've often thought about the value of using it as a tool for load work up.

What you're describing is more of a screening experiment for 2 variables: bullet jump and powder charge. I believe there's some merit to that, however, a full blown DOE would involve too many variables that would be very hard to screen for, with the most difficult being the shooter

 

[DocScott]

Nicely done CharlieNC! I spent many a day with statistical experimental design work to model and optimize chem plant processes and you are spot on with industry having moved beyond the "one variable at a time" method. Have you considered the possibility of using simplex optimization? While it doesn't yield as much information about the response surface, it might let you very rapidly approach the optimum. I've often thought of trying that and your work has given me a nudge in that direction. Well done, sir!

 

[6BRinNZ]

Well written - in almost plain english
It is an interesting approach - but how did 5 or 10 shot groups compare to validate?
curious minds want to know.

 

[CharlieNC]

To answer a few questions... One of the nice features of the DOE is that there is "hidden replication"; for example in this case for every powder charge, two different jumps were tested meaning there are repeats across the powder range (except the central point). That is one reason the fitted curves are a decent fit with the limited shots, bearing in mind that the scatter at a given charge is due to the real impact that the different jumps had at that charge. These smooth curves, not the raw data points, were used to see the location of the nodes (POI sensitivity); and it is unlikely that the answer would have been any different with more shots at each condition. Of course there would have been less noise for a point or so, but bear in mind we are looking at the smooth curve which is determined across the entire range of the shots rather than trying to read too much into each individual shot. There is a slight hill in the surface plot for the group size, and that is a limitation of only taking two shots per trial. In this case I am interested in achieving a consistent POI for tactical shooting; if the primary motivation was shooting small groups it would have been more informative to follow this trial up in a narrower range with more shots. With a balanced design such as this you could manually graph the key responses (like vertical POI) to observe the same behavior, or within Excel a scatter plot with the curve fitter would show the exact same. In the beginning it does take a bit of studying to become familiar with the principles, and this approach is not for everyone. And there are many other design options to evaluate as many variables as you want. The intent is to promote awareness to those who may be interested.

 

[michaelbz]

What software did you use? Did it have a DOE template?

 

[CharlieNC]

What software did you use? Did it have a DOE template?

I use Minitab and it does have a complete DOE template, analysis, etc.

 

[SenderoMan]

Well written and well explained! I agree with the above posts about the use of more shots to validate a possible load. I understand your tactical usage, but the end result is to find the best possible load combination in the least amount of time and resources. If you could change those parameters to include a five shot group and share with us, it would be greatly appreciated.
It would also be nice to plug in a known combination that is a known performer, i.e. 6BR and Varget and seems if this system validates that information.
Thanks for the write-up!

 

[swadiver]

ah, your takn all the fun outa it.

 

[Regnar]

I am of above average in the intelligence category and am decent at math. I am above average in the reloading game. This may be the cats meow and have distinct advantages over the methods most use but it is not something I could get into because of the complexity of it.

 

[Shynloco]

IMHO,
All the technical info and jargon are good and wonderful. But in the end, there are certain steps and processes that have been used and proven for finding what combination (of powder and jump/if jump works well at all) in your particular rifle. I've found three shot groups are good to use in the development process. But i'ts the five shot groups where the validation of that load development becomes the final step and deciding factor. I'm anything but a math genius and worse at ballistics detail. But it's my rifle that tells me everything that is reality, something a math equation is incapable of doing.

Alex

 

[CharlieNC]

I was apparently not clear that this is just an example of a DOE, and not the only way to do it. Instead of 5 different charges and jumps, you can do 20 charges x 3 jumps x 2 primers if you wanted to. And you can do 10 shots per item instead of 2, or any number. The options are unlimited, the key is the DOE will let you efficiently and thorughly explore the space you are interested more effectively than one variable at a time.

 

[GrocMax]

We talking about Sigma 6 here?

 

[michaelbz]

I too think it is a great write up. While I hear the concerns of 2 shot groups, I think people are missing the concept of curve fitting. Curve fitting depends on it nearest neighbors with decreasing weighting of data points further out. So it is not really a two shot group because the nearest neighbors are part of the group. The limited data points would be a problem if you had discontinuous sudden spikes, highly nonlinear behavior, or a periodic system above the sampling frequency. A single bad shot would probably not alter the results, but two or three bad shots would

 

[michaelbz]

Design of Experiments (DOE) has been around for 50+ years, and predates Six Sigma. It is used by everybody in science and engineering. It is well proven, but it is a tool, and as such it can be miss-used.