Crunching the Numbers
Date: Sep 18 2012
If you’re anything like me, one of the results of a good day at the range will be lots of data in the form of a bunch of shot-up targets and a scratch pad full of scribbled notes. With the use of rudimentary statistics, some objective sense can be made of these numbers in addition to subjective observations, such as whether a load is clean or dirty or whether it produces a thunderous fireball at the muzzle or seems pleasantly quiet. Without much calculating, it’s possible to note whether a load is faster or slower than another, or such things as whether it delivers shots into tight clusters or scatters them across the target. However, a gut feeling or simple observation will not give us a hard number to describe the differences in resultant data. Proper statistical analysis will allow specific comparisons and provide a guide for those seeking top performance from their handloads.
Lucky for us, smart guys developed the branch of mathematics called statistics that specifically deals with (according to Webster) “the collection, analysis, interpretation and presentation of masses of numerical data.” While Mark Twain might consider statistics in the same category as lies and damn lies, the use of certain statistical methods allows us to make specific conclusions regarding our results. Most of us already do a quick statistical analysis without knowing it when we note an unusually large extreme spread. Another common and simple operation is finding the average value to represent a group of numbers. Unfortunately, other very useful techniques of quantifying the characteristics of a set of numbers can get rather involved. Double lucky for us, though, other smart guys have seen fit to include some of these statistical functions in fairly common electronics, such as handheld calculators, and we don’t have to become math geeks to use these methods.
Before getting too far, however, some basic assumptions regarding our data need to be made in order for our conclusions to be valid. For reloaders, perhaps the most useful branch of statistics deals with data that is considered to be well behaved or “normally distributed.” Simply, we need to make sure our data compares apples to apples. This is actually quite rudimentary and not that difficult to assure. It doesn’t make sense to talk about the average velocity of a favorite .45 ACP load and include velocity figures from a .357 while doing the calculations. Well behaved data is graphically illustrated by the bell curve, where most of the points tend to cluster around center and yet allow for reasonable outliers. While proof by example does not work in mathematics, enough statistical analysis has been done on firearm performance that we can safely assume typical data will fit within the bell curve.
A basic analysis of normally distributed data will involve measurements of both central tendency and dispersion. This is a key concept and the reason for using statistics: Statistics measure data. Two common measures of central tendency, average and extreme spread, are quite useful to reloaders. Extreme spread is simple to measure and compare. To find it in a group of numbers, subtract the value of the smallest data point from the highest. A large extreme spread implies poor precision and can be used to flag a system with significant inconsistencies, whether terrible ambient conditions, bad bullets or a loose sight. While extreme spread can be used to single out particularly bad performance, keep in mind that it is based solely upon the two most extreme data points and ignores the rest. This does little to inform us about the nature of a group of numbers other than the extremes. The same phrase, “extreme spread,” is also commonly used to describe groups on target. In this instance, a measure of extreme spread is a bit more useful because a load with an extreme spread of 2 inches will always hit a 4-inch bull, so one might look at the measure of extreme spread as more of an indicator of whether a load will work or not.
Finding the value of the average for a group of numbers, while more involved than extreme spread, only requires addition and division: divide the sum of all the values by the total number of values. This is very useful to reloaders as it provides the single value that every other value tends to cluster around. While the average value does not give an indication of how specific data points are distributed, it does consider each data point. The value of the average can be used to represent each individual shot, allowing us to quantify the number group and make broader assumptions about how future shots will perform. The average is especially useful in regard to velocity as these readings should be uniform with minimal outliers and clustered around the average. This velocity average, taken with an average of case web expansion measurements, provides us with some of the best indirect indications as to the pressures our loads are developing.
After a quick glance at extreme spreads and calculating averages, measuring the dispersion of data gives us an idea of just how “well behaved” it is. The most common method of doing this is by calculating the standard deviation (SD). This calculation is a bit more involved, and I would suggest using a multifunction calculator or computer to do the math. This calculation quantifies the amount of deviation from the average of each point of data, a sort of “average deviation,” if you will. The SD goes hand in hand with the bell curve, where 68 percent (two in three) of the data points will lie within ±1 SD of the average, and roughly 95 percent of all the points will lie within ±2 SD of the average. This data forms the shape of the main body of the bell, and the remaining 5 percent forms the thin skirt at the base. That’s all fine and dandy but doesn’t really offer much clarification, yet. By taking the SD and dividing it by the average, we come up with a number called the coefficient of variation (CV). This will allow direct comparisons between different groups of numbers, as the CV is a dimensionless ratio, sometimes expressed as a percentage. I have found that velocity data from good rifle loads will show a CV in the range of 0.10 to 0.80 percent, with pistols trending slightly higher. Loads that produce CVs up to 2 or 3 percent can still perform well on target, but anything beyond that gets to be rather inconsistent and starts to show up in poor downrange performance.
With velocity data, we seldom go beyond extreme spread, average, SD and CV, perhaps due to the fact that velocity is the end result of an incredible number of influencing factors, and we seldom require a very specific velocity anyway. On the other hand, hitting the target means putting the projectile into a very specific spot. The only way to know exactly where a shot will land is to take it, leaving no ammunition for future use. The statistics of target analysis involve mathematical confidence and quantifying such things as reliability and predictability, and confidence can get rather involved and beyond the scope of this column.
To put it in very simple terms, increasing confidence in future results improves with more samples. The number of shots in a group and number of groups required can be approximated by what kind information is sought. To make rough sight adjustments or to get an idea of where groups will typically impact, a few groups will suffice. Again, this is taking more of a “does it work or not” measure like extreme spread. Quantifying the influence of a minor component change, such as comparisons between primers, will require many shots into many groups. When comparing factors that have a minute influence on a load, a subjective conclusion utilizing the brain’s amazing ability to process multiple inputs without knowing exactly how or why is as good a method as any.
Now let’s imagine we just got home from the range, have cleaned up and are ready to take a closer look at the raw data. For this exercise, here are the actual numbers from two different loads fired in the same SP101 .357 Magnum. For the sake of comparison, the only difference is the powder charge, a factor that can have a substantial influence on performance. The rest of the load consists of an 18 Brinell hardness number (BHN), 125-grain cast Lee flat-point, sized .358 inch, lubed with Lyman Moly, loaded to 1.560 inches OAL in Federal brass and sparked with Federal 100 Small Pistol primers.
Right away, we can guess that the Universal load was much more consistent – but by how much? And, what is the average velocity of each load?
Load 1: high of 1,037 - low of 806 = 231; not that good
Load 2: high of 1,172 - low of 1,142 = 30; much better
Load 1: 947.7 fps
Load 2: 1,157.8 fps
Standard deviations (courtesy of my calculator):
Load 1: 68.6 fps; 95 percent of all shots should be between 810.5and 1,084.9 fps (±2 SD)
Load 2: 7.8 fps; 95 percent of all shots should be between 1,142.2 and 1,173.4 fps (±2 SD)
Making a direct comparison of the standard deviations does not take into account the differences in average velocity, so if we create a dimensionless ratio by dividing the SD by the average, we get the CV:
Load 1: 68.6 ÷ 947.7 = 7.24
Load 2: 7.8 ÷ 1,157.8 = 0.67
Now, a direct comparison can be made. We see that the CV of load 2 is much less than load 1. At this point, a dyed-in-the-wool statistician would do the analysis required to measure just how much more consistent load 2 is, but for our purposes, it is simply enough to note that load 2 appears to be a much better performer than load 1. The differences in consistency in velocity also showed up in the groups on target. Load 1 did well to print into 6 inches at 25 yards while load 2 cut nice 1½- to 2-inch clusters. For whatever the reason, I think load 1 just didn’t develop enough pressure to make Unique burn consistently; it needs help. Perhaps a stronger primer or tighter crimp would help. By using statistical analysis, we’re able to quantify what we might already recognize, that load 2 clearly performs much more consistently. Now, instead of entering all the raw data into my data book, the result of this load comparison is recorded like shown in the table. A quick glance at the CV and accuracy columns tells me the Unique load is not a top performer.