GAO Recommendations on Body Armor Testing

Posted by:

To determine what effect, if any, the problems GAO observed had on the test data and on the outcomes of First Article Testing, the Army should provide for an independent ballistics evaluation of the First Article Testing results by ballistics and statistical experts external to the Department of Defense before any armor is fielded to soldiers under this contract solicitation. Because DOD did not concur with this recommendation, GAO added a matter for congressional consideration to this report suggesting that Congress direct DOD to either conduct such an independent external review of these test results or repeat First Article Testing.

 

To better align actual test practices with established testing protocols during future body armor testing, the Army should assess the need to change its test procedures based on the outcome of the independent experts’ review and document these and all other key decisions made to clarify or change the testing protocols during future body armor testing. Although DOD did not agree that an independent expert review of test results was needed, DOD stated it will address protocol discrepancies identified by GAO as it develops standardized testing protocols. DOD also agreed to document all decisions made to clarify or change testing protocols.

 

[NB: DOD quietly dropped its opposition to this GAO recommendation and an “independent experts’ review” of Army/DOD ballistic test protocols has started. One retired Army general was assigned to the panel, raising some doubt as to just how “independent” this panel truly is.]

To improve internal controls over the integrity and reliability of test data for future testing as well as provide for consistent test conditions and comparable data among tests, the Army should provide for an independent external peer review of Aberdeen Test Center’s body armor testing protocols, facilities, and instrumentation to ensure that proper internal controls and sound management practices are in place. DOD generally concurred with this recommendation, but stated that it will also include DOD members on the review team.

[NB: Quoting from footnote 1, page 77 of the GAO report: “… the most current Army Test Operating Procedure for testing body armor had not been updated since 1975. Test Operations Procedure (TOP), 10-2-506 Ballistic Testing of Personnel Armor Materials. So there it is in all its unvarnished ugliness — another key indicator of just how little the Army’s Leadership and defense contractors value the American Grunt. For testing life-or-death gear, i.e., personal protective equipment, use a 35-year old set of test protocols. Then, when forced by public pressure to address the issue, in typical Washington fashion, appoint a panel, a commission, “to study” the issue. How long will this study will take, and how many warriors will fall on the killing fields while the scandalous situation is being “studied” are two issues blithely ignored in this GAO report.]

Roger Charles

Editor and Senior Investigative Reporter for SFTT

Review the GAO Report: http://www.gao.gov/new.items/d10119.pdf

0

Body Armor Testing: What the GAO found

Posted by:

During Preliminary Design Model testing the Army took significant steps to run a controlled test and maintain consistency throughout the process, but the Army did not always follow established testing protocols and, as a result, did not achieve its intended test objective of determining as a basis for awarding contracts which designs met performance requirements.

In the most consequential of the Army’s deviations from testing protocols, the Army testers incorrectly measured the amount of force absorbed by the plate designs by measuring back-face deformation in the clay backing at the point of aim rather than at the deepest point of depression.

The graphic depicts the difference between the point of aim and the deepest point.

GAO Body Armor Test Explanation

[NB: So, there it is — GAO is telling the Congress that after two and one-half years, Army testing failed, and failed so abysmally that $121 Million in protective plates already produced and in warehouses awaiting distribution were designated unsafe to issue because their qualifying tests were unreliable.]

Army testers recognized the error after completing about a third of the test and then changed the test plan to call for measuring at the point of aim and likewise issued a modification to the contract solicitation. [NB: Yes, you read this sentence correctly. After having the error pointed out to them, Army test officials decided the “fix” was to change the test plan/requirements so that this flaw would be consistent throughout all testing!!]

At least two of the eight designs that passed Preliminary Design Model testing and were awarded contracts would have failed if measurements had been made to the deepest point of depression. The deviations from the testing protocols were the result of Aberdeen Test Center’s incorrectly interpreting the testing protocols. In all these cases of deviations from the testing protocols, the Aberdeen Test Center’s implemented procedures were not reviewed or approved by the Army and Department of Defense officials responsible for approving the testing protocols. After concerns were raised regarding the Preliminary Design Model testing, the decision was made not to field any of the plate designs awarded contracts until after First Article Testing was conducted.

During First Article Testing, the Army addressed some of the problems identified during Preliminary Design Model testing, but GAO observed instances in which Army testers did not follow the established testing protocols and did not maintain internal controls over the integrity and reliability of data, raising questions as to whether the Army met its First Article Test objective of determining whether each of the contracted designs met performance requirements. The following are examples of deviations from testing protocols and other issues that GAO observed:

  • The clay backing placed behind the plates during ballistics testing was not always calibrated in accordance with testing protocols and was exposed to rain on one day, potentially impacting test results.
  • Testers improperly rounded down back-face deformation measurements, which is not authorized in the established testing protocols and which resulted in two designs passing First Article Testing that otherwise would have failed. Army officials said rounding is a common practice; however, one private test facility that rounds told GAO that they round up, not down.

[NB: So, a private test facility rounds BFD up, benefiting the soldier while the U.S. Army rounds BFD down, benefiting whom? If you guessed “contractors,” you got it right. Yet, the Army and the Pentagon continue to chant their mantra: force protection is our number one priorityforce protection is our number one priority. True, combat is inherently dangerous and unpredictable — but adding risks and hazards by issuing sub-standard protective gear grounds this mantra into a hollow and empty phrase.  Especially if you allow this chant to accompany media reports of a fallen warrior’s funeral at Arlington National Cemetery or upon arrival of a flag-draped coffin at Dover AFB — “force protection is our number one priority…”.  It is simply an outrage that in the second decade of the 21st Century, American grunts still get the short straw when it comes to personal protective equipment?]

  • Testers used a new instrument to measure back-face deformation without adequately certifying that the instrument could function correctly and in conformance with established testing protocols. The impact of this issue on test results is uncertain, but it could call into question the reliability and accuracy of the measurements.
  • Testers deviated from the established testing protocols in one instance by improperly scoring a complete penetration as a partial penetration. As a result, one design passed First Article Testing that would have otherwise failed.

With respect to internal control issues, the Army did not consistently maintain adequate internal controls to ensure the integrity and reliability of test data. In one example, during ballistic testing, data were lost, and testing had to be repeated because an official accidentally pressed the delete button and software controls were not in place to protect the integrity of test data. Army officials acknowledged that before GAO’s review they were unaware of the specific internal control problems we identified.

As a result of the deviations from testing protocols that GAO observed, four of the five designs that passed First Article Testing and were certified by the Army as ready for full production would have instead failed testing at some point during the process, either during the Preliminary Design Model testing or the subsequent First Article Test. Thus, the overall reliability and repeatability of the test results are uncertain. Although designs passed testing that would not have if the testing protocols were followed, independent ballistics experts have not assessed the impact of the deviations from the testing protocols to determine if the effect of the deviations is sufficient to call into question the ability of those designs to meet requirements.

[NB; This sentence merits VERY close attention. GAO has raised the crucial issue of whether, for this life-or-death equipment, in use by DOD for over ten years, there is a problem with “the ability of those [protective plate] designs to meet requirements,” There is no dispute that in this sample of the five designs tested, four designs actually failed!! Notwithstanding that U.S. Army test officials certified these 4 failed designs as ready for full production.]

Vendors whose designs passed First Article Testing have begun production of plates. The Army has ordered 2,500 sets of plates (at two plates per set) from these vendors to be used for additional ballistics testing and 120,000 sets of plates to be put into inventory to address future requirements. However, to date, none of these designs have been fielded because, according to Army officials, there are adequate numbers of armor plates produced under prior contracts already in the inventory to meet current requirements.

Roger Charles

Editor and Senior Investigative Reporter for SFTT

Review the Report: http://www.gao.gov/new.items/d10119.pdf

0

Why the GAO Study on Body Armor Testing?

Posted by:

The Army has issued soldiers in Iraq and Afghanistan personal body armor, comprising an outer protective vest and ceramic plate inserts.   The Government Accountability Office (“GAO”) observed Preliminary Design Model testing of new plate designs, which resulted in the Army’s awarding contracts in September 2008 valued at a total of over $8 billion to vendors of the designs that passed that testing.

Between November and December 2008, the Army conducted further testing, called First Article Testing, on these designs. GAO is reporting on the degree to which the Army followed its established testing protocols during these two tests. GAO did not provide an expert ballistics evaluation of the results of testing. GAO, using a structured, GAO-developed data collection instrument, observed both tests at the Army’s Aberdeen Test Center, analyzed data, and interviewed agency and industry officials to evaluate observed deviations from testing protocols. However, independent ballistics testing expertise is needed to determine the full effect of these deviations.

[NB: To this day, we are not aware of any plans Congress has to direct an independent, transparent, “expert ballistics evaluation” of the personal protective plates issued to America’s frontline troops.  A Member or either the House or Senate Armed Services Committee must introduce directive language into the “National Defense Authorization Act for FY  2011” calling for requirements provided in the GAO report that will re-set appropriate testing protocols. For the benefit of congressional staff reading this report, GAO is pointing out that if Congress wants to know the full extent of the flawed testing, DOD and the Army must bring in outside experts to make this determination. Again, this takes legislation.  Stop and think for a minute about the full implications of this seemingly simple statement. GAO is not explicitly stating that the Army and DOD do not have the requisite engineers, or technicians. GAO is saying in classic read-between-the-lines Washington-speak that the Army/DOD consortium cannot be trusted to conduct an honest, scientifically-objective test. That the GAO felt the need to issue this blatant appeal for outside intervention is a sad commentary on the quality of our government’s commitment to ensuring our frontline troops wear only the “best-available” personal protective equipment. For those who have lost count, these frontline troops are approaching their ninth anniversary on the killing fields and the Army/DOD team has yet to demonstrate that it can conduct a reliable, repeatable ballistic test. BTW, GAO understandably did not address the issue of previous tests, but given the significant flaws identified by GAO in this two-and-one-half year investigation, any logical reader can justifiably question the validity of earlier tests. These questions include both the test protocols and the actual conduct of the tests (the competence and integrity of the testers). ]

Roger Charles

Editor and Senior Investigative Reporter for SFTT

Review the Report: http://www.gao.gov/new.items/d10119.pdf

0