Electronic Warfare: Additional Buys of Sensor System Should Be Delayed
Pending Satisfactory Testing (Letter Report, 09/27/96, GAO/NSIAD-96-175).

GAO reviewed the Army's program for acquiring the Intelligence
Electronic Warfare Common Sensor (IEWCS), focusing on whether the Army
was ensuring that the system performed acceptably before committing to
the system's production.

GAO found that: (1) the Army prematurely committed to low-rate initial
production of IEWCS systems to install on Advanced Quickfix (AQF)
helicopters; (2) the Army is planning additional low-rate initial
production that it has not justified; (3) the Army plans to enter
full-rate production without ensuring that the system meets minimum
acceptable performance requirements; and (4) unless the Army changes its
acquisition strategy, it could procure a system that requires redesign
and retrofit to achieve acceptable performance.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  NSIAD-96-175
     TITLE:  Electronic Warfare: Additional Buys of Sensor System Should 
             Be Delayed Pending Satisfactory Testing
      DATE:  09/27/96
   SUBJECT:  Electronic warfare
             Army procurement
             Advanced weapons systems
             Testing
             Concurrency
             Helicopters
             Military aircraft
             Product performance evaluation
IDENTIFIER:  Intelligence and Electronic Warfare Common Sensor
             Ground-Based Common Sensor--Light
             Ground-Based Common Sensor--Heavy
             EH-60 Helicopter
             Quickfix Helicopter
             Advanced Quickfix Helicopter
             AVR-2 Laser Warning System
             Desert Storm
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                    <info@www.gao.gov>                        **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to Congressional Committees

September 1996

ELECTRONIC WARFARE - ADDITIONAL
BUYS OF SENSOR SYSTEM SHOULD BE
DELAYED PENDING SATISFACTORY
TESTING

GAO/NSIAD-96-175

Electronic Warfare

(707129)


Abbreviations
=============================================================== ABBREV

  IEWCS - x
  DOD - x
  GBCS-L - x
  GBCS-H - x
  AQF - x
  LRIP - x
  IOT&E - x
  ORD - x
  ORDs - x
  AQFs - x
  IOT&Es - x
  TEMP - x

Letter
=============================================================== LETTER


B-272289

September 27, 1996

Congressional Committees

We reviewed the Army's $1.55 billion program for acquiring the
Intelligence Electronic Warfare Common Sensor (IEWCS) system.  Our
review focused on determining whether the Army was taking necessary
measures to ensure that the system demonstrated acceptable
performance before committing to its production.  We conducted this
review under our basic legislative responsibilities and are
addressing this report to the committees of jurisdiction.  The report
identifies problems and calls for corrective action that the
Department of Defense (DOD) has indicated an unwillingness to take. 
We are suggesting that Congress may wish to take the necessary action
to ensure that the DOD addresses the problems we have identified. 


   BACKGROUND
------------------------------------------------------------ Letter :1

The IEWCS system is intended to modernize the Army's signals
intelligence equipment at the division level and includes a common
suite of subsystems for use on three different platforms.  The system
for the Army's light divisions,\1 called Ground Based Common
Sensor-Light (GBCS-L), is to be mounted on high mobility multipurpose
wheeled vehicles.  For heavy divisions, the system is called the
Ground Based Common Sensor-Heavy (GBCS-H) and is to be mounted on a
derivative of the Bradley Fighting Vehicle.  The airborne version is
mounted on the EH-60 Quickfix helicopter and is called the Advanced
Quickfix (AQF).  (See figs.  1 through 3.)

   Figure 1:  Ground Based Common
   Sensor-Light

   (See figure in printed
   edition.)

   Source:  U.  S.  Army.

   (See figure in printed
   edition.)

   Figure 2:  Ground Based Common
   Sensor-Heavy

   (See figure in printed
   edition.)

   Source:  U.  S.  Army.

   (See figure in printed
   edition.)

   Figure 3:  Advanced Quickfix

   (See figure in printed
   edition.)

   Source:  U.  S.  Army.

   (See figure in printed
   edition.)

IEWCS is expected to be capable of intercepting enemy communications
signals, locating the source of those signals, and jamming them
electronically.  It is also expected to be capable of locating enemy
radars.  The Army started limited production of the GBCS-L in fiscal
year 1995 on an urgent basis to field a system with a specific
capability to counter a particular type of threat communications
system.  The Army had originally planned to upgrade seven EH-60
helicopters to the AQF configuration in 2 years of low-rate initial
production (LRIP).  It started LRIP of three AQF systems in fiscal
year 1996 and had planned to procure the remaining four systems in
fiscal year 1997. 

In our September 1995 report, we indicated that the Army's fiscal
year 1996 budget request to upgrade the EH-60 Quickfix to the AQF
configuration could be reduced because operational testing of the
AQF, needed to prove its effectiveness and suitability, was not
scheduled until fiscal year 1997.\2 Although the Army's fiscal year
1996 budget was approved, the DOD Comptroller considered our findings
in evaluating the Army's fiscal year 1997 budget request and reduced
the Army's planned second LRIP procurement of AQF systems from four
to one.  The Army justified the additional system as needed to raise
the total LRIP quantity to four systems desired for the AQF's fiscal
year 1997 initial operational test and evaluation (IOT&E). 


--------------------
\1 The Army has light and heavy forces.  Light forces include
nonmechanized infantry, airborne, and air assault units.  Heavy
forces include armor, mechanized infantry, and cavalry units. 

\2 1996 Defense Budget:  Potential Reductions, Rescissions, and
Restrictions in RDT&E and Procurement (GAO/NSIAD-95-218BR, Sept.  15,
1995). 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :2

The Army has prematurely committed to LRIP of the unproven IEWCS
system and plans an additional LRIP that is not justified.  In
addition, the Army has plans to approve additional production of the
system and enter full-rate production without demonstrating that it
can meet minimum acceptable operational performance requirements. 
Unless the acquisition strategy is changed, the Army risks becoming
committed to procuring an unsatisfactory system requiring redesign
and retrofit to achieve acceptable system performance. 


   PREMATURE LOW-RATE PRODUCTION
   OF IEWCS
------------------------------------------------------------ Letter :3

The Army decided to enter low-rate production of AQF systems in
November 1995 despite unfavorable user test results.  The decision
was linked to a test that was supposed to verify the operational
characteristics of the IEWCS on all three platforms.  The test
results (details of which are classified) showed that the system
would work occasionally under the right conditions, but failed to
demonstrate that the system was sufficiently mature to justify
production.  For example, the system had problems locating targets. 
On one occasion, a GBCS-L platform erroneously displayed a location
as being in northern Texas when the actual location was in southern
Arizona.  Further, the AQF version's performance was the poorest
against frequency hopping (low-probability of intercept) signals,
performing at a rate of only one-third that of the other two
platforms. 

Army program officials stated that the three AQF systems that were
contracted for in January 1996 and the one system to be contracted
for in fiscal year 1997 are necessary for the AQF's IOT&E.  However,
the first three systems are not scheduled to be delivered until June
1998 and the fourth is not scheduled to be delivered until even
later, both long after the scheduled September 1997 AQF IOT&E. 


   UNNECESSARY RISK IN PRODUCTION
   PLANS
------------------------------------------------------------ Letter :4

Even though the Army plans to conduct IOT&E on the GBCS-L, AQF, and
GBCS-H systems in stages over the next 3 years, the test results may
not necessarily affect production decisions because the Army has
taken the position that the performance criteria it has set in
operational requirements documents (ORD) for the IEWCS are not
absolute pass/fail measures.  Instead, the Army has stated that
"...[the criteria] represent estimates of performance for which a
failure to achieve a given criterion would require a careful
management reassessment of cost effectiveness and program options
during the program milestone decision review." Consequently, the Army
has given itself an option for proceeding into full-rate production
of IEWCS systems that may not meet minimum acceptable performance
requirements. 

According to DOD Regulation 5002.R, at each milestone beginning with
program initiation, thresholds and objectives expressed as measures
of effectiveness or performance requirements should be documented in
an ORD.  The threshold value is the minimum acceptable performance
requirement that, in the user's judgement, is necessary to satisfy a
need.  The objective value is what is desired by the user and what
the user is attempting to obtain.  The ORDs (which are classified)
for the IEWCS platforms have objectives.  However, the minimum
acceptable performance requirements (thresholds) have not been
specified for most of the objectives.  For example, one AQF ORD
objective is that the system must automatically process signal data
at the sensor for threat detection and identification in near
real-time.  However, there is no specification for how often this
must happen or how much data is to be processed to achieve minimum
acceptable performance.  Likewise, another ORD objective indicates
that the system must locate threat emitters (communications) to
within specified ranges, yet is silent regarding how often this needs
to happen in order to meet the minimum acceptable performance
requirement.  Thus, conceivably a system that could intercept at
least 1 out of 100 enemy messages could be considered to have met the
requirements. 

Decisions to enter and continue production without ensuring that
systems meet minimum acceptable performance criteria have
historically been made with adverse consequences.  As we reported in
1994,\3 DOD has repeatedly committed electronic warfare and other
systems to production without assurance that the systems would
perform satisfactorily.  Many of the weapon systems that started
production prematurely later experienced significant operational
effectiveness and/or suitability\4 problems.  As a result, major
design changes were often needed to correct the problems, costly
retrofits were required for many delivered units, and substandard
systems were sometimes deployed to field units. 

For example, the Army began production of its AVR-2 and AVR-2A laser
warning system\5 despite unfavorable test results and without
verifying that design changes to correct performance problems were
adequate.  In total, the Army spent as much changing the system's
design as it did on the system's original development and procured
over half of its total program quantity without completing
operational tests to ensure the system's satisfactory performance. 
On another electronic warfare system, the Army made design changes to
correct a serious shortcoming detected during operational testing but
failed to verify the adequacy of the changes in further operational
testing.  Subsequently, during Operation Desert Storm, the system
proved so defective that Army pilots stopped using it. 


--------------------
\3 Weapons Acquisition:  Low-Rate Initial Production Used to Buy
Weapon Systems Prematurely (GAO/NSIAD-95-18, Nov.  21, 1994). 

\4 Operational effectiveness refers to the ability of a system to
accomplish its mission in the planned operational environment. 
Operational suitability is the degree to which a system can be placed
satisfactorily in field use considering such factors as reliability
and maintainability. 

\5 The AVR-2 and AVR-2A laser warning system is installed in
helicopters to alert pilots to the presence of laser energy and
thereby provide protection against threat weapons that rely on lasers
for their operation. 


   RECOMMENDATION
------------------------------------------------------------ Letter :5

We recommend that the Secretary of Defense: 

  -- require the Army to cancel the planned fiscal year 1997
     procurement of one AQF system and

  -- require both that specific, measurable, minimum acceptable
     performance requirements be established for the IEWCS system and
     that the system demonstrate the capability to meet these
     requirements before proceeding with additional procurements. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :6

In commenting on a draft of this report, DOD disagreed with our
recommendation that the Army cancel its fiscal year 1997 procurement
of one AQF system.  DOD stated that while the one AQF system will not
be available to support fiscal year 1997 AQF testing, it is needed to
ensure that sufficient test articles are present to support a
multi-year testing program.  DOD also stated that the procurement
supports the establishment of an initial production rate for the
system sufficient to lead to full-rate production upon successful
completion of operational testing.  We found no evidence supporting a
requirement for the fiscal year 1997 system to support future
testing.  The Test and Evaluation Master Plan (TEMP) for the GBCS-L
states that only three platforms of each type (GBCS-L, AQF, and
GBCS-H) are required for operational testing.  The TEMP for the AQF
does not address the number of test articles required for testing the
AQF, and the Army has not prepared a TEMP for the GBCS-H. 
Furthermore, the Army could stretch the production of the three AQF
systems ordered last year, thereby maintaining a stable production
line until test results become available. 

DOD partially agreed with our recommendation requiring both that
specific measurable, minimum acceptable performance requirements be
established for the IEWCS system and that the system demonstrate the
capability to meet those requirements before the Army proceeds with
additional procurements.  DOD stated that it saw merit in this
recommendation.  It also agreed that the Army should establish key
performance parameters for the IEWCS system before conducting final
systems IOT&Es, but saw no need to require the successful
demonstration of those parameters prior to further procurements.  As
we reported, decisions to enter and continue production without
assuring that systems meet minimum performance criteria have
historically been made with adverse consequences.  Consequently, we
continue to believe that there should be a requirement that the
established key performance parameters be met prior to the
procurement of additional systems. 

DOD's comments are reprinted in their entirety in appendix I, along
with our evaluation. 


   MATTERS FOR CONGRESSIONAL
   CONSIDERATION
------------------------------------------------------------ Letter :7

In light of DOD's unwillingness to have the Army revise its IEWCS
acquisition strategy, Congress may wish to take the actions necessary
to limit AQF procurement until AQF systems successfully complete
operational testing and to require IEWCS' demonstration of
established key performance parameters prior to the procurement of
additional systems. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8

To address our objectives, we interviewed officials and obtained and
reviewed briefing, budgetary, and planning documents from the office
of the Project Manager, Signals Warfare, Vint Hill Farms Station, Va. 
We also visited officials, examined test sites, and obtained
explanations of test procedures and results at the Intelligence
Electronic Warfare Test Directorate, Fort Huachuca, Az.  We also
obtained, reviewed, and analyzed test reports prepared by the
Intelligence Electronic Warfare Test Directorate and test plans
prepared by IEWCS project office to determine whether the Army was
taking necessary measures to ensure that the IEWCS system
demonstrated acceptable performance prior to committing to
production.  We conducted our review between September 1995 and June
1996 in accordance with generally accepted government auditing
standards. 


---------------------------------------------------------- Letter :8.1

We are sending copies of this report to other appropriate
congressional committees; the Director, Office of Management and
Budget; and the Secretaries of Defense and the Army.  We will also
make copies available to others on request. 

Please contact me at (202) 512-4841 if you or your staff have any
questions concerning this report.  Major contributors to this
assignment were Jackie B.  Guin, Paul Latta, and Henry Arzadon. 

Louis J.  Rodrigues
Director, Defense Acquisitions Issues

List of Congressional Committees

Chairman
Ranking Minority Member
Committee on Armed Services
United States Senate

Chairman
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate

Chairman
Ranking Minority Member
Committee on National Security
House of Representatives

Chairman
Ranking Minority Member
Subcommittee on National Security
Committee on Appropriations
House of Representatives




(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
============================================================== Letter 



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)


The following are GAO's comments on the Department of Defense's (DOD)
letter dated August 12, 1996. 


   GAO COMMENTS
------------------------------------------------------------ Letter :9

1.  While a set of four Advanced Quickfixes (AQF) for one division
may give the Army flexibility in future tests, we found no evidence
that supports a requirement for this many test platforms.  According
to the Test and Evaluation Master Plan (TEMP) for the Ground Based
Common Sensor-Light (GBCS-L), only three platforms of each type
(GBCS-L, AQF, and Ground Based Common Sensor-Heavy (GBCS-H)) are
required for operational testing and the Army already has three AQFs
under contract, plus two additional developmental platforms
available.  The TEMP for the AQF does not address the number of test
articles required for testing the AQF and the Army has not prepared a
TEMP for the GBCS-H.  Consequently, we know of no requirement for
four AQFs for testing either now or in the future. 

2.  With regard to DOD's position that the additional procurement of
one AQF in 1997 "permits a more orderly increase in the production
rate," we note that the DOD Comptroller reviewed the Army's request
to produce four AQFs in 1997 and cut the request to a single system. 
Since a reduction in the production rate from three in fiscal year
1996 to one in fiscal year 1997 is not an "orderly increase in the
production rate," DOD may find it to be useful to stretch production
of the three AQFs ordered last year.  In this way, a stable
production line could be established and test results could be
available before the Army commits to additional production. 

3.  As we noted in the body of our report, decisions to enter and
continue production without ensuring that systems meet minimum
performance criteria have historically had adverse consequences. 
Thus, we continue to believe that there should be a requirement that
the established key performance parameters be met prior to the
procurement of additional systems. 

*** End of document. ***