4-17
earlier IT events that will require cooperatively-planned, SYSCOM- or program-approved IT plans
capturing test objectives, procedures, resources, and data collection and analysis requirements.
Unlike OT phases with a single test plan, CBTE IT may be accomplished using multiple test plans
across the program’s test continuum. IT objectives may be incorporated in the main body of the
test plans or be included as an appendix attached to the test plan. IT plan development will follow
the SYSCOM process, guidelines and format, and should clearly identify those portions of the
MBTD applicable to the identified test phase through alignment to the IEF Vignettes. The
TPM/OTD/OTC will actively participate in IT plan development, bring their operational expertise
to the process and advocating for the operational environment, threat surrogacy, and OT data
collection and analysis requirements. Each IT plan may not include the entire MBTD, but should
clearly articulate what portions of the design, by vignette run, are planned for accomplishment. It
must also provide a SUT/SoS description (or reference to the description in a corresponding
document), the relevant COIs, the measures, any known test limitations, a DMOT, and any data
scoring requirements and procedures unique to the test phase. This plan will include the tasks to
be performed and the associated conditions to be considered for the test, as identified in the IEF.
The program manager, or representative, will approve the IT plan for use by the program office
test team. At the conclusion of IT plan development, the TPM/OTD will conduct a CBTE Test
Plan Review Board (TPRB) with the division ACOS, Technical Director, 01B, 01C, 01D, and the
DOT&E Action Officer (AO). The TPM/OTD will brief the test objectives, the system’s current
hardware and software configuration, the IEF vignettes to be conducted, the DMOT, all aspects of
operational and threat realism, the data to be collected and its relevance to OT, and the data scoring
and analysis plan. The ACOS will sign a memorandum approving the IT plan for OT data
collection. For DOT&E oversight programs where the TEMP does incorporate the CBTE strategy,
DOT&E signature on the TEMP confers approval of the CBTE strategy and no additional approval
document will be required. For programs where the TEMP does not incorporate the CBTE
strategy, a document for DOT&E approval, such an IEF approval memorandum, will be required.
If deemed appropriate by DOT&E, a Data Collection Plan (DCP) may still be required.
4.4.6 Test Plan Changes:
For changes to test plans after initial review and concurrence, the program TPM/OTD will review
the changes in consultation with the assigned 01B CTF and 01C AO, to determine if the changes
affect the MBTD or the qualification of data for use in OT. If MBTD is not affected, the test plan
changes may be approved by the division ACOS without further review. If changes to the MBTD
are required, the TPM/OTD will submit an IEF update (revision or change letter) for
COMOPTEVFOR leadership approval and DOT&E concurrence (if required).
4.4.7 Data Collection, Scoring, and Analysis:
The TPM, OTD, or designated OT representative, should be onsite during the entire IT in order to
ensure that the required data are being collected in accordance with the approved IT plan. The
criteria and methodology for scoring data to support OT has not changed. In the CBTE construct,
incremental data scoring may be required to ensure events are completed prior to moving on to
subsequent test events or test phases. The frequency, conduct, and composition of scoring boards
should be discussed in the IT plan, MOA, or ITT Charter, and the TPM/OTD must ensure that all
stakeholders understand how the scored data aligns with OT objectives. Critical thought is
required when scoring IT data for OT use. Were the data affected by the system configuration,