As the military begins developing "modestly massive" training programs—for maybe 100 players—and then moves on to those with truly massive capabilities, the parameters of larger scale product testing change. Testing products while still in development is critical to ensuring that a program delivers as promised. So project managers need to know that reliable technologies and expertise already exist for collecting and analyzing input from very large numbers of people.

Test questions require more than yes and no responses from soldiers. Here is only one of many potential in-program scenarios that soldiers may face, and the types of detailed questions / comments that can be sought. Let's say a massive training program has one component that simulates various military services responding in a coordinated way. Some key issues to be tested at the squad level would be:

• How effective is the enemy AI? How smart do they fight? Do they fight in a realistic way? Is the enemy AI too predictable? Is it too easy to beat? Does it ramp up appropriately, so personnel can significantly increases their skills over time?
• Is the process engaging—does it feel real to personnel who've actually fought in that situation? If it seems canned, personnel will not fully engage in the training.
• Does the environment look and feel as it should? Is it complete, including all visual and auditory cues that you'd actually be alert for in such a situation—cues that would actually drive the squad's decisions and actions?
• Are technical standards being met? Is the frame rate great enough for animation to look smooth? Are there latency problems with thousands of participants at once?

The technology exists today to collect data from literally thousands of soldiers who are asked to test isolated components as they are being developed—and to analyze all that data quickly and efficiently. The obvious intended result is to end up with the most effective training program possible. The beauty is that now we can get input from people in all branches of the military, in all theaters of operation, from all over the globe easily, accurately and quickly.