I started my engineering career doing flight test… At least that is how I see it. My first actual engineering job was doing chip verification at Hewlett Packard (now HP) on a cache coherency chip they were developing for some of their high end servers. The HP job was around 2001, when HP still was a household name, although their decline into irrelevance had already started. Anyways, I was only a engineering co-op and my time at HP was short. So, my first actual job was as a flight systems engineer at NASA Dryden Flight Research Center in Edwards, CA, USA starting in 2006.

Dryden (now renamed to NASA Armstrong after Neil Armstrong who worked at Dryden in his early days at the agency) is the primary flight test facility for NASA Aeronautics. It is located on the grounds at Edwards Air Force Base in the Mojave Desert in Southern California. Edwards, for its part, is the primary developmental flight test facility for the US Air Force and home to many many first flights, X-planes, and aviation records. The NASA, Air Force partnership at Edwards has a long history beginning with the first ever supersonic flights with the X-1 and X-2 and flights of the X-15, the most famous high speed, high altitude, rocket powered, space plane ever built. Just to note, Neil A. flew the X-15 at Dryden and many other famous airplanes which was a major reason for his selection to the astronaut corp.

Neil and the X-15 Neil and the X-15

Anyways, as I have mentioned, I started as a flight systems engineer, which is a mixture of a pure systems engineer and a aviation computer engineer. My jobs included systems engineering tasks such as requirements development, process design, and verification planning, but also included computer engineering tasks such as coding, hardware design and implementation, and so on. It was actually a really great introduction to aviation research, systems development, and computer vehicle integration. After a while, I also started working as a flight test engineer, which amongst other tasks, meant I got to fly and run experiments in the backseat of F-18s, F-15s, F-16s, T-34s, etc. Being a NASA flight test engineer was quite the honor and exposed me to the operational side of flight testing. So, I feel like I have a rather well-rounded view of flight test and this blog post is about some of the high level things I have learned as a flight tester and how to get to flight test.

Me Me as a flight test engineer

First, let’s discuss what I am not going to talk about. There are two fundamentally different forms of flight test. There is operational flight testing and developmental flight testing. For operational testing, a system is introduced into a vehicle configured as close as possible to a normal operational vehicle. The pilot’s job is then to perform standard activities and see where the system brakes or see how it integrates into his normal operational tasks. Operational testing occurs at the end of the system development life cycle and is sometimes one of the last steps before a system is fielded in a fleet of vehicles. We are not going to talk about operational flight testing. Instead, we are going to talk about developmental flight testing.

In developmental flight testing, a system is integrated into a special test aircraft and oftentimes tested in otherwise very controlled situations. Keeping tight control of the parameters of a particular test point in flight is the hallmark of a great developmental test pilot. For example, a developmental test point might dictate that the airplane be at an altitude of 33,500 ft., at 520 knots, 10 degree right wing down bank, full standard power, with full wing fuel tanks when the system under test is initiated. Getting all of these aircraft parameters right is called being “on condition” and is sometimes pretty challenging. Luckily for us, NASA has some of the best pilots in the world, and so us engineers could dream up challenging conditions and they would deliver.

Nils NASA test pilots, led by Nils Larson, preparing for a mission

Another categorization for flight test is between manned and unmanned vehicles. For unmanned vehicles much of the support systems required for safety and test efficiency don’t exist. The systems can be integrated into the vehicles in a much more free way, because if the system does something bad no one will get hurt. This isn’t the case for manned flight test. In manned flight test, great care has to be taken to assure the safety of the test pilot and to ensure that the system being tested can be bound or easily disconnected by the test pilot. This is because in manned developmental flight test, you never assume that the system will work as expected. In fact the opposite is true. You must assume that the system will fail spectacularly and design your approach to make that safe (in general this is called fail-safe).

This doesn’t mean that you just put any system on the aircraft and hope it works. Instead for manned developmental flight testing a series of simulations must be used to build the system maturity up to the point of flight test. The lowest level simulation capability is called software-in-the-loop (SITL) batch sim testing. At this level, an approximation of flight dynamics is incorporated into test structure where you can manually set the aircraft flight state and run multiple (sometimes thousands) of pre-programmed test points. The system itself is only a software module attached to the simulator. This kind of testing is really good for unit level tests and sometimes initial system level testing.

The next level of simulation capability is hardware-in-the-loop (HITL) batch sim testing. In this paradigm, the system software is loaded onto the actual hardware that will run in the airplane. An interface between the batch sim and the hardware must be developed that mimics the timing and variables that will be delivered to the system on the actual airplane. Cross compiling the software for the actual hardware is sometimes a tedious process and debugging on hardware is harder (or sometimes not possible), so HITL testing should only be begun when the SITL testing is stable and all the bugs are understood and most fixed. In HITL you will find timing bugs or interface bugs you would never find with a SITL version, so this stage is critical.

The next stage gets us closer to flight test, in fact it is practice for flight test. It is the piloted simulation stage. This stage can either be run as SITL or HITL, however it is preferable to run it as HITL. This stage serves really two purposes. It is the first time that a human pilot is integrated with the system. Human pilots are great aviation software testers. They are able to uncover bugs that the batch sim hasn’t found yet, because humans have their own way of doing things that is difficult, or maybe impossible, to program a computer to do. So, the exact timing or coordination of a maneuver, including the inexactness and free flowing of a human pilot is often able to make system states change in unexpected ways. The second purpose of piloted simulation is to do detailed practice of the actual flight test points that will be run. At NASA our rule was that each test point had to be flown in piloted simulation before it could be flown in the actual aircraft. This allows the pilots the ability to practice getting on condition, or even to practice dealing with failures or abnormalities on the ground where safety isn’t an issue.

Sim

Of course, in order to practice the flight test points in the sim, they have to be developed first. This process is called flight test planning. It involves extensive design work by the whole team, including safety and management personnel. The idea in developing and scheduling the test points is to “expand the envelope.” This means that the flight test points will start at the most benign conditions possible, so that any bugs can be dealt with in the easiest way possible. If those tests are successful, then the flight test points progress to harder and more dynamic maneuvers. Expanding the envelope in this way is critical for safety, as it is much easier to deal with a system fault going straight and level at altitude, than it is barrelling down a canyon close to mach 1! By the time you do the dynamic, harder test points, you want to have quite a bit of confidence that the system will work as you expect it to work, so a flight test buildup is key here.

Flight testing is a crucial step in the development of any airborne system, so much so that many people don’t consider a system as viable until it has been through at least one stage of flight test. There are just so many ideas that work well “on paper” but when they are put into the actual airplane, don’t work at all. In fact, Dryden’s motto was “To separate the real from the imagined.” I think this sums up the purpose of flight testing perfectly. However, one must realize that flight testing is a later stage in the testing process that has many different types of simulation along the way.