Security Testing with Holodeck

Holodeck is an invaluable tool that helps development teams harden their applications and gain an advantage against hackers. It is used by companies like Microsoft, Symantec and Adobe to test their applications and provide the deepest, most effective testing possible. Holodeck is featured in major software testing publications such as How to Break Software and How to Break Software Security, and is used in many University Software Engineering curriculums to teach students the most effective ways to uncover security and functional defects.

Holodeck can be used to conduct a variety of security testing techniques in the software development process, including:

CRASH THAT APPLICATION
One of the goals of a hacker is to crash an application and see if it leaves any sensitive data behind. For example, if a program creates a temporary file that holds sensitive data and the application crashes, is the file left open on the disk? In many cases yes it does and it’s a goldmine for hackers, plus it’s a relatively simple exploit. Simulating the conditions for a crash is often more difficult than one would assume, but with Holodeck it’s simple.

Force the Application to Operate in Low Resource Conditions
Applications often assume they own all system resources and are free to consume as needed or wanted. Limiting quantities allows testers to quickly crash an application and expose temporary files. Holodeck allows testers to virtualize the application under a test’s runtime environment, and quickly limit the availability of memory, disk space and network bandwidth. There are several scenarios where limiting these resources can cause a crash, for example:

	Failing to check return values for memory allocations/NULL pointer dereferencing
	Trying to write files to a full hard disk
	Badly framed or missing data due to network I/O lag

In each of these cases, and dozens more, the application may crash because there’s no error handler in place to handle the situation, or the application isn’t engineered in a way to operate effectively in a constrained environment. In any case, the goal of crashing the application is quickly reached and the analysis of the resulting environment can proceed.

Fuzz Test the Application’s I/O Vectors and Force it to Consume Corrupted Data
Poorly coded applications will crash when they get fuzzed [randomly corrupted] data, typically trying to process the data without checking to see if it is correct and complete. For example, if a word processor just assumes that a .doc file is its own and it turns out to be a binary image file, the word processor will try and display the bits as a document and likely crash. Fuzzing can be applied to all the places that an application gets input data including file streams, network streams, the GUI and the registry. Holodeck gives testers the ability to quickly virtualize the application under the test’s runtime environment and corrupt any or all of the application’s input vectors, forcing it to consume bad data and crash.

EXPOSE THAT DATA PIPE
Another goal of hackers is to gain access to data in the application while it is running. There are several techniques that may be employed, and most take the form of intercepting various data streams and redirecting them to places where they can be exploited. There are several areas where hackers look for these “data pipes”, including test APIs and unencrypted network streams. Holodeck gives testers the ability to quickly and simply discover and monitor both of these pipes.

Discover Unprotected Test APIs
Test APIs are constructs used by software teams to drive applications in the testing process. They generally expose most, if not all, of an application’s functionality, including data access. A common flaw in many applications is not disabling or removing the test interfaces from the shipping application, providing a simple exploit vector for hackers. Holodeck provides complete API discovery and monitoring support that allows testers to identify and log all modules an application loads, the API calls it uses and their sequence. Knowing which calls an application makes out of the set of all the calls available and which it doesn’t is significant, particularly if they are owned by the application itself. The portion of the API that does not get called may very well be an application’s test API or some other vector into the application.  Holodeck allows testers to quickly expose these sensitive points and enable their lockdown.

Discover Unencrypted Network I/O
Data that passes over the network should always be encrypted so that it cannot be viewed by network sniffers or other similar tools. Additionally, data should be encrypted before it is passed to any public API such as Winsock, the Windows socket API. Holodeck allows testers to monitor both point to point network streams and the public APIs that an application calls. This allows testers to analyze data that is passed from their application to the network and across the wire - trapping potentially exploitable I/O vectors in the process.

MUCH, MUCH MORE....
There are a plethora of ways that hackers try to attack applications and gain access to sensitive data - we have called out just a few of them above. Holodeck allows testers to create runtime environments that expose all the points that a hacker may try to exploit and give the software development team the information needed to lock their application(s) down. Holodeck is a software development team’s best defense against hackers.  The best defense is a good offense, and Holodeck provides the best offense possible.