Traditionally, dynamic malware analysis is done with Virtual Machines (VMs). VM technology provides many advantages and conveniences for malware analysis. Snapshots, multiple OS images, and disk imaging are some of the features that make virtualization a natural fit for malware analysis. This allows VM-based systems to be easily automated to analyze a large number of samples with a relatively small investment in hardware.
Virtualization and malware analysis almost sounds like a marriage made in heaven. But the truth is that anyone using VM-based malware analysis sandboxes has actually been behind the curve for a long time. An ever-increasing sophistication of malware, including recently revealed APT attacks, has made it clear that malware can be incredibly smart and easily outwit VM-based sandboxes.
Increasingly, malware employ the use of anti-analysis techniques. This means that malware attempt to detect when they are being analyzed by AV systems, allowing them to change their behavior so as not to trigger any alarms. VM-detection routines used for this purpose are commonly traded amongst malware authors.
Some researchers and companies have become aware of the weakness of VM-based analysis and have designed their systems around emulation instead. In fact, we considered it ourselves. With emulation, an entire target computer, including its CPU and other hardware are simulated within another computer. This allows a sandbox to become immune to most VM-detection routines. Unfortunately, this typically results in massive performance problems. And worse still, there are anti-emulation techniques like timing attacks that malware can employ to detect even these systems.