The Rise of Rootkit-Based Malware:
Why anti-spyware and anti-virus software is no longer enough

by Drew Robb

There was a time there a little while back when it seemed that the security software companies finally had the hackers, criminals and unscrupulous businesses on the run. After years of difficulties with viruses, worms, Trojans and spyware, it appeared that maybe, just maybe the anti-virus (AV) and anti-spyware (AS) vendors were at last gaining the upper hand.

Such hopes, however, proved to be futile. The malware writers once again have the upper hand and AV/AS vendors have been caught napping. According to CERT, a security research arm of Carnegie Mellon University, reported vulnerabilities in applications jumped by over 50 percent in 2005 after three years of little or no change.

The situation is so bad that none of the AV/AS vendors can remove regenerating malware or detect rootkits. Even the ones that claim rootkit detection capabilities do it almost exclusively based on signatures, something that can be easily bypassed by newer strains of malware that use polymorphism (this means, literally changing into many forms) or incorporate more advanced rootkit technology.

And the bad news is that the situation is not expected to get better anytime soon. According to research by AV vendor McAfee, one in seven malware incursions use rootkit technology to obfuscate their actions. By 2008, over 84 percent of all malware are expected to be disguised by rootkits. That could create havoc unless new tools are brought in to augment today's inadequate security perimeter.

Evolving Menace of Hidden Malware

While the fundamental approach to malware prevention, detection, and removal has not changed in the past several years, the threats have evolved dramatically. A big driver is the fact that the spyware industry now generates several billion dollars in revenue each year and is backed by organized crime. They harness a variety of covert tools to generate advertising revenue, capture bank account and credit card information, and steal corporate information. The impact can be devastating.

In particular, rootkits can make detection near impossible by existing AV/AS products. The best example of a rootkit is one used by Sony to prevent copyright violations. The vendor didn't tell anyone it had placed a hidden policing mechanism on home computers without permission. Yet not a single AV/AS vendor detected it.

Rootkits by themselves, however, are not necessarily evil. But the technology can be used by malware to actively hide itself and escape detection. Using the stealth cloak provided by rootkits, spyware can operate undetected. The attacker can then remotely install or modify components, steal locally stored personal information and even use the compromised machine for illegal activities.

Another evolving malware menace is a keylogger. As well as recording keyboard strokes, it can steal critical information that is not even stored locally on the computer. For example, a keylogger can steal a person's credit card number when the user enters their credit card number for a legitimate online transaction. It can also steal passwords and use them to gain unauthorized access into the network.

"Rootkit-based malware are viewed as the kings of malware," says Jayant Shukla, CEO of Trlokom Inc., a Monrovia, CA-based security software company. "They are hardest to detect and remove, cause considerable damage to the network infrastructure and pose an unprecedented risk to personal information."

Rootkits typically enter via free software downloads (the rootkit is secretly bundled with another "free" application) or by exploiting an application vulnerability via web browsers, IM clients, VoIP clients or e-mail.

The Windows Metafile (WMF) exploit, for example, infected users via banner ads when they visited a web site, sometimes even at reputable web sites. Although it came in via the Web browser, it took advantage of a vulnerability in a component of the operating system that was not even accessible directly. Shukla terms these as "Ricochet attacks." Such attacks will gain popularity because they make it possible to exploit vulnerabilities in any component of the operating system.

Detection and Elimination

Fortunately, there are a few tools on that market that either detect or cleanse rootkits. Winternals Software of Austin, TX, has a free detection utility known as RootkitRevealer (www.sysinternals.com/utilities/rootkitrevealer.html). It spots registry and file system discrepancies that may indicate the presence of a user-mode or kernel-mode rootkit. Webroot Software Inc. offers Spy Sweeper for $30 as a consumer solution that blocks spyware and some rootkits (www.webroot.com/consumer/products/spysweeper/). Trlokom has developed an enterprise and consumer tool called SpyWall to defend against web-based external attacks and infection from malware. The client version is priced at $14.99 (www.trlokom.com/product/spywall.php).

"Webroot say they can handle rootkits, and Trlokom has created an entire product to address the rootkit and zero-day attack problem," says Natalie Lambert, an analyst at Forrester Research. "As that is its focus, Trlokom probably does a better job in detecting and handling rootkits."

SpyWall addresses rootkits via behavioral techniques and by isolating the web browser to prevent spyware infections. Untrusted programs are run in a sandbox where damage is contained, analyzed and eradicated. It costs $14.99.

One customer is Chun Yu Works Inc., a producer of nuts and bolts with a manufacturing facility in Chino, California. Its Windows PCs were being devoured by malware. As a result, the IT person had developed the habit of carrying around anti-spyware tools on a thumb drive for that inevitable moment when yet another end user reported an infection or performance slow. But addressing desktop casualties one after another was a thankless task.

"We spent too many hours every week handling spyware attacks on our desktops," says Roberto Wong, network administrator at Chun Yu Works. "It was taking so long to handle some machines that we began to wonder if it might be cheaper just to supply infected users with a new workstation."

The company adopted Trlokom software and Wong noted an immediate shift in threat-based challenges.

"After we put in SpyWall, we didn't get any more infection for six months," says Robert Wong, network administrator at Chun Yu Works. "We have had no problems at all with WMF, rootkits or other similar vulnerabilities."