April 2010

McAfee, ClamAV, McAfee: Why keeping security software up to date is mandatory, not optional

1: On March 31st, McAfee's support for their V1 format antivirus DAT files ended, and with it come the end-of-life for VirusScan 8.0. Plenty of warning had been given, and they'd even extended the end-of-life date by three months. Yet, come April 1st, McAfee's Community Forums were filled with tales of people whose antivirus had ceased updating. There was a twist to this tale, however. One customer was using a V2-supporting antivirus program, but it still wasn't updating. It turned out that updating to a later version of the McAfee Agent solved his problem.

2: The ClamAV Users' mailing list has become a bit of a flame-fest of late, all due to Sourcefire's withdrawal of support for versions pre-0.95. They'd announced all this six months ago, but users were still caught unawares. Unfortunately, the developers effectively "turned off" older versions when they issued a pattern update using a data format only supported by ClamAV 0.95 and later. And how the affected users howled.

3: Back to McAfee yesterday, and we find peoples' PCs being screwed up by a bad DAT update. Those who escaped unscathed were those who had read, comprehended, and implemented a setting change discussed in the last VirusScan 8.7 patch release notes. More luck than anything else, in this case, though.

Notice a common theme to all of these?

Due diligence, or lack of it. Not by the vendors, but by the users of their products.

Security software, by its very nature, requires frequent updates. Improvements to increase reliability, detect new classes of threat, and so on. It is critical to the security of an organisation's assets.

So why, oh why, are people sticking their heads in the sand and not keeping things up to date?

Fools.

McAfee Mayhem

Yesterday afternoon McAfee released their 5958 anitivirus DAT pattern and unwittingly unleashed a denial of service attack on thousands of PCs around the world.

The update mistakenly detected the W32/Wecorl.a virus in the system file svchost.exe, promptly quarantining it, and rendering the affected PCs almost useless.

The McAfee users' forum was soon full of posts from upset customers.

This afternoon, McAfee sent out an email to its security alerts mailing list from Dave DeWalt, President and CEO, full of spin over this incident.

"In the past 24 hours, McAfee identified a new threat that impacts Windows PCs. Researchers worked diligently to address this threat that attacks critical Windows system executables and buries itself deep into a computer's memory.
The research team created detection and removal to address this threat. The remediation passed our quality testing and was released with the 5958 virus definition file at 2.00 PM GMT+1 (6am Pacific Time) on Wednesday, April 21"

So far so good, and thanks for telling us how this came about, much appreciated.

"The research team created detection and removal to address this threat. The remediation passed our quality testing and was released with the 5958 virus definition file at 2.00 PM GMT+1 (6am Pacific Time) on Wednesday, April 21."

The timing seems right, good, good.

"McAfee is aware that a number of customers have incurred a false positive error due to this release. Corporations who kept a feature called “Scan Processes on Enable” in McAfee VirusScan Enterprise disabled, as it is by default, were not affected."

Nice spin, but that doesn't quite wash. From the Virusscan 8.7i Patch 3 release notes:

Issue: With the improved functionality of the on-access scanner memory scan, lower and middle ranged systems may see a performance impact at startup and after a successful AutoUpdate of the engine or DATs. Currently the Process on enable option is enabled by default on the shipping version of VirusScan Enterprise 8.7i. McAfee recommends that in a managed environment, disable this option prior to deployment of the Patch, until the impact of memory scanning can be determined for your environment. It is not possible to maintain both the more comprehensive scanning that comes with Patch 1 and later, and the former level of scanning. Therefore, only the more comprehensive scan is used.

NOTE FOR CURRENT AND NEW USERS:

* The Patch installation does not modify current settings to disable the Process on enable option.

* The VirusScan 8.7i NAP and extension that are included with the Patch do change the McAfee Default policy, but do not modify the My Default policy, or any custom policy settings that were made prior to the check-in of the new NAP/extension.

* The VirusScan Enterprise 8.7i Repost with Patch now installs with the Process on enable option disabled, unless the Maximum Security option is selected during the installation.

The emphasis in red is mine. So no, it was not the default apart from clean installs of VSE 8.7i Patch 3 repost into a virginal ePO. Tut (to put it mildly).

The CEO's email continues:

"Our initial investigation indicates that the error can result in moderate to significant issues on systems running Windows XP Service Pack 3."

That seemed to be the consensus on the McAfee forum, too.

"The faulty update was quickly removed from all McAfee download servers, preventing any further impact on customers. We believe that this incident has impacted less than one half of one percent of our enterprise accounts globally and a fraction of that within the consumer base."

I'm not sure how quickly it was removed, a notification timed at 12:47pm CDT (18:47 GMT+1) stated it had been removed. I think several customers would quibble about the precise meaning of "quickly" in this context.

Less than one half of one percent of a very large number is still a large number. Actual numbers would have been more meaningful.

"McAfee teams are working with the highest priority to support impacted customers. We have also worked swiftly and released an updated virus definition file (5959) within hours and are providing our customers detailed guidance on how to repair any impacted systems."

Within hours? Bad DAT released at 2pm GMT+1, replacement after 7pm GMT+1. So that's 5 hours between releases. Could a solution have reasonably been released sooner? The spinmeister doesn't tell us. 5 hours is a long time when people have PCs dying all around them.

As a result of this fiasco, widely respected Windows commentator Ed Bott no longer recommends McAfee security software. Ouch, that's going to hurt.

Securosis' Mike Rothman asks Who DAT McAfee Fail?. Not a totally stupid analysis, but he fails to consider reality. He suggests delayed updates as a workaround, giving sysadmins time to react to duff updates. But, McAfee, on a good day, releases DAT updates once daily. Not good enough to catch new threats, alas. But it is worse than that. New detections can take up to three days to appear in the DAT files. Yes, really! And even more delay in fighting malware's the last thing we need.

Oh yes, I almost forgot to tell you. We were lucky. I'd read that bit in the release notes and configured McAfee's ePolicy Orchestrator appropriately. Only one PC (out of several thousand) misbehaved, detecting the non-existent virus. Fortunately, McAfee Virusscan's attempt to clean it had failed, leaving the PC in a healthy state. I suspect that detection was the result of a rare (but known) bug where policies aren't correctly applied. Needless to say, McAfee's antivirus was stripped off and reinstalled, without any further issues.

This blog post is an expanded version of a post originally made in the McAfee Community Forums

Postscript, April 27th: Over at PCMag's "Security Watch", Larry Seltzer has some interesting comments on the Lessons of the McAfee False Positive Fiasco. Be sure to follow the links in his story. Neither he nor Ed Bott cover the "end user responsibility" angle that I do, alas.