McAfee SiteAdvisor Blog

It is the software with a million names: Spyware. Adware. Contextual advertising software. Behavioral targeting code. The ungainly but lawyerly Potentially Unwanted Program. Malware.

I’m not raising the nomenclature issue to be flip. Being labeled “spyware" can mean millions of dollars in lost revenue for a program’s publisher. Labeling something “spyware" can mean millions of dollars in legal fees for the one doing the labeling. The money issue alone makes these important debates to have, no doubt.

But for the average Web consumer, all this name calling is supremely unhelpful. When a user is facing a download decision, he just wants to know whether it’s going to muck up his machine. This spring, SiteAdvisor set out to develop a way to alleviate the mystery (and the misery) that goes along with these decisions. 100,000+ tested downloads later, we think we’ve got something that will really help the average Web user. In fact, when it comes to popular downloads, we believe we’ve got the only truly objective, comprehensive dataset on what they do to users’ computers.

Testing, Testing, One, Two, Three
Before I can tell you how we test downloads, I need to tell you what downloads we test. For SiteAdvisor purposes, a download is a program which can make your computer do something significant. In geekspeak, we look for executables like exe’s, scr’s and msi’s. Compressed files are also extracted and scanned for executables.

Now there are lots of files that can be downloaded that we don’t test for. At least not yet. For example, we don’t analyze audio or video files or Microsoft Word documents or graphic formats. So we’re not testing Jane’s resume or John’s photos from his trip to the Grand Canyon. If you think there’s a file format we should be testing, let us know. And if there’s a specific download you’d like us to test, if you’re curious about an untested download from MyFavoriteGames.com, for example, submit the link by going to their SiteAdvisor summary page.

Mount Up, Troops
So, on to the tests themselves. Once again, our ‘bots take center stage. Every day, thousands of times a day, our brave digital warriors power up their PCs and go forth to expose themselves to the best and worst the Web has to offer.

Once we find a program to download, we install it onto a “clean" PC. What’s a clean PC? SiteAdvisor designed a system using "virtual machines" that allows us, in effect, to use a "new" computer once and only once to test one and only one download. This way, we are absolutely certain that whatever happens to that machine can only be the result of that one software installation.

How bad is it, Doc?
After we find and install the program, we run the computer through a series of tests, measuring and documenting our findings at each step of the way. Essentially, we’re taking the computer’s temperature. Is it sick? If so, how badly?

With the program running, we put the PC through its browsing paces, visiting a series of Web sites selected because they’re popular and because they’re the kind of sites (i.e. travel, financial, gaming) that commonly trigger advertising. We also look for and document whether our browser settings have changed. For example, have our home page or search engine defaults been reset? Our goal is to show you how your browsing experience will be affected if you install the software in question.

We also summarize the download’s overall impact on a computer by displaying its 1-to-10 Nuisance Score. The one above is for an Aaliyah screensaver we downloaded from EntertainmentWallpaper.com. The Nuisance Score is SiteAdvisor’s proprietary synthesis of all the data we’ve collected on a download. It’s an at-a-glance guide to help you decide whether to download a program. Low scores result from minor nuisances like changed home pages. Higher scores result from bundled things like adware or viruses. Bundling more than one low-score nuisance can push a rating into the red zone as well.

You talking to me?
Often, malicious or annoying software can be identified by its digital "signature," the unique changes it makes to a computer's operating system. Since we use new computers for each download, our system registry always starts clean. If we detect any changes made there or to our system files, we show you every addition, deletion and modification. ScenicReflections offers a "Soothing Sunsets" screensaver, for example, that may look quiet on your monitor, but behind the scenes, it's anything but.

Likewise, SiteAdvisor watches and documents which network servers are contacted by the downloaded program. The presence of network traffic alone does not signal badness. It’s which servers are being called and how many of them are associated with malware. Again, the goal of this data is to give you a common sense check against software that takes "liberties" with your Internet connection. For example, we downloaded one program that contacted more than 50 servers.

Best Face Forward?
Like my email blog earlier in the week, this is another long piece of writing, but I had a lot of ground to cover. I hope it gives you a good sense of how we arrive at our test results for program downloads. One question that I get a lot is whether our ratings ever change. Some people point to the noise being made these days by contextual advertising companies who claim they’re cleaning up their acts. One of the great things about working here is that we can put those claims to the test. But that’s for the future.

--Shane Keats