Thanks, Reviewers! (Part 2 of 2)
Posted by Jonathan Cohen at 02:46 PM
Just before Christmas, we announced a new milestone – 100,000 reviewer comments and counting! Today we’ll highlight ten tips for making the most of your site review: what types of comments might prompt SiteAdvisor to change one of its Web safety ratings? Which reports, though informative, won’t result in a modified Web safety rating?
But first, a quick clarification about how reviewer comments affect a site's rating. Reviewer comments never automatically change a site's rating. All comments are reviewed by SiteAdvisor to determine when a preponderance of credible evidence indicates a ratings change may be warranted. That means that a given site is not subject to an automatic ratings change by, say, a malicious competitor who posts unsubstantiated negative comments.
So what types of reviews are most likely to be deemed credible, and thus have an impact on a site's rating?
Five Ways to Make Your Reviews Stand Out
1) Identify a scam site we haven’t found yet:
SiteAdvisor staffers try to flag scam sites that would be nearly impossible for our automated ‘bots to catch. For example, we warn about work-at-home sites that promise fantastic pay-outs in return for up-front payment, ringtone sites that employ automatic rebilling without full or adequate disclosure and other sites that sell products or services which are normally available for free. But we can’t find them all. Submit a review about a scam site and clearly explain why it is misleading or deceptive. It’s extra helpful to judge it based on criteria developed by third parties like the U.S. Federal Trade Commission.
2) Refer to the research of other well-established security researchers:
We don’t monitor all the white hats out there conducting their own research. For example, if you find a site with a security breech and the experts at SpywareWarrior, VitalSecurity, or CastleCops agree with you, add a link in your comment to their findings.
3) Weed out False-Positives:
A false-positive refers to a site that received a red rating but actually deserves a green one. This doesn’t happen often, but when it does, it’s often because our automated testing flagged a download from a utilities site or a security researcher. For example, we rated factbites.com as yellow until a user wrote in to clarify that it is a research oriented site that should have been green.
4) Strength in numbers:
The more users who complain about a site rating (particularly users with high reputation scores), the more seriously we’ll consider a reported Web safety issue. That said, while we respect what every reviewer has to say, we don’t always agree. (See the newgrounds.com example from Part 1 of this feature)
5) Be prolific and insightful:
Here’s a case where it pays to do your homework and write often. The more often you submit a review and the more insightful your reviews, the higher your reputation score will become.
Rating posts affects reputation.Every time a fellow reviewer clicks “yes,” your reputation accumulates more points. Get enough points and your reputation score (out of maximum of 9) goes up.
The higher your reputation, the more weight we will give to your comments. To see our list of reviewers ranked by reputation score, click here.
Two Things to Avoid
1) “Bad” Language:
Profanity and flame wars are not helpful for site ratings, and we'll often remove posts with such language.
2) Vendettas:
This is not the forum to conduct a personal vendetta against a site. If you had a bad e-commerce experience, by all means, share. Include as much detail as possible: was the customer service non-existent? Was the advertising misleading? How? Did the product never arrive? But if you and the site owner are headed for court, leave the depositions with the lawyers. We reserve the right to truncate or remove long, rambling rants, particularly when they become personal.
Three Possible Reasons Your Review Didn’t Change the Site’s Rating
1) Inability to confirm the data:
We encourage you to include details in your posts like your virus scanner alerts, but sometimes, we can’t replicate the result with our own scan. Since we re-test on a regular basis, if the site truly has a safety issue, we’ll catch it sooner or later and document the issue.
2) Not enough time to confirm the data:
Many of our most prolific community members paste the headers and text from spammy e-mails they believe they received from the site in question. With e-mail in particular, however, it takes time to prove that a sign-up at that site results in spammy e-mail, and we'll generally want to recreate and document the issue ourselves before changing a site's rating. (Take a look at the extensive process we go through here). But such comments are still helpful, and allow us to prioritize sites for additional testing. Where there’s smoke, there’s often fire.
3) Not enough data:
We aren’t likely to change a site’s rating based on a single reviewer experience. Especially for e-commerce sites, a single bad experience, no matter how egregious, could be a fluke. We typically require a critical mass of reviewer feedback to accumulate before we change a site’s rating.
Your Suggestions
SiteAdvisor’s Web safety rating system is far richer because of the invaluable human component provided by our volunteers reviewers. We’d like to know your suggestions for Reviewer Central. You can let us know through the comments section of this post.
