…continuedHow We Conduct S&T Test Reports
Frequently Asked Questions
The Internet provides a global public forum where readers can communicate in an instant, not only with us, but also with each other. We receive many e-mail messages about S&T Test Report each month so many that we can't answer them all and our reviews are a frequent topic of discussion in newsgroups and other online venues. Certain questions come up repeatedly; we'll answer a few of them here.
"What role, if any, does the manufacturer of a product have in its review?"
As already noted, if a new product is scarce or especially costly, we might borrow a sample from the manufacturer or distributor for S&T Test Report. Obviously this alerts the company that its product is being reviewed. Another way a vendor might learn about an upcoming review is if we call with questions that came up during our bench or field testing, or to verify current pricing and availability. But we never share the contents of a review with a vendor in advance of publication, and usually a company that knows about an upcoming review doesn't know which issue it will appear in.
Manufacturers sometimes encourage us to review their products and sometimes they actually discourage us! But we decide what equipment to review based on reader interest and on our own perceptions of which new products are especially significant in today's astronomy marketplace.
"But aren't you constrained in what you can write about products because so many manufacturers and dealers advertise in your magazine?"
Yes and no. We are constrained to get our facts straight. Both readers and advertisers look critically at what we write, so we take great care that every number, statement, and test result gets checked and double-checked for accuracy. But we are not constrained to say only positive things about the products we review, and we don't.
Many consumer publications perhaps even most of them are paid for mainly by ad revenue. But that's not the case with Sky & Telescope. In its early years the magazine carried very little advertising, mainly because there were hardly any commercial suppliers of telescopes and accessories. While there are many astronomy vendors and plenty of advertising in S&T today, most of our revenue still comes from subscriptions and newsstand sales. So if you believe that money talks, then the loudest voice is that of our readers, and it always has been.
We sometimes joke that we know we've written a fair, balanced, and accurate review when we hear equal amounts of praise and criticism from both readers and advertisers!
"Why don't you ask for input from people who've actually bought and used the equipment?"
We do! We get lots of reader feedback on new products, and we read additional users' comments on the Internet in the course of preparing a review. But we believe you get a better value from S&T Test Report if we have our own experts do the tests and write up the results in a manner that's consistent from issue to issue.
There's nothing preventing a consumer from writing a fair review, but sorting the good ones from the poor ones is not easy for those not already familiar with the equipment in question. And it's usually impossible to tell whether the reviewer has a personal agenda or experience with any other equipment.
As already explained, most of the time we anonymously purchase the equipment we test, and we test it the way consumers use it, over a period of months, rather than publishing first impressions based on a night or two of casual use.
"Why do I sometimes find comments on the Internet from amateur astronomers who pan equipment that S&T has reviewed favorably?"
Many online reviews get written because the author is either very excited or very disappointed in a piece of gear. Rarely do you see a review by someone who is simply satisfied. As a result, much of the writing revolves around supporting the idea that a given product is either absolutely amazing or utterly worthless. In our experience, most equipment falls somewhere in-between.
When we search the Internet for comments on a product before publishing a review, we look for issues (both good and bad) that are mentioned consistently. It's a mistake to judge a product based on a composite picture assembled from random reports of problems by different people using different instruments.
It's certainly true that not every telescope comes out of the box in perfect condition. But unless there's an overall indication of shoddy quality something we would definitely mention in a review a random problem such as a stripped screw or a small ding in the paint usually isn't worth mentioning. We will definitely cover something major, such as dead electronics in a Go To telescope.
We constantly work at improving the quality of our product reviews through internal editorial assessment, input from optical designers and equipment manufacturers, and most importantly, feedback from readers. Our goal is to keep S&T Test Report second to none.