Frequently Asked Questions

WhatNotToSing.com is the Internet's most complete public reference site on the popular U.S. television series American Idol. The heart of the website is the WNTS.com Performance Database containing information on every season, episode, contestant, performance, song, and artist in the show's run. We also feature news, analyses, essays, and other Idol-related features, mostly pertaining to the data we've collected. [Back to Top]

We've been collecting data weekly since Season Four in 2005. Ratings for Seasons Two and Three were calculated retroactively from surviving web sources and archives. WhatNotToSing.com went online in 2007, and the full database was opened to the public in early 2008. [Back to Top]

Quite honestly, we started Project WNTS because we grew tired of watching one promising contestant after another kill their once-in-a-lifetime opportunity by making inferior, foolish, or downright suicidal song choices. Many Idols are kids or not far removed, and some have no real familiarity with the entertainment industry. Choosing the right material and arranging and presenting it to one's best advantage are skills that music professionals take years to hone.

We could have just set up a website to publish our opinions, but what good would that do? (Most of you have heard the old saying about opinions, which isn't suitable for repeating on a family site.) Instead, we elected to trade in hard facts by applying our mathematical and musical backgrounds to the effort. By studying what has and hasn't worked in the past, we hope to enable future contestants to make better, more educated choices.

For the two young ladies who in particular motivated us to get off our butts and start this project, see the dedication on our About Us page. [Back to Top]

WhatNotToSing.com was founded by three longtime American Idol fans with long and varied backgrounds in mathematics, music, and web design. Brian and Nick are old enough to be the fathers of most of the contestants. Amy, who was in high school when we started the project, is our resident singing expert. She's now too old to audition for the show. When we started, we were all living in the Philadelphia area (hence the many references to Philly sports teams, Wawas, and other trappings of life in and around the City of Brotherly Love).

These days, only Nick remains in the Delaware Valley; Brian is back in his native Canada most of the year, and Amy lives in Florida. [Back to Top]

It's complete in the sense that every season, episode, contestant, performance, song, and original artist is in there. We even keep it up-to-date in real time during every episode whenever possible, so that visitors can quickly see who's performed particular songs before. However, it'll never be totally complete, because we're always coming up with new pieces of information to throw in there. [Back to Top]

No. There's quite a bit of data that we can't make available just yet. Sometimes it's because the information is incomplete or not up to our standards, other times because we haven't gotten around to building the necessary search pages. If there's a particular report that you need, and you think we might be able to generate the information for you, let us know. [Back to Top]

A primary artist is one who is most widely associated with a song by the general public. If we asked you who sang "Stairway To Heaven", we're going to wager that your first answer isn't going to be Dolly Parton (though she did cover it, along with many others.)

A secondary artist is one who recorded a notable but perhaps not as widely known version of the song. Generally speaking, it has to meet one of these criteria:

1. The artist had a Billboard Top 10 (or so) hit with it, but they're clearly not the first artist most people think of when they hear the song. (Examples: Cyndi Lauper for What's Going On; Aretha Franklin for I Say A Little Prayer.)
2. The artist is noteworthy and their unusual arrangement was the basis of an Idol performance. (Examples: Live for I Walk The Line; Chris Cornell for Billie Jean.)
3. The artist is noteworthy and they were the original songwriter, performed a well-known cover, or fall into that broad "because it seems right" category. (Examples: Carole King for You've Got A Friend; Van Halen for You Really Got Me.)

Songs may have multiple primary and/or secondary artists. A great example is Because The Night: We've credited both Patti Smith Group and 10,000 Maniacs as its primary artists since both had popular hits with it, and we list Bruce Springsteen as a secondary artist because he co-wrote it and frequently plays it live. [Back to Top]

Because Idol's cover of Hollywood Week is too spotty and inconsistent. Many performances never air because of time considerations, lack of copyright clearance, or the producers' general capriciousness, and the ones that do air usually last no more than 15 seconds. We list (but do not rate or cross-reference) audition songs for each contestant where we can, and we'll consider doing the same for Hollywood songs someday. If anyone cares to compile and donate a comprehensive list, that would definitely get the ball rolling. [Back to Top]

Many viewers have observed that American Idol often engages in some curious verbal gymnastics when it comes time to announce the results. Our solution to this conundrum is, quite honestly, to take our best guess. If Ryan says there's a Bottom Three, or if the results show unfolds in such a way as to suggest that the trio left standing on the stage were actually the three-lowest vote getters, we go with that. If Ryan says he's announcing results at random, we believe him. In borderline cases, we go with what Wikipedia says. In any case, we only track the B3/B2 during the Finals, not the Semifinals. Going any deeper than that is a fool's game. The way we look at it, there are few people on earth who are so laden with wisdom that it's worth hanging on their every word, and we're pretty darn sure that Ryan Seacrest isn't among them. [Back to Top]

Please do! Our goal is to keep the database as accurate as possible. Just send us an email. [Back to Top]

Every performance in the WhatNotToSing.com database is assigned an approval rating, which represents how much (or not so much) the Internet fans of the show liked the performance. Rating values, which are usually displayed in boldface on the site, range from 0 to 100, with 50 being average.

Seasons, episodes, contestants, songs, and artists also have approval ratings, which are calculated from the average of the relevant performances. [Back to Top]

Ratings are calculated from reviews of each episode posted on publicly-accessible websites, newsfeeds, blogs, forums, message boards, chat rooms, and other online sources. We sample dozens of sites each week and tally opinions statistically. [Back to Top]

It's easier to list the sources we don't use. We don't count the judges' opinions nor any posted on the official American Idol website (so as to remain fully independent of sources controlled by the show.) We don't consider fan sites that advocate for or against a particular contestant, nor any site that has an obvious bias or agenda. We visit only free websites or ones with open signup policies; any site that requires us to pay money to access, we skip. Finally, like any good pollsters, we don't count the opinions of the WhatNotToSing.com staff, nor our families' or friends'.

Other than the above, pretty much anything goes. [Back to Top]

A few are focused solely on AI, others on Reality TV in general. But most sites we've used have nothing to do with the show at all – they're online newspapers, community message boards, hobby sites, tech sites, blogs, and wherever else we find folks discussing the latest episode of American Idol.

We've pulled opinions from at least 200 different sites over the years. [Back to Top]

Back in American Idol's glory days: no. We chose a different sampling of sites each week because there was no shortage of sources. These days, however, we'd be lying if we said we didn't visit the same sites pretty much every week. We do check Twitter nowadays, though, which gives us a certain amount of variety. And, since 2013, we've relied heavily on our "Review Crew" of independent AI fans who send us their reviews via email or Facebook. [Back to Top]

Never fewer than 200, and usually 300 or more. [Back to Top]

Again, this has changed over time. We used only to consider opinions timestamped between the performance and the start of the results show, to avoid the raw emotions of a shocking elimination from skewing the numbers. As the number of online sources dwindled, we had to make allowances. And, since the AI reboot when results are announced in real time at the end of the show, it's obviously impossible.

Fortunately, few people these days take American Idol eliminations as being affronts to the very fabric of civilization, as some did when the series started. Our "Review Crew" are particularly hardy souls, and virtually all take pride in the integrity of their ballot history. If there's been any problems in the past ten or so seasons, we've not noticed.

After the Finale, and before we freeze the ratings for good, we used to perform one last fine-tuning based upon the Idolsphere's "Best Of The Season" and "Worst Of The Season" wrapups. Alas, those too have gone the way of the carrier pigeon. So, the only adjustment nowadays is our annual Post-Season Normalization, which merely fits the just-concluded season's ratings into the historical distribution of past seasons. We do this because, quite honestly, we might never see a 90 rating again otherwise -- the variety of songs and genres on the show have made it such that consensus is difficult to achieve and thus very high scores are exceedingly uncommon. Normalization essentially adjusts for this, and it usually allows 87s and 88s to become 91s and 92s. That keeps everyone happy. The season is then considered closed, and no further adjustments are made. [Back to Top]

In part, but many other factors go into an approval rating as well: song choice, presentation, the judges' remarks, the contestant's behavior and composure, personality, past performances, and how well other contestants performed that night. Singing quality is the single most significant component, but it's far from the only one. We measure how well America liked the whole package. [Back to Top]

"No, no, a thousand times no!" American Idol voting patterns are far too complex to boil down to a single number. What matters instead is how well a contestant motivates viewers to vote for him or her. Strange as it may seem (unless you've watched the show for years), how much they liked or disliked a performance on a particular night isn't always the primary factor in their decision. Therefore, never mistake approval ratings for a voting prediction system. [Back to Top]

That's not a question. [Back to Top]

Because that's how Internet fans rated them the night of the show. If performances grow or shrink in stature over time, whoopee. We can't measure that, we don't try to measure that, and until the day comes when American Idol allows us fans to vote on past seasons retroactively, we couldn't care less about measuring that.

If it makes you feel any better, there are a few performances in our database that we happen to think are overrated or underrated, too. That's to be expected in any consensus-based poll. We believe we have a solid and accurate methodology, and we apply it uniformly week in and week out. In the specific cases of Crocodile Rock and Build Me Up Buttercup, which are the two performances we're asked about the most, you can click on their respective links to read our analysis of why we think they scored as they did. But no, we're not going to change their ratings years after the fact to bring them in line with some perceived conventional wisdom (which is a moving target anyway.) [Back to Top]

When it comes to entertainment ratings, most folks are more familiar and comfortable with a scale of 1 to 5 stars. So, just to be nice, we often express ratings in terms of stars, by dividing our 0 to 100 scale into five 20-point intervals:

	Stars	Numeric Rating
	1 star	0 to 19
	2 star	20 to 39
	3 star	40 to 59
	4 star	60 to 79
	5 star	80 to 100

It's a statistical measure (abbreviated σ) of how much variation there is in a data set. Put simply, the higher the value, the more the reviewers disagreed. A love-it-or-hate-it performance, such as Blake Lewis's beat-boxed You Give Love A Bad Name, will have a high standard deviation, while one that was universally liked or disliked, e.g. Bo Bice's a cappella In A Dream, will have a low σ.

On WhatNotToSing.com, the average standard deviation of a performance is about 18. [Back to Top]

Because when we were originally preparing the past seasons' ratings, we didn't locate enough surviving sources on the Internet to meet our standards. This was particularly true for the semfinals and wildcard episodes, which aired before American Idol's TV ratings took off. To supplement our data, we established the Season One Review Crew, a panel of impartial AI critics and fans to help us review the performances from that long-ago season. Thanks to their invaluable input, we were able to finalize the AI1 ratings and publish them in May of 2008. [Back to Top]

It refers to the online community of American Idol fans: bloggers, forum posters, web columnists, roundtable panelists, etc. The word is probably derived from "blogosphere". We didn't come up with it, and we don't know who did, but it's a useful term. If you'd like to claim credit, drop us a line. [Back to Top]

Nonexistent. The Internet doesn't need another public Idol board; it has plenty of good ones already. Besides, our goal is to remain independent of both the show and its community of online reviewers. The last thing we need is to offer forums of our own, thus discouraging fans from posting reviews on other websites. [Back to Top]

Again, there are enough good (and not-so-good) American Idol-related websites out there that we feel no particular need to become yet another media clearinghouse, particularly as all images and audio from the show are copyrighted. Like everyone else over the age of 5, we know where to find Idol media clips on the Web when we need them. We've included a few photos from the show here and there that we think qualify under 'fair use' guidelines. [Back to Top]

Most "shocking" Idol eliminations are anything but; they usually match some well-established, fairly simple patterns. To make it easier for Idol fans to recognize the danger signs in the future, we borrowed the names of a pair of old kiddie TV shows to describe the two most common scenarios.

The Mighty Mouse Syndrome is the simpler of the two. When an established contestant turns in their first utter disaster of a performance, they are virtually never sent home or even in the Bottom Three. That's because his or her terrified fanbase mobilizes like crazy to rescue them. You can easily imagine their fans chanting Mighty Mouse's famous slogan: "Here I come to save the day!" But contestants beware: this is not a well you can go to too often, because fatigue and disappointment limit the number of times the MMS can save your skin.

The Sesame Street Syndrome is more complex, but more interesting (and, more powerful.) When two contestants produce a train wreck on the same night – and we don't mean merely "poor", we mean "godawful" – we've shown empirically that over two-thirds of the time, neither one will be sent home. It's like the Mighty Mouse Syndrome on steroids: not only are the two respective fanbases scared, but they sense weakness in the other contestant. "OK, Ernie was bad," they reason, "but Bert was just as terrible. All we have to do is throw Ernie just enough votes to get past Bert, and he'll be safe." Bert's fans reason exactly the opposite, of course. So who winds up going home the next night? Kermit, unexpectedly and undeservedly. Hence the name. [Back to Top]

Throughout the site, links into our database pages are tagged with icons so that you know what section of the database you're entering:

This helps avoid confusion, particularly since Performances and Songs usually share the same name, and because a few former Contestants now have their own Artist pages too. [Back to Top]

You're using an old browser that doesn't fully support the modern PNG image format, and we'll lay dollars to donuts that it's Internet Explorer 6. It takes a tremendous amount of additional work for web developers to support the long-obsolete IE6 fully. The WhatNotToSing.com design staff has compromised on this matter - we've wasted invested a lot of time ensuring that the site is fully functional in IE6, but little to no time trying to make it appear attractive in it. (These days, with auto-updated browsers and apps, this problem has pretty much disappeared, but you can't imagine the grief it caused us back in the day.) [Back to Top]

Because we're a research and analysis site, not a fansite, so we try to keep things formal. It's easier for everyone to stay objective if we don't sound too chummy with the Idols, none of whom we've ever met personally anyway. (But yes, when discussing contestants among ourselves, we say "Carrie and Bo" like everyone else, not "Underwood and Bice". We're not that prissy.) [Back to Top]

If you'd like to volunteer your talents on WhatNotToSing.com, be it as an author, editor, music historian, analyst, graphic designer, or any other role, just drop us a line. We'd love to have you join the team. [Back to Top]