Obviously Metacritic doesn't have the revenue stream to just toss out prizes like Netflix can, but they should hold some sort of open contest to discover the best formula for ranking films/music/TV/etc. Metacritic is certainly loads better than rottentomatoes, but they are still quite imperfect.
Take Amadou & Mariam's Welcome to Mali. It's the highest-rated album of the year, with a 92 on Metacritic, which is basically the equivalent of an 11.0 on Pitchfork. And it's seriously an amazing album, and likely deserving of that score. But it's not even available in most U.S. stores, as it is currently an import; as such, not many reviews are available. And clearly, the reviewers that are more inclined to review such music (African pop-rock-ska) are predisposed to liking such music in the first place. And so eight outlets review it, and all of the reviewers love it, and probably knew they would love it before they ever hit Play.
Whereas TVOTR's Dear Science received an 88 on Metacritic (which is obviously an equally stunning score in the scheme of things). And it is probably even more stunning than Welcome to Mali's 92, considering it is based on the opinions of 40 different reviewers. Yet a cursory comparison of the scores doesn't really tell the whole picture, and as such, it becomes a futile enterprise to do things like year-end rankings.
Obviously I have no idea what algorithms would be used for this, just talking out loud. Obama probably won't get around to it in his first hundred days.