This is definitely the flaw of Metacritic.....and why I'd vouch for independent sites which incorporate multiple ratings over corporate ones, though I would like to believe the best...

Oddly though, this means I put more trust in the sites with less traffic, though yes, not just any site. I guess middle-range and up is what we should go for when talking about which sites should be counted in the MC ratings.

Interesting piece -- props for investigating.

So does this mean KZ2's metascore would be indescribably lower if not for whatever they're talking about here? That's unfortunate.

Though isn't that the case for every game? Every single game gets reviewed by every publication out there and most of them would be lower level sites and not the normal IGN and Gamespot etc... If anything take out some of the lower level reviews and the metacritic score wouldn't change much because it has all be universally positive.

I love how the GameGrep community posts intelligent comments. Meanwhile, over on the N4G thread, the comments sound something like:

"BALALSKDJF;LOASIDJGOPIJ KILLZONE 2 GREASESTEST GAME EVAR!!!!SDFLKASJDF;LKAJ

It is true that some lower level blogs have more pull than probably necessary on metacritic, but it can't be singled out to Killzone 2. It was reviewed by about 54 sources as of now, and this of course includes all of the major websites that people trust.

Gears 2 was reviewed by 88 sources and MGS4 by 81, meaning that they're had more lower level sites weigh in with their opinion than KZ2.

And I'm sure that once the game releases the review count will go up as reviewers who aren't recognized enough to get the game pre-release will publish their reviews.

And I remember reading recently that MC scores do get weighted depending on the source credibility.

KZ2's metascore has been dabbling among the 90-93 range, so I can imagine that there's at least a universal consensus. I just hope it stays above 92; the last thing the PS3 gaming community needs is an extra influx of fanboys...

I'm with dmf on this one. The thing to consider on a site like metacritic is the average not who is reviewing it. Even if a handful of lower level fanboy sites did get on there with ridiculusly high scores it would still average out when the other sites panned it. And it's not as if KZ2 is getting bad reviews from the "big" sites anyway. So this one minor site hardly boosts the average at all.

Ultimately to me this acticle comes off more along the line of Rigney complaining about his sites lack of recognition rather than a real flaw in Metacritic. This would only be a flaw if it wasn't an average. Don't worry Rigney I'm sure you could get your KZ2 review on Metacritic too if you wanted.

I usually check IGN, Gamespot, and Meta scores for a game before I buy it - and even before that I rent it first to gauge my own take on it over theirs/on par with theirs, then make the decision.

I do know that Metacritic are currently going over their listings of sites on their approval system.

We've been asked to submit a survey twice now, and the detail they now ask is impressive. It has been known that their system can be artificially raised or lowered in some cases by factions within the the system. So seeing that they are actively doing something about it gives me some hope.

Ideally we'll be allowed into their ranking system . .

Another thing, the PS3 site, has an Alexa ranking of 606k odd . . which really isn't THAT bad in the greater scope of things. It means they have an average of around 1000 - 1500 ppl a day viewing the site.

Sure, it's not in the 200k's yet, but given time it could well get there. As ranking up there in the 200k and especially the 100k's mark does take some time. And unless you've got BUCKETS loads of money and backing, then getting the coverage you need to get that high is huge.

Also remembering that when you start on Alexa, your site is usually around the 25 - 28th Million position before it starts to move . .

Plus a lot of smaller and more dedicated site are unlike the corprate run sites that rely on the backing and advertizing of the publishers that keep them going. The smaller sites are the ones in there with the other gamers. The ones you'll find in threads like this, on forums, in blogs and on sites etc. Rather than having to meet targets and levels handed down from management and not even get to spend a huge amount of time actually online or playing games.

So if anything, those smaller sites have a better grasp of the gaming world, as they are in it every day . . .

EDIT: Lol, I definitely just posted a comment in the wrong article.

Sorry...

I think some people (I don't mean anyone here) puts to much stock on the metacritic scores.