MadTracker - Forum - The dodgy review system.....

Color scheme

Profile

FAQ

Forum

General discussion

The dodgy review system.....

Reply to topic

Post new topic

Goto page 1, 2, 3 Next

em22
Registered User

Joined: 06 May 2003
Location: Manchester, England

Posted: Mon Jul 12 2004 15:36 The dodgy review system.....
Okay, my tune was a bit crap but I'm slightly concerned to how the reviews were done. Some people were revied by 3 people, some by 5, some by 4...... Even if the results are the average worked out from the reviewers, there really shouldn't be any difference in the way in which the tracks were reviewed. i would like an explanation on how you see this as fair. Cheerio.
em22

Yannick
MadTracker Author

Joined: 16 Apr 2003
Location: Belgium

Posted: Mon Jul 12 2004 15:43
As not all the reviewers could write reviews for the whole 95 entries, it's logical that some entries have less reviews than others. On the other hand we did our best to have a maximum of reviews for the high-ranked songs, to have the fairest ranking at the top. (Btw, big thumbs up to Sunbuster who managed to review everything!) Yannick

Inge
Man-At-Arms

Joined: 04 May 2003
Location: Nieuw Lekkerland @ Holland

Posted: Mon Jul 12 2004 18:49
I understand your point of criticism. Let's see if this can make things more bearable: The only sensible explanation implies that you trust all judges to be equally objective and talented in their style of reviewing and giving marks. This would imply that it is irrelevant who reviewed you. We did make sure that all high entries got atleast three reviews. Furthermore, I for myself did listen all entries, and checked the places and actual reviews. If I thought that an outcome was unbalanced, I would add a review. If I thought that the current reviews already said everything worth saying, I wouldn't write a review. Inge
Care for a game of Monopoly?

InHousePussy
Registered User

Joined: 13 Nov 2003

Posted: Mon Jul 12 2004 19:17
quote Inge-The only sensible explanation implies that you trust all" so what ur saying is;there is no standard at all...just taste..."?? when one person gives a track a score of say 4 and the otherone gives an 8 on lets say tech. where is the cred and stability...??? Me don't know...

Inge
Man-At-Arms

Joined: 04 May 2003
Location: Nieuw Lekkerland @ Holland

Posted: Mon Jul 12 2004 19:44

inhousepussy wrote:

quote Inge-The only sensible explanation implies that you trust all" so what ur saying is;there is no standard at all...just taste..."??

when one person gives a track a score of say 4 and the otherone gives an 8 on lets say tech. where is the cred and stability...???

Me don't know... Cool

Oh my. The good old 'defend yourself if you have judged a compo because a certain amount of users will not agree on the system'. Let's try again.

I haven't said for a single moment that there is no standard. What I did say, is that I trust all individual reviewers to be gifted enough to write a good, constructive review without too much backgroundnoise due to subjective influences. Whether Sunbuster, Novus, Mindplay, Yannick or I reviewed a song, the actual outcome will be more or the less the same. Three reviews should atleast be enough to get the average scoring balanced, and it should clear out the backgroundnoise.

Ofcourse individual scores will differ much for certain tracks. But look at the top 10. The reviews there are - generally speaking - very homogenous. This indicates that the judges have reviewed quite similar, despite their differences in background and style. Extrapolate that red line, and you will come to the conclusion that three reviews suffices in most of the times.

And yes: some songs will have got lower scores due to individual preferences. But if a certain song really was good, then it will have got good scores (and vice versa, for that matter).

Inge

Care for a game of Monopoly?

InHousePussy
Registered User

Joined: 13 Nov 2003

Posted: Mon Jul 12 2004 21:11
don't get me wrong Inge,,This is a thought that i think alot of ppl r thinking...And btw,I get U'r meaning..np.. But the thing I'M thinking of is(and not beacuse of my score..come on man..i ain't no kid,and i know and hear my entry wasn't top notch) that in all other compos there is a set standard when it comes to the tech. scores,and then i start wondering;is there a tech standard at all???, cus u know this prog in and out..or at least u should..since ur the jury...(no?) and thats all... A'it?? (And btw dont patronize ppl..it's not nice.. ) ok? Just to end this on a good note; brill show...
Last edited by InHousePussy on Mon Jul 12 2004 22:32; edited 1 time in total

Sunbuster
Registered User

Joined: 05 May 2003
Location: Finland

Posted: Mon Jul 12 2004 21:29
so, we once again ended up at this...

InHousePussy
Registered User

Joined: 13 Nov 2003

Posted: Mon Jul 12 2004 21:39

Sunbuster wrote:

so, we once again ended up at this... Sad

That was NOT the point at all...

just count me out this if thats ur viewpoint..

i'm done... Dancing

Inge
Man-At-Arms

Joined: 04 May 2003
Location: Nieuw Lekkerland @ Holland

Posted: Mon Jul 12 2004 21:47

inhousepussy wrote:

Sunbuster wrote:

so, we once again ended up at this... Sad

That was NOT the point at all...

No harm done, appearantly, since we probably miscommunicated.

About the techstandard you refer to: please explain further. I simply have no clue what you refer to.

As for the patronizing tone: you might be able to imagine that a *tremendous* amount of time has been investigated into this project. I'm not going to defend our system endlessly, and thus my voice of communication might be somewhat top-down instead of equally leveled. I'm too proud of the results to get sick and tired of endless debating. If I misunderstood you, then accept my apologies. If you think we did a bad job: pay me, and I'll give up my parttime job to write reviews for everyone Wink

Inge [who didn't pay Sunbuster to write reviews for everyone, btw Wink

]

Care for a game of Monopoly?

InHousePussy
Registered User

Joined: 13 Nov 2003

Posted: Mon Jul 12 2004 21:55
Thanx for that Inge. what i mean is ; the reviewers should hold the highset technical knowhow when it it comes to tracking..if not, even i could be a reviewer of tech.quality..and not have clue at all.. btw -how much do u want??

Inge
Man-At-Arms

Joined: 04 May 2003
Location: Nieuw Lekkerland @ Holland

Posted: Mon Jul 12 2004 22:05

inhousepussy wrote:

what i mean is ; the reviewers should hold the highset technical knowhow when it it comes to tracking..if not, even i could be a reviewer of tech.quality..and not have clue at all.. Wink

I haven't used MadTracker for a year, apart from beta-testing. I do have something like three or four years of hardcore MadTracker experience, so one might say that I understand the basics quite well Wink

What's more important: I know what the limitations are of a sample-based tracker. If one was able to overcome these things, then I was technically all happy.

I hardly even looked into the tracks, since I mainly based my reviews on the actual quality of the tracks. I based the technical rating on how it sounds: does the production impress me, can I hear that attention has been payed to overall sound, is the sound quality optimized, do I hear adding value based on track fx and so on. It's all about the music Smile

inhousepussy wrote:

btw -how much do u want?? Wink

The world, my dear, the world Very Happy

Inge

Care for a game of Monopoly?

InHousePussy
Registered User

Joined: 13 Nov 2003

Posted: Mon Jul 12 2004 22:14
and that's just what i'm talking about...do u see...or do u listen...and who does what..? dang...i didn't even start this topic..but...now i know how things work whithin the jury... enough... -Ceri

mikx
Registered User

Joined: 05 May 2003
Location: Melbourne

Posted: Tue Jul 13 2004 04:56
i see your point IHP. you're right, and i think for the next compo maybe it would be better for there to be a protocol to follow when judging the track's 'technicality'. perhaps instead of just listening to how technical it sounds, have a look at the way it was produced and actually see how technical the producer was in terms of using channel effects, etc.. i realise there would be a difference of opinion in this too- for example one judge might have a different opinion of technicality than another. hmm.. difficult issue. i personally thought the judging was pretty well done, and it has been like this for 2 years now, so quit whinging! mikx

Walter Vos
Registered User

Joined: 04 May 2003
Location: Groningen, The Netherlands

Posted: Tue Jul 13 2004 07:49
I didn't participate, but I think the judging went really well this time, last time all technoish tunes were reviewed to the ground, because somebody obviously didn't know much about that genre. This I think it's much better. I do have some doubts about the technical thing now though. But I do think what Inge says is ok too. I you're impressed with what is coming out of this tracker then it is technically good. So... well, what's my point now?
what is hip? www.waltervos.nl

Sunbuster
Registered User

Joined: 05 May 2003
Location: Finland

Posted: Tue Jul 13 2004 10:18
ihp: the sad smiley was mostly directed at the starter of this topic, and on the fact that the reviewing system once again got under fire without there being any real suggestions as to how it should be improved (which now have surfaced later in the topic) well, I for one did go into how the mt module was built up. If someone got a great drum beat for instance, but achieved that by using loops and didn't state where those loops came from I might have deducted a point, since someone else could have made (and probably did) the loops. So a small reminder for next year, if you use loops, state if you made them yourself or took them from somewhere Another example, if a track was very heavy on channel fx, I checked if the fx for several channels was routed to one channel whenever possible. If not, then some points were deducted. In the technical score I also took into account the overall mix to some extent. So if you otherwise had a great track, but it was very centered, or one instrument stood out a lot from the others, then you might have lost a point or two on that.

Reply to topic

Post new topic

Page 1 of 3

All times are GMT + 1 Hour
Goto page 1, 2, 3 Next