Rescoring of 2010 Olympics

[Baron Vladimir]

Well, what you personally want to value more is a completely different topic to debate on! It's not the point in what i personally value more if i judge in a system where many different things are equally valuable.

 

[gkelly]

No, even the new system is not universal in those things you are talking about, it is actually an opposite of it. Because for a +4 GOE on jumps or spins there is at least 3 different ways to do it, for +3 GOE element there are more than 10 equally valuable ways. For a 8 in a component there are more than 10 different ways to get there etc etc That's why the new system doesn't pretend to value one quality more than another per se (like the old one did), and you can't claim how one quality is better than another, because you can get to the similar degree of quality in many different ways by having many different criteria (actually written) to told you that. In this system it is written that two different things can be equally better, so there is no point to 'debate' how one is better than another.

This is about what the judges value in awarding GOEs . . . according to what the guidelines list as positive bullet points.

And also about the ranges of negative GOE reductions based on the severity of errors.

Both of which can be in the eye of the beholder to some extent.

6.0 system never said you can't "get to the similar degree of quality in different ways".

True. The 6.0 guidelines listed a number of things to take into account, but they didn't officially say what to give more or less weight to.

Also, with the CoP system you absolutely can "claim one quality is better than another". It's still up to the judges if they want to value height in a jump more than an arm position,

Yes . . . though it's more a matter of where each judge draws the line between yes an element qualifies for that bullet point or no it doesn't.

or a good spiral as a better transition than turn, or a certain spin position as better than another, or whatever else in terms of the program components.[]

Yes, plenty of room for judges to disagree there in the same way they could disagree in 6.0 evaluations.
Without any of them being wrong.

the new system DOES "value one quality more than another" - look at the footwork rules for example. There are many other different ways to do footwork, but the system only rewards a very limited set of movements/turns instead.

That's how the system defines the levels, and also what the well-balanced program rules allow and how many points the Scale of Values allots the various elements. I.e., all about the base value.

Which is different from what the system/rules says about GOEs.

However, now that there are some positive GOE bullets designated as mandatory for >+3 GOE, the guidelines do now specify that some qualities are more important than others.

But there's still room for judges to set their own standards as to what qualifies as "effortless," for example (or any other mandatory bullet point).

 

[Skatesocs]

Kolyadafan, have you received all the scores? I'm curious about when the actual scoring will be discussed.

 

[kolyadafan2002]

Kolyadafan, have you received all the scores? I'm curious about when the actual scoring will be discussed.

not quite yet, I'm getting them in dribs and drabs.

 

[Skatesocs]

not quite yet, I'm getting them in dribs and drabs.

It's your own project, of course. But I would've imposed a dead line by now. At least to stunt the OT discussion, before the scores were out.

 

[kolyadafan2002]

It's your own project, of course. But I would've imposed a dead line by now. At least to stunt the OT discussion, before the scores were out.

Many people have promised me the scores by a certain date, and I'm expecting to have them all by next week Wednesday, and I will have the scoresheet ready by the end of next week.

 

[Skatesocs]

Ooh, soon enough! :thumbsup: I want to see how many agree with me lol

 

[kolyadafan2002]

Ooh, soon enough! :thumbsup: I want to see how many agree with me lol

If anybody else wants to run a similar project, it would be interesting to see the variation we would have if 9 other users judged, and to see how the results would be effected between groups of different people.

 

[Skatesocs]

If anybody else wants to run a similar project, it would be interesting to see the variation we would have if 9 other users judged, and to see how the results would be effected between groups of different people.

As I said, I am more interested in the people who tried to do this exactly as the judges do. When I rescore an event, I always take three viewings (tech, PCS, overall review) and spend several minutes on each element and component. Humans are bad at multitasking - there's research that indicates this repeatedly. My results could be vastly different from the judges, because I have enough time to do it in a different way than they do. I do think my results are usually better - and sometimes the judges give an entirely nonsensical result, like this event - but given that in singles skating currently they have to deal with 17 things (7 jumps, 3 spins, 1 step sequence, 1 choreography sequence, 5 components) in an LP, I'd say it's nearly impossible to arrive at a great result with this time pressure, even with a certain amount of expertise. It is not shocking that they often default to reputation, politics, biases, nationalism, audience reaction, so on and so forth. Any system based off human observation and heuristics will be limited in this manner, anyone who claims otherwise is simply incorrect.

I don't want to judge this event again, and I even said in my "wrong olympics results" thread it was just for fun - my issue is rather with the the quality of the skating that people keep propping up as amazing, which probably seeps into popular opinion and thence to the judges ears - but it would be interesting to see what people came up with here. We often even have better camera angles than the judges do, the judges' perception of skating and jump quality will be affected by perspective and angles of incidence. I'd rather run a different event - but again, for fun, and especially if we all agree to do it exactly like the judges do.

 

[lzxnl]

If anybody else wants to run a similar project, it would be interesting to see the variation we would have if 9 other users judged, and to see how the results would be effected between groups of different people.

Which Olympics would others want to rescore themselves? Sochi 2014? That would be a bloodbath.

 

[Skatesocs]

Which Olympics would others want to rescore themselves? Sochi 2014? That would be a bloodbath.

Why would you bring it up then lol

 

[Jeanie19]

Which Olympics would others want to rescore themselves? Sochi 2014? That would be a bloodbath.

2002 would be a good one to do.

 

[kolyadafan2002]

Which Olympics would others want to rescore themselves? Sochi 2014? That would be a bloodbath.

I was speaking of 2010, and comparing the results between different groups of judges to see if they'd be similar.

 

[CanadianSkaterGuy]

That's not what a universal scale means, and 6.0 system never said you can't "get to the similar degree of quality in different ways". Also, with the CoP system you absolutely can "claim one quality is better than another". It's still up to the judges if they want to value height in a jump more than an arm position, or a good spiral as a better transition than turn, or a certain spin position as better than another, or whatever else in terms of the program components.

You're contradicting yourself too, because the new system DOES "value one quality more than another" - look at the footwork rules for example. There are many other different ways to do footwork, but the system only rewards a very limited set of movements/turns instead.

But the judges don’t set the level - they only mark grade of execution. Also, there is nothing preventing a skater from doing footwork the way they want to. The system is designed to reward certain types of complexity and has a framework by which skaters execute their programs. Yes there are parameters but it doesn’t remove creativity altogether. If a skater wants to do 7 spins in their free program and no jumps, they can do so. But the element rules are to establish a baseline for which skaters can earn base value and score points, and an even playing field (hence why things like the Zayak rule exist). A structure is needed for the purpose of comparing skaters and then judges reward or deduct based on execution — and it’s far less arbitrary than 6.0 where judges make a conscious effort to mark skaters as relative to each other instead of as absolutes (which is more ideal for objectivity). Now that skaters are being marked more as absolutes, discussion is negligible as a trained judge should be able to independently assess a program. Debating the merits of skaters in the middle of a competition on serves to get others to shift or second guess their opinions. Then it becomes a case of who has the loudest voice or is the best debater - and for a skater’s fate to be determined by whether they have good debaters on the judging panel who will go to bat for them simply is not fair. Especially skaters from lesser federations or without national representation on the panel.

 

[Blades of Passion]

Also, there is nothing preventing a skater from doing footwork the way they want to. If a skater wants to do 7 spins in their free program and no jumps, they can do so.

And receive a Level 1 because of the silly rules and no points for those elements, and automatically lose because of it? Oh okay. It's a ridiculous assertion, like telling people they could become any kind of doctor they want, but then only paying people for being dentists. The long program rules should definitely allow for more flexibility (aka, let someone do less jumps and more spins if they want to, just count an equal number of elements for each competitor), there's no reason not to, other than ignorance on the part of the ISU.

and it’s far less arbitrary than 6.0 where judges make a conscious effort to mark skaters as relative to each other instead of as absolutes (which is more ideal for objectivity). Now that skaters are being marked more as absolutes, discussion is negligible as a trained judge should be able to independently assess a program.

It's not arbitrary at all to mark skaters relative to each other. The main point of judging a competition is to accurately assess who deserves to place where. If the placements end up being wrong, then the system is not working. Yes, it's good to judge on an absolute scale, so as to have a clear method of seeing what everything is worth, but that scale is always existing in relation to the perception of what the "absolute best" is, which often includes comparing a skater to the best executions ever. It's also not quite possible to always mark such things with absolute precision, arts are not exact in being quantified, we are always making approximations of worth. Therefore, if we are approximating the worth of one competitor's choreography or skating skills at a 7.5, then it's a good thing to compare those qualities to another performance at the same competition, so as to most accurately asses what each one should be worth.

Then it becomes a case of who has the loudest voice or is the best debater - and for a skater’s fate to be determined by whether they have good debaters on the judging panel who will go to bat for them simply is not fair. Especially skaters from lesser federations or without national representation on the panel.

That's. Already. How. It. Is.

The "loudest voice" and "debate" simply comes from the biased, self-interested federations and is told to the judges, whom they control. Skaters from lesser federations, or who have less political support, or whatever it may be, would be benefited by having an independent panel of experts who intelligently analyze programs.

 

[CanadianSkaterGuy]

2002 would be a good one to do.

This actually would be interesting to score using IJS elements. Of course, the comparison isn't the same (lol, I'm guessing there would be so many level 1 spins/footwork), but from an elements standpoint I wonder if Goebel could have beaten Plushenko (or even Yagudin) with an IJS panel.

 

[CanadianSkaterGuy]

And receive a Level 1 because of the silly rules and no points for those elements, and automatically lose because of it? Oh okay. It's a ridiculous assertion, like telling people they could become any kind of doctor they want, but then only paying people for being dentists. The long program rules should definitely allow for more flexibility (aka, let someone do less jumps and more spins if they want to, just count an equal number of elements for each competitor), there's no reason not to, other than ignorance on the part of the ISU.

Yes, because a level 1 footwork is easier than a level 4 footwork. We don't live in some fantasy world where the most exquisite double toe loop should be on par with an unremarkable quadruple toe loop in terms of value towards a win. Like any sport, you are free to execute the difficulty you want - but harder/riskier difficulty deserves to be rewarded. That is the sport aspect of figure skating.

A skater can do a string of quick footwork later on for the purpose of throwing it in there - that's what the ChSp is in there for. If they want to they can do a 4th spin (indeed a couple skaters do their 3 spins and then a 4th scratch spin to classically close their free program even though it's not worth anything). And GOE is an equalizer where a skater who can't do the hardest jumping passes can still stay in the game thanks to good execution on spins and whatnot. Sometimes a level 3 footwork that's well done will score better than a level 4 footwork that was frantically trying to get all features.

And face it - the system is now all about getting points. You could bemoan the fact that the most exquisite skater in the world with the best choreographed program that can only do doubles will never have a shot to win against those who can do quads, but that's how it should be - until there's an artistic competition where it doesn't matter your level of athleticism is marginally important compared to artistry, but such a competition would be driven hugely by subjectivity and essentially a battle of exhibitions worthy of fluffy 90's pro competitions but definitely not the Olympics.

And receive a Level 1 because of the silly rules and no points for those elements, and automatically lose because of it? Oh okay. It's a ridiculous assertion, like telling people they could become any kind of doctor they want, but then only paying people for being dentists. The long program rules should definitely allow for more flexibility (aka, let someone do less jumps and more spins if they want to, just count an equal number of elements for each competitor), there's no reason not to, other than ignorance on the part of the ISU.

It's not arbitrary at all to mark skaters relative to each other. The main point of judging a competition is to accurately assess who deserves to place where. If the placements end up being wrong, then the system is not working. Yes, it's good to judge on an absolute scale, so as to have a clear method of seeing what everything is worth, but that scale is always existing in relation to the perception of what the "absolute best" is, which often includes comparing a skater to the best executions ever. It's also not quite possible to always mark such things with absolute precision, arts are not exact in being quantified, we are always making approximations of worth. Therefore, if we are approximating the worth of one competitor's choreography or skating skills at a 7.5, then it's a good thing to compare those qualities to another performance at the same competition, so as to most accurately asses what each one should be worth.



That's. Already. How. It. Is.

The "loudest voice" and "debate" simply comes from the biased, self-interested federations and is told to the judges, whom they control. Skaters from lesser federations, or who have less political support, or whatever it may be, would be benefited by having an independent panel of experts who intelligently analyze programs.

No it is not. The panel does not in real-time discuss the skaters throughout the competition and if they do it might be a benign comment after that skater has been assessed like, good skate! Not - that's the best skate we will see today, and I hope people agree with me and mark accordingly! Certainly in real-time they don't turn to each other and say "Hey, was that jump in time with the music? Was that jump pre-rotated?" They have their own monitors to review technical details and they shouldn't be chatting about it or looking over each others' shoulders. It would be particularly bold of judges to communicate with each other during a competition and indeed judges have been historically sanctioned for shady attempts to "align" or "sway" other judges to favour a particular skater(s).

 

[CanadianSkaterGuy]

And receive a Level 1 because of the silly rules and no points for those elements, and automatically lose because of it? Oh okay. It's a ridiculous assertion, like telling people they could become any kind of doctor they want, but then only paying people for being dentists. The long program rules should definitely allow for more flexibility (aka, let someone do less jumps and more spins if they want to, just count an equal number of elements for each competitor), there's no reason not to, other than ignorance on the part of the ISU.



It's not arbitrary at all to mark skaters relative to each other. The main point of judging a competition is to accurately assess who deserves to place where. If the placements end up being wrong, then the system is not working. Yes, it's good to judge on an absolute scale, so as to have a clear method of seeing what everything is worth, but that scale is always existing in relation to the perception of what the "absolute best" is, which often includes comparing a skater to the best executions ever. It's also not quite possible to always mark such things with absolute precision, arts are not exact in being quantified, we are always making approximations of worth. Therefore, if we are approximating the worth of one competitor's choreography or skating skills at a 7.5, then it's a good thing to compare those qualities to another performance at the same competition, so as to most accurately asses what each one should be worth.
.

The logical flaw in this is that skaters who come earlier are assessed according to only those who have skated. So not only the first skater has nobody to be compared with, but the assessment is contingent on who has already skated which by luck of the draw could be a slew of strong skaters or poor skaters.

Plushenko skated 10th in the Vancouver 2010 SP. Everyone else who was placed in the top 10 skated 16th or later in the order. So how was a judge supposed to score Plushenko "relative" to the best skaters yet to come? And of the skaters before Plushenko, save for Amodio who skated 4th (and placed 11th), every one of the skaters before Plushenko placed 21st or lower. The guy who came last was 2 skaters right before Plushenko. So if a judge has that skate relatively fresh in their mind, is it fair then to shower Plushenko with amazing marks just because he was superior to the relatively mediocre skaters preceding him?


And it brings up a good point anyways for this exercise because for the SP we weren't watching the skaters the way the judges in 2010 watched them; there were other skaters in between, and floods, and warmups and a load of other variables that could change how they assessed the 2010 Olympics and how we did. For the purpose of this exercise, we all only watched the 8 skaters kolyadafan listed - we didn't watch the other skaters in the SP when assessing them. And the order of the LPs were different. For the record, I watched each skater's SP then LP... instead of all 8 SPs, followed by all 8 LPs (because I was trying to assess each competitor as independently as possible).

Regarding skate order: interestingly, in the 2010 FS, the flight of skaters right before Lysacek all skated poorly - Chan fell and had tight jumps, Brezina had pops and a fall, Amodio had several errors, Kozuka fell, Van der perren had a lot of pops and a zayak even, Ten had stumbles his skating/programs weren't at the level we know he eventually was capable of. So, really, NOBODY was lighting it up yet and the judges were hankering for a good skate from someone -- perhaps that might have augmented the marks Lysacek got for a clean FS versus the guys before him skating well? By comparison, Plushenko had to skate after Weir/Takahashi/Lambiel, who are typically known as skaters with good artistry, and they all skated pretty decently - so maybe that tempered how high Plushenko's marks ended up being?

 

[Ziotic]

I can’t wait to share more thoughts as all scores are supposed to be in on Wednesday.

Having not watched the event the first time around I can say that I felt some people were gifted marks based on reputation.

 

[Blades of Passion]

Yes, because a level 1 footwork is easier than a level 4 footwork.

The current rules for footwork do not encompass the full range of difficulty that is a possible. It's very possible to do a footwork sequence that is objectively around the same level of difficulty as what a current Level 4 might be, but that footwork will not get the credit it deserves. The same can also be said about spins to a degree.

And face it - the system is now all about getting points. You could bemoan the fact that the most exquisite skater in the world with the best choreographed program that can only do doubles will never have a shot to win against those who can do quads, but that's how it should be - until there's an artistic competition where it doesn't matter your level of athleticism is marginally important compared to artistry, but such a competition would be driven hugely by subjectivity and essentially a battle of exhibitions worthy of fluffy 90's pro competitions but definitely not the Olympics.

Arguing in extremes only and missing the idea. Yes, skaters are trying to do things to get points, but there are other ways that deserve to score points outside of the framework that currently exists. Aside from allowing for more variety, there also certainly can be an "artistic" program at the Olympics (the idea already has significant support in the ISU) that has less of a focus on jump rotations and element levels.

No it is not. [judges having their opinions swayed at competitions]

Yes it is. The judges are told how to score certain skaters by their "bosses" and other such "lobbyists" before the competition starts. Skaters are currently held under the thumb of these politics, with big federations having the most sway, making the judging less objective. As for tech panels, they are able to talk and give their opinion about aspects of the tech elements throughout the entire competition, that's a fact. So any claim of "judging would be hurt by judges talking to each other" is something that's already affecting competitive skating and always will as long as there's a tech panel.

The logical flaw in this is that skaters who come earlier are assessed according to only those who have skated. Plushenko skated 10th in the Vancouver 2010 SP. So how was a judge supposed to score Plushenko "relative" to the best skaters yet to come?

You're supposed to set as accurate of a mark as possible within the objective scale and then reference that as needed. Nobody is getting judged relative to skaters to come, rather they should all be getting judged within the framework of an overall objective scale, but WHILE being mindful of scores that were already given, so as to properly rate each performance in relation to each other. Perhaps there is added difficulty to keeping many performances in mind while assessing each mark, but a great judge should be sufficiently capable of handling this task, and referencing the numbers would only serve to improve judging consistency and mindfulness.