Justifying claims of under/over scoring | Page 3 | Golden Skate

Justifying claims of under/over scoring

Perdita

Final Flight
Joined
Dec 15, 2014
Still, I think there is also room at the table for fans who just want to say, "Oh that performance was so beautiful and sublime; it touched my heart; she hit every note of the music; she deserves a 10 in PCSs and I don't care what some old ISU document says!" :yes: ;)

Oh yes, of course :yes: If I have my own rulebook, I'm not gonna give 6s and 7s for Deniss Vasiljevs. He deserves 10s!

ETA: That was a bit of an over statement... I take that back and say he deserves 9s at least :biggrin:
 
Last edited:

drivingmissdaisy

Record Breaker
Joined
Feb 17, 2010
Where is the proof that skaters automatically get lower PCS for skating in an earlier group?
I'm sure it happens... but I DON'T believe it happens as often as some people claim.

I don't think it is possible to "prove" the claim, but I do think it is quite common. We have often seen performances from very good skaters (Mao's LP in Sochi, Yuna's SP at 2013 Worlds) in which lower PCS that we expect to see for those skaters is observed. For Mao in particular, I think her SP PCS was actually better than her LP PCS even though the LP was a much better performance. However, she skated last in the SP and very early in the LP in Sochi. I don't know how else to explain such a discrepancy in her own PCS scores in that single event other than skate order. I don't think it's automatic, but I just think the judges mentally "anchor" PCS scores relative to skaters they have just seen.
 
Last edited:

Alex D

Record Breaker
Joined
Sep 23, 2013
Should several of us try to analyze the programs?

Maybe some of us will see differences in Asada's favor that you didn't notice.

Maybe we can speculate about advantages she may have had in those areas that aren't as obvious on video as in person.

Maybe you can explain in detail why you thought Hongo's program was better.

We can each agree that we might miss something or prioritize some criteria differently than each other or than the judges.

Maybe some of the judges thought Hongo was better in those areas but the way the averaging worked out Asada came out ahead.

Maybe some judges did miss some important qualities that we can see because we're better at analyzing musical interpretation and all the Choreography criteria that do come across on video than they are.

We can still respect the process while disagreeing with specific results.

Or we can just assume that our own assessments are the only right ones and anyone who disagrees with us, official or unofficial, must be wrong. But if fans disagree with each other about specific programs, does that mean we have to question each other's honesty?

I think I made my point clear, :dev2: Nobody backs up his claims, but complains about my view - sorry, but where are the discussions here that we want by the OP? This is exactly my problem and most likely by the OP too. If some people analyze stuff, we get called out for it, but those who do not analyze anything, are always right and supported. :scratch2:

Mao skates beautifully and makes it look very calm and effortless, she has deeper edges, better one foot skating ... this for instance, justified her SS advantage. But, giving her 0.50 or 0.60 more at INT and CH, is not right. Again I explained why. Hongo´s elements align perfectly to the music and the different phrases of it, look at her three jump combination, you can not place it better. She is one with the music, these smiles when she goes into the Irish dance steps, you see the joy of the music right in her eyes and you clearly see that what she is skating to, was there right from the start and not something made up on the ice, exactly as it has to be based on ISU regulations. Her program has a steady progress, soft at the beginning, powerful and full of highlights towards the end.

Look at Rostelcom, the program there was nowhere near that. She looked tired, her elements were off beat and rhythm, therefore rightfully low PC´s.

Mao has a beautiful way of skating to music, there is no doubt about it, INT has to be high, her facial expression in the LP was ace and she portrait that Japanese woman on the ice greatly - but again, why the big gap? Another aspect is the choreography, Asada made several errors and you saw it and as soon Adelina does them, we have the haters come out and talk bs about her.

Where are those people now? Why is it always the Russian skaters who are pointed at? If Adelina would had scored so high with a faulty program, we would have had the same chatter again like at Rostelcom, but Mao is completely safe of that, not fair.

Really, the construction of the program suffered, it was not that well aligned anymore. Compare her LP with the SP and you will see two completely different skates, one is amazingly well performed, the other is clearly not perfect, yet gained these outstanding scores, while the perfect Hongo or Mao´s SP - was so bad, that both performances had to be scored as low as they were?

Truth is, Asada did profit from the late starting position at the LP and her name, just like she suffered by the early one at the SP. Hongo suffered in the LP by skating against Mao, the champion - who needs to make a comeback as she is needed for this sport. This is not a secret, it is well know that the big names are pushed to get the headlines, look at Hanyu.

Honestly, I love Asada and I always felt she was unjustified only the second behind Kim, but this discussion here is not going anywhere, if nobody of you finally contributes to what the OP wants.

Yes I am salty, as I did expect more from this thread, but it is again the same old story, that we had at Rostelcom, fans support their skaters, haters complain and it is unfair to the OP to start this now again.
 

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
I think her SP PCS was actually better than her LP PCS even though the LP was a much better performance. However, she skated last in the SP and very early in the LP in Sochi. I don't know how else to explain such a discrepancy in her own PCS scores in that single event other than skate order. I don't think it's automatic, but I just think the judges mentally "anchor" PCS scores relative to skaters they have just seen.
Again. This is the type of unsubstantiated claim I am talking about.

Why, in terms of PCS criteria, do you think the LP was "a much better performance"?

(The only "justification" you gave was: "I think her SP PCS was actually better than her LP PCS even though the LP was a much better performance")

I am not saying you are right or wrong, just that you have not provided anything to back up what you say. Once again, my view is, you need to make a case in terms of the actual PCS criteria that the judges are supposed to be using, vs. how that matches up (or doesn't) with the specifics of the performances, video footage, and how others at the same competition were scored.

But worse than that - you simply assume you are correct, and then go on to create generalised explanations for the unsubstantiated "discrepency" as you see it...

ie. that it is "quite common" for scores to be decided/influenced by skate order.

Lots of people do this. They will also invoke other generalised "explanations" such as politics, federation influence, location, reputation, conspiracy etc etc. And pretty soon everybody believes these that things are true, that they happen routinely. Even though they have not been proven or justified - they have been created with no solid foundation.

Once again, I'm not saying you are right or wrong about the particular case you mentioned... but before creating "explanations", we need proper evidence that there is something that needs to be explained!

Simply saying "I think" performance X was "better" than performance Y... is not enough!
 
Last edited:

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
AlexD, I liked your last post, you made some good points, and you actually made a case for what might be a scoring discrepency...

I don't think gkelly was refuting your claim - I took it as just pointing out that PCS is difficult and subjective by its nature, and that it's possible there are aspects you didn't see or consider - and bringing others into a discussion/analysis of the matter could be helpful/productive?

Especially since we don't necessarily know how the judges see things. I've mentioned before that listening to Yukiko Okabe's commentary was sometimes surprising! eg. she once pointed out using video footage, exactly why a seemingly great (and crowd-pleasing!) performance by Daisuke Takahashi did not earn the PCS one might have expected... and conversely what aspects of some other competitors' skating deserved good PCS even though their performances did not seem very "engaging"...
 
Last edited:

gkelly

Record Breaker
Joined
Jul 26, 2003
AlexD, I liked your last post, you made some good points, and you actually made a case for what might be a scoring discrepency...

I don't think gkelly was refuting your claim - I took it as just pointing out that PCS is difficult and subjective by its nature, and that it's possible there are aspects you didn't see or consider - and bringing others into a discussion/analysis of the matter could be helpful/productive?

Exactly.

For myself, I'm less interested in analyzing why some fans are more forgiving to some skaters than others. That's going to vary by which fans you happen to be discussing with, anyway. And we'll each probably be more senstitive to criticisms of our own favorites.

I'm more interested in analyzing the actual skating and why it might have been scored the way it was. Sometimes I can't end up justifying the panel's scores and I just have to chalk it up to judges seeing something I didn't see (more likely) or missing something I did. I do think that skate order and expectations do have some unconscious effect on judges, but I don't think there's anything nefarious about that, it's just the price you pay for relying on human beings to evaluate qualities that by their nature require human evaluation. And I think those effects also play an unconscious part in fan evaluations, as well as fan desires to root for underdogs.

AlexD, you give some good reasons why you think Hongo deserved high CH and IN scores for that event, and a few reasons why you think Asada's performance there deserved not so high scores for those components. I'd love to get some discussion going with analysis from multiple viewers, so we can consider from multiple points of view rather than just jumping to the conclusion that the judges did a bad job, intentionally or otherwise.

I'm sorry I didn't watch and mock-score those programs at the time, so any analysis I give now will be less unbiased than if I'd tried it unspoiled.
 

drivingmissdaisy

Record Breaker
Joined
Feb 17, 2010
Again. This is the type of unsubstantiated claim I am talking about.

Why, in terms of PCS criteria, do you think the LP was "a much better performance"?

(The only "justification" you gave was: "I think her SP PCS was actually better than her LP PCS even though the LP was a much better performance")

Simply saying "I think" performance X was "better" than performance Y... is not enough!

True. But there was a disparity in the marks for the same skater in the same competition. The PCS would indicate that the SP with 3 major errors was skated better than the clean LP. At least we would expect the clean performance to match the performance with 3 major errors, wouldn't we? On what basis do you feel like the LP PCS should be lower? Either one has to be better than the other or both the same, but the panels judged the LP PCS to be lower.
 

cl2

Final Flight
Joined
Nov 9, 2014
AlexD, I liked your last post, you made some good points, and you actually made a case for what might be a scoring discrepency...

I don't think gkelly was refuting your claim - I took it as just pointing out that PCS is difficult and subjective by its nature, and that it's possible there are aspects you didn't see or consider - and bringing others into a discussion/analysis of the matter could be helpful/productive?

Especially since we don't necessarily know how the judges see things. I've mentioned before that listening to Yukiko Okabe's commentary was sometimes surprising! eg. she once pointed out using video footage, exactly why a seemingly great (and crowd-pleasing!) performance by Daisuke Takahashi did not earn the PCS one might have expected... and conversely what aspects of some other competitors' skating deserved good PCS even though their performances did not seem very "engaging"...

Do you have a link to said Okabe's commentary? I'm not familiar with her commentaries, and would be very interested to listen to her (vastly more expert than mine) analysis.
 

yuki

Final Flight
Joined
Mar 2, 2012
True. But there was a disparity in the marks for the same skater in the same competition. The PCS would indicate that the SP with 3 major errors was skated better than the clean LP. At least we would expect the clean performance to match the performance with 3 major errors, wouldn't we? On what basis do you feel like the LP PCS should be lower? Either one has to be better than the other or both the same, but the panels judged the LP PCS to be lower.

Mao's LP PCS in Sochi was higher (although not by much) than her SP PCS on all 5 categories.
 

rain

Record Breaker
Joined
Jul 29, 2003
Personally, I blame the IJS. Under ordinal judging I was perfectly content to say, I thought this performance was better than that, Sometimes the judges agreed, sometimes they didn't, and sometimes there was a split panel with some judges agreeing with my opinion and others having a different view.

Regardless, i could always list what I thought were cogent reasons for my opinion, while also listening to what others thought. The viewer felt engaged and invested.

Now…well, we can't argue with the computer that adds up the scores, so all we can do is say meaningless things like, "I thought she should have gotten 8,75 instead of 8.25 in choreography, or "I didn't think he satisfied the bullet point for smooth knee action," or "82.3 for that!? I wouldn't give it a tenth more than 80.4."

Yes, and this is why I'm not a fan of the whole notion of the original post in the first place. I think we can, very legitimately, still say "I thought that performance was better than the other" and it is just as valid oftentimes as a complicated post full of numbers that a poster has arbitrarily assigned to a skater — and far more interesting and easier to read.

And I think it is often obvious to the naked eye without quantifying it to the decimal point when somebody is over or under-scored. Other times there are more intricate arguments to be made.
 

cl2

Final Flight
Joined
Nov 9, 2014
I think I made my point clear, :dev2: Nobody backs up his claims, but complains about my view - sorry, but where are the discussions here that we want by the OP? This is exactly my problem and most likely by the OP too. If some people analyze stuff, we get called out for it, but those who do not analyze anything, are always right and supported. :scratch2:

Mao skates beautifully and makes it look very calm and effortless, she has deeper edges, better one foot skating ... this for instance, justified her SS advantage. But, giving her 0.50 or 0.60 more at INT and CH, is not right. Again I explained why. Hongo´s elements align perfectly to the music and the different phrases of it, look at her three jump combination, you can not place it better. She is one with the music, these smiles when she goes into the Irish dance steps, you see the joy of the music right in her eyes and you clearly see that what she is skating to, was there right from the start and not something made up on the ice, exactly as it has to be based on ISU regulations. Her program has a steady progress, soft at the beginning, powerful and full of highlights towards the end.

Look at Rostelcom, the program there was nowhere near that. She looked tired, her elements were off beat and rhythm, therefore rightfully low PC´s.

I agree with gkelly's suggestion for us to all field our perspectives in as precise a manner as possible, to enable open discussion.

So I'll begin by write down a few thoughts regarding INT of Rika's Riverdance program. Can't spend my whole Sunday on FS analysis, so I'll focus only on a narrow aspect: what I perceive to be lacking in her INT which might be costing her some points. This is an opinionized analysis based on the information I can access (namely, youtube videos), and based on my personal layperson's aesthetic ideal of Riverdance. Note, I'm not currently trying to analyze any other aspect of her program, nor comment on the fairness of her PCS, nor compare her with her competitors. And, I will welcome your comments or critique of my analysis.

So here goes...

Rika's performance (in CoC) was certainly feisty and in tempo. Particularly in the fast, rhythmic portions in the second half of the program, her movements were well accentuated to the music, with good feel for the rhythm. However, I'd like to see more refinement and clarity with the composition of the shapes/configurations of her body positions. For example, her toe tipetty-tap steps (what are those called?) are a bit loose and needs to have clearer leg/knee positions. In some of her hands-on-hips positions, she does not actually have her hands on her hips, but simply near her hip area with her elbows and shoulders pushed forward, leading to wobbling and poorly defined angular shape intended by the bent elbow. (From watching the Riverdance, the aesthetic of tap dance seems to be to keep the knees close together and upper body dead still.) In several of her sharp, brisk movements, and even on her crossovers, the wobbling of her extended arms/legs detracts from the sharpness of the movement and the visual clarity of the intended shape. Perhaps brisker arm motion to attain the intended position will allow an added split second to visually define the intended shape. (As an aside, I think that if the music were to be of a lyrical nature, the same "wobbling" movements that's not good for sharp choreography would look more like "breathing with the music", which is good for flow-y choreo.)

So, Rika's interpretation is overall very good, hitting the major points in timing, rhythm and energy, but just short on the finer details of shape and spatial body positions that would make it perfect.
 

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
Do you have a link to said Okabe's commentary? I'm not familiar with her commentaries, and would be very interested to listen to her (vastly more expert than mine) analysis.
Unfortunately, no. The times we have seen her commenting on performances, they were not online videos - we have a Slingbox in Japan connected to a TV/DVR, which allows us record/watch Japanese TV from here in the UK.

The DVR automatically records all winter/water sports programs (the category that figure skating falls under)... and there is a constant stream of those filling up the DVR hard disk... so we don't even keep recordings after we watched them - we have to keep clearing them out to make room for new ones...
 

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
I think we can, very legitimately, still say "I thought that performance was better than the other"
Yes, you can say that, but it will be indistinguishable from posts by the ubers, the haters, and the ignorant.... who will post according to their agendas rather than any "real" justification... or selectively pick out certain aspects that are "better" while ignoring (or unaware) of other aspects that are not better and the affect scoring... etc etc.

Certainly, "I thought that performance was better than the other" cannot be used to hold judges accountable, or show there's something amiss with the scoring. It has to be in the terms that the judges use, or it isn't at all "legitimate" in that context.

and it is just as valid oftentimes as a complicated post full of numbers that a poster has arbitrarily assigned to a skater — and far more interesting and easier to read.
"I thought that performance was better than the other" might indeed be as valid as a "complicated post full of numbers that a poster has arbitrarily assigned to a skater"... but only in that NEITHER would be valid at all. They'd both be as meaningless as each other.

The numbers in a properly-made case for under/over scoring would NOT be "arbitrary" or made up. They'd come from the competition protocols, and from the criteria that judges are supposed to use. Those are not arbitrary, and they are necessary to make a reasoned argument.
 
Last edited:

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
Personally, I blame the IJS. Under ordinal judging I was perfectly content to say, I thought this performance was better than that, Sometimes the judges agreed, sometimes they didn't, and sometimes there was a split panel with some judges agreeing with my opinion and others having a different view.
It is not helpful to wish we were back in the 6.0 era. Because we are NOT back in the 6.0 era!

That is not how scoring is done any more, and we can't use 6.0 judging to hold IJS judges to account.

Now…well, we can't argue with the computer that adds up the scores, so all we can do is say meaningless things like, "I thought she should have gotten 8,75 instead of 8.25 in choreography, or "I didn't think he satisfied the bullet point for smooth knee action," or "82.3 for that!? I wouldn't give it a tenth more than 80.4."
You think it is meaningless to refer to the actual bullet points the judges are supposed to use, for assigning GoE etc?

Perhaps it would be meaningless, if people behaved like the straw men you invented. What was that anyway? An attempt to discourage or discredit properly-made arguments that refer to the actual bullet points and criteria that the judges are supposed to use? Some kind of subversion in the hopes that it will trigger a return to 6.0 scoring?! For someone calling themselves "Mathman", I find myself frequently surprised that you post so many illogical ideas and "non sequiturs"...

Whatever, the meaningless things in your straw man examples are NOT "all we can do".

We can say "I thought she should have gotten 8,75 instead of 8.25 in choreography..." without it being meaningless... if we point out how well she met the PCS criteria vs other skaters in the same competition, eg. if another skater scored the same for CH, without meeting the criteria so well etc. (see AlexD and Cl2 posts above)

And we can list the bullet points we thought they met (or didn't meet), to see if eg. the GoE for a given element seems justified or not.
 
Last edited:

rain

Record Breaker
Joined
Jul 29, 2003
The numbers in a properly-made case for under/over scoring would NOT be "arbitrary" or made up. They'd come from the competition protocols, and from the criteria that judges are supposed to use. Those are not arbitrary, and they are necessary to make a reasoned argument.

I don't think I've ever seen a poster who didn't pick and choose which bullet points to use/ignore (and often weigh what they think the relative importance should be) when trying to make their point when they start to make up their own scoresheets on the skaters — because they're always trying to justify their own feeling/opinions on the matter. Sometimes the arguments are more legitimate than at other times, but what it comes down to, always, is whether your opinion coincided with the judges' opinion(s) or not, whether you litter your argument with decimal points or not.

Look, I just don't think anyone here should be telling other posters that unless they're willing to sit down and write up their own protocol spreadsheets that they should sit down and shut up because they don't have anything legitimate to say or valuable to add to the conversation. I find that kind of exclusivity unpalatable.
 

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
Look, I just don't think anyone here should be telling other posters that unless they're willing to sit down and write up their own protocol spreadsheets that they should sit down and shut up because they don't have anything legitimate to say or valuable to add to the conversation. I find that kind of exclusivity unpalatable.
Its not about shutting people up, and I do understand that some performances are visibly "better" than others in some ways, with no need to look at protocols etc. And I have no problem with people saying so.

But once people start claiming actual overscoring or underscoring... and making implications with such claims (incompetance, systemic unfairness, conspiracy, cheating etc)... without proper justification... then I have a problem.

Then I think it's bad for the sport, and leads to all sorts of online unpleasantness... and the cases where complaints really COULD be justified... are buried, indistinguishable from the "noise" because nobody makes reasoned arguments, in the proper terms.

Instead of rallying people, or triggering action by or against the ISU... the ISU gets to write off all complaints as just a few more drops in the ocean of fan-wars, ignorance etc.

Speaking of the ISU, I don't think things would be so bad if they would be more "transparent" about scoring - hold reviews or at least periodic case-studies of scoring, and make them publically available. The "closed" and secretive nature of ISU judging makes people understandably suspicious (especially given past scandals)... but also makes it doubly important to use properly reasoned arguments, if we are to detect any mistakes or wrong-doing regarding scoring...
 
Last edited:

rain

Record Breaker
Joined
Jul 29, 2003
Its not about shutting people up, and I do understand that some performances are visibly "better" than others in some ways, with no need to look at protocols etc. And I have no problem with people saying so.

But once people start claiming actual overscoring or underscoring... and making implications with such claims (incompetance, systemic unfairness, conspiracy, cheating etc)... without proper justification... then I have a problem.

Then I think it's bad for the sport, and leads to all sorts of online unpleasantness... and the cases where complaints really COULD be justified... are buried, indistinguishable from the "noise" because nobody makes reasoned arguments, in the proper terms.

Instead of rallying people, or triggering action by or against the ISU... the ISU gets to write off all complaints as just a few more drops in the ocean of fan-wars, ignorance etc.

Well, I don't believe for a moment that the ISU cares at all what anybody on this message board says. I do think unfounded accusations of corruption are a problem, but truly, you're just not going to change some people's minds, no matter how reasoned your argument.

I just think that this board is for skating fans of all levels — those who have scoured the ISU rulebook and those who haven't, equally. Trying to make some posters feel as if their opinions are inferior because they can't, or don't want to, quote specific bullet points and GEOs every time they want to state an opinion is unfair, as is calling their opinions mere noise.

I'd argue that posts full of numbers dissected to death, particularly when they're being used to try to crush other posters' opinions, can be just as unpleasant as posts that don't contain a lot of justification for an opinion, just in a longer-winded way.
 

YesWay

四年もかけて&#
Record Breaker
Joined
Sep 28, 2013
I just think that this board is for skating fans of all levels — those who have scoured the ISU rulebook and those who haven't, equally. Trying to make some posters feel as if their opinions are inferior because they can't, or don't want to, quote specific bullet points and GEOs every time they want to state an opinion is unfair, as is calling their opinions mere noise.
With respect - if someone claims that scoring has been incorrect... but they can't justify that claim, in the same terms used by the judges to assign those scores... then their opinion is indeed, by definition, "inferior" to a properly reasoned claim that is made in those terms.

And I am of the view that they really shouldn't make such claims, if they can't actually back them up. And certainly not in the emphatic and vehement ways that we see in many cases!

I'd argue that posts full of numbers dissected to death, particularly when they're being used to try to crush other posters' opinions, can be just as unpleasant as posts that don't contain a lot of justification for an opinion, just in a longer-winded way.
We can never stop fan wars... but at least posts with numbers have a chance of being corroborated or refuted with logical arguments. Unlike "the other sort" of arguments...!
 
Last edited:

cl2

Final Flight
Joined
Nov 9, 2014
I sense a tension between people who wish to be able to express their opinions without further explanation, and people who wish to see an academic analysis of skating. No one is preventing the former group of people from expressing their opinions in other parts of the forum. I know that building supporting arguments is hard and time consuming, and not everyone wants to do that. But I think the OP wanted this thread to be talking about the latter, so expressing unsupported claims in this thread is off-topic.

I personally want to see some level-headed analysis of FS too, and was hoping this thread will be a good avenue for that. But if this thread devolves into a quibble about whether someone is allowed to air their opinion or not, then I'm leaving.
 
Top