Judging math details

QuadThrow said:

No definitly not. That would mean the judges know the exact difference between a 7.75 and a 7.50. But noone knows. ...

That means the system is too sensitive and judges mark skaters more accurately than a human is probable able to..

Click to expand...

I think that the points made by gkelly in post #39 are quite vaild, however (9.75 ). Phycholgists tell us that the human brain is capable of distinguishing about 7 different comparative categories. The IJS asks judges to distinguish among 41, from 0.0 to 10.0. This is obviously impossible.

But if we begin with the observation that, based on a judge's experience with judging hundred of contests, this performance was somewhere around the 7s or 8s, well, now there are only eight gradations, from 7.0 to 8.75 to distinguish among. This is about the right number that a judge can handle.

Also, although we can consistently distinguish only about 7 or 8 pre-determined categories, when we directly compare one against another we can always determine that this one is better than that one, even if the difference is very small. (This is a virtue of ordinal judging.) If we give a 7 to all of the skaters, it is useful to be able to say that even though they are all 7s, this one is a little more more 7 than that one.

gkelly said:

Remember that the 10.0 scale covers all of skating, where 0 represents not doing anything at all ...

Click to expand...

I remember one time a dance couple had to withdraw before an event because of being injured in practice.. When it was their turn they skated onto the ice, bowed to the audience, and left. One judge gave them a positive score, 0.25, in presentation.

moriel said:

Imho actually better control. Because, in case of 7 vs 8, it is a more objective difference. It is easier to evaluate. It is easier to distinguish.
When you give people too many gradations, it doesnt make it easier, it makes it harder.

Click to expand...

This judging system is extremely ambitious and overestimates the ability of human beings. Oh and the judges better make sure to stay on the ISU time schedule

There's a disconnect between mathematical precision and judging a figure skating--something which needs to cover many elements and put out a product in minutes. I thought 6.0 was more transparent about that, but flawed in many other ways. COP has helped in many ways, but the picture that it paints of comfort in numbers is illusory.

Any possible judging system that humans could devise to capture all possible aspects of what makes one performance "better" than another is going to be flawed.

The question is, which system has fewer practical limitations, which has fewer theoretical flaws, which aspects are so important we need them to be as flawless as possible and which flaws are we willing to live with?

Once a system is in place, it's useful to identify its drawbacks and see if we can think of ways to overcome them without losing its advantages.

I don't think it's very useful to focus on one or two specific flaws and discount the whole system on that basis. If you want to get rid of the whole system as it currently exists, you would need to replace it with another comprehensive whole system. Which is bound to have some flaws of its own.

gkelly said:

Any possible judging system that humans could devise to capture all possible aspects of what makes one performance "better" than another is going to be flawed.

The question is, which system has fewer practical limitations, which has fewer theoretical flaws, which aspects are so important we need them to be as flawless as possible and which flaws are we willing to live with?

Once a system is in place, it's useful to identify its drawbacks and see if we can think of ways to overcome them without losing its advantages.

I don't think it's very useful to focus on one or two specific flaws and discount the whole system on that basis. If you want to get rid of the whole system as it currently exists, you would need to replace it with another comprehensive whole system. Which is bound to have some flaws of its own.

Click to expand...

Definitely. I think the general thrust of my comments at least has been that COP should be simplified somewhat so that judges can wrap their head around their jobs more easily. That or split some of the duties to make it more manageable. Or give the judges more time--which will never happen because of TV.

I think in general all viewers should be more aware of the limits of any judging system.

Mathman said:

I think that the points made by gkelly in post #39 are quite vaild, however (9.75 ). Phycholgists tell us that the human brain is capable of distinguishing about 7 different comparative categories. The IJS asks judges to distinguish among 41, from 0.0 to 10.0. This is obviously impossible.

But if we begin with the observation that, based on a judge's experience with judging hundred of contests, this performance was somewhere around the 7s or 8s, well, now there are only eight gradations, from 7.0 to 8.75 to distinguish among. This is about the right number that a judge can handle.

Click to expand...

I remember some judge was explaining and giving advices for judging and the process is very simmilar with that. What make their kognitive job easier is - 1) they dont need to compare every part of one skating with another, so its easier to concentrate to every single skating 2) some desicions are almost automatic, cause they had it so many times in their expirience 3) they are more than 10 judges who decide, so if some miss something, someone else will see it 4) they are using (or should use) technology to make some desicions... Mistakes are always possible of course, and for example in football where judges need to concenrate to a much less things, or even in track and field where they are concentrating on just one, some desicions could be very problematic and wrong. But some of those mistakes in judging are just human, and it would be not normal if they dont happen from time to time.

GGFan said:

This judging system is extremely ambitious and overestimates the ability of human beings. Oh and the judges better make sure to stay on the ISU time schedule

There's a disconnect between mathematical precision and judging a figure skating--something which needs to cover many elements and put out a product in minutes. I thought 6.0 was more transparent about that, but flawed in many other ways. COP has helped in many ways, but the picture that it paints of comfort in numbers is illusory.

Click to expand...

I´m all for simplifying the work for judges in some thing, but making them more accountable for other things.
For example, make PC scores integer (because GOE is, so why not), and require more details on the scores instead, such as checkboxes for GOE and so on.

moriel said:

For example, make PC scores integer (because GOE is, so why not)...

Click to expand...

In 6.0 judging the presentation score (also the technical score) was graduated in tenths, from 0.0 to 6.0 -- 61 grades. So I guess having only 41 in the IJS is a step in the right direction. :yes:

Mathman said:

I think that the points made by gkelly in post #39 are quite vaild, however (9.75 ). Phycholgists tell us that the human brain is capable of distinguishing about 7 different comparative categories. The IJS asks judges to distinguish among 41, from 0.0 to 10.0. This is obviously impossible.

But if we begin with the observation that, based on a judge's experience with judging hundred of contests, this performance was somewhere around the 7s or 8s, well, now there are only eight gradations, from 7.0 to 8.75 to distinguish among. This is about the right number that a judge can handle.

Also, although we can consistently distinguish only about 7 or 8 pre-determined categories, when we directly compare one against another we can always determine that this one is better than that one, even if the difference is very small. (This is a virtue of ordinal judging.) If we give a 7 to all of the skaters, it is useful to be able to say that even though they are all 7s, this one is a little more more 7 than that one.

Click to expand...

This would imply that the current GOE system i.e. +3 to -3 is just about perfect - I have wondered about about the proposed +5 to -5 GOEs and how easy it will be to tell the difference between them, and this seems to explain it. However marking things out of 10 e.g. a film or a TV program, I would always want 0.5 differences e.g. 7.5 rather than 8.0 so I don't quite see how this fits in, but it's not too bad from a PCS perspective, even with 7.5 you can sometimes think it's actually a bit higher or maybe a bit lower so 0.25 differences are probably just about right.

Miller said:

However marking things out of 10 e.g. a film or a TV program, I would always want 0.5 differences e.g. 7.5 rather than 8.0 so I don't quite see how this fits in, but it's not too bad from a PCS perspective, even with 7.5 you can sometimes think it's actually a bit higher or maybe a bit lower so 0.25 differences are probably just about right.

Click to expand...

I think it's about right, too. The argument would be something like this. First there is a coarse division into categories like Outstanding, Very Good, Good, etc. This gives you a general range of available scores for a particular competition. For instance, for a Grand Prix event you pretty much know that almost all the skaters will be Good (in the 7s) or Very Good (in the 8s). So except for acknowledged superstars like Hanyu or Medvedeva, and except for unexpected meltdowns, you have maybe 8 or so possible scores to work with, 7.0 to 8.75, which is about right.

By the way, I used to wonder whether that was why figure skating was judged on a scale from 0 to 6 (7 possibilities), rather than from 1 to 10. I found out (on this board ) that I was all wet about that. The reason actually had to do with how school figures were scored.

Mathman said:

I think it's about right, too. The argument would be something like this. First there is a coarse division into categories like Outstanding, Very Good, Good, etc. This gives you a general range of available scores for a particular competition. For instance, for a Grand Prix event you pretty much know that almost all the skaters will be Good (in the 7s) or Very Good (in the 8s). So except for acknowledge superstars like Hanyu or Medvedeva, and except for unexpected meltdowns, you have maybe 8 or so possible scores to work with, 7.0 to 8.75, which is about right.

By the way, I used to wonder whether that was why figure skating was judged on a scale from 0 to 6 (7 possibilities), rather than from 1 to 10. I found out (on this board ) that I was all wet about that. The reason actually had to do with how school figures were scored.

Click to expand...

Actually, your description gives me an idea. They should change it to: the judge first selects a category on the Outstanding, Very Good, Good and so on scale, and then gives some sort of an additional score, kinda like GOE. Then the computer converts it all back to a 0-10 scale. Could be interesting, and could be displayed like that, making PCs more understandable for the casual viewers (because right now, when you show a 9.25, one has no clue, but then it could be written as very good + small bonus)

moriel said:

Actually, your description gives me an idea. They should change it to: the judge first selects a category on the Outstanding, Very Good, Good and so on scale, and then gives some sort of an additional score, kinda like GOE. Then the computer converts it all back to a 0-10 scale. Could be interesting, and could be displayed like that, making PCs more understandable for the casual viewers (because right now, when you show a 9.25, one has no clue, but then it could be written as very good + small bonus)

Click to expand...

For ease of notetaking, judges could just continue thinking in terms of the numbers. The screen where they select the component scores could include both the numbers and the verbal descriptions.

The numbers are necessary for the calculations as well.

Casual viewers aren't going to look at the protocols or even the summary of scores on the ISU site where the panel's averages for each component are displayed. It wouldn't hurt to include a key on that page for the

Casual viewers just see the total PCS for all five categories added together announced during the Kiss and Cry.


How would you propose to announce scores during the event?
Back at the very beginning of IJS the K&C announcements used to include each program component separately, but then they decided it took to long to announce and was slowing down the events too much. "Outstanding plus a small bonus" takes even longer to say aloud than "9.25." For one score it doesn't make much difference, but for 5 scores times 24 or 30 skaters it would add several minutes to the whole event.

Interested fans who want to know more but are still learning might benefit from more explanation during the event and certainly from pointers to where to find the detailed scores for this event and a permanent easy-to-find place to find explanations of what the numbers mean.

This is a much smaller population than casual fans, but it's a place where the ISU could encourage some casual viewers to become real fans. Make the detailed scores easy to find online, make an overview of the scoring system very easy to find online and very easy to understand, with clearly visible links to well-organized sources of detailed scoring information, and require broadcasters to announce where to find them every so often.

Print out the scoring overview documents and make them available at live events.

Use jumbotrons etc. at major live events to show short visual + verbal explanations of how the scoring works and what each score means, to be shown before the event begins/during resurfaces, mixed with more fun segments.

Should the ISU produce short explanatory videos in English, and also in Russian, French, Japanese, and Chinese?

Oh, i meant the format not to make it easy for judges, but to make it easy for people to watch.
There are many gradations in casual viewers, btw. There are ones that, for example, actually follow competitions, but find the scoring too complicated to bother with protocols and arent willing to spend their time watching explanatory videos and all. They just like watching FS or cheer for a few skaters.

At any rate, again. If scores require watching so many explanatory videos to understand, the scores are bad.

I really dont understand why people are so unfriendly to make things less complicated. Btw, if there was a PC classification such as a described, it could be announced on rink too, because it would not be so long, and it would actually add to people`s understanding.

I think the problem with PCS is that it's just one number that doesn't mean a lot. However with TES at least at the big events you get an idea of how it's mounting up plus if you were to follow the example of Japanese TV you could even show the individual items as they go through, plus at the end TES doesn't vary too much unless there's loads of URs in which case you'll often get the commentator explaining what's going on which in turn explains things to the more casual viewer.

The only way I could think of explaining PCS to the casual viewer would be to somehow divide into 3 i.e. the Rule of 3 which is always about how people can take in 3 things at once but not more - see politicians for example when give speeches, they do it all the time. Hence if you had PCS as 'Skating Skills'. 'Transitions' (which would need explaining) and 'Artistry' i.e. the last 3 components, then you might get somewhere, plus I always think people naturally understand things out of 10.

However to display these at the end would probably mean screen overload so how you might do it I don't know, but splitting it into 3 things might be a good idea, plus maybe even get the judges to concentrate on the last 3 components more rather than always giving them in a certain plus or minus range. It's always annoying IMO when an 'artistic' skater doesn't get rewarded in the last 3 categories, but someone who rather skates through the music does, but gets high PCS because of their skating skills and transitions.

Miller said:

The only way I could think of explaining PCS to the casual viewer would be to somehow divide into 3 i.e. the Rule of 3 which is always about how people can take in 3 things at once but not more - see politicians for example when give speeches, they do it all the time. Hence if you had PCS as 'Skating Skills'. 'Transitions' (which would need explaining) and 'Artistry' i.e. the last 3 components, then you might get somewhere, plus I always think people naturally understand things out of 10.

Click to expand...

This is a good idea, in my opinion. The "Artistry" component would be easy to understand, plus it would make violinists, ballet dancers, sculptors, etc., mad because they do not regard figure skating as having any "artisti." merit -- not like playing the violin, dancing in a ballet or creating a statue.

As for Transitions, I think that this is not hard to understand. It means what the skaters do in between the jumps and spins. Some skaters do a lot of cool stuff, others don't do much at all. To me, "skating skills" is much harder to explain. Isn't everything they do an exhibition of skating skills?

Mathman said:

Thids is a good idea, in my opinion. The "Artistry" component would be easy to understand, plus it would make violinists, ballet dancers, sculptors, etc., mad because they do not regard figure skating as having any "artistic." merit -- not like playing the violin, dancing in a ballet or creating a statue.

Click to expand...

We want these three components together to equal about 1/3 of the total score? That is currently the case (except for those guys doing quads with exceptionally high base values and also good GOEs)

As for Transitions, I think that this is not hard to understand. It means what the skaters do in between the jumps and spins. Some skaters do a lot of cool stuff, others don't do much at all. To me, "skating skills" is much harder to explain. Isn't everything they do an exhibition of skating skills?

Click to expand...

To educate casual viewers:

Skating Skills is everything the skaters do with the edges of their blades on the ice.

You could list the current Skating Skill criteria to clarify what is meant, in abbreviated form or with full explanations.

It would help if there were video examples of the kinds of things that judges look for (and listen for) under this component, in several short snippets that networks could play during resurfaces and warmups of live events, going in and out of commercials, etc., and that arenas could play on the jumbotron at such times. It's not something that can be summed up once in a few seconds in a way that nonskaters will instantly get it, but if broadcasts spent as much time pointing out skating qualities as they do pointing out which jumps skaters do and where/why they run into trouble on the jumps, viewers would quickly get a sense of what this component refers to.

Much of what is being judged under this component is much more obvious live, even to an untrained observer, than it is on video, and more obvious up close than from the upper mezzanine of a large arena.

We can't get all casual viewers right up next to the ice at elite events. But it would be possible to make closeup videos, with blade sounds amplified and no music to cover them up, as educational tools to give the best possible sense on video of what the basis of the sport of figure skating is all about.

The scoring system alone cannot do that. It's a matter of giving audiences the benefit of the doubt that they will understand if they're shown what to look for.

I think the problem of why so few nonskaters really understand what skating is about is that TV commentary focuses on the things that are easy to see (jumps and disruptive jump errors, spin and spiral positions, artistry including facial expression which is really peripheral) and rarely makes any attempt to address the more important fundamental skills.

If commentators say something like "The judges are looking at what the skaters are doing with their feet" but don't show viewers what that involves, the implication is that those things shouldn't matter to the viewers. But since they do matter to the skaters and to the results, it would be more respectful to viewers to invite them into the knowledge base rather than setting up a dichotomy between what those mysterious judges care about vs. what audiences care about.

That's really a network commentary issue. But the ISU could help by producing instructional videos geared toward fans that networks and local organizing committees could use.

gkelly said:

We want these three components together to equal about 1/3 of the total score?

Click to expand...

Actually, I would consider having only two PCS. SS+TR together (we could call the combined component "Skating Skill and Transitions"). And the other three together, called "Performance." Each of the two combined categories would be worth 25% of the total.

We would hope that the commentators would try harder to drive home that the distinction between element scores and program scores is that the former gives points for each technical element and the latter gives points for the program as a whole.

I would not be optimistic that the casual channel-flipper would care much about all the things that the judges are looking for under Skating Skills. I think if we mention just the bare minimum -- speed, effortless power, confident glide -- and transitions are things like Ina Bauers and steps before a jump -- that would be enough. The audience is probably not really interested in "depth of edge" and "rhythmic knee action," and "she just did a three-turn," nor are they able to notice these things even when pointed out by the commentators.

As for the combined "Performance" component I think that the majority of the audience is at the level of "Aw, ain't that purdy"; I know I am.

Yes, most people who watch skating on TV just want to enjoy the performances without worrying much about the details.

TV networks cater toward that majority of casual viewer and steer away from technical explanations for the most part.


The ISU and member federations make rules to serve their immediate constituents, who are the skaters, the officials, the federations, people who have spent hours and years of their lives in rinks. People who understand care about the technical details.

Lower-level skaters, and parents, are part of the latter group, many still learning how it all works. If they compete in events that use IJS, they soon learn how it works, but there is a learning curve.

Skaters at levels that don't use IJS, or former skaters who are not actively involved may not understand the details of the new-ish scoring system very well, but they do know more about skating technique than nonskaters. And if they're interested in following the current elite sport as fans, they have a head start in understanding the rules.

Fans who watch on TV or online and want to know more, and maybe sometimes buy tickets to elite events but never skated themselves, often come into the process with expectations shaped by the TV commentary more than of the reality of the sport as it's actually practiced. There aren't really any official means designed to educate enthusiastic fans about what's really being judged. Unofficial outlets such as Golden Skate where skater fans and non-skater fans discuss together are probably the best place to learn.

I do think that the ISU would do well to put together materials to invite enthusiastic fans into the knowledge base.

But I don't think it's the job of the scoring system to take on that task.

Mathman said:

Actually, I would consider having only two PCS. SS+TR together (we could call the combined component "Skating Skill and Transitions"). And the other three together, called "Performance." Each of the two combined categories would be worth 25% of the total.

We would hope that the commentators would try harder to drive home that the distinction between element scores and program scores is that the former gives points for each technical element and the latter gives points for the program as a whole.

I would not be optimistic that the casual channel-flipper would care much about all the things that the judges are looking for under Skating Skills. I think if we mention just the bare minimum -- speed, effortless power, confident glide -- and transitions are things like Ina Bauers and steps before a jump -- that would be enough. The audience is probably not really interested in "depth of edge" and "rhythmic knee action," and "she just did a three-turn," nor are they able to notice these things even when pointed out by the commentators.

As for the combined "Performance" component I think that the majority of the audience is at the level of "Aw, ain't that purdy"; I know I am.

Click to expand...

I would agree with this, or certainly the weighting part of it. Are transitions a bit too different to be regarded as in the same category as skating skills, or would the judges be fine this? Over to those who have actually done this. However if you followed the Rule of 3 as above you could have a 25/25/50 split and get exactly the same result, it's just what you think is better from a presentational point of view, 2 things, or 3 with the commentators saying the Artistic/Interpretation/Presentational component is half the total, or double that of the others, whatever might be best for a more non-mathematical audience.

Finally you've the subject of upping the PCS part vs TES. I guess that's a subject for another day, but I would agree provided that there's enough notice for skaters who have concentrated, quite understandably, on the TES side of things not losing out overnight to those that have not.

I must say i dont agree with some of you on PCS side and i dont think components should be clustered. First, by reading the rules and waching the skaters i see transitions component include variety/creativity and some skaters can express that as their own original and differential factor, which i consider as artistic concept (so i dont see the last 3 components as more or less 'artistic' than transitions). Second, i think last 3 components are not in the same league and can differ from skaters to skaters (not very much, but in 0.5 - 1 range, which i think is enough). How they are defined (and judged) i see those 3 components relate on different things - performance is more related to audience, composition to ice surface and interpretation to the music rythm... If some may do cluster analise of judges score im pretty sure results wouldnt be those 2 clusters most people think it be...

As a side note, keep in mind that, appearently, the judges DO NOT DIFFERENTIATE the different PCs categories, basically just giving some score for all fields, with minor variation.
So all you people who keep saying "omg, the PCs categories are a necessity, they are needed to differentiate the subtle aspects", the judges dont.