Hypothetical discussion : How would you reduce national bias in judging figure skating

[snowed]

AFAIK, you can measure the strength and angle with which the blade presses the ice, distribution of the skater's body weight along the blade as well as speed and direction of movements by using sensors.

What kind of sensors? Attached to the blade, or under the blade? I wouldn't step on the ice with something glued on my blades that could fall off. Or would the skaters have to buy the sensors themselves and mount them ahead of time?

IMHO, the biggest problem is an obvious reluctance to use technology on the part of ISU and skating officials. They might at least be up to date with the research which is taking place despite their visible lack of interest .

I am not ISU judge, and I am very reluctant to the possibility of using AI or sensors technology at this point. I'll be for sure open when it will develop in the future

for example, Japanese researchers have already developed technology to measure edges with the use of a smartphone camera and sensors attached to the skater's body, not the boots or blades, which is much less intrusive. And rather inexpensive, as they say.

I looked at the Japanese study, it looks like a school project to me. They didn't use sensors but an algorithm to calculate alignment of knees, hips, shoulders from the video in the video. Plus, if you look at the video they present, it is taken from a frontal view so those "points on the body" could be identify.

As for "wired" skaters, sensors are mostly a wireless technology, lol.
Besides, they would not be more "wired" than anyone who goes to a TV studio and I have not seen many people running away in horror from such a pain and humiliation when lured to appear on the screen., not many viewers turning their TV off in disgust at a show's "wired" hosts and guests (which they mostly do not see anyway). Of course, it is a different kind of "wired" but still..
As for distraction, I guess we should first let skaters try the technology and see what they say, whether they find it difficult to adapt or not at all, whether they would rather be "wired" and scored based on accurate objective measurements, or not "wired" and assessed based on what the panel sees or sees not on their screens...
And still, probably there are many improvements which could be achieved without "wiring" if some IT specialists expert in the state-of-the-art technology were employed.

Well, in the cement building like skating rings (or high rises) wireless signal is not reliable. I'm actually aware a wireless judging system was developed by a former skater or judge, on their own money, to rent to competition, and it just didn't work (the data didn't always transmit from the judges tablets to the server). A new state of the art skating rink in my area was built with wireless speakers... they don't work, there was a skating competition I went to watch and for almost every program the music stopped mid program. I have more examples...

At this point I would be happy if ISU would add one more camera, or use the commercial video that the spectators see along with their tech video

 

[Mathman]

The issue with that is that skating would become more and more similar. Just like what we see in the rhythm dance these days or in pairs short programs where the so many elements are so bunched up that everyone ends up doing the same thing.

Interestingly, that was the whole idea behind the short program: everyone would do the same thing (just klike figures) and then the judges would decide who did it best. In the free program the competitors were allowed more, well, freedom to showcase their various strengths.

 

[4everchan]

Interestingly, that was the whole idea behind the short program: everyone would do the same thing (just klike figures) and then the judges would decide who did it best. In the free program the competitors were allowed more, well, freedom to showcase their various strengths.

well... the reality now is that both programs look the same and both focus on jumps : what is different " style points "

 

[snowed]

I do not think the national bias is the biggest problem in judging, because the highest and lower marks are thrown away. Of course I agree the national bias should be monitored, as it seems it is done already to some extent.
The biggest problem I see, is the inconsistent "calls" for edges and underrotations, supposedly because of lack of a second video to review.
Plus I think ISU should make an effort in educating/ explaining the rules to the public, especially the components. I am putting a good effort in understanding and I cannot say I fully understand. I am also suspicious of corridor" judging, in the sense that judges award components scores based on previous achievements or high technical content. This could hide incompetence from judges, or we, as fans, may simply don't understand the complexity of judging.

 

[4everchan]

I do not think the national bias is the biggest problem in judging, because the highest and lower marks are thrown away. Of course I agree the national bias should be monitored, as it seems it is done already to some extent.
The biggest problem I see, is the inconsistent "calls" for edges and underrotations, supposedly because of lack of a second video to review.
Plus I think ISU should make an effort in educating/ explaining the rules to the public, especially the components. I am putting a good effort in understanding and I cannot say I fully understand. I am also suspicious of corridor" judging, in the sense that judges award components scores based on previous achievements or high technical content. This could hide incompetence from judges, or we, as fans, may simply don't understand the complexity of judging.

Let me remind you a story, not in figure skating but in artistic swimming.

Sylvie Fréchette, Barcelona, 1992. She was the reigning world champion and heavy favourite to win gold.
There was a new keypad system utilized for the first time. Judges were not used to it.
Back then, they had figures and then a solo.
Sylvie performed her figure. The Brazilian judge wanted to give her 9.7. However she entered 8.7 She tried to correct her mistake right away but didn't succeed. She told the referee... guess what, the referee was American and the main competitor for gold was... American...
The referee wouldn't hear it... The mark stood up...

Well well.. it should be okay right? Because the lowest mark gets thrown away right?
That's what I did hope. But nope... In a tight sport like artistic swimming and especially in the figures, Sylvie ended up in 4th place. Yes... in 4th place. The point is that yes, the lowest mark can get thrown away but in that case, what was going to be perhaps Sylvie's highest mark, became her lowest mark... and thus everything shifted significantly.

A protest was made... but guess who was handling protests ? The same referee... the American.

In the end, Sylvie won the solo segment and nearly got the gold... but lost... by the smallest margin thinkable...

After numerous appeals, the IOC awarded Sylvie with a gold medal. The American swimmer got to keep hers.

Here, the bias was probably made by the referee.
The scoring system was new. A judge made a mistake (let me see, probably a volunteer).
Even with the lowest mark removed, it changed the results.

I am telling you all this because it does matter when judges play favourites, even when their scores get tossed up.

I watched a recent video of Sylvie now at 56, who went to Barcelona. She suffered from PTSD and has NO MEMORY of Barcelona games. She went there to try reconquering some of her moment. The PTSD is not linked to the judging mistake. Days prior to her departure, she found her fiancé dead in their house. He had committed suicide.

I am not writing all this to say : look at the bad and mean American referee. I am writing all this to show that we cannot think lightly about marks getting tossed because in some events, the gold medal is decided by the tiniest of margin (for instance Ice dance in 2014 - Weaver and Poje lost out the gold by 0.02 points) One bad mark from a biased judge or one way too favourable mark from another may have made the difference here.


If you want to know more about Sylvie : it's in French but it's a wonderful movie.

Sylvie Fréchette – Retour à Barcelone | Sports | Radio-Canada.ca

« Je veux retrouver des pièces importantes de ma vie. » Une invitation l'a ramenée à Barcelone 31 ans après les Jeux olympiques dont elle n’a aucun souvenir.

ici.radio-canada.ca

 

[snowed]

Let me remind you a story, not in figure skating but in artistic swimming.

Sylvie Fréchette, Barcelona, 1992. She was the reigning world champion and heavy favourite to win gold.
There was a new keypad system utilized for the first time. Judges were not used to it.
Back then, they had figures and then a solo.
Sylvie performed her figure. The Brazilian judge wanted to give her 9.7. However she entered 8.7 She tried to correct her mistake right away but didn't succeed. She told the referee... guess what, the referee was American and the main competitor for gold was... American...
The referee wouldn't hear it... The mark stood up...

Well well.. it should be okay right? Because the lowest mark gets thrown away right?
That's what I did hope. But nope... In a tight sport like artistic swimming and especially in the figures, Sylvie ended up in 4th place. Yes... in 4th place. The point is that yes, the lowest mark can get thrown away but in that case, what was going to be perhaps Sylvie's highest mark, became her lowest mark... and thus everything shifted significantly.

A protest was made... but guess who was handling protests ? The same referee... the American.

In the end, Sylvie won the solo segment and nearly got the gold... but lost... by the smallest margin thinkable...

After numerous appeals, the IOC awarded Sylvie with a gold medal. The American swimmer got to keep hers.

Here, the bias was probably made by the referee.
The scoring system was new. A judge made a mistake (let me see, probably a volunteer).
Even with the lowest mark removed, it changed the results.

I am telling you all this because it does matter when judges play favourites, even when their scores get tossed up.

I watched a recent video of Sylvie now at 56, who went to Barcelona. She suffered from PTSD and has NO MEMORY of Barcelona games. She went there to try reconquering some of her moment. The PTSD is not linked to the judging mistake. Days prior to her departure, she found her fiancé dead in their house. He had committed suicide.

I am not writing all this to say : look at the bad and mean American referee. I am writing all this to show that we cannot think lightly about marks getting tossed because in some events, the gold medal is decided by the tiniest of margin (for instance Ice dance in 2014 - Weaver and Poje lost out the gold by 0.02 points) One bad mark from a biased judge or one way too favourable mark from another may have made the difference here.


If you want to know more about Sylvie : it's in French but it's a wonderful movie.

Sylvie Fréchette – Retour à Barcelone | Sports | Radio-Canada.ca

« Je veux retrouver des pièces importantes de ma vie. » Une invitation l'a ramenée à Barcelone 31 ans après les Jeux olympiques dont elle n’a aucun souvenir.

ici.radio-canada.ca

I don't understand how, if that mark was tossed, still influenced the result...
Also, in figure skating, as I understand, the electronic system doesn't accept changes after the marks are sent in, it is not up do the referee to change the mark. Of course a report can be made. Are you sure it is not the same, or was not the same, in the swimming case? Are you sure it was national bias from the American referee and not an electronic system (or just rules not allowing changes) problem?
I'm happy to hear she won in the end...

 

[Mathman]

Okay. So some people, like yourself, think it's all good...

Anything can be improved What I object to is the bundling or the two totally different concepts "bias" and "corruption."

Corruption is bad. Corruption implies evil intent. Throw the bums out! Wax indignant about corruption wherever you find it.

Bias means generally anything that prevents you from making accurate predictions about a population (for instance, the population of all possible marks that all possible judges would be expected to give to a figure skatimng performance) from a sample (what scores did this particular panel give, or this individual judge.) Gkelly's post #39 above illustrates the difference.

There are be good reasons why data-oriented people wish to tamp down bias. But, yeah, I do think that most figure skating judges fall into the category of good people rather than evil, or at least so-so on that spectrum..

 

[4everchan]

Anything can be improved What I object to is the bundling or the two totally different concepts "bias" and "corruption."

Corruption is bad. Corruption implies evil intent. Throw the bums out! Wax ibdignant about corruption wherever you find it.

Bias means generally anything that prevents you from making accurate predictions about a population (for instance, the population of all possible marks that all possible judges would be expected to give to a figure skatimng performance) from a sample (what scores did this particular panel give, or this individual judge.) Gkelly's post #39 above illustrates the difference.

There may be good reasons why data-oriented people would wish to tamp down bias. But, yeah, I do think that most figure skating judges fall into the category of good people rather than evil, or at least so-so on that spectrum..

This thread is about bias not corruption (I guess the solution to make pro ISU judges without association to federation could also reduce corruption, so fair enough)

I agree that most judges are probably rather good I also agree that all of us are biased one way or another. I know my biases and they are quite strong and obvious

However, the point of this discussion is how could we still have the sport we love with a lower impact of personal/cultural/nationalistic biases ?

 

[4everchan]

I don't understand how, if that mark was tossed, still influenced the result...
Also, in figure skating, as I understand, the electronic system doesn't accept changes after the marks are sent in, it is not up do the referee to change the mark. Of course a report can be made. Are you sure it is not the same, or was not the same, in the swimming case? Are you sure it was national bias from the American referee and not an electronic system (or just rules not allowing changes) problem?
I'm happy to hear she won in the end...

Simple example.

8.7, 9.0. 9. 1 9. 2 .9.5

the top and lowest marks are tossed... Result 27.3

9.0 9.1 9.2 9.5 9.7

tossing the outliers

Result 27.8 that's a whole .5 difference which is huge in many sports.

I was young in 1992 and the way I recall the event is that the Brazilian judge tried to talk right away to the referee to let her know she tried to correct her score but it didn't work... They did have an extensive discussion. It was an honest mistake from the judge who apologized to Sylvie later on. The referee wouldn't hear any of it. Based on that, the Canadian OC made a protest as well... the same referee was in charge of hearing the protest...

 

[Mathman]

This thread is about bias not corruption (I guess the solution to make pro ISU judges without association to federation could also reduce corruption, so fair enough).

Maybe so maybe so. I just did an experiment. I typed "corruption" into the GS search field. 10 pages. Then I tried" bias" Ten pages.

 

[4everchan]

But you increased all the mark on the second set...
8.7 9.2 9.5 9.3 9.6 and 9.7 9.2 9.5 9.3 9.6, would have 1st mark the 8.7 and 9.7 tossed both times for being lowest and highest, the rest of the marks are the same....
I mean there could be 8.7 9.2 9.5 9.3 9.8 and 9.7 9.2 9.5 9.3 9.8, and 9.8 would be tossed up for highest mark on both cases and then the 9.7 (the 9.2 would be tossed as the lowest) would make a difference in the second set

Maybe @Mathman can check the theory that in order to positively influence the score, a judge should try to have the second highest marks, not the highest mark... then they would not be tossed and positively affect the average. But if this is true, I would think the federations would have made this calculations already and their judge would not score so high. This would mean to me that judges that mark high don't do it to influence the marks and they don't do it on purpose.

i didn't at all. i used the example with the brazilian judge.

in the first set, i gave the erroneous 8.7

In the second set, I gave the 9.7 she wanted to give instead of the 8.7

all the other marks are the same.

Back then, if my memory doesn't fail me, that 9.7 would have been the highest mark... and it then became the set's lowest mark as an 8.7

 

[snowed]

i didn't at all. i used the example with the brazilian judge.

in the first set, i gave the erroneous 8.7

In the second set, I gave the 9.7 she wanted to give instead of the 8.7

all the other marks are the same.

Back then, if my memory doesn't fail me, that 9.7 would have been the highest mark... and it then became the set's lowest mark as an 8.7

I've just realized you were correct and deleted my post, I wasn't able to edit it...

 

[4everchan]

I've just realized you were correct and deleted my post, I wasn't able to edit it...

not a problem the brazilian judge wasn't able to edit her score either LOL all is good

 

[CanadianSkaterGuy]

I would have their associated country abbreviation next to their judge number directly on the protocols next to the scores they gave out. Ideally their name too.

Usually we have to dig/cross-reference with the Judges List to see who submitted what.

If it’s right on the protocols then it will be more obvious who is giving what scores and which may be cases of national bias, thus discouraging these judges from being obvious outliers.

Under 6.0 you could at least see the nationalities of each judge so if they favoured their own or a rival the commentators could point that out. With an aggregate score it hides impropriety and with a protocol not showing the nationalities of each judge it’s harder to discern national bias.

Although if you can tell the nationality based on the scores then national bias is probably evident.

 

[4everchan]

I would have their associated country abbreviation next to their judge number directly on the protocols next to the scores they gave out. Ideally their name too.

Usually we have to dig/cross-reference with the Judges List to see who submitted what.

If it’s right on the protocols then it will be more obvious who is giving what scores and which may be cases of national bias, thus discouraging these judges from being obvious outliers.

Under 6.0 you could at least see the nationalities of each judge so if they favoured their own or a rival the commentators could point that out. With an aggregate score it hides impropriety and with a protocol not showing the nationalities of each judge it’s harder to discern national bias.

Although if you can tell the nationality based on the scores then national bias is probably evident.

Sure but with skating scores available one click away, we get that easily, and as you say, when looking at protocols, sometimes I see something fishy and I am able to guess the nationality of the judge, and verify it quickly... So I don't think it would be enough.

 

[Mathman]

I would have their associated country abbreviation next to their judge number directly on the protocols...

Under 6.0 you could at least see the nationalities of each judge so if they favoured their own or a rival the commentators could point that out.

I think that there is a tension between instances of national bias and the perception of widespread skulduggery that may be harbored by fans, either casual or intense. It would be natural for the ISU to want to keep control of this in their own hands. While accepting their responsibility to identifiy instances of judging bias and to take whatever actions they deem appropriate, nevertheless they want to avoid taking a public relations hit that adversely affects the reputation of the whole sport.

I can certainly see why the ISU would not want a commentator to point out to the TV audience, "Look at those ridiculously high marks that the judge from Slovenia gave to his own skaters. What a crooked sport. Why are you watching this travesty anyway?"

 

[4everchan]

I think that there is a tension between instances of national bias and the perception of widespread skulduggery that may be harbored by fans, either casual or intense. It would be natural for the ISU to want to keep control of this in their own hands. While accepting their responsibility to identifiy instances of judging bias and to take whatever actions they deem appropriate, nevertheless they want to avoid taking a public relations hit that adversely affects the reputation of the whole sport.

I can certainly see why the ISU would not want a commentator to point out to the TV audience, "Look at those ridiculously high marks that the judge from Slovenia gave to his own skaters. What a crooked sport. Why are you watching this travesty anyway?"

But this is already possible. In this era, info is right at your fingertip. Even in GS, in any competition thread, we have the brilliant @Andrea82 who provides us details about the tech panel and the judges. So if GS fans know, you bet the media do too actually, some commentators read the competition threads of Golden Skate

 

[Skating91]

At this point I would be happy if ISU would add one more camera, or use the commercial video that the spectators see along with their tech video

It's amusing people talking about AI and sensors on skates when operating something as simple as a second, third, forth camera for the officials is beyond the capability of the ISU at the moment.

This is one sport that could benefit from cameras, replays, slow motion more than any other sport in the world, and the officials are left with one camera shot to work with and the fans get fuzzy "HD" Youtube feeds in dark arenas.

They couldn't make a beautiful sport look more ordinary if they tried.

If after giving the tech panel access to a second, third, forth angle they can't get take off and landing calls right, then maybe start looking at AI and sensors.

I think the better solution would be just to hold them accountable, make them explain their decisions after each event, try to educate them if wrong, if all else fails simply fire them as you would an incompetent official in any other sport.

 

[snowed]

It's amusing people talking about AI and sensors on skates when operating something as simple as a second, third, forth camera for the officials is beyond the capability of the ISU at the moment.

This is one sport that could benefit from cameras, replays, slow motion more than any other sport in the world, and the officials are left with one camera shot to work with and the fans get fuzzy "HD" Youtube feeds in dark arenas.

They couldn't make a beautiful sport look more ordinary if they tried.

If after giving the tech panel access to a second, third, forth angle they can't get take off and landing calls right, then maybe start looking at AI and sensors.

I think the better solution would be just to hold them accountable, make them explain their decisions after each event, try to educate them if wrong, if all else fails simply fire them as you would an incompetent official in any other sport.

At this point the tech is instructed to call what they see, either live either on video, not what they think, guess or even know is... A skater that we all know flutzes all the time, will get away with the lutzes that are not in the line of sight live or video. At this point I would not fault the tech panel... they do what they can. Plus there are 3 of them, they vote 2 against one.

 

[Skating91]

At this point the tech is instructed to call what they see, either live either on video, not what they think, guess or even know is... A skater that we all know flutzes all the time, will get away with the lutzes that are not in the line of sight live or video. At this point I would not fault the tech panel... they do what they can. Plus there are 3 of them, they vote 2 against one.

The only thing is that there is a young Korean skater who the officials suddenly become very confident with giving edge calls every single time, and there's a Georgian skater they pick apart everything other than a completely rotated landing is little more than +1 for GOE. Yet as you say there are recidivists with lutz take off edges (these can be identified in real time, in slow motion with a bad camera angle, etc) but the officials seems to miss it almost every time. I find this very odd.

On the flip side, I watched an American skater recently with 5/7 dirty landings in the free skate and received an enormous score, I saw something similar with a Belgian early in the season. I'm not making this up I even posted screen captures demonstrating the cheated landings but none of it affected scores.

It's all very strange to me these discrepancies.