Hypothetical discussion : How would you reduce national bias in judging figure skating

[gkelly]

AFAIK, you can measure the strength and angle with which the blade presses the ice, distribution of the skater's body weight along the blade as well as speed and direction of movements by using sensors.

In theory, yes.

But I don't think the technology currently exists to do this in a way that is unobtrusive both for the skaters (would the sensors be attached to the blades or boots? or under the ice?) and for the viewers (judges and spectators -- would there need to be equipment that obstructs views?), and then would be affordable on a large scale for skaters and competitions at all levels.

It would be interesting to start a pilot project to develop such technology. But I think it would be decades away from being ready for widespread use.

Probably some other features too and they will probably grow in numbers with each year. I guess this could bring answers at least to some of your questions.
As for counting turns, steps etc. I guess a simple app using slow mo good quality video could do it much quicker and with higher credibility than another volunteer.

Again, if you think it's possible with current technology, please develop it, or hire a tech expert to do so.

Otherwise, I think it's easier said than done -- that the eyes of skating experts can do a better job than technology that could be ready for use by, let's say, summer 2026.

 

[Magill]

In theory, yes.

But I don't think the technology currently exists to do this in a way that is unobtrusive both for the skaters (would the sensors be attached to the blades or boots? or under the ice?) and for the viewers (judges and spectators -- would there need to be equipment that obstructs views?), and then would be affordable on a large scale for skaters and competitions at all levels.

It would be interesting to start a pilot project to develop such technology. But I think it would be decades away from being ready for widespread use.


Again, if you think it's possible with current technology, please develop it, or hire a tech expert to do so.

Otherwise, I think it's easier said than done -- that the eyes of skating experts can do a better job than technology that could be ready for use by, let's say, summer 2026.

Look, I am just a fan and this is a fan forum for chatting where people exchange thoughts, ideas and opinions, not for professionals to offer and gain commissions. Currently I am not being employed by ISU to develop any technology or do any research for them, including desktop research, or develop any specific design or plan. Neither do I seek such employment, nor am I the right person for the job. But ISU has a separate fund for technology development, there are lots of IT specialists who would happily do it for them for money. BTW are you aware how they have been using this fund? Cause as for now, you cannot really see it during competitions where all this money has been going to....
And as to your first question, for example, Japanese researchers have already developed technology to measure edges with the use of a smartphone camera and sensors attached to the skater's body, not the boots or blades, which is much less intrusive. And rather inexpensive, as they say.

 

[gkelly]

Look, I am just a fan and this is a fan forum for chatting where people exchange thoughts, ideas and opinions, not for professionals to offer and gain commissions. Currently I am not being employed by ISU to develop any technology or do any research for them, including desktop research, or develop any specific design or plan. Neither do I seek such employment, nor am I the right person for the job. But ISU has a separate fund for technology development, there are lots of IT specialists who would happily do it for them for money. BTW are you aware how they have been using this fund? Cause as for now, you cannot really see it during competitions where all this money has been going to....
And as to your first question, for example, Japanese researchers have already developed technology to measure edges with the use of a smartphone camera and sensors attached to the skater's body, not the boots or blades, which is much less intrusive. And rather inexpensive, as they say.

I don't know how how the ISU has been using this fund.

Good to know that some progress has been made somewhere.

I expect such technology will be used in the future. I just doubt it's as close to ready for widespread use as soon as fans might hope.

 

[Magill]

I don't know how how the ISU has been using this fund.

Good to know that some progress has been made somewhere.

I expect such technology will be used in the future. I just doubt it's as close to ready for widespread use as soon as fans might hope.

IMHO, the biggest problem is an obvious reluctance to use technology on the part of ISU and skating officials. They might at least be up to date with the research which is taking place despite their visible lack of interest .
I am sure there are many little things which could be measured objectively and beyond doubt using already existing and not excessively expensive technology, not to mention things like investing in a second camera for comps to avoid the blind spots on the one currently in use...

 

[DancingCactus]

I would have on spare judge to switch out judges when a skater from their country is on the ice. This can't prevent them from underscoring a rival skater, but at least they wouldn't be able to overscore their own skaters. Imagine a football match between Germany and Argentina with a German or Argentinian referee. The very notion is absurd.

 

[icewhite]

Good to know.

You can find the current ice dance GOE guidelines in this document:

https://www.isu.org/figure-skating/rules/fsk-communications/31370-isu-communication-2569/file

They're set up differently from the singles/pairs guidelines. Is this closer to what you would want to see for those disciplines as well?


Suggestions?



Maybe each PCS bullet point should get its own score. Then they could be weighted so that if the ISU decides that "Power and speed" is twice as important as "Variety of edges, steps, turns, movements and directions" (or vice versa), the system would weigh them against each other rather than asking judges to figure out their relative weight in coming up with an overall Skating Skills score.


Please do. I'm curious about what you're thinking.

I like the idea of each bullet point getting its own score with a definite rate of importance. It would surely take a little longer, but I am totally willing to wait for a more accurate and fairer score.

I would also like to have a clear definition in how far mistakes and the bv should affect the PCS. At the moment it is an argument that is brought up when it's convenient: "but xy was clean!", "but xy had much more difficult jumps!"
Maybe the default deduction for a fall should be higher and maybe there should be a set deduction for 3 or more elements with negative GOE.
Otherwise mistakes on the elements would not affect the PCS.
It should be clearly stated that PCS are independent from the bv and TES - or, if people don't agree with that, it should be clear by how much the PCS shall be affected (although that does not make sense to me, but the most important factor here for me is that it should not be up to subjectivity how much it does affect the PCS). Otherwise it becomes (and often is) a joker argument you can use anytime you want.

 

[Mathman]

To me, there are a number of issues that we need to be more clear about before we rush in where angels fear to tread.

For instance, would we want to have skaters wired up with sensors which would allow us to determine the precise distribution of weight along the blade at each instant? We could, after all, replace the human skater with a robot (who would then judge herself) and cut out the middleman altogether.

More seriously, to me the whole question of bias is not so cut-and-dried as is ought to be (define "ought" ). I was very intrigued by the discussion on the other thread of the case involving the Canadian official who, as an Olympiic judge, gave generous marks not only to Virtue and Moir over Papadokis and Cizeron but also to the other two Canadian dance teams Weaver and Poje and Gilles and Poirier.

"Bias" is a well-defined and precisely measurable mathematical concept that attaches to collections of data independent of context. But what if the judge had said, "I am biased in favor of innovation and imaginative use of movement and concept in ice dance. P&C are wonderful skaters, but their programs do have a certain sameness. Gilles and Poirier, on the other hand, their programs bring a breath of fresh air to a stodgy sport, plus they skate with spirit and elan. If I gave higher marks to the Canadian ice dancers than my fellow judges did, perhaps it is because I hear a different drummer.

Would we want the ISU to say, aha, you admit it, you're fired for bias?

This case is also a good test of the hypothesis that such judging anomalies drive fans away from the sport in dismay and disgust. I, for one, as much time as I spend on Golden Skate , had never heard of this "scandal" until a few days ago, nor had I ever heard of anyone named Leanna Caron, nor can I name a single national federation president, not even the USA. (And neither can any of my friends or the random guy walking down the street or my cousin Earnie )

 

[noskates]

I think as long as there are humans involved there will always be charges of bias - even if there isn't any, And wiring skaters and using technology just - to me - seems a distraction to the sport.

 

[Mathman]

About the cost, if let's say 100 judges would be required and each paid an anual salary of $US 100,000 (the average salary of an NFL referee is $205,000), that's 10 million dollars. Or would 60 at $90,000 be enough? A substantial chunk of change for an organization with an anual operating budget of 16 million, but still withing the range of what is possible (they could cash in some of their long-term investments in anticipation of increased revenue in the future).

But there is also the question of whether we want to strengthen the power of the central authority at the corresponding diminution of the prerogatives of the membership. In a labor dispute, who is "right," workers or management? In politics, do we want a dictator and hope for a benign one, or do we want democracy with its own built-in peccadillos and dangers?

 

[Magill]

As for "wired" skaters, sensors are mostly a wireless technology, lol.
Besides, they would not be more "wired" than anyone who goes to a TV studio and I have not seen many people running away in horror from such a pain and humiliation when lured to appear on the screen., not many viewers turning their TV off in disgust at a show's "wired" hosts and guests (which they mostly do not see anyway). Of course, it is a different kind of "wired" but still..
As for distraction, I guess we should first let skaters try the technology and see what they say, whether they find it difficult to adapt or not at all, whether they would rather be "wired" and scored based on accurate objective measurements, or not "wired" and assessed based on what the panel sees or sees not on their screens...
And still, probably there are many improvements which could be achieved without "wiring" if some IT specialists expert in the state-of-the-art technology were employed.

 

[4everchan]

To me, there are a number of issues that we need to be more clear about before we rush in where angels fear to tread.

For instance, would we want to have skaters wired up with sensors which would allow us to determine the precise distribution of weight along the blade at each instant? We could, after all, replace the human skater with a robot (who would then judge herself) and cut out the middleman altogether.

More seriously, to me the whole question of bias is not so cut-and-dried as is ought to be (define "ought" ). I was very intrigued by the discussion on the other thread of the case involving the Canadian official who, as an Olympiic judge, gave generous marks not only to Virtue and Moir over Papadokis and Cizeron but also to the other two Canadian dance teams Weaver and Poje and Gilles and Poirier.

"Bias" is a well-defined and precisely measurable mathematical concept that attaches to collections of data independent of context. But what if the judge had said, "I am biased in favor of innovation and imaginative use of movement and concept in ice dance. P&C are wonderful skaters, but their programs do have a certain sameness. Gilles and Poirier, on the other hand, their programs bring a breath of fresh air to a stodgy sport, plus they skate with spirit and elan. If I gave higher marks to the Canadian ice dancers than my fellow judges did, perhaps it is because I hear a different drummer.

Would we want the ISU to say, aha, you admit it, you're fired for bias?

This case is also a good test of the hypothesis that such judging anomalies drive fans away from the sport in dismay and disgust. I, for one, as much time as I spend on Golden Skate , had never heard of this "scandal" until a few days ago, nor had I ever heard of anyone named Leanna Caron, nor can I name a single national federation president, not even the USA. (And neither can any of my friends or the random guy walking down the street or my cousin Earnie )

You didn't hear about the scandal because it's not really a scandal Was Leanna Caron suspended for out of line judging ? No. And you are correct : judges are allowed to appreciate different styles of skating and to have a global picture of Mrs Caron's scoring, one would need to analyze every single one of her scores for both the RD and the FD and see really how she marked everybody else. I believe the ISU did that : she wasn't suspended. She didn't receive a warning. Some people complained, the ISU made a rule to make sure that high ranked officials from federation would not serve as referee or tech controllers... That's all.

The reason I opened this thread is to discuss ways the ISU could depart from the old system which is prone to National bias. Could we stick to the topic as we have discussed Leanna Caron extensively in the other thread

 

[Skating91]

It would almost be better to have less judges maybe one or two but doing it on a professional basis all over the world and held accountable. The tech panel just needs better technology I apparently can spot things on the Youtube feed easier than they can, so this is a major and inexcusable oversight in the sport.

And no, we don't need AI or anything like this just an extra monitor with the Youtube feed for the tech panel, or an extra camera.
Technology like a camera is very inexpensive now and lasts for many years there is just no excuse for having skaters cheating on take offs and landings and getting away with it because they jump out of view of the judges and the sole camera.

 

[el henry]

As for "wired" skaters, sensors are mostly a wireless technology, lol.
Besides, they would not be more "wired" than anyone who goes to a TV studio and I have not seen many people running away in horror from such a pain and humiliation when lured to appear on the screen., not many viewers turning their TV off in disgust at a show's "wired" hosts and guests (which they mostly do not see anyway). Of course, it is a different kind of "wired" but still..
As for distraction, I guess we should first let skaters try the technology and see what they say, whether they find it difficult to adapt or not at all, whether they would rather be "wired" and scored based on accurate objective measurements, or not "wired" and assessed based on what the panel sees or sees not on their screens...
And still, probably there are many improvements which could be achieved without "wiring" if some IT specialists expert in the state-of-the-art technology were employed.

For me, "Wired" in colloquial English does not mean wires trailing from skates, LOL to coin a phrase. It means all "sensored up". I agree with @noskates that it could be a distraction.

Then again, I find the tech box an unnecessary distraction that if I were able in horror to banish from my screen, I would.

I also would find that very distracting as a skater, but I am not a skater. And I don't think skaters are nearly as obsessed with what fans call accurate objective measurements as fans are, but I would be willing to learn.

So I would spend this hypothetical unlimited pot of money on: Skaters, do *you* want to be all sensored up? Do you want sensors all over the rink? What do you, the skaters, think? What do you think should be measured? Real money for an anonymous, professionally drafted questionnaire.

(This for me is separate from appearance of bias with national judges, and what our hypothetical unlimited pot of money would accomplish there).

Those results would guide me.

 

[4everchan]

We discussed AI at length... there is already a thread about it. No need to rehash all the same arguments.

I was asked to start a thread specifically on how to remove national bias by replacing the judging panel by professional ones.... can we leave the sensors out of this for now and stick to that very specific topic ? I believe it does deserve a proper discussion and I am afraid that once again the thread will drift towards AI and sensors

I am all for better use of technology but let's assume for now that the ISU is not going there.

What are other ways to make judging less biased ?

 

[mrrice]

I think as long as there are humans involved there will always be charges of bias - even if there isn't any, And wiring skaters and using technology just - to me - seems a distraction to the sport.

This is an absolute fact. As a judge myself, it has much more to do with the panel. US coaches have different values. Some value tech content more than others. It is rare to see them go against what they've seen on the GP circuit.

 

[4everchan]

This is an absolute fact. As a judge myself, it has much more to do with the panel. US coaches have different values. Some value tech content more than others. It is rare to see them go against what they've seen on the GP circuit.

Yes. That's correct. Different values from different cultural visions of the sport. Growing up in Canada, I watched a lot of Canadian skaters... and I see the sport differently than someone from another country. But that's the point : if the judges were hired by the ISU and trained by the ISU wouldn't that remove the nationalistic cultural views of the sport ?

A bit like in ice dance : the ISU is definitely directing a specific trajectory when removing patterns and adding so many not-leveled CHOREO elements. They want the sport to be less rigid and freer in order to build programs that connect more with the audience. In some ways, the ISU is pushing the boundaries itself. (This is not a thread to discuss ice dance specifically, I am bringing it up to show how the ISU is already calling the shots on what they want to see in the sport).

So in light of that, I believe the ISU would be able to define precisely what are good skating skills, good composition and performance/execution. I believe that bullet points could get even more refined. Calls on jumps could get better with very strict guidelines : some tech specialists already do so. Professional judges = higher standards of objectivity.

There will always be room for subjectivity in figure skating (and many sports). But the problem for me is when it becomes nationalistic. We have enough with politics, no need for it in sport.

 

[Mathman]

The reason I opened this thread is to discuss ways the ISU could depart from the old system which is prone to National bias. Could we stick to the topic?

Sorry for my part in meandering off. But I do think that in considering of the question,"What should/can the ISU do to reduce national bias?" it is legimate to hold the opinion that this problem is not as bad as it is perceived to be, that the ISU is already doing quite a bit on this front , or that whenever "judging" occurs, there will be compliants of bias no matter what you do.

For instance, on the sub-question of, "How can the ISU make the judging criteria for PCSs more precisely defined and more objectively measurable?" an exceptable answer might be "nothing, nor would it be a good thing even if possible." For me, I am actually rather imporessed with a lot of the verbiage that the ISU has come up with on this front over the years.

For musical interpretation the skaters are expected to employ "finesse to reflect the details and nuances of the music." There is nothing the slightest bit vague about this sentence. I know what "finess" means. I know what "nuances" are. I know what the word "music" refers to (sort of). The only thing that is fuzzy is, how should figure skating judges procede to measure the extent to which a performce is "outstanding," or "excellent" or "very good" on this point.

(Not, by the way, that there is anything wrong with a rousing bull-in-the-china-shop performance to Tchaikovsky's 1812 Overture. )

 

[4everchan]

Sorry for my part in meandering off. But I do think that in considering of the question,"What should/can the ISU do to reduce national bias?" it is legimate to hold the opinion that this problem is not as bad as it is perceived to be, that the ISU is already doing quite a bit on this front , or that whenever "judging" occurs, there will be compliants of bias no matter what you do.

For instance, on the sub-question of, "How can the ISU make the judging criteria for PCSs more precisely defined and more objectively measurable?" an exceptable answer might be "nothing, nor would it be a good thing even if possible." For me, I am actually rather imporessed with a lot of the verbiage that the ISU has come up with on this front over the years.

For musical interpretation the skaters are expected to employ "finesse to reflect the details and nuances of the music." There is nothing the slightest bit vague about this sentence. I know what "finess" means. I know what "nuances" are. I know what the word "music" refers to (sort of). The only thing that is fuzzy is, how should figure skating judges procede to measure the extent to which a performce is "outstanding," or "excellent" or "very good" on this point.

(Not, by the way, that there is anything wrong with a rousing bull-in-the-china-shop performance to Tchaikovsky's 1812 Overture. )

Okay. So some people, like yourself, think it's all good... let's hear suggestions from people who have some then and think improvement is deeply needed

 

[gkelly]

Suppose country A's skating culture has for decades consistently placed high value on soft knees/effortless flow, beautiful spiral positions, and beautiful layback positions?

And country B's skating culture has consistently placed high value on absolute speed throughout the program, deep outside edges on lutz takeoffs, and strong flying positions in flying camel spins.

The skaters from country A will tend to be quite strong in the areas that A's culture values, and the judges from A will tend to reward them for it, and to penalize B skaters for what they consider unattractive positions and effortful-looking skating.

Meanwhile, B's judges will reward B's skaters for the strengths valued by B and will penalize A's skaters for relative lack of speed and for flutzes.

Where the elements are concerned, this might not make more than 1 GOE point of difference here and there for the respective strengths and weaknesses.

But on Skating Skills, A's judges are likely to give their highest marks to different skaters than B's judges do. In fact, they will tend to give their highest marks to skaters from their own country.

If the different bullet points under Skating Skills each get separate scores from each judge, it might become clear which judges are rewarding which kinds of strengths (or penalizing which kinds of weaknesses).

If those judges mark their home skaters most highly, is that national bias or corruption? Or is it just rewarding what they have learned over their long careers to value as "good skating"?

Will it be clearer if people can see that the A judges strongly reward soft knees and effortless regardless of which country the skaters are from, and B judges strongly reward total speed regardless of national affiliation?

Suppose a judge changes affiliation from the country where they learned to skate and where they learned to judge, to a different country that happens to have a different skating culture, or to being purely beholden to the ISU.
You could insist that international judges must break all contact with the skating community in the country where they grew up so that they're not allowed to stop by a local rink let alone judge a local competition, and not allowed to watch skating on TV with their own country's commentators, and explicitly forbidden talking about skating with friends from their home country.

But these judges would still probably tend to share the values of the skating community they started out in. I don't think that can be avoided, although of course serving on panels with judges from around the world will expose them to other ways of valuing the various criteria.

 

[4everchan]

Suppose country A's skating culture has for decades consistently placed high value on soft knees/effortless flow, beautiful spiral positions, and beautiful layback positions?

And country B's skating culture has consistently placed high value on absolute speed throughout the program, deep outside edges on lutz takeoffs, and strong flying positions in flying camel spins.

The skaters from country A will tend to be quite strong in the areas that A's culture values, and the judges from A will tend to reward them for it, and to penalize B skaters for what they consider unattractive positions and effortful-looking skating.

Meanwhile, B's judges will reward B's skaters for the strengths valued by B and will penalize A's skaters for relative lack of speed and for flutzes.

Where the elements are concerned, this might not make more than 1 GOE point of difference here and there for the respective strengths and weaknesses.

But on Skating Skills, A's judges are likely to give their highest marks to different skaters than B's judges do. In fact, they will tend to give their highest marks to skaters from their own country.

If the different bullet points under Skating Skills each get separate scores from each judge, it might become clear which judges are rewarding which kinds of strengths (or penalizing which kinds of weaknesses).

If those judges mark their home skaters most highly, is that national bias or corruption? Or is it just rewarding what they have learned over their long careers to value as "good skating"?

Will it be clearer if people can see that the A judges strongly reward soft knees and effortless regardless of which country the skaters are from, and B judges strongly reward total speed regardless of national affiliation?

Suppose a judge changes affiliation from the country where they learned to skate and where they learned to judge, to a different country that happens to have a different skating culture, or to being purely beholden to the ISU.
You could insist that international judges must break all contact with the skating community in the country where they grew up so that they're not allowed to stop by a local rink let alone judge a local competition, and not allowed to watch skating on TV with their own country's commentators, and explicitly forbidden talking about skating with friends from their home country.

But these judges would still probably tend to share the values of the skating community they started out in. I don't think that can be avoided, although of course serving on panels with judges from around the world will expose them to other ways of valuing the various criteria.

I would think that Canadian judges value good skating skills and flow. They don't necessarily reward their own skaters that much because not all of the current skaters have these qualities. As a matter of fact, they tend to mark them quite low (single skaters mostly). So it can go both ways.

I think that the training of professional ISU judges would indeed address all of these cultural differences in skating. Judges would learn how to value them and priorities would be given. For instance, blade work would be placed higher than let's say, connection with the audience or armography

But indeed, if fans see the diverse skating cultures, the ISU would also take that into consideration when training the judges.

The issue with that is that skating would become more and more similar. Just like what we see in the rhythm dance these days or in pairs short programs where the so many elements are so bunched up that everyone ends up doing the same thing.