Why do more skaters not file protests against the results?

[yuumagical]

The enthusiasm shown here for AI judging has really got me thinking. The more I learn about AI systems, the less enthusiastic I am about them being used in terms of judging figure skating.

These types of systems often use machine learning using preexisting data sets. Obviously this data would come from elements done under the current system. But how would this data be treated? Would the positive GOE on a perfectly executed toeloop be treated in the same way as the same amount of positive GOE on a saltoe? Would factors such Eteri bonuses in preexisting data skew the data in a way that would be antithetical to a fair judging system? The type of data that serves as a reference point is crucial to the failure or success of an AI judging system.

This is a bit more farfetched of a worry, but if an AI system were to judge component scores as well, I worry that such a system could favor certain styles of costumes or even certain physical traits.

I'm not saying that an AI system can't be used for judging in figure skating, but that doing so would require quite a bit of precaution.

 

[lilimum]

The enthusiasm shown here for AI judging has really got me thinking. The more I learn about AI systems, the less enthusiastic I am about them being used in terms of judging figure skating.

These types of systems often use machine learning using preexisting data sets. Obviously this data would come from elements done under the current system. But how would this data be treated? Would the positive GOE on a perfectly executed toeloop be treated in the same way as the same amount of positive GOE on a saltoe? Would factors such Eteri bonuses in preexisting data skew the data in a way that would be antithetical to a fair judging system? The type of data that serves as a reference point is crucial to the failure or success of an AI judging system.

This is a bit more farfetched of a worry, but if an AI system were to judge component scores as well, I worry that such a system could favor certain styles of costumes or even certain physical traits.

I'm not saying that an AI system can't be used for judging in figure skating, but that doing so would require quite a bit of precaution.

Perhaps AI is not the right expression. I think in figure skating we don't need a self learning system but a system of sensors, cameras and software to assist / replace at least the technical controllers. If this system is calibrated based on textbook jumpers like Nathan Chen or Hanyu would be sufficient and there shoulc be introduced a mark for poor take-off, similar to wrong edge and underrotation

 

[Magill]

Perhaps AI is not the right expression. I think in figure skating we don't need a self learning system but a system of sensors, cameras and software to assist / replace at least the technical controllers. If this system is calibrated based on textbook jumpers like Nathan Chen or Hanyu would be sufficient and there shoulc be introduced a mark for poor take-off, similar to wrong edge and underrotation

Nagoya University et al has published a study on the automatic edge assessment when jumping Lutz, low-cost, conducted with a simple smartphone and a 3D-motion-analysis software, resulting in a 3D jump simulation. It has been linked in Yuzuru Hanyu's FF as it refers his thesis on AI / sensor technology use in FS. What they propose still needs improvement but it shows things are happening and the resulting technology might come sooner than many of us think, and not be necessarily expensive after all. In fact, they say it might be used by skaters themselves for training purposes, to get technology-assisted objective feedback on their edges.
They also mention a more elaborate earlier study, also by Japanese researchers, using 6 cameras located outside of the rink, too expensive and time consuming in setting-up to be used by amateurs for self-training purposes, they say, but would do for competitions, maybe.
I wonder if ISU has ever contacted the researchers to seek ways to work with them or at least experiment with putting their work into practice. If they have done it with Lutz, probably they can easily extend their work to cover all the jumps, I guess, they just need funds and, possibly, someone interested in putting their results into practice.
The link shows the summary which has been published in an open-access platform, there is a built-in link in it leading to a video, and the paper itself can be downloaded for free using a button in the top right-hand corner. Food for thought!

[2310.17193] Automatic Edge Error Judgment in Figure Skating Using 3D Pose Estimation from a Monocular Camera and IMUs

 

[DizzyFrenchie]

The enthusiasm shown here for AI judging has really got me thinking. The more I learn about AI systems, the less enthusiastic I am about them being used in terms of judging figure skating.

These types of systems often use machine learning using preexisting data sets. Obviously this data would come from elements done under the current system. But how would this data be treated? Would the positive GOE on a perfectly executed toeloop be treated in the same way as the same amount of positive GOE on a saltoe? Would factors such Eteri bonuses in preexisting data skew the data in a way that would be antithetical to a fair judging system? The type of data that serves as a reference point is crucial to the failure or success of an AI judging system.

This is a bit more farfetched of a worry, but if an AI system were to judge component scores as well, I worry that such a system could favor certain styles of costumes or even certain physical traits.

I'm not saying that an AI system can't be used for judging in figure skating, but that doing so would require quite a bit of precaution.

This is why I hadn't used the word AI. Many people use it for automated processes, but I wished to be clear that I was only speaking of calibrating cameras/softwares after ISU rules as they are, not after how skates are judged at the moment; which anyway wouldn't bring the least result, as both tech calls and judging are perfectly inconsistent, unless one adds a "personalised overlay" for each skater... or how could an AI "learn" that Yuzuru Hanyu's Olympic 4A, missing about 100°, is < while Nathan Chen's Olympic solo quadruples and quad combinations, all missing about the same 100° (as he had done since 2020-2021), musn't be called at all? This to take the same two Men skaters as Mathman, although we were initially speaking of Women, but the rules and tech panels and judges and authorities are the same.

 

[icewhite]

I think AI is not needed or very helpful for identifying jump rotation. It would be useful for PCS. And I am not in favour of implementing it now... but I can very well see it being developed for exactly that purpose. But of course, yes, the problem with AI is always the data you feed it with. So that data would have to be selected very carefully.

For simple counts of rotation and identifying correct jumps normal technology would be enough. The problem I see is not the technology but the aspect that's already the problem now: It is not clearly enough defined how a jump should look like exactly. Where is the exact cut off for when a rotation starts? When exactly do we count it as landed? What exactly does it mean it needs to be jumped over the toe pick?
As long as these clear definitions do not exist we can only have a lenient panel that's in dubio pro reo. Only if a UR is clearly visible to the naked eye in real time it can be called. But technology does not work like that.
Or... you use AI after all. Then you don't need those clear definitions. But it would also not really solve the underlying issue imo. Skaters would still not have a fully reliable insight into whether their jump is correct or not. You would just give in to "correct is what's judged as correct".

 

[gkelly]

If you go to watch protcols from previous years, very few had edge calls. Some got > or >>, but minority. There are protocols from 2005 or something like that and anybody can go and check, you don't have to guess.

Most of the symbols for error calls didn't exist in 2005.

Originally any jump that was underrotated by >90 degrees was "downgraded" to a jump from the same takeoff with one lower revolution. So underrotated 4T was called as 3T and that's all you'll see on the protocols.

That caused some problem with the computer seeing Zayak violations where the skater had clearly attempted a quad not another triple, so that's when the < symbol was introduced, around 2005.

It was several years later that the distinction between underrotation < and downgrade << was introduced, and the q was added only a couple of years ago.

I don't remember whether e calls were added to the protocols from the beginning or whether that was a later addition ca. 2008 or 2010. Definitely the ! call came later.

So the ISU has been trying over the years to be more precise in having the tech panel call varying severity of these errors.

How consistent the panels are in seeing the errors and calling them once the symbols were available to be shown on the protocols (and on the judges' screens) has varied, often depending on the particular members of the panel.

Undoubtedly there will be more technological solutions added in the future. But the trick will be finding a solution that is affordable and can physically be implemented in all venues used for skating competitions at the level(s) where the technology is to be used. If it's just giving more input (e.g., additional camera angles, technological application of the camera views in addition to or instead of human eyes) to add the same calls in the same way, that could be used only for championships and GP if it's too expensive/resource intensive for JGPs and lesser events in smaller local rinks.

 

[icewhite]

Most of the symbols for error calls didn't exist in 2005.

Originally any jump that was underrotated by >90 degrees was "downgraded" to a jump from the same takeoff with one lower revolution. So underrotated 4T was called as 3T and that's all you'll see on the protocols.

That caused some problem with the computer seeing Zayak violations where the skater had clearly attempted a quad not another triple, so that's when the < symbol was introduced, around 2005.

It was several years later that the distinction between underrotation < and downgrade << was introduced, and the q was added only a couple of years ago.

I don't remember whether e calls were added to the protocols from the beginning or whether that was a later addition ca. 2008 or 2010. Definitely the ! call came later.

So the ISU has been trying over the years to be more precise in having the tech panel call varying severity of these errors.

How consistent the panels are in seeing the errors and calling them once the symbols were available to be shown on the protocols (and on the judges' screens) has varied, often depending on the particular members of the panel.

Undoubtedly there will be more technological solutions added in the future. But the trick will be finding a solution that is affordable and can physically be implemented in all venues used for skating competitions at the level(s) where the technology is to be used. If it's just giving more input (e.g., additional camera angles, technological application of the camera views in addition to or instead of human eyes) to add the same calls in the same way, that could be used only for championships and GP if it's too expensive/resource intensive for JGPs and lesser events in smaller local rinks.

I suppose the costs would speak in favour of AI. You would not need so many sensors and cameras/ actual hardware equipment.

 

[icewhite]

In general if such things are implemented I would like to see them for the other elements and skating overall. We look at every single jump very closely, and of course that makes sense if some calls make such a big difference in points. But at the same time we do not care about extreme differences in the execution of step sequences?

 

[snowed]

In the short program and free skate Kaori received mostly 3's and even one judge gave a +4 for the following 3Lz jumps.

If a skater was given an edge call for a lutz, why wouldn't they simply protest the result and provide clear photographic proof as I have done that a precedent has been established where a lutz doesn't require an outside edge.

To me this is very straightforward.

It's not a criticism of Kaori. She does not require such favours in order to win.

In my knowledge, for Lutz the edge has to be outside up to when the weight starts to be transferred on the toe pick of the vaulting foot. In these pics the weight is already transferred, so not a good choice. Also in my knowledge the tech panel looks at the slow motion video not still pics exactly because they can better the moment when the weight is transferred. By any change you can post slow motion small clips of these take offs?

 

[eppen]

Most of the symbols for error calls didn't exist in 2005.

Originally any jump that was underrotated by >90 degrees was "downgraded" to a jump from the same takeoff with one lower revolution. So underrotated 4T was called as 3T and that's all you'll see on the protocols.

That caused some problem with the computer seeing Zayak violations where the skater had clearly attempted a quad not another triple, so that's when the < symbol was introduced, around 2005.

It was several years later that the distinction between underrotation < and downgrade << was introduced, and the q was added only a couple of years ago.

I don't remember whether e calls were added to the protocols from the beginning or whether that was a later addition ca. 2008 or 2010. Definitely the ! call came later.

So the ISU has been trying over the years to be more precise in having the tech panel call varying severity of these errors.

How consistent the panels are in seeing the errors and calling them once the symbols were available to be shown on the protocols (and on the judges' screens) has varied, often depending on the particular members of the panel.

Undoubtedly there will be more technological solutions added in the future. But the trick will be finding a solution that is affordable and can physically be implemented in all venues used for skating competitions at the level(s) where the technology is to be used. If it's just giving more input (e.g., additional camera angles, technological application of the camera views in addition to or instead of human eyes) to add the same calls in the same way, that could be used only for championships and GP if it's too expensive/resource intensive for JGPs and lesser events in smaller local rinks.

A couple of choronological additions.

< was introduced in the 2006-7 season and it was used until the end of 2009-10 season to indicate downgrade. Early IJS was superharsh for jumps in this way and it was advisable to try only superconsistent jumps.

e started to get used in 2007-8 and ! appeared in 2008-9.

The distinction between an underrotation < and downgrade << was introduced in the 2010-11 season. This was combined with an overhaul of how jumps were scored in general and this system encouraged risk-taking in jumps - hence the abundance of quads in the late 2010s.

q was added in the 2020-21 season.

I usually try to sit on the judges side a little above them to get a good view of what the choreo is like and how the skaters are performing. This means that you also get to observe how the judges work.

So, a minor detail on what the two judging panels see or cannot see. The judging screen looks roughly like this even if this pic is from before the -5 to +5 system and these days they use mostly laptops:

On the left you have a list of elements and during the program, the red flags for review appear on the left side - the decisions by the tech panel. When they have reviewed the element, the flag is removed if all's well or then a red sign on the right side of the element indicates that there is call of some sort on it. I am pretty sure the judges can also see what the call is - otherwise it would make no sense. So, they know all the time what the techies think, but can also view the elements on video for themselves.

The tech panel uses headphones with microphones so they can speak to each other without problems. Sometimes they talk a lot, sometimes not so much, depends on the people, I guess.

There have been seasons when it has seemed that the different flaws with jumps have been observed more strictly than other times. As if the ISU had decided that this season just about everything possible gets called. I have not done any statistics, but there seemed to be that kind of a trend a few years ago. Like a reminder to the skaters and coaches to pay attention to edges and jump rotation...

 

[Skating91]

People are talking about the implementing AI.

Sorry, but AI is not needed to identify something as simple as the wrong take off edge on a lutz. Twice they missed it on the weekend across two programs for the best female skater in the world. Even Mark Hanretty who is polite a man as can be is even basically saying the lutz has a wrong edge. He's as puzzled by this as all of us.

AI is not needed to identify that as a wrong edge. This is simply sheer incompetence. If not incompetence, I have to assume corruption. What else could it be? If they can't get this right then how can we expect them to get anything else right (look at Isabeau's program 7/7 jump components had a dirty landing but only reflected in 2/7 jumps).

If there was some kind of challenge system or if skaters felt emboldened to file protests when they were robbed, then yes one or two tournaments would be chaos but this would soon work itself out. Simply the technical controller would be fired after the chaos caused.

Really, there needs to be some kind of internal review after every event at the ISU and if the technical controller misses something as obvious as Kaori's wrong edge on the lutz in both the short program and free skate, that technical controller should be suspended at the minimum if not his/her accreditation revoked.

This is so easy to fix.

Thankfully none of this affected gold at the GP final because GOATamoto is in a class of her own, but not judging Loena's short program correctly definitely robbed Hana of a well deserved silver, and that is simply wrong. She might never get this chance again. It's terrible imo. I have screenshots of post of Loena's short program to prove it.

This is not a criticism of Loena, a bronze would still be a great result and there is no shame in it.

I just want impartial, honest judging like everyone else.

 

[Skating91]

In my knowledge, for Lutz the edge has to be outside up to when the weight starts to be transferred on the toe pick of the vaulting foot. In these pics the weight is already transferred, so not a good choice. Also in my knowledge the tech panel looks at the slow motion video not still pics exactly because they can better the moment when the weight is transferred. By any change you can post slow motion small clips of these take offs?

Second from the left you can see the pick has not hit the ice and a very clear inside edge.

Sorry, but this is all very obvious to me in real time.

If you look at the women's free skate thread in real time I identified Isabeau not rotating any of her jumps, then I posted proof afterwards showing I was indeed correct.

I can identify all of this in real time from a Youtube stream and I have no qualifications. It's not that hard.

https://twitter.com/chiburahakkai/status/1020618307873804288

A flat edge at take off is a mistake. This is an inside edge, so a major error. Instead mainly +3 for these jumps.

 

[snowed]

At the moment, tech panels and judges are unfair to the point that a great many would need lengthy protests.

But do you think that the rules of the sport are wrong or do tech panels and judges apply the rules wrongly? Is it possible to have a poll? I'll put my answers in parenthesis
1. are the rules of figure skating totally wrong? (no)
2. can the rules be improved (yes)
3. are the tech panels and judges unfair on purpose (like intentional, let's excuse unintentional bias and let's not talk about criminal corruption) so they bend the rules? (no)
4. do tech panels and judges understand the rules? (yes)
5. do the tech panel and judges have enough time and tools (enough videos) to judge correctly, review everything, think of everything? (no)
6. do we, as casual fans, understand the rules precisely (no)

So are the tech panel and judges unfair?

 

[snowed]

Nagoya University et al has published a study on the automatic edge assessment when jumping Lutz, low-cost, conducted with a simple smartphone and a 3D-motion-analysis software, resulting in a 3D jump simulation. It has been linked in Yuzuru Hanyu's FF as it refers his thesis on AI / sensor technology use in FS. What they propose still needs improvement but it shows things are happening and the resulting technology might come sooner than many of us think, and not be necessarily expensive after all. In fact, they say it might be used by skaters themselves for training purposes, to get technology-assisted objective feedback on their edges.
They also mention a more elaborate earlier study, also by Japanese researchers, using 6 cameras located outside of the rink, too expensive and time consuming in setting-up to be used by amateurs for self-training purposes, they say, but would do for competitions, maybe.
I wonder if ISU has ever contacted the researchers to seek ways to work with them or at least experiment with putting their work into practice. If they have done it with Lutz, probably they can easily extend their work to cover all the jumps, I guess, they just need funds and, possibly, someone interested in putting their results into practice.
The link shows the summary which has been published in an open-access platform, there is a built-in link in it leading to a video, and the paper itself can be downloaded for free using a button in the top right-hand corner. Food for thought!

[2310.17193] Automatic Edge Error Judgment in Figure Skating Using 3D Pose Estimation from a Monocular Camera and IMUs

If I understand well, they use 10 points on the body, so shoulders versus hips alignment (not points on the blade) to track the edge. Am I wrong?

This is very interesting to me and it makes sense because the edge comes from whole body alignment and lean towards the inside of the circle, but that makes me think if all these very deep outside edges (for Lutzes entries ) from forcefully bending the ankle towards outside but having the body lean towards inside of the circle are ultimately outside edges?

The Lutz is special because it is the only jump that counterrotates (from the entry edge), the outside entry edge should be a "proof", an instrument to check the counterrotation... but it is really the counterrotation that makes a Lutz not the outside edge. If the outside entry edge is "fake"/ forced is there counterrotation there? I'm exaggerating to make the point...

 

[DizzyFrenchie]

In my knowledge, for Lutz the edge has to be outside up to when the weight starts to be transferred on the toe pick of the vaulting foot. In these pics the weight is already transferred, so not a good choice. Also in my knowledge the tech panel looks at the slow motion video not still pics exactly because they can better the moment when the weight is transferred. By any change you can post slow motion small clips of these take offs?

Tech panels aren't allowed to check prerotation with slow motion, they must do it at regular speed, that rule was added a few years ago; but there's no prohibition for viewing the takeoff in slow motion to check the edge after all.

 

[DizzyFrenchie]

But do you think that the rules of the sport are wrong or do tech panels and judges apply the rules wrongly? Is it possible to have a poll? I'll put my answers in parenthesis
1. are the rules of figure skating totally wrong? (no)
2. can the rules be improved (yes)
3. are the tech panels and judges unfair on purpose (like intentional, let's excuse unintentional bias and let's not talk about criminal corruption) so they bend the rules? (no)
4. do tech panels and judges understand the rules? (yes)
5. do the tech panel and judges have enough time and tools (enough videos) to judge correctly, review everything, think of everything? (no)
6. do we, as casual fans, understand the rules precisely (no)

So are the tech panel and judges unfair?

1. No, as a whole they look good to me.
2. Certainly yes. Particularly, very controversial changes were made at the infamous 2022 ISU Phuket Party Congress, voted thanks to methods which to me amount to a fraud, and it was livestreamed!
3. Clearly, yes, or why some chronic faults on some skaters would never be seen, while others are scrutinised to get the smallest occasional error, sometimes inventing an error?
4. I believe that as a whole, yes, but they don't always show it.
5. I agree. That's why technological help would come handy. Do you remember the 2020 European Championships long wait in the Kiss & Cry for Gabriella Papadakis & Guillaume Cizeron? A member of the tech panel later explained that the camera had been out of order (if I remember correctly) and that usually 80% of their time was devoted to counting rotations... I mean, why, in the XXIth Century? Let machines count rotations and tech panels have time for the rest!
6. Indeed, I have several times misunderstood rules. Yet most of them are written very clearly.

 

[snowed]

Tech panels aren't allowed to check prerotation with slow motion, they must do it at regular speed, that rule was added a few years ago; but there's no prohibition for viewing the takeoff in slow motion to check the edge after all.

What I was saying, is that I think that tech panels checks edges in slow motion not still pics. Do you know differently?
I didn't know tech panel is not allowed to check for prerotation in slow motion, thank you for the information....

 

[DizzyFrenchie]

What I was saying, is that I think that tech panels checks edges in slow motion not still pics. Do you know differently?
I didn't know tech panel is not allowed to check for prerotation in slow motion, thank you for the information....

They're supposed to check prerotation at normal speed only. In fact, not all prerotation, but the possibility of a downgrade for forward (backward for an Axel) take-off. It's p.19 in this year's handbook:

https://www.isu.org/figure-skating/rules/sandp-handbooks-faq/31570-technical-panel-handbook-1/file

 

[eppen]

Out of curiosity looked at some stats based on what has been entered into this database: https://fses.sakuraweb.com/ (includes mostly international comps and JPN nats).

The search options include wrong edge which makes it possible to see some trends in 3Lzs

2017-18 1258 attempts, 171 edge calls = 14% (calls for Russians 13 = 8% of calls)
2018-19 1170 attempts, 122 edge calls = 10% (calls for Russians 23 = 19% of calls)
2019-20 1220 attempts, 215 edge calls = 18% (calls for Russians 29 = 13% of calls)
2020-21 415 attempts, 58 edge calls = 14% (calls for Russians 2 = 3% of calls)
2021-22 1407 attempts, 202 edge calls = 14% (calls for Russians 3 = 1,5% of calls)
2022-23 1476 attempts, 263 edge calls = 18% (obviously no calls for Russians)

Kaori has been jumping 3Lzs most of her career, until 2021-22 5 to 9 attempts per season. She got edge calls to most of them, on average 70% of the attempts. 2021-22 13 attempts with 7 calls (53%) and 2022-23 17 attempts with 6 calls (35%). This season so far 11 attempts with 5 calls (45%).

 

[lariko]

Hear me out now, could it possibly be....

Judge A thinks the jump was landed cleanly and thus they scored it that way.

And random fans on the internet can say what they want and post all the screenshots they want, but that is what the judge saw in the moment? and that is how they will score it?

That concept makes the most sense to me.

Tech panel. Judges correct their marks based on tech panel evaluation.