Ice scope data shows zero correlation between height & GOE

Since we actually have an objective measure of jump size now, I decided to compile the data from ice scope for the men's SP 3As and the GOEs they earned into a spreadsheet. For the sake of making the data more comparable, I only included 3As with positive GOE.

Here's what I found. (For comparison, here's a graph of the relationship between PCS and 3A GOE. The correlation between PCS and 3A GOE was 0.66, which is general considered a strong association. The correlation between total planned jump BV across both programs and 3A GOE was 0.58, and the correlation between PCS and total planned jump BV was 0.69)

1. The correlation between jump height and GOE is virtually zero (r=0.03). You can see it in the scatterplot here. In fact, if we remove Yuzuru Hanyu for being an outlier (he cleared the next highest jump height by 5 centimeters, more than one standard deviation), the correlation between jump height and GOE becomes negative (r=-0.23). In other words, if your name is not Yuzuru Hanyu, the higher you jump your 3As, the lower your GOE tends to be, according to the ice scope data for this competition. This effect is stronger than the relationship between jump distance and GOE and landing speed and GOE.
   
2. The correlation between jump distance and GOE is very small. (r=0.11) Here's the scatterplot. While there is a positive relationship, only about 1-1.5% of the variation in GOE scores can be explained by by variation in jump distance. Note again that this is weaker than the negative relationship between height and GOE for all skaters excluding Yuzuru Hanyu.
   
3. The correlation between landing speed and GOE is the highest of the three things ice scope measured, but is also very small. (r=0.12, veryyy slightly more than distance) Here's the scatterplot of landing speed versus GOE. Again, the relationship is weaker than the negative relationship between jump height and GOE if you exclude Yuzuru. Landing speed explains only about 1.5% of the variation in GOE.

In essence, if you want to predict how much GOE someone will get for a successful jump, the jump height, jump distance, and landing speed will tell you virtually nothing, whereas the skater's PCS and planned jump BV is a much more reliable indicator.

(Oh, and here are the average stats for all men.)

• Average height: 0.59m (standard deviation: 4.1 cm)
• Average distance: 2.87m (standard deviation: 0.39 m)
• Average landing speed: 14.71 m/s (standard deviation: 3.03 m/s)
• Average GOE: 1.75 (standard deviation: 0.92)

You can find all the data for skaters who successfully completed (positive execution) their 3A in the spreadsheet. Notable figures (highest in each category bolded, second highest italicized):

Also, if someone happens to have a link to a collection of the ladies' and pairs jump data, I would highly appreciate it.

Shanshani said:

Since we actually have an objective measure of jump size now, I decided to compile the data from ice scope for the men's SP 3As and the GOEs they earned into a spreadsheet. For the sake of making the data more comparable, I only included 3As with positive GOE.

Here's what I found. (For comparison, here's a graph of the relationship between PCS and 3A GOE. The correlation between PCS and 3A GOE was 0.66, which is general considered a strong association. The correlation between total planned jump BV across both programs and 3A GOE was 0.58, and the correlation between PCS and total planned jump BV was 0.69)

1. The correlation between jump height and GOE is virtually zero (r=0.03). You can see it in the scatterplot here. In fact, if we remove Yuzuru Hanyu for being an outlier (he cleared the next highest jump height by 5 centimeters, more than one standard deviation), the correlation between jump height and GOE becomes negative (r=-0.23). In other words, if your name is not Yuzuru Hanyu, the higher you jump your 3As, the lower your GOE tends to be, according to the ice scope data for this competition. This effect is stronger than the relationship between jump distance and GOE and landing speed and GOE.
   
2. The correlation between jump distance and GOE is very small. (r=0.11) Here's the scatterplot. While there is a positive relationship, only about 1-1.5% of the variation in GOE scores can be explained by by variation in jump distance. Note again that this is weaker than the negative relationship between height and GOE for all skaters excluding Yuzuru Hanyu.
   
3. The correlation between landing speed and GOE is the highest of the three things ice scope measured, but is also very small. (r=0.12, veryyy slightly more than distance) Here's the scatterplot of landing speed versus GOE. Again, the relationship is weaker than the negative relationship between jump height and GOE if you exclude Yuzuru. Landing speed explains only about 1.5% of the variation in GOE.

In essence, if you want to predict how much GOE someone will get for a successful jump, the jump height, jump distance, and landing speed will tell you virtually nothing, whereas the skater's PCS and planned jump BV is a much more reliable indicator.

(Oh, and here are the average stats for all men.)

• Average height: 0.59m (standard deviation: 4.1 cm)
• Average distance: 2.87m (standard deviation: 0.39 m)
• Average landing speed: 14.71 m/s (standard deviation: 3.03 m/s)
• Average GOE: 1.75 (standard deviation: 0.92)

You can find all the data for skaters who successfully completed (positive execution) their 3A in the spreadsheet. Notable figures (highest in each category bolded, second highest italicized):

Skater	Height(m)	Distance(m)	Landing speed(m/s)	GOE
Yuzuru Hanyu	0.7	3.62	15.3	3.43
Nathan Chen	0.58	2.66	17.1	2.74
Shoma Uno	0.51	3.44	18.3	3.09
Mikhail Kolyada	0.65	2.50	11.8	2.97
Vincent Zhou	0.58	2.69	16.7	1.6
Jason Brown	0.60	2.35	14.6	2.51
Boyang Jin	0.57	2.55	16	2.51
Morisi Kvitelashvili	0.60	3.51	17.0	1.83
Slavik Hayrapetyan	0.64	3.13	17.9	0.11

Click to expand...

So basically what this study says is that judges don’t really give high GOE based on the quality of the jump itself but rather on reputation and better technical content? Wow what a surprise!!!!! *end sarcasm here*

I can't look at the numbers in detail right now, but I just wanted to thank you for doing this work! I really appreciate when folks like yourself go above and beyond to get us actual data to look at instead of just relying on our gut feelings.

Can’t wait for human judges to be replaced.

GoodMorningGrapeseed said:

Can’t wait for human judges to be replaced.

Click to expand...

I think this data makes clear enough why judging the height/distance bullet should, at the very least, be done by computer. We clearly have the technology. Height and distance are objective measurements. Why should this be up to the judges? We can even do cool things like make it fractional. So the higher/further you jump, the more points out of, say, 1 whole point you get.

I've never thought that the judges grade GOE based on the bullet points, regardless of what the ISU document says; I mean, who has the time to go through each bullet point and take 20 seconds to assess each bullet point individually? Basically, it's just about how good the landing position was, and then tack on +1 or +2 if there was height, distance, difficulty and all of that, and subtract for any errors. That's pretty much the long and short of it.

There is one major problem with using the exact measurements of height, distance and speed to relate it to GOE: skaters are different sizes. Obviously technique and quality of a jump directly impact the height and distance, but a skater who is 5'0'' is unlikely to jump as high as someone who is 5'7''. Body type also factors in. Doesn't mean both skaters don't have excellent jumps, just means that they don't have the same physical abilities.

Is 'Axelcam' an official instrument in judging?

Also, GOE reflects things that can be measured objectively (i.e. height, distance, speed into and out of the jump) and those that cannot (landing quality, difficulty of entry, air position, etc.). I don't know how you'd standardize for the latter.

drivingmissdaisy said:

Also, GOE reflects things that can be measured objectively (i.e. height, distance, speed into and out of the jump) and those that cannot (landing quality, difficulty of entry, air position, etc.). I don't know how you'd standardize for the latter.

Click to expand...

This is a great point. However, don't the revised GOE rules say something to the effect of if good height/distance is not present, the other bullets rewarded should be limited? Can't remember the details exactly, but it's something along those lines.

silver.blades said:

There is one major problem with using the exact measurements of height, distance and speed to relate it to GOE: skaters are different sizes. Obviously technique and quality of a jump directly impact the height and distance, but a skater who is 5'0'' is unlikely to jump as high as someone who is 5'7''. Body type also factors in. Doesn't mean both skaters don't have excellent jumps, just means that they don't have the same physical abilities.

Click to expand...

Is this true? There are some tiny ladies with big jumps (Liza - all of her jumps are huge and she is very short), and Shoma is also one of the shortest men and has a high 3A. Midori Ito as well. Alena Kostornaia is much shorter than Trusova and Shcherbakova, but her jumps are not small at all.

Part of sport is physical ability. Some people have better physical ability than others.

It's funny - I thought this height/distance/speed tracker was very interesting to see, but it seems now certain people are losing it online/Twitter about the correlation between height/distance/speed on landing and why it's not translating into better GOE for the skaters they like -- or worse GOE for the skaters they don't like.

Very good height and very good distance is just one bullet of six that corresponds to the GOE of a jump. Just because a skater had superior/inferior height and distance relative to others doesn't mean that determines the GOE of the jump.

For example, what are the transitions leading into it? Is it in line with the music structure? Does the skater have good air position?

Also, are there any deductions elsewhere in the jumping pass? For example Hanyu's 3A is normally a +5 when executed as well as he can.. but this one had a bit of a lean and was scratchy on the landing (forward on the toe pick) - hence the relatively low landing speed), which explains the +4s in spite of having the best height/distance.

And another interesting variable, what is the amplitude relative to the skater's height? How do you compare a relatively diminutive skater (e.g. Uno - 1.58m) who does a 3A with 0.51 m height and 3.44 m distance to a taller skater (e.g. Kvitelashvili - 1.80 m) doing a 3A of 0.6m height/3.51 distance? Presumably a shorter skater jumping the same height/distance of a taller skater will look "bigger" to a judge.

It's also not like they're pulling out rulers... so it's kinda weird that some people are harping on the judges for not taking these stats into account -- stats based on ONE sample size (the World 2019 men's SP), mind you. And no judge is like "Well, so and so jumped this big, and so I need to bear that in mind when I'm assessing very good height/distance for the next skaters."

It makes for interesting discussion and analysis of course... but some of the comments I've seen trying to leverage this data are just cringe-worthy, and fail to account for a ton of variables - the most obvious being other GOE bullets/deductions that factor into the overall GOE a judge assigns a jump.

- - - Updated - - -

silver.blades said:

There is one major problem with using the exact measurements of height, distance and speed to relate it to GOE: skaters are different sizes. Obviously technique and quality of a jump directly impact the height and distance, but a skater who is 5'0'' is unlikely to jump as high as someone who is 5'7''. Body type also factors in. Doesn't mean both skaters don't have excellent jumps, just means that they don't have the same physical abilities.

Click to expand...

Oops, you beat me to it!

This is fantastic! I would love for scoring to be rooted in data/stats. Are these stats publicly available? Is there an API anyone knows of and can share?

Edwin said:

Is 'Axelcam' an official instrument in judging?

Click to expand...

No. Nor should it be, IMO. It's great for public interest, though. I get they use it on the axel because that's the jump everyone does, but some skaters have a weak axel and some skaters have a strong axel, so it does give the impression that a certain skater who has low stats is a poorer jumper in general than one who has higher stats.

This cam has been applied to other jumps when a skater hasn't been successful on their axel, so I wish they would apply that to the skaters' biggest elements. Like, I don't really care about Brown's 3A - I care about the amplitude of his 3F, for example since that was obviously his biggest and best jump.

rlopen said:

So basically what this study says is that judges don’t really give high GOE based on the quality of the jump itself

Click to expand...

There are, of course, plenty of other considerations based on the quality of the jump itself other than size.

satine said:

This is a great point. However, don't the revised GOE rules say something to the effect of if good height/distance is not present, the other bullets rewarded should be limited? Can't remember the details exactly, but it's something along those lines.

Click to expand...

For each type of elements, there are now six possible positive bullet points. Of those, three are considered "mandatory" for earning GOE higher than +3.

If you meet all three of those bullet points and also at least one other, you can earn GOE of +4 or +5.

If you are missing any of those three, then your GOE is capped at +3, even if you have met all five of the other criteria.

It is up to each judge individually to determine which bullet point each jump has met.

1) very good height and very good length (of all jumps in a combo or sequence)
2) good take-off and landing
3) effortless throughout (including rhythm in Jump combination)

The other three are
4) steps before the jump, unexpected or creative entry
5) very good body position from take-off to landing
6) element matches the music

So yes, if your jumps are not very big, you would be missing one of the mandatory bullet points and should not score higher than +3. You could score as high as +3 if you have at least three other bullets.

On the other hand, even if you have the biggest jump ever seen, if you don't have a good takeoff and landing or if the element isn't effortless, you can't earn higher than +3. And you would still need at least two other bullet points, mandatory or otherwise, to get to +3.

Height and distance alone will only get you one +.

Size is not the only determinant of high jump GOE, nor is it the single most important one.

If any criterion could be considered the most important, it would probably be "Effortless throughout," since that one is mandatory on all kinds of elements, not only jumps.

There was Ice Scope analysis as far as I remember where it was shown that this system is not doing measurements of jumps correctly in many cases. Just because it has pretty graphic image over TV translation with sci-fi looking numbers - doesn't mean these numbers should be trusted at all.
So, considering the above, - making any deductions from this data is too early at this stage of this system development. It's just pretty little feature to entertaing viewers (and just really distracting IMO) for now. Hence this topic has no sense.

CanadianSkaterGuy said:

It makes for interesting discussion and analysis of course... but some of the comments I've seen trying to leverage this data are just cringe-worthy, and fail to account for a ton of variables - the most obvious being other GOE bullets/deductions that factor into the overall GOE a judge assigns a jump.

Click to expand...

The human eye is excellent at perceiving a jump's height, distance and speed, and combining it instantly with other perceptions of the jump like rotation, air position and speed ... plus the other characteristics you mentioned like how well a jump expresses the music, tempo, tone, mood. Replacing the human factor ever with machines wouldn't be the way to go.

I really enjoyed the graphics, though. It was interesting, like diagramming a sentence is interesting. Not to measure up whether the judging was done right or not, but as a demonstration of the mechanics. Also, it gives perspective on just how high, far, and fast a jump travels ... something that can be somewhat distorted by watching video. I sometimes stop a replay in the middle of a pairs throw jump, just to help my eye measure what I'm looking at. Seeing the differences, such as the effect of a tall pairs girl like Ashley Cain being thrown to where her feet are at the level of her partner's waist, is kinda fun. So now I pay more attention to the overall effect of a jump, and I notice when the girl's feet only reach to her partner's groin level. I would say I enjoyed the graphics and information on the pairs throw jumps more than the singles.

These stats have been quite illuminating. I wonder if there is technology to determine other stats too like the speed of rotation for a spin or rotational lift, and general average speed across the ice (and if that relates to skating skills as much as we think it does).

rlopen said:

So basically what this study says is that judges don’t really give high GOE based on the quality of the jump itself but rather on reputation and better technical content? Wow what a surprise!!!!! *end sarcasm here*

Click to expand...