5 years of Ice Scope

[Anna K.]

Ice Scope is a tracking system for figure skating that measures speed and distance based on image analysis. This system is only used for media content, not for judging. It was developed by Fuji Television Network Inc. and Qoncept Inc.. and it was first applied in 2019 during the WC in Saitama when data about height, speed, and distance of jumps was added to slow-motion jump replays on TV.
Since then, Ice Scope has been regularly used by Japanese television for media purposes and I know at least one scientific study - Determinant analysis and developing evaluation indicators of grade of execution score of double axel jump in figure skating - about the relationship between the judged score and kinematic parameters of a figure skating element, which is based on data retrieved by this technology.
It's been 5 years since Ice Scope system is known to us all.
Do you have any thoughts about it? Any suggestions or wishes for the future, maybe?

 

[Mathman]

Ice Scope is a tracking system for figure skating that measures speed and distance based on image analysis. This system is only used for media content, not for judging. It was developed by Fuji Television Network Inc. and Qoncept Inc.. and it was first applied in 2019 during the WC in Saitama when data about height, speed, and distance of jumps was added to slow-motion jump replays on TV.
Since then, Ice Scope has been regularly used by Japanese television for media purposes and I know at least one scientific study - Determinant analysis and developing evaluation indicators of grade of execution score of double axel jump in figure skating - about the relationship between the judged score and kinematic parameters of a figure skating element, which is based on data retrieved by this technology.
It's been 5 years since Ice Scope system is known to us all.
Do you have any thoughts about it? Any suggestions or wishes for the future, maybe?

More like this!

If anyone doesn't want to read this long technical PDF, an adequate summary in in the first few lines. Can we use the three easily measureable quantities horizontal distance, verticle height, and landing speed of a jump to predict the GOE? The conclusion of the case study is, yes in the case of horizontal distance, not so much for the other two.

I wonder if this technology can be adapted to measure things like smooth flow and security of the exit edge (not just speed).

 

[Anna K.]

I wonder if this technology can be adapted to measure things like smooth flow and security of the exit edge (not just speed).

This technology only measures distances and, based on this (distance and time), calculates speed.

In theory, we could look for a correlation between specific exit speed/speed regained after exit and with what we understand as "running edge". However, to calculate the speed with this technology, we need to select a specific distance. In case with the "exit speed", this distance is unclear - at least for me. How big section is "exit", exactly?

 

[Mathman]

I am far from an expert in the actual technology. But if you have a fast enough camera, then even the tiniest distance is enough to cslculate the (nore or less) instantaneous velocity. If you have a bunch of these you can measure change in velocity -- that is acceleration plus miniscule changes in direction. And if you have a bunch of those, then you can get a reading of changes of changes of speed and direction -- i.e."wobbliness or jerkiness" (Mathematically, derivatives, 2nd derivatives, and 3rd derivatiives.)

So it seems like quantifying the "smoothness" of the exit should be possible. I would assume that the measurements would be takien starting right from the instant the blade touches down (however that is determined?) and continuing for a few tenths of a second.

I would also expect a negative correlation between jump height and exit speed. If you jump straight up as high as you can, you will not be going very fast along the ice when you come down. If you could measure the angle at which the skater launches into the air, the the distance can be calculated from the height and vice versa. However, skaters, lika basketball players "hanging in the air" for a slam dunk, can fool human observers by maneuvering the arms and legs about the center of gravity.

Edit. I believe that Yuzuru Hanyu wrote his University thesis on this topic. (I also recently discovered that former skater Takihiko Kozuka wrote his amster's thesis on a comparison of figure skating jumps with similar jumps done on land.)

 

[Anna K.]

I am far from an expert in the actual technology.

Take five from another far from an expert (which means, nitpickers are welcome)

But if you have a fast enough camera

I don't think that camera speed has anything to do with this system; pixels (frames of the video) do. According to the article:
In this system, two 4 K cameras (3840 × 2160 pixel) are used to capture skaters’ jumps (30 fps) (Figure 2 ①). By photographing the entire 60 m × 30 m ice skating rink, the system calculates how many centimetres each pixel in the image corresponds to (Figure 2 ②,③). Kinematic parameters are calculated based on the pixels of the images.
Hence the smallest measurable distance would be that provided by 1 frame. In the article it was defined, when they calculated the landing speed:
Landing speed (km/h): Calculate the distance travelled five frames (Figure 3 B) after the frame where the heel edge completely landed on the ice (Figure 3 A). One frame is approximately 0.033 s, as the video is taken at 30 fps. Calculate the landing speed by dividing the travelled distance by the time.

So, if we want that

even the tiniest distance is enough to calculate the (more or less) instantaneous velocity. If you have a bunch of these you can measure change in velocity -- that is acceleration plus minuscule changes in direction. And if you have a bunch of those, then you can get a reading of changes of changes of speed and direction -- i.e."wobbliness or jerkiness" (Mathematically, derivatives, 2nd derivatives, and 3rd derivatives.)

So it seems like quantifying the "smoothness" of the exit should be possible. I would assume that the measurements would be taken starting right from the instant the blade touches down (however that is determined?) and continuing for a few tenths of a second.

we can theoretically add it to the future wishlist, if the technology measures smaller distances/differences between more frames. Currently (or at the moment when the above article was published), the smallest measurable distance is the distance made in 0.33 of a second.

Edit. I believe that Yuzuru Hanyu wrote his University thesis on this topic. (I also recently discovered that former skater Takihiko Kozuka wrote his master's thesis on a comparison of figure skating jumps with similar jumps done on land.)

I wonder if these studies are available in open sources somewhere
I understand that they are originally written in Japanese but there often exists also English versions of scientific works.

 

[Magill]

I wonder if these studies are available in open sources somewhere
I understand that they are originally written in Japanese but there often exists also English versions of scientific works.

Here's the link to a short summary of Yuzuru Hanyu's thesis - in Japanese - from the Waseda University open source repository.
The abstract - a few sentences only - is originally in English.
The first, key sentence: "The wireless inertia sensor type motion capture does not interfere with the operation of the device wearer."
While it addresses key worries expressed by many posters on similar threads here, I am pretty sure Yuzuru checked it thoroughly while skating / jumping himself and I think he can be trusted on this
He used the device not just for assessing the typical jumps parameters, but also to asses the proper take off, prerotation, underrotation etc.

https://waseda.repo.nii.ac.jp/records/64787

Here's a YouTube vid summarizing the content of his thesis

Yuzuru Hanyu Thesis from Waseda | 3D Motion Capture, Physics, Unfair vs AI judging in Figure Skating

In late April 2021, Waseda University published a summary of graduation thesis written by Yuzuru Hanyu. Within days, it's attracted over 20k downloads from a...

www.youtube.com

I am not aware of any official translation. There are unofficial translations to English which I am aware of but as they have not been copyrighted, they cannot be shared, quoted, or even brought up for discussion in public. Maybe it would be worth it to have ISU finance the proper copyrighted translation as the work comes from someone with unquestionable expertise both in theoretical and practical issues of the matter, and I guess many people would be interested to read. The above YouTube video has close to 300 000 views, more than many medal winning performances. Actually, you might be surprised they have not done it yet. Hello, ISU?

 

[Mathman]

I don't think that camera speed has anything to do with this system; pixels (frames of the video) do.

That's what I meant by "camera speed": how many frames per second are recorded. The typical video camera uses about 24 frames per second. But in 2011 researchers at the Massachusetts Institute of Technology, studying particles moving at the speed of light, developed technoplogy that records a trillion frames per second. So we could measure how much ice is covered in each trillionth of a second of a skater's performance.

A couple of years later years later this record was broken by the University of Tpkyo (4 trillion FPS) and the current fastest camera is at the University of Lund (Sweden): 5 trillion.

(I imagine that it would be expensive for the ISU to buy this technology for use in figure skating scoring. But maybe we could butter up the Swedes by naming it the Ulrich Salchow Whiz-Bang camera amd the Lund people would donate one.)

 

[Anna K.]

That's what I meant by "camera speed": how many frames per second are recorded. The typical video camera uses about 24 frames per second. But in 2011 researchers at the Massachusetts Institute of Technology, studying particles moving at the speed of light, developed technoplogy that records a trillion frames per second. So we could measure how much ice is covered in each trillionth of a second of a skater's performance.

A couple of years later years later this record was broken by the University of Tpkyo (4 trillion FPS) and the current fastest camera is at the University of Lund (Sweden): 5 trillion.

(I imagine that it would be expensive for the ISU to buy this technology for use in figure skating scoring. But maybe we could butter up the Swedes by naming it the Ulrich Salchow Whiz-Bang camera amd the Lund people would donate one.)

I see.
It doesn't look to me that we need to measure trillionths of second though. In the above study, they used 3 frames per second (selected by the algorithm I guess) of 24 available. So we already have a reserve.
There are other problematic aspects related to the image analysis of such minor moves: we need a quality image where this move is clearly seen from a proper camera angle. This is not what we always get from videos that are made for media purposes.
The other aspect is manual data input. The system allows measurements but the parameters of measurements need to be set manually. In other words, a human needs to look up the frames, click frame A, click frame B, and click "calculate". Hence it makes sense to reduce data to as few frames as possible and focus on larger distances measured.

 

[Anna K.]

I have not posted any wish of my own yet. Here you go.

Talking about the existing Ice Scope system, I wish Fuji TV had an official Ice Scope website in English (Japanese is also fine, we can use Google translations) where all measurements done during the competitions were made available.
Even if there was as little content as the length and height of jumps and throw jumps already displayed during the competition, it would be great material to view and compare. And it would save a lot of time for superfans who otherwise have to go through the whole video recording of the competition to check these data manually .

Please, Fuji TV, do it! You will popularize you system, which may result in other televisions buying or renting it for them. It will be good for you.

 

[Mathman]

I see.
It doesn't look to me that we need to measure trillionths of second though. In the above study, they used 3 frames per second.

Looking ahead, what would be cool is if the technology could be super-charged to measure not just distances and times, but also changes in these parameters from instant to instant. This would translate into such figure skating comcepts as power/acceleration and control and stability of edges.

Of course we don't need a trillion frames per second unless the skater is moving at lithe speed of light. But even with 24 frames per second these things could be estimated fairly well just by comparing the changes from frame 8 to frame 9 with the changes from frame 20 to frame 21. That way fans could get real-time measurements not only of height and distance, but also a quantified measure on "smoothness of flow out of the landing" and other GOE bullets.

(I am not 100% sure that this would be a good thing, though. Some things may be better judged by humans eye than measured by machines.)

 

[Anna K.]

That way fans could get real-time measurements not only of height and distance, but also a quantified measure on "smoothness of flow out of the landing" and other GOE bullets.

(I am not 100% sure that this would be a good thing, though. Some things may be better judged by humans eye than measured by machines.)

Everything that has a material form can be measured. Another question is, do we want to measure it or not and if such measurements are worth the effort.

If we had measurements (even not necessarily real-time) that would allow predicting GOE and PCS, it would be groundbreaking in the sense that it would prove that GOE and PCS has material basis to start with. We could view GOE and PCS not as something that judges make up in their heads to manipulate results but as something that happens on the ice and has a material form. I think it would be good even if it was merely used in post-competition analysis for the needs of athletes and media and did not influence the actual judging.
After all, it is always interesting to compare what our eyes say with what our measurement tools say

 

[Mathman]

Everything that has a material form can be measured. Another question is, do we want to measure it or not and if such measurements are worth the effort.

A thought-provoking statement and question. We might almost resurrect the riddle , "What is art?"

Can the degree of enigma-ness of the Mona Lisa's smile be measured? (9.9 0ut of 10 on the inigma-ness scale?_Woud it be of any use to analyze the gradation of pigment from one molecule to the next (which I'm sure has been done by art enthusiasts)?

To me, an important distinction between the TES side and the PCS side is that the first measures things that actually happen, while the second is mostly a study in the relation between the performer and the viewer. Different viewers result in different values and different scores.

Computers (and science i general) are godd at breaking something down into its constituent atoms. Human brains seem to be better at putting the atoms together and making the Statue of David out of them. Researchers in artificial intelligence aspire to bridge the gap.

I was always intrigued by the bullet point for "Performance" (under the 5 program components scheme) that read: "the skater makes an invisible connection with the audience." I understand, sort of, what they were trying to say. However, the connection was not only invisible, it also could not be smelled, tasted, heard, or experienced tactilely, and as for the instruments of technolgy, what measurement would we want to make? We can only hope to measure what the skater actually puts on the ice; we cannot measure "connections" between what the skater puts on the ice and the sensibilities of other entities.

On the other hand, there is the "smell of the grease-paint, roar of the crowd" phenomonon that adds up to an enjoyable experience at the theater.

Anyway, the study posted in the OP puts some meat and bones on this topic. I hope other GSers will join in. And thanks to Magill for the Hanyu links.

 

[Anna K.]

It is probably healthy to resurrect riddles like "What is art?" or "What is love?" time by time and apparently each time we'll get a slightly different answer. I don't think that the subject of this topic really requires it though.

A technology is a tool. An application of a tool needs a purpose and this purpose doesn't necessarily have to be a construction of a new Mona Lisa (although Leonardo might have liked this idea, given his obsession with proportions and ratios).

If we have a tool, what do we want this tool to do? It has been discussed in this forum several times. There are certain aspects of element quality that are hard to evaluate by an eye, i.e., prerotations of jumps, correct entry edge of lutz/flip. A technology could help judges in that.

However, a technology can help not only judges, but also athletes and their coaches. In fact, if a technology is used by judges, it should also be available for athletes when they do their homework. Or, if a technology is available, athletes can use it as a "judging simulator" for training even if it is not actually used by judges Such way, a technology can be a tool that helps to improve the quality of skating. And, if a technology can be used to improve their jumps, then why can't it be used to improve their glide?