Modernization of PCS score | Golden Skate

Modernization of PCS score

Elucidus

Match Penalty
Joined
Nov 19, 2017
I think it's a widely known fact that PCS system in its current form is a joke. It's relic of 6.0 past - moreover it's not working as intended but serves as mere reputation indicator. Everyone knows it, I know it, you know it, judges, skaters, coaches knows it. Nevertheless attempt to improve PCS judging was denied in last congress - which concerns me greatly. Still, whether it's accepted unspoken rule or not - it doesn't make it unneccessary to trying fix it eventually. In current form it's just bad for the sport.
Here what I would suggest to change - to address current PCS issues to an extent, at least:

1) IN and CO marks should be abolished entirely. They wouldn't serve their purpose anyway. Noone have clear understanding how they should work - neither viewers nor even judges.Their ambiguity and fuzziness just makes temptation to turn them into reputation tools that much stronger. Their functions can easily be relegated to PE category. "Performance" is much more easy to get and universal concept. It should depict how good the skater sells his program, his level of artristry and whole impression of the performance. It's easy to understand (spectators standing ovations is good sign of great PE mark, for example) and not so convenient to manipulate in that case - everyone can tell whether judges were right or not since great performances are pretty obvious even for layman eyes. Which can't be said about IN and CO marks at all - thus inviting judges to manipulate them as they see fit. And tbh, even in theoretical fair execution of that marks - it's just not possible to interpret them precisely due to their highly subjective nature. IN for example isn't possible to be higher than 5 IMO for anything - since lion share of any program interpretation lies in their music and costume - everything else is just set of standart elements. If the program have clear "down to the earth" theme - is one thing. But what about highly abstract music? It can't be interpreted at all - and any mark for it is just pure fantasy on judges part :shrug:

2) all remaining 3 PCS categories (PE, TR, SS) should be assessed separately and independently. They shouldn't be affected neither by reputation nor age of skaters or their TES mark. For example, marks can be like this:

Kovtun PE 8 TR 6 SS 6
Messing PE 9 TR 8 SS 9
Zagitova PE 9 TR 9 SS 7
Medvedeva (in current form) PE 7 TR 6 SS 7
Kostner PE 8 TR 6 SS 9
Tuktamysheva PE 9 TR 6 SS 6
Trusova PE 8 TR 8 SS 7
Scherbakova PE 9 TR 9 SS 6
Kostornaia PE 9 TR 9 SS 9
and so on.

3) one can say that the above marks are still somewhat subjective and can be easily manipulated. While I agree - here I suggest some mechanisms to address such issues. PE mark is the most subjective of them all. However some new rules can be implemented to keep it at bay. For example for each fall PE mark should be lowered by one - i.e. with one fall it can't be higher than 8, with two - higher than 7 etc. Even for Messing :biggrin: There shouldn't be exceptions. Moreover it should be mandatory rule and not mere recommendation - it should be blocked by program on PC until the judge would mark the PE correctly in case of falls deductions.
For TR - each transition can be counted manually during all skate even by tech panel member and there can be some mandatory mark range of TR for each transitions count.
SS - it's somewhat constant category anyway. Still there can be some rule like the skater with level 4 step sequence can't get SS lower than 7, for example. To encourage junior skaters mostly.

Considering the above tweaks - new PCS score will become significantly less subjective since it is more transparent and easily verifiable (only one of the three components remains somewhat subjective). Couches and skaters would easily understand on what aspect they should work more in case of not great PCS score - which is pretty important too. Also I would revert already existing particular TES rule to an old form:

4) UR rule of jumps - 90 degree UR should be acceptable again. In current form strict UR rule is just bad for the sport. It destroys many talented skaters and gives too much power to tech panel considering huge effect it can have on overall score. IMO if the blades on landing are mostly backwards - it's ok jump. If forward - it's not ok. Simple and easily understandable. Even if some skaters would rotate some of the jumps on ice - let them be. It's still much better than what we have now when visually perfect performance with clean jumps would get low score suddenly. It's awful, confusing and highly detrimental to the sport - for spectators and fans as well as for skaters and couches too.
 

DSQ

Record Breaker
Joined
Apr 14, 2018
Country
United-Kingdom
I don’t know I think PE can be in spite of a fall. I do agree that the 9 cap on PE should be mandatory but otherwise an amazing performance can shine through any mistakes.

As for the new UR rule killing the sport that’s to be seen imo. Some might say it doesn’t go far enough because it doesn’t take into account pre rotation. I think skaters should see it as a new challenge. Skaters who were fully rotating their jumps weren’t getting any sort of GOE advantage over skaters who consistently were on the edge and least under the current rules all jumps are 100% okay.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
I would love to have a thoughtful, nuanced, respectful discussion of program components. Can we do that here, or is hyperbole the order of the day?

A couple of points:

For example for each fall PE mark should be lowered by one - i.e. with one fall it can't be higher than 8, with two - higher than 7 etc. ... Moreover it should be mandatory rule and not mere recommendation - it should be blocked by program on PC until the judge would mark the PE correctly in case of falls deductions.

This would only make sense if everybody were starting out with scores in the 8s and 9s.

That is not true even when you look at the whole field at Grand Prix events or World Championships, let alone lesser senior level events. (Theoretically juniors or lower levels could use different rules than seniors. Currently all levels follow the same rules for program component scoring, although the factors or the number of components used may be different at lower levels in domestic events in different federations.)

Way back ca. 2003-04 it was exactly the rule that judges were supposed to lower the PE score by 1 point for each fall. That didn't happen consistently and even if it did it would be impossible to know whether judges were actually applying it or not. Which is why the ISU removed that rule and replaced it with the mandatory 1.0 fall deduction taken off the Total Segment Score.

Suppose Skater A falls once and earns a PE score of 8.0, for example, from Judge Z. How do you know whether that judge felt that
*the performance as a whole, as actually performed, was worth 9.0 even with the fall, but subract 1 point just to comply with the rule
*would probably have been worth 9.0 without the fall, so just subtract 1 point automatically
*the performance as a whole, as actually performed, was worth 8.0, which reflected the fall and every other good and not-so-good quality of that performance, so no need for an additional subtraction
*the performance as a whole as actually performed, including the fall, was worth somewhere around 8.5, so score it 8.0 to comply with the rule

Judge Z also awards 8.0 to skater B who doesn't fall. We have no idea whether she thought the two performances were identical in overall performance quality or honestly thought that A was still better overall but the rule doesn't allow her to reflect that considered opinion.

Meanwhile, lower in the standings, above-average but not medalworthy skater C is having a pretty good day and judge Z is thinking the PE score should be 7.0. Then skater C falls on the last element or the closing pose, but in such a way that doesn't ruin the performance in Z's opinion. So she still awards the 7.0 (no subtraction), or maybe 6.75 -- both legal if the rule is no score higher than 8.0 for one fall. Not that this skater was going to earn 8.0 even without it.

And skater D is overall not as strong at performance but doesn't fall. Judge Z awards 6.0 for PE.

Maybe judge Z thinks that the difference in overall performance quality between A and B is approximately the same as between C and D, even taking into account the falls by A and C. Maybe this judge would happily score skater A well above 9 with no falls. However, this rule requires that the rule-required score for skater A be no higher than the judge's honest score for skater B, whereas the judge is perfectly free to reflect her honest opinion about the similar difference between C and D.

And at the bottom of the standings (let's say we're at Four Continents or a lesser senior B event), skater Q is maybe able to perform well enough to earn 4s for PE on a good day. This is a bad day technically and he falls 4 times. Is the judge required to give a PE score in the 0s? Or would any score lower than 5 be legal -- even the same 4.5 this judge might give this skater on a good day?

For TR - each transition can be counted manually during all skate even by tech panel member and there can be some mandatory mark range of TR for each transitions count.

Easier said than done. Care to design a rubric for counting (and evaluating the quality of) every possible move between elements and leading into and out of elements that skaters might perform during a freeskate? And how the tech panel member busy counting between elements is supposed to keep up with the rhythm of the element calling at the same time?

SS - it's somewhat constant category anyway. Still there can be some rule like the skater with level 4 step sequence can't get SS lower than 7, for example. To encourage junior skaters mostly.

There are skaters who can achieve the required features for level 4 step sequences but whose power, depth of edge, etc., as well as the difficulty and quality of the skating in the other ~3 minutes of the program, are nowhere near 7.0 "good" quality.

4) UR rule of jumps - 90 degree UR should be acceptable again.

This has nothing to do with program components, so it's a discussion for a different thread.
 

moriel

Record Breaker
Joined
Mar 18, 2015
The last congress showed us that ISU is not interested in transparent and fair judging.
Higher GOE weights, for instance, is another place where judges can manipulate scores.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
In evaluating the processes of evaluation adopted by the ISU over the past century and a quarter, I think we need to take into account that:

*All skating evaluation is qualitative, and therefore "subjective" in the sense that it is subject to individual human perception and human thought processes, which will not be identical even between two judges or two fans with identical training who are sitting right next to each other (or watching the same video).

*Qualitative assessments are continuous variables. Numerical scores are digital. The process of turning qualitative assessments into digital scores is also necessarily subject to individual human thought processes.

*Evaluating the quality of a single school figure or a single jump or spin or other discrete technical move can be relatively straightforward, although some elements are complex enough that each skater's execution can demonstrate an individual mix of strengths and weaknesses even within single elements. Human judgment is required to determine how to balance different strengths and weaknesses in varying degrees into a single score.

*There are some things that can be quantified by counting or measuring, but even determinations of which skills are more difficult than others and by how much is a subjective process.Everyone is going to have a slightly different opinion about what should be worth more or less or not considered at all. The Scale of Values and the breakdown and multiplying factors for program components are a consensus of opinion (of the technical committees and the federation representatives at the ISU biennial meetings) about relative difficulties and importance of various elements.

*Whole programs are far more complex than isolated elements. There are many many variables that go into evaluating just one aspect of a whole performance, let alone combining several (or all, under 6.0 judging) into a single score. Again, individual human perception and thought process and value judgments will always come into play in the honest evaluation.

All this being true, any attempt to design a scoring system for whole programs will necessarily be subjective. Someone's opinions are going to factor heavily into all decisions. If any one of us tries to design a system on our own, the system that we come up with will reflect our personal opinions. Input and negotiation among experts with different opinions will result in a better consensus than only one person's point of view, but at some point decisions need to be made and some individuals' opinions will be overridden or left out.

And whatever system is established, the officials tasked with implementing that system are going to have their own perceptions of what they see and their own methods of integrating the various aspects of a complex performance

You can't have a judged sport without human judgment. And you can't capture quality without relying on human judgment.

With that reliance comes differences of opinion, i.e., subjectivity.

Is it preferable to focus the scoring system primarily on those aspects of the sport that can be quantitatively measured and counted, while minimizing focus on what makes a performance "good" or "better"?

Or should the focus of the scoring system, especially the scoring for aspects of each program as a whole, be on finding ways to allow experts to exercise their educated judgment about as many qualities as possible among those that the skating community considers intrinsic to good skating, and to find mathematical ways to build a consensus out of differing valid opinions?

And what would constitute "transparency" when it comes to those experts rendering their qualitative judgments?
 

Elucidus

Match Penalty
Joined
Nov 19, 2017
And at the bottom of the standings (let's say we're at Four Continents or a lesser senior B event), skater Q is maybe able to perform well enough to earn 4s for PE on a good day. This is a bad day technically and he falls 4 times. Is the judge required to give a PE score in the 0s? Or would any score lower than 5 be legal -- even the same 4.5 this judge might give this skater on a good day?
The rule I proposed would only put top limit on how much you can award the skater with PE score in case of falls. It's not actually points deductions. In case of 4 falls the max limit would be 8-4=4. Feel free to try to convince me that performance with multiple falls can be solid anyway (I would strongly disagree with it despite borderline cases such as Messing skates under Charlie Chaplin) - however the rule is directed against cases such as Kostner's or Medvedeva's disaster skates at Euro/RusNat with sky-high PCS - since it's just blatant favoritism and nothing else.
In case of weak skaters with potential 6 or 7 as PE mark - one or two falls shouldn't make any difference and TES points deductions should be enough penalty for it. It's up to judges whether lower the PE mark for it or not (and they probably should) but it's not rule in that case. Rules are supposed to block any extremal abusing of the system - and in case of weak skaters usual PCS recommendations and common sense would do the work, I believe. You should understand though that the exact numbers are not final. It can be changed to "in case of one fall no more than 9 PE, two falls - 8 PE and so on". What matters here is principle of limiting PE mark with each fall more strongly.

Easier said than done. Care to design a rubric for counting (and evaluating the quality of) every possible move between elements and leading into and out of elements that skaters might perform during a freeskate? And how the tech panel member busy counting between elements is supposed to keep up with the rhythm of the element calling at the same time?

Just put the extra guy in tech panel to only count transitions number and nothing else. Quality of transitions shouldn't matter (isn't the quality more related to SS anyway?) since the guy is there only to determine TR mark limit range (let's say the skater with only 10 transitions can't be awarded with TR higher than 6 for example, or for example that 30+ transitions guarantees range between 9-10 TR - something like that) - final mark, quality of transitions etc. is for the judges to assess. They just couldn't award the TR mark outside of the range the tech panel provides. In the end TR range would be pretty constant value for tech guys as well since the program and number of transitions are pretty much the same for particular skater during the season. The idea is just to avoid significant manipulation of the score regardless of actual skate. It allows for example to get high enough TR mark even for junior skaters - which fixes "reputational judging" to a degree already.

There are skaters who can achieve the required features for level 4 step sequences but whose power, depth of edge, etc., as well as the difficulty and quality of the skating in the other ~3 minutes of the program, are nowhere near 7.0 "good" quality.
In that case the rule is about low limit. In case of Lv4 StepSequence it's really hard to imagine the skaters with lower than 7 SS doing it. Even if there are some skaters like that currently - most likely it's due to the fact that the current PCS are not awarded properly anyway. Noone assess actual SS properly when watching some junior's skating (which I am trying to fix here). I believe the skater who can do it - regardless of age or reputation should be awarded at least 7 SS since ability to meet requirements is clear sign of high-level skating skills (Lv4 requirements includes quality of turns, edges etc. too btw). On other hand the skater who can do only lv3 StpSqc - can still get even 10 SS. It's up to judges in that case. As I said before - the purpose of such rules is to avoid abusing, favoritism or negligence to an extent - not to release the judges from their work )

Suppose Skater A falls once and earns a PE score of 8.0, for example, from Judge Z. How do you know whether that judge felt that
*the performance as a whole, as actually performed, was worth 9.0 even with the fall, but subract 1 point just to comply with the rule
*would probably have been worth 9.0 without the fall, so just subtract 1 point automatically
*the performance as a whole, as actually performed, was worth 8.0, which reflected the fall and every other good and not-so-good quality of that performance, so no need for an additional subtraction
*the performance as a whole as actually performed, including the fall, was worth somewhere around 8.5, so score it 8.0 to comply with the rule
Is that really that important? Noone cares about what judges were actually think - there isn't any way to know for sure and will never be. What matters is only final score and how fair it reflects the quality of actual skating in the eyes of majority.
 

Miller

Final Flight
Joined
Dec 29, 2016
In terms of simplifying Program Components I did an exercise after last years Cup of China, the 2017 one, where I looked at all the values for all the skaters across all the disciplines - the thread is still out there somewhere, but the result of looking at the raw data was that it was quite possible to reduce the components to at least 4, and quite possibly 3 i.e. SS, TR and a combined PE/CO/IN that was worth 50% of the marks with SS and TR 25% i.e. only just different from the current 20%. Alternatively you could have SS, TR, PE and a combined CO/IN that were all worth 25%.

The reason for this was that all the values were very similar in the first place - there was never any gap bigger than 1.00 between the highest and lowest component scores for any skater/couple. TR was always a bit lower than SS (there was no obvious reason for this), while PE, CO and IN were always very similar, except where the skater made serious mistakes and it seemed to affect the rest of their performance in which case PE would be lower, but never more than about 0.50. Finally CO and IN were almost always identical, and especially so in Pairs and Dance. The biggest gap only ever occurred for really expressive skaters like Elena Radionova when it might be 0.25. The reason for this was speculated that it might be that CO could reflect the ability of a skater/their federation to afford an expensive choreographer and there might be some unwritten rule at work to not necessarily reward a skater for this, just keep it at the same level as IN.

Also since then whenever I've looked as PCS values I've always found IN and CO are so close they might as well be one, I don't think I've ever seen anything bigger than about a 0.35 difference, which is not many marks at all when you think about it - in Men's each 1.00 in PCS component works out at 2 marks in the final LP total so even the max of 0.35 in the highest scoring discipline of all works out at only 0.7 marks, and more often than not it is much much smaller than this.

Hence just purely on a numbers basis you could combine the 5 components into at least 4, and possibly 3, and not get the kind of 'corridor judging' that you tend to get - with 3 components certainly, it should certainly be possible to concentrate on each one individually - the so called 'rule of 3', and not do something like think of what SS should be then plus or minus it which seems to be what you often get, though I do note that Elizaveta Tuktamysheva had a couple of very low TR values from a couple of judges at Skate Canada, 6.25 in the SP, which is a good sign that at least some judges are prepared to stick their necks out a bit, plus it also ties in with what the OP is trying to hopefully achieve i.e. SS, TR and whatever else judged separately and hopefully more accurately rather than being all one and the same which is what the COC exercise tended to reveal.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
In case of weak skaters with potential 6 or 7 as PE mark

The definitions of 6 and 7 are "above average" and "good."

"Weak" is 3.

https://www.usfigureskating.org/content/PC Chart combined SPD.pdf

The scoring system needs to work for the whole range of skill levels it applies to, not only to above-average seniors and outstanding juniors.

It can be changed to "in case of one fall no more than 9 PE, two falls - 8 PE and so on".

That would be saner, and closer to what the ISU actually instituted this year (after having nothing to say about the effect of falls on PCS since ca. 2004 or 05 when they introduced the fall deduction.

Just put the extra guy in tech panel to only count transitions number and nothing else. Quality of transitions shouldn't matter since the guy there only to determine TR mark limit range (let's say the skater with only 10 transitions can't be awarded with TR higher than 6 for example, or for example that 30+ transitions guarantees range between 9-10 TR - something like that)

So the limits are based solely on quantity? The tech panel counter would have nothing to say about quality, variety, or difficulty of the transitions, or the intricacy or continuity with which they are woven into the program?

I would like to investigate more closely how something like this might work and how it might actually lead to "better" evaluations of skating transitions, as opposed to simply "more quantitative/objective" evaluations. Since I personally do not think that counting quantity comes close to telling the whole story, and obviously neither does the ISU, I'm not sure that restrictions based on quantitative counting would necessarily lead to better evaluation. But it would be interesting to try to brainstorm ways to make Transitions scoring better, without deciding at the outset that focusing on quantity is the way to go.


In that case the rule is about low limit. In case of Lv4 StepSequence it's really hard to imagine the skaters with lower than 7 SS doing it.

At 2018 Junior Worlds, eleven men and two ladies achieved StSq4 in the short programs. Their average Skating Skills scores ranged in the low 5s to low 7s.
In the free skates, there were five men and four ladies who achieved it, with SS scores ranging from high 6s to high 7s.

Also at 2018 Worlds SP, one man who achieved level 4 placed 33rd in the SP with 6.11 SS and one lady placed 23rd with 5.61 SS.

Even if there are some skaters like that currently - most likely it's due to the fact that the current PCS are not awarded properly anyway. Noone assess actual SS properly when watching some junior's skating

Who is "no one"?
I get the impression that you're not very familiar with junior or non-elite senior skating. Try attending some large non-elite events to get a better sense of the full range of skating skills at these levels and below, and take that into account when designing your system.

Or at least watch some whole JGP or Junior Worlds or senior B events on video, although Skating Skills and some aspects of Composition are the hardest to distinguish on video.

Before you decide that people must be doing something wrong, try learning what they know and doing what they do from the inside and see what would really be involved in doing it right, or how "right" would best be defined.
 
Joined
Jun 21, 2003
Here are my thoughts on this subject. First, the idea of combining Presentation, Choreography and Interpretation into one mark (we could call it "the second mark" for short ;) ) is a good idea. There is no reason to have three repeated scores all the same when one will do.

However, the following part of the proposal needs to be worked out in greater detail. IMHO:

Their functions (Choreography and Interpretation) can easily be relegated to PE category. "Performance" is much more easy to get and universal concept. It should depict how good the skater sells his program, his level of artristry and whole impression of the performance. It's easy to understand (spectators standing ovations is good sign of great PE mark, for example) ...

I think that if we judge skaters on "level of artistry," then we are back to square one as far as objectivity is concerned. Plus, we do not want to see figure skating competitions won or lost by how loudly the audience claps for their favorites (or by viewers voting on the Internet, etc.).

Transitions:

Just put the extra guy in tech panel to only count transitions number and nothing else. Quality of transitions shouldn't matter since the guy there only to determine TR mark limit range (let's say the skater with only 10 transitions can't be awarded with TR higher than 6 for example, or for example that 30+ transitions guarantees range between 9-10 TR - something like that).

I agree with gkelly that just counting how many Mohawks a skater does, without regard to variety and quality, is missing the whole point of this component. I would like it better if SS and TR were combined into one component (called, in my favorite ISU phrase, "the full skating vocabulary"). So it would go like this:

Elements (including GOES) 50%
Presentation 25%
SV (skating vocabulary) 25%

There would still be plenty for the judges to do and for fans to argue about. Is this skater's Charlotte more praiseworthy than that skater's split jump?

[Weird factoid that I found out by clicking around the Internet. Professional show skater Charlotte Oelschlagel's name (oil-beater) refers to someone whose job is to extract oil from olives. :) ]
 
Last edited:

Elucidus

Match Penalty
Joined
Nov 19, 2017
The definitions of 6 and 7 are "above average" and "good."

"Weak" is 3.

https://www.usfigureskating.org/content/PC Chart combined SPD.pdf

The scoring system needs to work for the whole range of skill levels it applies to, not only to above-average seniors and outstanding juniors.
Ok. Anyway for such weak skaters their PE mark would be affected only with 5+ falls. I don't see any unfairness here.

So the limits are based solely on quantity? The tech panel counter would have nothing to say about quality, variety, or difficulty of the transitions, or the intricacy or continuity with which they are woven into the program?

I would like to investigate more closely how something like this might work and how it might actually lead to "better" evaluations of skating transitions, as opposed to simply "more quantitative/objective" evaluations. Since I personally do not think that counting quantity comes close to telling the whole story, and obviously neither does the ISU, I'm not sure that restrictions based on quantitative counting would necessarily lead to better evaluation. But it would be interesting to try to brainstorm ways to make Transitions scoring better, without deciding at the outset that focusing on quantity is the way to go.

As I already said above (I edited the post) I have a feeling that quality of transitions is more related to SS than TR. Skater with great SS would do great transitions whether they are small in numbers or not - wouldn't he? Since basically transitions are just basic skating elements which are related to SS. In that case their quality should be already assessed by SS mark and what remains is only their number. How they are vowen into jumping elements is determined by jumps GOE bullet point too.
Nevertheless what I meant is that the tech panel should help only with determining the range of possible TR mark. The quality of transitions and the final mark should still be judges prerogative. They just will be limited to what extent they can manipulate the score.
I get the impression that you're not very familiar with junior or non-elite senior skating. Try attending some large non-elite events to get a better sense of the full range of skating skills at these levels and below, and take that into account when designing your system.
Low-level skating is pretty middle-case whereas the suggested rules directed against system abusing and favoritism- which is manifested mostly at top level. Low-level skaters should be fine. If anything - they can only benefit from it since they can get somewhat more PCS than usual.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
As I already said above (I edited the post) I have a feeling that quality of transitions is more related to SS than TR. Skater with great SS would do great transitions whether they are small in numbers or not - wouldn't he?

The quality of what he does with the blade would probably be high, although the variety and difficulty and continuity from one move to the next might be low. However, judges should be free to reflect that in their scores.

Or vice versa.

But let's say you have two skaters with similar skating skills. One has great flexibility and extension and the other is fairly musclebound.

They both perform the same spirals and spread eagles and split jumps in their programs. The first hits beautiful positions and the second does not. Shouldn't the former earn a higher TR score than the latter, on the strength of quality, even if the SS score is the same and so is the countable content of the transitions?

Since basically transitions are just basic skating elements which are related to SS. In that case their quality should be already assessed by SS mark and what remains is only their number.

Not at all.
Variety and difficulty are also part of the TR component. It might be possible to come up with some quantifiable metric for these criteria, but that adds more complexity to the numbers than just quantity and quality.

And then you have intricacy (old guidelines) or continuity of the movements (new guidelines).

Skater A does a spread eagle, crossover to back outside edge, step forward into double axel.
Skater B does a spread eagle directly into a double axel.

They have both done spread eagles (and both done double axels). The quantity of the same. The quality of the spread eagles may be the same -- or A might even have a better spread eagle on its own merits. But B has more intricacy in the connection between the spread eagle and the 2A and deserves to be rewarded for it.

Skaters C and D both do the same series of turns into a triple loop. C goes from the preceding strokes into the turns with no hesitation and jumps directly from the exit of the last turn, all with a relaxed but energized upper body. D hesitates for a second after the preceding strokes and sets her arm/shoulder/back position stiffly before starting the series of turns; on the final back outside edge she again hesitates, possibly letting both feet glide on the ice for a moment while she sets her arms and shoulders, and then takes off for the loop.

Both have performed the same steps and the same jump. But C had a lot more continuity to her movements than D did.

How they are vowen into jumping elements is determined by jumps GOE bullet point too.

Yes, that can be true. Although if the rotation and landing are failed badly enough (by both skaters, let's assume) to warrant straight -5s, that might cancel out any advantage that the skater with the more intricate/more continuous entrance.

Or more difficult entrance, as the case might be.

Nevertheless what I meant is that the tech panel should help only with determining the range of possible TR mark. The quality of transitions and the final mark should still be judges prerogative. They just will be limited to what extent they can manipulate the score.

What is the difference between "manipulating the score" and "applying judgment to award an appropriate score"?
How can a predetermined rubric pass judgment on thought processes that haven't taken place yet?

Low-level skating is pretty middle-case whereas the suggested rules directed against system abusing and favoritism- which is manifested mostly at top level. Low-level skaters should be fine. If anything - they can only benefit from it since they can get somewhat more PCS than usual.

It could be useful to come up with guidelines for how to score TR for skaters with high quantity but low quality or vice versa, for high quality but low difficulty, high variety but low quality, etc. All of which would necessarily be more complex and should be determined by more flexible methods than just counting.

Should we look at some examples to brainstorm what might work?
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Reading all those criterias (https://www.isu.org/figure-skating/...96-program-component-chart-id-sp-2018-19/file) judges are looking for when they are scoring PCS, i don't see how Interpretation of the music's, Composition's and Performance's criterias could be combine in one score. The only way to make program components composed of lesser number of categories is to put Composition's criterias in other categories and go with 4 categories of PCS. Purpose/idea/mood can be a part of PE, Pattern/Ice coverage part of SS, Use of space and design of movements part of TR, Phrase and form part of IN, and Originality part of TR, PE or IN. So, i think they could go with 4 categories: SS, TR, PE, IN by that logic...
I don't see how they can make some of those categories to be judged on more objective merrits knowing that process of the judging as a human activity can't be 100% objective. Actually, i think a lot of those categories are indeed judged on objective merrits! You can see those objective merrits - by sitting in the judges position in the arena (not all of that by the TV screens tho) - in lot of criterias of SS and TR, but also in PE - you can see skaters involvement, projection, personality in his/her body movements and facial expression and you can see variety in movements and energy, and also by hearing the audience's response, in CO - you can see the pattern's intricacy, ice coverage, and variations/spatial design in body movements, in IN you can see if movements are in time of the music beat/rhythm. You can see how much time of the program skaters are working to fullfill those criterias and how much time of the program is spent just as preparation for the required elements/jumps. So, a lot of those criterias are defined on objective merrits and just a small number of criterias are defined in a way to actually include judges subjective preferences. The only problem is that every judge can't see everything of those in that amount of time, but that is not because they are too 'subjective', but because their cognitive systems are not that perfect.
 
Joined
Jun 21, 2003
The only way to make program components composed of lesser number of categories is ...

To me, that's the problem in a nutshell. All these subcategories and bullet points are just blah, blah, blah. They supposedly offer some artsy-sounding words of guidance, but I don't think that experienced professional figure skating judges pay much attention to them.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
To me, that's the problem in a nutshell. All these subcategories and bullet points are just blah, blah, blah. They supposedly offer some artsy-sounding words of guidance, but I don't think that experienced professional figure skating judges pay much attention to them.

Well, i don't think they sound or they are defined to be that artsy at all... But overal impression of some observer who is watching a person skating to those 'not so artistic' criterias can be that he is watching something artistic indeed :biggrin:
 

Elucidus

Match Penalty
Joined
Nov 19, 2017
But let's say you have two skaters with similar skating skills. One has great flexibility and extension and the other is fairly musclebound.
It's debatable, actually. I would say that quality of transitions is fully dependant on SS. If you know how to move your blades properly - quality of all nonjumping elements will improve accordingly - including continuity and effortlessness. Flexibility is another matter - but it's more related to spins anyway, with their own GOE bullets.

They both perform the same spirals and spread eagles and split jumps in their programs. The first hits beautiful positions and the second does not. Shouldn't the former earn a higher TR score than the latter, on the strength of quality, even if the SS score is the same and so is the countable content of the transitions?
Most likely - yes.

Skater A does a spread eagle, crossover to back outside edge, step forward into double axel.
Skater B does a spread eagle directly into a double axel.

They have both done spread eagles (and both done double axels). The quantity of the same.
No, the quantity is not the same. Skater A did more transitions already, didn't he? With more higher number of transitions variety criteria is followed automatically, btw ;)

Skaters C and D both do the same series of turns into a triple loop. C goes from the preceding strokes into the turns with no hesitation and jumps directly from the exit of the last turn, all with a relaxed but energized upper body. D hesitates for a second after the preceding strokes and sets her arm/shoulder/back position stiffly before starting the series of turns; on the final back outside edge she again hesitates, possibly letting both feet glide on the ice for a moment while she sets her arms and shoulders, and then takes off for the loop.

Both have performed the same steps and the same jump. But C had a lot more continuity to her movements than D did.
Well, C would hit jump's positive GOE bullets for effortlesness and creative entry, while D only for entry - maybe. The thing is - many criterias of each PCS category are duplicated in other categories and thus are becoming superfluous. Simplicity is the key to pefrection. Current system is too complex and in dire need to be more accessible for skaters, spectators, judges.

What is the difference between "manipulating the score" and "applying judgment to award an appropriate score"?
How can a predetermined rubric pass judgment on thought processes that haven't taken place yet?
...

It could be useful to come up with guidelines for how to score TR for skaters with high quantity but low quality or vice versa, for high quality but low difficulty, high variety but low quality, etc. All of which would necessarily be more complex and should be determined by more flexible methods than just counting.

Hm, I see your point. I agree that need to separate the work of TR assessment into both panels looks.. cumbersome and not very intuitive. You know what? We can transfer all TR assessment job to tech panel entirely, can we? Let it be completely numerical value of quantity of transitions and that's it. Figure skating is the sport and in need to have more objective values if we want that this sport will be respected among others. The eternal stigma of some ice show/ice dance travesty is more than enough already. If you think about it - only a mere point or so divided Zagitova's deserved victory and Medvedeva's reputational victory. If some judge would have a little bit different mood that day - the greatest victory would turn into greatest shame this sport can have. When fair efforts are not awarded. The same can be said about RusNat results.. if not the accidental falls of senior top skaters - the young quadsters would never have their fair victory ever. The falls were lucky in a sense - that let the figure skating keep it's fragile image of fair sport a little bit more. But how much long this luck will last? Why this sport should depend on favorites falls or challengers quads to be able to even compete on equal terms with reputational monsters? I want that each and every young skater would know firmly - your efforts will be awarded, learning difficult jumps is not in vain, this is still a sport. I am deeply convinced that difficult programs should win. And I am always frustrated when I see that some reputable skater with much more easy content winning over tremendous work and efforts of others put in their performance - only because the first have somewhat more beatiful dress, and maybe some gloss and polish in his movements. All of that is disgusting. Example? Tarakanova and many others great skaters humiliated and brushed off - all in favor of Medvedeva's training run-through masked somehow as performance of her life. Just.. a disgrace for fs as a sport discipline.
That's said, returning to previous point, I think that the main problem of current PCS is their complexity. Judges just doesn't have enough time to assess all that "variety and fluidity" crap - especially in real time. They have enough work with assessing current elements GOE's. I am 100% sure that they didn't even try to assess all PCS categories according to guidelines - they just don't have time for that. They just put numbers predermined earlier, before the skate, for each skater -looking in some special reputational chart, or something ) With maybe little corrections in case of falls ) Therefore all your objections regarding proper assessing of so called variety etc. are just.. not very convincing ) Considering that all that assessment never worked in the first place (there are tons of examples when skaters with poor or little number of transitions were awarded with high TR mark and so on) - what makes you so certain that they will work in a new system?
If ISU just don't want to keep extra panel of judges for PCS marks - which last congress showed (and they have valid reasons for that albeit mostly financial) - I would say let's ease their work! Certainly they would have enough time to assess properly only two PCS category after the skate, wouldn't they? Some TR qualities such as fluidity etc. can be relegated partly to SS, partly to GOE bullets - others can be abolished as superfluous. Of course you can say that with such mechanical approach where only number of transitions matters - everyone and their moms will begin to pack their programs with as much transitions as possible - and that wouldn't be very good for variety of skating. While it's definitely a valid point - I have to remind you that currently it is already works in a sense. I mean, let's take Zagitova's programs for example. Why do you think she could get so high-sky PCS for her clean programs? While I agree that reputation is playing a part of it - I would name main reason though - transitions. Skating with so many transitions is insanely difficult. I doubt there are any other senior skater existing who could repeat her programs at all. Judges knows it and they feel compelled to award it properly. On other hand having so many transitions don't leave any time to hold poses, show gliding etc. So SS mark should suffer accordingly. If judged properly (and there is a hope with fewer PCS categories that it can be the case) there wouldn't be so much motives to put too many transitions knowing that you risk your SS mark. Also exact quantity of transitions for each TR mark is yet to be defined - so it can be that the number for 9-10 mark wouldn't be that high. And even if it will - it can become one extra motivation to develop figure skating in that direction too. Why not? ) I would gladly observe how new programs would change - changes means progress after all.

i.e. SS, TR and a combined PE/CO/IN that was worth 50% of the marks with SS and TR 25%
I would change it to PE 25%, TR 25%, SS 50%.
Figure skating is not ice show. Pantomiming and dresses shouldn't win the competitions. Skating skills should matter more. After abolishing figures school - skating abilities of athletes degraded tremendously. It's one way to fix the situation. With great SS great PE will follow anyway even with mediocre programs (example: Kostornaia).

I think that if we judge skaters on "level of artistry," then we are back to square one as far as objectivity is concerned. Plus, we do not want to see figure skating competitions won or lost by how loudly the audience claps for their favorites (or by viewers voting on the Internet, etc.).
PE will always be the most subjective part of skating marks, I am afraid. It's not possible to change it as long as we want the figure skating be artistic sport and not just jumps. All we can do to keep high level of objectivity is to reduce the weight of PE mark.
Also, maybe you will be surprised, but number of votes/views is pretty objective way to assess performance strength. There are tons of great and classical music pieces and songs with millions of hits which proves my point.

I agree with gkelly that just counting how many Mohawks a skater does, without regard to variety and quality, is missing the whole point of this component. I would like it better if SS and TR were combined into one component (called, in my favorite ISU phrase, "the full skating vocabulary"). So it would go like this:

Elements (including GOES) 50%
Presentation 25%
SV (skating vocabulary) 25%

There would still be plenty for the judges to do and for fans to argue about. Is this skater's Charlotte more praiseworthy than that skater's split jump?

Well, that's one way to change the system too, ok. However I disagree that repeating only mohawks over and over would do the trick. The PE mark is bound to suffer significantly in that case. It's not all that simple since all PCS categories are dependent on each other - and lack of something in one will definitely be reflected in other. So in the end the final mark should be balanced. In theory, at least.
On other hand if repeating mohawks would still be beautiful and interesting to watch (which is pretty much impossible - but let it be for arguments sake) - let it be awarded anyway. Variety for the sake of variety is not a good thing either. Everything should have its own purpose, each tool you are using should let you show what you want. If you can draw beautiful drawing with only mohawks - I would only applaud to such high-level "painter" and he would be worthy every points he would get :)

P.S.: btw TR and SS while having some similarities - are still too different things, even contradicting at times - so the idea of combining them is.. debatable, to put it mildly. For example too many transtions destroys opportunities for gliding and vice versa. In your system the judges wouldn't see any difference between Kostner's and Zagitova's skating.. which isn't right in my opinion. Not that there isn't any difference already by current system PCS between them but still :biggrin:
 

gkelly

Record Breaker
Joined
Jul 26, 2003
Thanks for the interesting discussion.

I would say that quality of transitions is fully dependant on SS.

Dependent yes, but not "fully dependent." Especially the quality of the edge-based transitions.

But transitions also very much include glides in flexibility positions, in which case both the edge quality and the quality of the positions contribute to the overall quality of the move.

Transitions also include nonlisted jumps, some of which (e.g., split jumps) highlight the position in the air. Better positions make for a higher quality move. Without much direct relation to skating skill.

Flexibility is another matter - but it's more related to spins anyway, with their own GOE bullets.

For this purpose, I'm talking about flexibility in transitional field moves between the elements.

A skater with better flexibility is likely to showcase in it both in transitional moves and also in spins. But that doesn't mean there's no point in rewarding it outside of spins.


No, the quantity is not the same. Skater A did more transitions already, didn't he? With more higher number of transitions variety criteria is followed automatically, btw ;)

I hope your wink here means you're joking.

Crossovers and most common/easiest the step or turn to get onto the takeoff edge of the jump are generally not counted as transitions.

Jumping directly from a spread eagle position requires a difficult shift of balance. Putting the extra step or two in between to get back to the easy standard entry removes that difficulty.

I agree that need to separate the work of TR assessment into both panels looks.. cumbersome and not very intuitive. You know what? We can transfer all TR assessment job to tech panel entirely, can we? Let it be completely numerical value of quantity of transitions and that's it.

OK, so how do you define a transition?
And do you really count one very easy move as being worth exactly the same as one very difficult move (for purposes of this score)? If quantity is the only criterion, one that the ISU has not included at all in their definition, it would be pretty much a meaningless score.
Add a "difficulty and variety of the connecting moves" bullet to the Skating Skills score, let the intricacy of transitional moves (or other elements) leading directly in or out of elements be reflected in the GOEs alone, let the quality of the in-between moves be reflected in Performance/Execution, and let the continuity of movement be reflected in both SS and PE.

But that's not going to make those components any more objective than they are now; it will make those components them slightly more complex and therefore more subject to differences of opinions about how to weight the various criteria they comprise.
If some judge would have a little bit different mood that day - the greatest victory would turn into greatest shame this sport can have.

It has always been true that the sport balances both difficulty and quality, and different judges and different fans weight them somewhat differently. And on any given occasion a skater who is better at one might be or significantly better or just a little better that day. So it's always been true that, in relatively close contests, exactly which judges were on the panel and maybe what kind of mood they were in could make the difference between who places first and who second. That's not a travesty; it's the nature of a complex and qualitative sport.

That's said, returning to previous point, I think that the main problem of current PCS is their complexity. Judges just doesn't have enough time to assess all that "variety and fluidity" crap - especially in real time.

Well, if you consider variety and especially fluidity to be crap, you're dismissing over a century's worth of consensus within the skating community as to what constitutes good skating.

If you're looking for an objective sport with objective scoring, maybe an elements-only competition would be more to your taste. Or a different sport entirely.

They have enough work with assessing current elements GOE's. I am 100% sure that they didn't even try to assess all PCS categories according to guidelines - they just don't have time for that.

I'm 100% sure that many, probably most, judges try very seriously to apply all the PCS categories according to guidelines and that if you asked a judge to explain in detail how they arrived at their scores, especially an articulate one who is also good at putting their thought processes into words, you'd be surprised at how many details they noticed and weighed thoughtfully against the criteria.

Thought is quick. Documenting the thought processes is time consuming. Separate scores for 7-12 elements and 5 separate components gives more detail than the 2 scores of 6.0 judging. But if a judge notices 100 or more different details about a given program, they're not going to have time to write down or click/type in a computer every one of those 100 observations. Even if they take time to give a detailed verbal explanation afterward, they would summarize several separate but related observations into a single comment.

Considering that all that assessment never worked in the first place (there are tons of examples when skaters with poor or little number of transitions were awarded with high TR mark and so on) - what makes you so certain that they will work in a new system?

I think that the current IJS system works better for this purpose than the two-mark system under 6.0. It would probably be possible to come up with an even better system, but we don't yet have good proposals for moving forward toward better assessments. Simple counting up quantity with all moves valued equally, and then using the totals to restrict the judges ability to exercise their judgment, would not be a step forward.

If ISU just don't want to keep extra panel of judges for PCS marks - which last congress showed (and they have valid reasons for that albeit mostly financial) - I would say let's ease their work! Certainly they would have enough time to assess properly only two PCS category after the skate, wouldn't they?

There are two basic ways you could reduce the current 5 components (which currently comprise >20 individual bullet points) down to 2.

One would be to just throw away all the criteria under 3 of the existing components, so that there might only be 10 total criteria for judges to consider. That would certainly be easier to assess quickly. But it would also mean that many considerations that have been part of the oral and written traditions of what constitutes good skating and good performance quality would no longer officially be part of the scoring at all. So if some of those criteria were important to you ("you" meaning any given stakeholder, whether skater, coach, official, fan, etc.) but no longer supposed to be considered in the scoring, and skaters therefore stop trying to be good at those things, then some of your favorite aspects of the sport might become much harder to find in anyone's performances.

The other approach would be to keep all the criteria in the written guidelines but just to split them into 2 summary scores instead of 5. So each of those 2 scores would have more bullet points to weigh against each other, maybe up to 12 per score instead of 3-6. Which means it would take longer to come up with a well-thought-out number for each of those 2 scores.

This might save a little time for the judges, but it wouldn't make their assessments any more objective or accurate. Probably the opposite.

Meanwhile, the length of the wait between the end of the program and the announcement of scores is usually more dependent on the tech panel reviews than on judges scoring the components.

Some TR qualities such as fluidity etc. can be relegated partly to SS, partly to GOE bullets - others can be abolished as superfluous.

So if I'm a fan who loves transitions and you abolish most of the criteria that make them valuable, skaters are probably going to stop doing much interesting skating between the elements and the programs will become less interesting to me. My loss, huh?

(And also a loss to skaters who excel at skills that have not been incorporated into the scale of values compared to those whose best skills are jumps and spins but not much else.)

I would change it to PE 25%, TR 25%, SS 50%.
Figure skating is not ice show. Pantomiming and dresses shouldn't win the competitions. Skating skills should matter more.

I agree with this. :)


Also, maybe you will be surprised, but number of votes/views is pretty objective way to assess performance strength.

However, fans' interest is often driven by nationalism or by personal preferences often based primarily on choice of music, regardless of how well the performance actually uses skating abilities, even more than is true for judges. So it's an objective way of assessing how popular a performance is but not of assessing how well skated it is.
 
Joined
Jun 21, 2003
In terms of simplifying Program Components I did an exercise after last years Cup of China, the 2017 one, where I looked at all the values for all the skaters across all the disciplines - the thread is still out there somewhere, but the result of looking at the raw data was that it was quite possible to reduce the components to at least 4, and quite possibly 3

In fact, statistically the five components can be reduced to just one. Just copy the SS score 4 times and for TR, put down .5 less.

TR was always a bit lower than SS (there was no obvious reason for this) ...

I always guessed that the reason is that judges feel more confident in counting the transitions and saying to themselves, "Hey, that program didn't have any transitions at all, to speak of."

Whereas they are less secure in saying, "That program didn't interpret the music worth a darn!"
 

Miller

Final Flight
Joined
Dec 29, 2016
In fact, statistically the five components can be reduced to just one. Just copy the SS score 4 times and for TR, put down .5 less.

Reading all those criterias (https://www.isu.org/figure-skating/...96-program-component-chart-id-sp-2018-19/file) judges are looking for when they are scoring PCS, i don't see how Interpretation of the music's, Composition's and Performance's criterias could be combine in one score. The only way to make program components composed of lesser number of categories is to put Composition's criterias in other categories and go with 4 categories of PCS. Purpose/idea/mood can be a part of PE, Pattern/Ice coverage part of SS, Use of space and design of movements part of TR, Phrase and form part of IN, and Originality part of TR, PE or IN. So, i think they could go with 4 categories: SS, TR, PE, IN by that logic...

True. Though PE could be less than CO and IN if the skater hit difficulties and it affected their overall performance, though presumably it wouldn’t be too difficult to adjust an overall ‘Performance’ component if you did have a case like this.

Interesting what Baron Vladimir says about redistributing part of the CO component to other components. In fact if say Pattern/Ice Coverage went to SS and Use of Space and Design of Movements to Transitions you’d kind of be increasing their weightings to 25% anyway, hence a 25% SS, 25% TR, 50% ‘performance’ component would be even closer to what you get now in terms of a score and the only question is whether you go with that, or as Baron Vladimir says a 4 component one of SS, TR, PE and IN, with all of them presumably at 25%.

Re the latter the only question then would be whether you get the level of accuracy you desire for things like SS and TR, or whether you’ll still get an overall value with plus or minusing on top. However as a simple first step it might not be a bad way to go. Remove something that always has the same value as one of the other components i.e. IN for whatever reason, and then take it from there as necessary.
 
Top