COP: How important is judge selection?

[moriel]

So then, right think should be not to include judges who have their own representatives. Ladies judging panel should be for example MEX, NZL, UZB, LTU, BUL, AUT, NOR, GBR, ESP. Mens JAR, PHI, TPE, BLR, FIN, CRO, SVK, POL, TUR etc etc

Something like that - or some pattern where the judges cancel each other's bias. Lets say the top contenders on a given event in ladies are russian and japanese. If there is a russian judge and a japanese judge, russian one will give higher scores to russians, and lower to japanese, and the japanese will do the other way around. On average, it is fine.

Overall, i think the judge's pannel is not the issue, there are enough judges really to make the higher or lower scores given by a single judge mildly irrelevant. The tech pannel is the problem.

 

[4everchan]

If we consider that.

If geography is so important, then it makes much more sense that Kazakhstan would favour China, who is their neighbour (!) and not Spain, which is about eight countries away from them, for instance. Kazakhstan chooses to be part of the Four Continents, so I think we should respect that they identify themselves as belonging there and not in Europeans. And if we stay on the respectful side, perhaps we can consider that they might not favour anyone because of geographical or cultural reasons and strive to judge fairly.

I also find it strange that we are focusing so much on the ladies, which is the one discipline in which 'Europe' has been dominant the last few years. In all other three, they are not. How can that be if European judges dominate and they are all so eager to favour each other? Perhaps because they are not nearly as culturally similar or inclined to favour each other as some suggest.

actually... my memory is failing me but it has been mentioned either by Denis or Misha that they wished their country was part of the Europeans... and the bond with Russia is strong with both these skaters... as a matter of fact, many skaters representing former soviet republics were born in Russia. So, one can expect that their judges were perhaps even former skaters skating in Russia or trained there... one could also expect that if they may feel closer to China than Spain, they certainly have very little to do with North America

Finally, geography is very important when a sport includes performance and skills. Geography in fine arts is a huge factor and I believe that to some extent, it is the same with figure skating. There are "local" styles in skating as well. For instance, Canadian ladies have been praised for their huge jumps... Russian and Chinese pairs have been praised for their very long arched high throws as opposed to North American pairs who got for smaller, less dangerous throws. There is a very clear balletic aspect to both Russian Ice dance and Pairs.... We can see that balletic approach to ice dance in other European countries... which is not as common in North America...

So yes, in that sense, geography is important as, culturally, an individual is often going to develop an interest based on what they know or have personally experienced. There was a whole lot of discussion between skating styles comparing USA and Canada a while back, as both federations focus on different things.... for instance, the USA still use 6.0 in some competitions etc...

I didn't intend to make a whole post about geography or anything like that, it's another topic altogether, but humans all develop different tastes and expectations based on their experiences.... even likes and dislikes for food will vary from region to region.... so in my opinion, it's fair to think that North Americans may at times be disadvantaged when a panel is made mostly of European judges.... and for this purpose, I think it's fair to think that KAZ and UZB would favour skaters from nearby countries rather than a skater from Canada or the USA.

You only need to look at 6.0 ordinals to see how the judging panels were so geographically divided back then into "west and east" to understand what I mean. Very easy to see. COP hides that a bit more but I do believe it's still there....

 

[eppen]

In the spring already, I took a look at the PCS in the GP series for men by analysing how judges from different countries have treated their own and the rivals (posted in March, I think, but I had the file still, so can do it again here).

I took the results of the Grand Prix series because there the judges are from different continents in each competition. I looked at the results from Chan, Chen, Fernández, Hanyu and Uno, so each had three competitions and all were at the GP final.

The material is small, only 30 scores altogether (but nothing like this could have been be done before). And of course, all the result from the GP series should be anlysed to get an idea how each judge scores in general – low, high, medium. But I did not have time to do this.

I calculated the individual scores for each judge for both parts of the competition and compared it to the trimmed average (i.e. the official result) – sort of familiar from what e.g. Skating Protcol does. If the judge’s score was over the average, the score was regarded positive and vice versa. Then I counted together plusses and minuses to get an overall positive/negative/cannot decide result for each country.

The skaters were judged by judges from 14-15 different countries. Each skater had a similar set of countries represented in the panels: 4-5 from Asia, 2 from the Americas, 5-8 from different parts of Europe and then ISR, AUS and NZL. The judges were usually different persons for each competition.

Chan: JPN, CHN, CAN, USA, ISR, RUS 3 times; CZE twice; KOR, UZB, NZL, POL, ESP, FRA, ITA once; 14 countries (Asia 4, Americas 2, Russia/E Europe 2, W Europe 3; ISR, NZL)
Chen: JPN, USA, RUS 3 times; CAN, ESP, ISR, FRA, ITA twice, CHN, KAZ, KOR, UZB, LAT, SVK, AUT, BEL once; 15 countries (Asia 5, Americas 2, Rus/E Europe 3, W Europe 5; ISR)
Fernández: JPN, USA, RUS, ESP, FRA 3 times; CAN, ISR twice; CHN, KAZ, KOR, UZB, LAT, BEL, ITA, SWE once; 15 countries (Asia 5, Americas 2, Rus/E Europe 2, W Europe 5; ISR)
Hanyu: JPN, CAN, USA, ISR, RUS 3 times; CHN, ITA twice; KOR, UZB, CZE, LAT, SVK, AUT, FRA, ESP once; 15 countries (Asia 4; Americas 2; Rus/E Europe 4; W Europe 4; ISR)
Uno: JPN, CAN, USA, ISR, RUS 3 times; CHN, ESP, FRA twice; KAZ, KOR, AUS, LAT, ITA, SWE once; 14 countries (Asia 4, Americas 2, Rus/E Europe 2; W Europe 4; ISR, AUS)

And how did the skaters get scored? “Positive only” means that the skater received only or mostly positive (above average) scores in each competition; negative is the opposite = below average all the time. Divided means that there were both negative and positive scores. Neutral means that the country gave equal number of positive and negative judgements.

Chan
Positive only 7: CAN, JPN, CHN, KOR, UZB, ISR, RUS, ITA (highest scores from CAN, CHN, JPN, UZB, RUS)
Negative only 3: CZE, ESP, FRA (lowest scores from CHN, CAN, USA, RUS, CZE, POL)
Divided 3: USA, NZL, POL

Chen
Positive only 3: JPN, LAT, SVK (highest scores from JPN, CHN, KOR, CAN, USA, SVK, AUT, FRA)
Negative only 3: ISR, RUS, BEL (lowest scores from CHN, BEL, FRA, ESP, ITA)
Divided 9: USA, CAN, CHN, KAZ, KOR, UZB, AUT, ESP, FRA

Fernández
Positive only 6: ESP, CHN, KOR, UZB, LAT, RUS (highest scores from ESP, KOR, UZB, USA, ISR, LAT, FRA)
Negative only 3: CAN, BEL, SWE (lowest scores from ESP, JPN, USA, FRA)
Divided 6: JPN, KAZ, USA, ISR, FRA, ITA

Hanyu
Positive only 6: JPN, UZB, ISR, LAT, SVK, AUT (highest scores from JPN, UZB, CAN, RUS, LAT)
Negative only 3: USA, RUS, ESP (lowest scores from CHN, CAN, ISR, ESP, ITA)
Divided 5: KOR, CAN, CZE, FRA, ITA
Cannot decide CHN

Uno
Positive only 5: JPN, KAZ, ISR, LAT, FRA (Highest scores from KAZ, ISR, LAT, FRA)
Negative only 3: CAN, AUS, SWE (lowest scores from KOR, USA, CAN, AUS, ISR, ESP)
Divided 4: KOR, USA, RUS, ITA
Cannot decide CHN, ESP

General observations:
1) All in all, there are relatively few differences in the composition of countries in the panel for each skater. Asia and Europe were both particularly strongly present. Maybe a few more Western European countries than Eastern. The ever present ISR was slightly surprising.
2) Apart from Chen, each skater got 5-7 positive results and 3 negative ones. Chen divided the judges most.
3) Most skaters got positive results from 3-4 European countries (ISR among them) and 2-4 Asian countries. Negative or mixed scores come usually from a combination of North American and European judges.
4) The top players were strongly supported by their own judges by scoring their own positively and the rival negatively (or mixed): Hanyu could do no wrong in front of the Japanese judges (twice the highest score; Uno got one negative score from his own). Fernández and Chan could falter in the eyes of their own judges, but the overall support is clearly there (both got once highest, once lowest score from their own). Their own judges did not generally give positive scores to the main rivals. Chen was the only one to not receive unanimous support from his own judges. This support is not done by huge over- or underscoring, but rather the scores for own/rival skaters are neatly within the positive/negative range apart from a couple of highest/lowest scores.
5) There does not seem to be any clear continental divides here. Fernández is the only European in the analysis and he gets the highest scores from Eastern European and Asian judges, his "own" side of the European continent does not show that much love towards him.

I have taken cursory looks in the ladies, and the general trend seems to be the same as in point 4: score your own high, rival low. No clear continental divides.

At some point I also read scientific articles analysing IJS scores in the first 4-5 years of its existence when you could not combine the country and score and the outcome was basically that if there was a judge from the skater's country in the panel, then he/she/they were likely to get a higher score. In the old days, the Cold War played a major part on both sides for scoring, but I don't know if that division is still as effective as many people seem to think.

Also, if we are talking about PCSs, the continual rise of the scores kind of started in 2013 when Chan and Hanyu started getting all 9s and 10s, Fernández joined that club in the latter part of the 2014-5 season.

In the ladies, before the Worlds 2013 no skater had gotten 9s as an average PCS score (there might have been occasional ones, but not enough to raise the trimmed mean over 9), but then Yuna Kim got mostly 9s.

2014 Asada got more 8s than 9s, Kostner got mostly 9s.

2015 both those ladies were out and even the winner (Tuktamisheva) got only 7s and 8s (and btw she skated in two tons of competitions winning them just about all and her PCS did not rise that much because of her consistency - I cannot imagine how that would have worked for Kostner as claimed above).

2016 three ladies got mostly 9s as PCS: Gold (SP only, of course), Wagner and Medvedeva.

2017 was the first time in the Worlds when the winner got all 9s for PCS - Medevedeva of course. Osmond received mostly 9s also for FS at least.

So far, 7 ladies have gotten very high PCS scores in the Worlds, all in the past 5 championships. 5 of them are non-European, 2 European. In the men, the situation is similar, the majority of the high PCS scores have gone to non-European skaters (Chan, Hanyu and now Uno) with one European tagging along. Based on these numbers, I find it hard to see how European judges scoring European skaters too high somehow can be blamed for the current situation - they were mostly overscoring the non-Europeans, right?!?

In the men, the situation began to change at about the time when TES started to rise which could mean that ISU might have started to think that it would be a good time to start using the whole scale of the PCS to make the TES and PCS somehow comparable. The TES in the ladies side has not perhaps risen quite so drastically, but the first to get high scores were respected champions, so reputation scoring is also a possibility. 2016 sticks to the eye with two US skaters getting scored very high - homeground advantage maybe? 2017, Med had been unbeatable for her 2 senior years... It is difficult to explain the situation with only one thing. It is possible that ISU have made a decision to use the scale differently - like 10.00 does not equal absolute perfection, but it can be awarded if the skater ticks all the boxes in the category?

The origin of the skater was also taken up - now that people can represent almost any country in any competition apart from the Olympics, does it matter? Many of the skaters representing small feds come from the US, Canada or Russia. Alex Krasnozhon (US) was born in St Petersburg - does this mean automatically that the Russian or Eastern European judges will score him higher?

And lastly, if the small federations were to take up all the judging, would they do it any differently? There might be no national bias, but as they are trained by the ISU, how would it change things? This is a conservative sport that changes slowly and the traditions are transmitted to the newcomers as well as to the reps from old federations.

E

 

[moriel]

Yep. Similar patterns with goe: give higher goe to your own, and lower to competition.
If the continent has any effect on the results, its way too small to be visible compared to national bias.

 

[4everchan]

Interesting but limited data as of course one cannot compare a great outing by skater x versus an event where he didnt do as well. I would be the kind of judge to give my countryman a good score when skating well and give them tough love score i e low when underperforming.

Also the data being limited for time reasons i am sure to the top 5 makes it harder to see the influence on lower levels. Also the data being limited to gp makes it even harder since obviously the pool of skater is limited and usually the fields are stronger so it skewes the averages.

Nevertheless, interesting work Eppen. Thanks for sharing

 

[Tulipstar]

actually... my memory is failing me but it has been mentioned either by Denis or Misha that they wished their country was part of the Europeans... and the bond with Russia is strong with both these skaters... as a matter of fact, many skaters representing former soviet republics were born in Russia. So, one can expect that their judges were perhaps even former skaters skating in Russia or trained there... one could also expect that if they may feel closer to China than Spain, they certainly have very little to do with North America

Finally, geography is very important when a sport includes performance and skills. Geography in fine arts is a huge factor and I believe that to some extent, it is the same with figure skating. There are "local" styles in skating as well. For instance, Canadian ladies have been praised for their huge jumps... Russian and Chinese pairs have been praised for their very long arched high throws as opposed to North American pairs who got for smaller, less dangerous throws. There is a very clear balletic aspect to both Russian Ice dance and Pairs.... We can see that balletic approach to ice dance in other European countries... which is not as common in North America...

So yes, in that sense, geography is important as, culturally, an individual is often going to develop an interest based on what they know or have personally experienced. There was a whole lot of discussion between skating styles comparing USA and Canada a while back, as both federations focus on different things.... for instance, the USA still use 6.0 in some competitions etc...

I didn't intend to make a whole post about geography or anything like that, it's another topic altogether, but humans all develop different tastes and expectations based on their experiences.... even likes and dislikes for food will vary from region to region.... so in my opinion, it's fair to think that North Americans may at times be disadvantaged when a panel is made mostly of European judges.... and for this purpose, I think it's fair to think that KAZ and UZB would favour skaters from nearby countries rather than a skater from Canada or the USA.

You only need to look at 6.0 ordinals to see how the judging panels were so geographically divided back then into "west and east" to understand what I mean. Very easy to see. COP hides that a bit more but I do believe it's still there....

Good you mention the West-East divide. That alone already tells us that 'belonging to Europe' wasn’t the deciding factor. I find it very strange to suggest that European countries would all now be under the sway of Russian cultural influences, as was done in the thread this thread was based on (not by you). Western Europe is much more influenced by American influences than by Russian influences. People here are much more likely to understand the languages spoken in the US and Canada than they are likely to know Russian. We have watched countless of movies, TV-series and sports from North America, and are much more likely to recognize and identify with cultural practices from North America than from Russia. Most people I know probably have not even watched one Russian movie or know much more about the country than that it’s big and Napoleon got trashed there. It's a shame really, seeing how much Russia has to offer in many ways.

I am not making the same statement about American influences on Eastern Europe, as I don’t want to make claims that are not based on facts but assumptions about places I know little about. Though, if I follow the logic here, I should probably identify with them closely, understand them better and want them to win over North American countries solely because they are European.

I can see what you say about nearby countries having similar styles and understanding each other better, but that effect lessens the more countries you pass. Kazakhstan can’t both favour countries just because they’re nearby, but then also any random European country, because that country is European but is so far away from Kazakhstan that the people there often don’t even know where Kazakhstan is.

Added to this, anyone who lives in Europe would know that when it comes to sport, nearby countries often specifically don’t favour each other and rather have a faraway country winning than their big rival next door. Personally, I like it that Belgium is at least sending skaters to the Olympics as they are a small country that we are close to. But that is as far as it goes. Beyond that, I would cheer much more for a skater from a country that has never sent anyone than a random skater just because they are from Europe. I will also support my favourites, who come from anywhere, including North America.

Frankly, I find some of the things that have been said on this topic quite insulting to smaller countries. It’s like we are all supposed to be under the sway of bigger countries and are all heavily biased depending on what continent we are in.
And like Neenah16 says, there is a lot of cherry picking of data to fit a narrative. The fact is that North Americans are not always the best at everything. When they don’t win something, it doesn’t automatically mean there is a big conspiracy against them. Perhaps there is a skater from elsewhere that is better.

Funnily enough, even in ladies (since this conspiracy only seems to focus on this discipline for some strange reason) the second, third and fourth place in Worlds 2017 were all North Americans. Perhaps those evil, all prevalent European judges are not as biased and detrimental to the chances of North Americans as you thought.

 

[OS]

I love the ongoing attempt to normalise and discount the well known established fact that regional judging panel can slant marks, especially given the history of this sport.

The ongoing systematic anomalies which have only happened in the ladies in the last 5 years, has resulted in the greatest inflation for young EUROPEAN skaters contrary to historic judging trends, where youngsters have to earn their PCS through proven improvements and results. It is like one rule for Russia, one for the rest of the world.

How is it 'cherry picking' when I used the ENTIRE 5 consecutive years of ladies final FS panel to showcase regional slants judging makeup? These are such basic 101s, I am surprised people still waste their effort to justify under political correctness to imply no possibility favouritism due to x y z, at the same time being complete hypocrites saying rest of the world representation should be fine with only makeup 35% of all judge panel and then it is fair.

The past 5 consecutive years has established a systematic pattern of

- overwhelming majority of consecutive SAME EU judging fed presentation on the final judge panel for the ladies. (many as 4/5 years, 3/5 years) Bear in mind, you just need 2 judges to slant the result.

- consecutive 5 years overwhelming majority 8/9 European judges representation with 'consecutive' the key anomaly. European championships have historically established itself having the greatest inflation among all elite competitions, contrary to all these generalisations of current affairs that have very little to do with how figure skating world actually work in real life. ie/ NOT driven by politics socio economics at large, but by federation interests, relationships, loyalty, culture etc.

- consecutive 5 years of under-representation of non-EU (REST OF THE WORLD) on the same judge panel, average only 1 per year representation. Judges inherent slants are supposed to cancel each other out. By having a one-sided panel with under-representation from the minority, it cancels this effect entirely. Hence again, perpetuate the slant pattern.

1 year you can discount it as unlikely (17% chance of 8/9 majority based on 26 international judges, non EU judges makes up 7), 2nd year it became anomalies (3% happening), by 3rd year consecutive, 4th year consecutive, 5 consecutive years it became 0.001% happening. If by true randomisation, it has a 1 in 36 millennium + years of happening by random draw. Yet it has happened.

The fact people like to ignore the fundamental conflict of interests at various levels in this sport is also puzzling to me, since it goes against the very idea of fairplay.

------------------

The stats and improbability are self evident and plainly defies purpose of randomising draw to ensure everyone gets equal opportunity to judge, to prevent favourtism, or chances of repeat representation.

This is an old interview by an Olympic judge worth a read.

https://figureskate.wordpress.com/2010/03/08/patrick-ibens-interview/

 

[frskate]

this event was judged under the 6.0 system
as far as i remember the referee was Swiss and the assistant East German
the judgment of the free skating was more exciting and more difficult.....

 

[gkelly]

I love the ongoing attempt to normalise and discount the well known established fact that regional judging panel can slant marks, especially given the history of this sport.

Yes, that has happened and undoubtedly still happens. But it's more complicated than just Europe vs. the rest of the world.

The ongoing systematic anomalies which have only happened in the ladies in the last 5 years, has resulted in the greatest inflation for young EUROPEAN skaters contrary to historic judging trends, where youngsters have to earn their PCS through proven improvements and results. It is like one rule for Russia, one for the rest of the world.

European skaters, or Russian skaters specifically?

Do you find French or Finnish or Italian or German or Slovakian skaters' scores to be inflated compared with Japanese or American or Canadian or Chinese or Taiwanese ladies?

Also, have the scores of non-European skaters also risen during that period, just not to the same degree? If so, it could be a combination of a trend toward score inflation coinciding with the rise of an especially talented cohort of Russian skaters. Do we see the same thing happening among the men, where Russians/Europeans are not currently dominant?

this event was judged under the 6.0 system
as far as i remember the referee was Swiss and the assistant East German
the judgment of the free skating was more exciting and more difficult.....

Which event are you referring to?

 

[Mathman]

This is an old interview by an Olympic judge worth a read.

https://figureskate.wordpress.com/2010/03/08/patrick-ibens-interview/

The part of the interview that I was most interested in was this:

Interviewer Tony Wheeler: So do you like this system better than the 6.0 system? Explain.

Skating judge Patrick Ibens: WelI, I have mixed feelings here cause it has both good and bad aspects.

Some of the good aspects are that skaters finally had to work their footwork. Back to clean edges, etc. Also, as I mentioned, they get rewarded for every element and not only for difficult jumps! The negatives are that all elements look alike, especially the spins and steps. There is also not much time left for much creativity.

What I hate the most about this system is that it is made to save the “not-so-good” judges, while the really good judges who are marking the way it’s meant to be (every component separately) risk the chance of being out of the corridor of average marks, and risk getting some assessments. A judge who basically does not know anything can give all the wrong marks or completely guess and their marks fall into an average! But someone who wants to have wide margins between components might be singled out for doing so. For example, when scoring the first three groups at the World Championships, you give between 5.50 and 7.00 and you are in the safe corridor. When the last groups come on the ice, give between 7.00 and 8.50 and you’re safe again!

To bring things up to date, we should say: to be safe, give 9.0 to 9.75 in the final group.

What makes me smile, though, is that every skating judge that has ever been interviewed says, I'm 100% honest, but those other guys!

 

[Mathman]

Yes, that (slanted judging reflecting regional bais) has happened and undoubtedly still happens. But it's more complicated than just Europe vs. the rest of the world.

I think this is the key question. Back in the day, skaters from the USA were quite happy to see judges from (West) Germany, Netherlands and Italy on the panel, but not so much Canada (you never know with those Canadians! )

Fortunately, with judges' anonymity being done away with, we will soon be able to conduct all sorts of statistical studies of national and regional bias again, as were done under 6.0.

 

[Mathman]

Am I reading the LP protocols from Rostelcom right? Did every single one of the Belarussian judge's PCS for Medvedeva get thrown out (9.50, 9.50, 10.00, 10.00, 10.00), thus protecting the scores of the Russian judge (9.50, 9.50, 9.75, 9.75, 9.75)?

 

[4everchan]

Am I reading the LP protocols from Rostelcom right? Did every single one of the Belarussian judge's PCS for Medvedeva get thrown out (9.50, 9.50, 10.00, 10.00, 10.00), thus protecting the scores of the Russian judge (9.50, 9.50, 9.75, 9.75, 9.75)?

impossible :scard7:

 

[xeyra]

Am I reading the LP protocols from Rostelcom right? Did every single one of the Belarussian judge's PCS for Medvedeva get thrown out (9.50, 9.50, 10.00, 10.00, 10.00), thus protecting the scores of the Russian judge (9.50, 9.50, 9.75, 9.75, 9.75)?

Would have been exactly the same if the BLR judge had given 9.75 where they gave 10.

 

[Mathman]

Would have been exactly the same if the BLR judge had given 9.75 where they gave 10.

Better safe than sorry.

Edit: By the way, wasn't there supposed to be a new rule this season that a skater could not get a 10 if she had a fall? Or is that just for ice dance?

 

[eppen]

I love the ongoing attempt to normalise and discount the well known established fact that regional judging panel can slant marks, especially given the history of this sport.

The ongoing systematic anomalies which have only happened in the ladies in the last 5 years, has resulted in the greatest inflation for young EUROPEAN skaters contrary to historic judging trends, where youngsters have to earn their PCS through proven improvements and results. It is like one rule for Russia, one for the rest of the world.

Ach, OS, cherrypicking is really bad, indeed. You looked at the 5 past worlds in ladies FS and got the panels mostly correctly European in majority. However, in 2013 there was a South African judge in the SP, in 2014 an Australian judge in the SP, 2016 and 2017 altogether 3 non-European in the panels. Only 2015 was completely European. The non-European judges in the SP panels did also inform the outcome of the competition. (The biggest European federation, Russia, had a judge in the ladies only once in 5 years! JPN was present 3 times...)

But you did not discuss what did these European judges do in the competitions at all, except to claim that they were boosting the PCS scores for European skaters. But who were the ladies in the top 10, say? Each year, there has been 3 European skaters and 7 non-European skaters in the Top 10. Earlier we established that the winners were 2-3 non-European-European. Looking at these numbers I really find it hard too see any evidence towards European judges favouring European skaters very strongly.

You looking at ONLY the compostition of the panels is cherrypicking the data that you want to believe and present. The countries are only one part of the whole process, but you should have the honesty to look at also what the panels do and present also that data instead on mysteriously imply who knows what kind of wrong-doing.

I have tried to present the judge selection process, what countries the skaters represent and how the judging panels in all four disciplines relate to that selection of countries, how the judging has gone in the GP series last year which was the first time we have been able to connect score and judge, plus have even looked at the development of PCS scores in singles skating. The data can be analysed in different ways and none of the results seem to point towards blatant major misconduct.

I also have to point out that the probabilities calculated above are purely hypothetical or even imaginary, because as I tried to say (twice, I think), as far as I know not all of the countries in those lists of 26-30 participants in the draw for judges participate in the draw for all four disciplines - not all of the federations have enough ISU judges for that. It is not indicated in the ISU reports which countries participate in which category. The draw is probably done from a smaller pool, whose size remains unknown. Hopefully no one else cites the probabilities anywhere else!

Oh, and the only young European female skater whose PCS score has risen very quickly in the Worlds level is Evgenia Medvedeva as far as I can see. Kostner can hardly be described as young? Pogorilaya and Radionova have gotten fairly good PCS scores, but not any better or worse than say the young Japanese ladies. Tuktamysheva got fairly low PCS scores in her championship yea 7s and 8s. The over 9s in the past 5 years I listed above were mostly seniors skaters, such as Kim, Asada, Kostner, Wagner and in a way also Osmond who is over 20. Even her PCS scores were discussed as high last season IIRC.

OS, if you wish to really make your point on in the PCS inflation for European (or is it just Russian?) ladies in the past 5 years, the judging panel composition is not enough evidence for it. You should also show what the PCS scores have been and how they have developed for different skaters year to year from different countries in the past 5 years. And also show what the pattern was for skaters from different regions in the 5 years before that under what kind of judging panels to establish the change. The judging panel composition in the Worlds for the past 5 years alone is simply not a sufficient argument to draw the conclusions you have drawn. You also have to show the numbers for the actual PCS scores to make your conclusions stick...

E

 

[Miss Ice]

Am I reading the LP protocols from Rostelcom right? Did every single one of the Belarussian judge's PCS for Medvedeva get thrown out (9.50, 9.50, 10.00, 10.00, 10.00), thus protecting the scores of the Russian judge (9.50, 9.50, 9.75, 9.75, 9.75)?

Yeah, that's definitely not a coincidence.

 

[eppen]

One more thing came to mind concerning the ladies FS and PCS, its alleged changes in the past 5 years, and their connection to judging panels with European majorities.

1) Is there a change in panel composition for Worlds around 2013? I checked the 5 years before 2013 (2008-2012) and it seems that the panels were drawn from a very similar pool of countries as these days. The proportions between European-non-European countries was just about the same as in 2013-2017: 2-3 non-European and 9-11 European countries (you have to look at both panels and 11-13 judges were used). The exception was 2010 when there were 5 non-European countries and 6 European countries. So, no changes there in the past 10 years and you can probably go as far back as mid-1990s to find similar panels (that's when the European map was redrawn to its current state more or less.)

2) The top 10 ladies in that period came from 5/5 or 4/6 European/non-European countries with the winner from non-European countries in all other years except in 2012. The ratio was higher than in the past 5 years when only 3 Europeans could be commonly found in the top 10.

3) The PCS scores rewarded to top 3 were always below 9 for everyone in 2008-12. In 2008 everyone was in the 7s (Kim, Asada, Kostner, Nakano), in 2009 Kim, Ando and Asada received 8s, in 2010 Kim got 7s and 8s, others 6s and 7s. In 2011, Kim got 8s, the rest 6s, 7s and 8s. In 2012 Kostner gor 8s, others 7s and 6s.

4) The Russians entered the top 10 in 2009, but got to the podium the first time in 2012 (Leonova). Russia got 2 ladies to top 10 in 2011 and 2012.

5) In the period 2013-17, 9s started to appear, as I explained already above. In 2013, Kim (a seasoned veteran, if I may) was the first lady to get 9s as PCS from the mainly European panel. In 2014, Asada and Kostner got 8s and 9s and at this point, both were in their nth season in seniors. In 2015, the first Russian gold medalist in ages, Tuktamysheva, toop the top spot with 7s and 8s in her PCS score sheet. She started her senior career in 2012-3 and this was her 3rd year in the seniors - compared to the earlier champions in similar timelines (particularly Kim and Asada) her PCS were on the low side. In 2016, the American ladies got the homeground advantage with a lot fo 9s from the mainly European panel, with Medvedeva taking the gold with mostly 9s in her PCS. That was her first year in the seniors. The same for 2017 for Medvedeva with Osmond in the 9s also.

Now, based on analysis of BOTH the panel composition and development in the PCS side by side, there does not seem to be a clear connection between the panel composition and the alleged rise in PCS scores. Nor is there an obvious tendency to score young Russian skaters highly in the Worlds level - Medvedeva seems to be the big exception here.

However, based on what I have done earlier in the singles skating, there seems to be a tendency to score PCS higher for some of the top skaters at least and it started in the fall of 2013 with the men in the GP series, followed a bit more slowly in the ladies in the past couple of years. This happens mostly to seasoned skaters (Chan, Hanyu, Fernández, Kim, Asada, Kostner, Wagner) with Medvedeva as the big exception.

The other Russian ladies remain in low to mid-8s for the most part (Radionova, Pogorilaya; the others have not been as constantly in the worlds in their senior careers). Pogorilaya got lower 8s for the first FS outing in the Worlds in 2014, Radionova got upper 7s in 2015 in her first Worlds. A steady, but not too stellar rise in PCS for these two in their Senior careers. There are a lot of Russian female skaters around, but not all of them enjoy similar success and get as high PCS as Medvedeva - she was already a big Junior star.

In men, there has been a lot of discussion of PCS scores for some newcomers to Senior ranks, Uno two years ago and Chen last year. Uno has gone from upper 8s in 2016 to upper 9s and 10s in 2017. His fast upgrade is very similar to Medvedeva's who has now started to get 10s in her 3rd year, unprecedented for ladies. Chen's PCS rose several points within his first season in the seniors and this has been criticized.

The quick PCS rise does seem to happen, but it concerns only few top skaters such as Uno and Medvedeva.

To really see if there is truly a general inflation, a much wider analysis to all skaters for the past decade should be done.

The main conclusion here is that the connection between European-dominated panels and higher and faster PCS scores for European/Russian ladies does not hold; simply because the general composition of the panels has not changed in the past 10 years and the trend of regular high PCS does not seem to exist.

E

 

[eppen]

Am I reading the LP protocols from Rostelcom right? Did every single one of the Belarussian judge's PCS for Medvedeva get thrown out (9.50, 9.50, 10.00, 10.00, 10.00), thus protecting the scores of the Russian judge (9.50, 9.50, 9.75, 9.75, 9.75)?

The Belarus judge also gave very high scores to Kostner and Radionova. In the men FS, the JPN judge scored Chen very high and was this then done in collusion to protect the second highest score from the US judge?

This is also something I don't care too much seeing, because one score from almost any judge can be interpreted in a variety of ways. But you gotta a look what they do with the other skaters to get perspective on how they score. In the JGP series in Brisbane this fall, in the men there was a Russian judge who gave the highest score to all the top 11 in the both SP and FS and then in the ladies an Australian judge who gave the bottom scores to all the top 12 ladies in the FS.

E

 

[Neenah16]

I think the solution to the judging problem is quite straight forward and easy (in theory), and it is individual accountability. It should not matter where the judges come from and which federation they belong to, they should at all times be evaluated as individuals because that is how they are judging. The ISU can have all the data on how each judge is scoring in every competition they have been in (even smaller events), so they should be able to look for patterns in scoring and evaluate judges accordingly. Maybe they can commission a research team to look into the data and create a software that can flag certain behaviours and patters for investigation. based on their evaluation, judges who have problematic scoring should not be included in big competition and of course those who are caught cheating should be banned for life.

Naturally, this requires real research done by professionals who can put up a fair evaluation system for the judges, which would require the ISU to spend few thousand dollars, but I do think it is doable and the result is worth it.