COP: How important is judge selection?

[Sam-Skwantch]

Why so? I don't see any advantages of that...but see disadvantages and risks of collusions

I was just pointing out that the consistency would be nice. I'm not too worried about national bias and all that. I think it's frequently overstated and besides...look how deeply it runs in its fan base. I'm convinced that several posters here pick their favorites first and foremost based on nationality. Hardly surprising to me that judges for nefarious reasoning or not will value the skills their federation likley values.

FWIW: I'd love to see the judges scored via some sort of system and that's how they get events instead of by representing national federations alone. I'm not the person to develop such a system. Just making an observation.

 

[Mathman]

* whole post *

Wow, thanks for all the research. Very cool.

The judges for the 2017 Worlds were selected by the customary draw in 2016 (http://isu.org/communications/519-2048-judges-draw-by-number-championships-2017-rev/file) and 26 countries participated (not necessarily to all the disciplines, btw): 17 European countries (AUT, BEL, BLR, CZE, FRA, GER, GBR, ITA, LAT, LTU, NOR, RUS, SLO, ESP, SWE, SUI, UKR), 6 Asian countries (CHN, ISR, JPN, KOR, TUR, UZB), 2 North American (CAN, USA) and Australia. 17-9 for Europe means that more European judges were likely to be drawn.

I think that Israel and Turkey are regarded by the ISU as "European." They send skaters to the European Championships and not to Four Continents. That would make the ratio more like 19 European and 7 for everyone else. On the typical panel of 9 judges we would expect about 6 or 7 European judges and 2 or 3 non-European. About 3% of the time, the panel will be all European.

Each individual country, European and non-European alike, has about a 35% chance of being seated on any given 9-judge panel.

 

[Mathman]

FWIW: I'd love to see the judges scored via some sort of system and that's how they get events instead of by representing national federations alone. I'm not the person to develop such a system. Just making an observation.

I think this would be a very challenging proposal. In figure skating, judges are evaluated largely by how close their marks are to those of all the other judges (the dreaded "corridor"). If you are like everyone else, that's good. If you stand out from the crowd, that's bad.

We would have to develop a system for some "master judges" to judge the judging. But then who judges the master judges?.

(When I was a lad the 4H clubs would bring their livestock to the county fair to be judged. Who raised the best calf, who raised the best pig? In addition to the livestock competitions, there was a judging competition. Youngsters would compete with each other in judging livestock. These judging contests were judged by professional livestock buyers who were employed by the meat-packing companies.

This was a big deal because the winners of the livestock judging contest could then apply for well-paying jobs as apprentice livestock buyers -- sort of like "going pro.")

 

[Sam-Skwantch]

It would be interesting if there was a" judge referee" who gets selected to score at each event and the "judges scores" were graded against the referee's marks. You could even grade the referees

 

[Mathman]

It would be interesting if there was a" judge referee" who gets selected to score at each event and the "judges scores" were graded against the referee's marks. You could even grade the referees

I think that the referee does score along with the judges. After the event there is a meeting between the referee and the judges where presumably the referee asks the judges to justify their marks, if they are way off from the marks of the referee and the other judges.

If I am not mistaken, the referee's marks are included in the average against which each judge's marks are compared. If there is an "anomaly," the referee conveys that information to the ISU judges oversight committee. Three anomalies during the season and you get some sort of demerit.

Something like that.

 

[Sam-Skwantch]

Well I'd love to know what the referees scores are then. I'm very rarely upset by scores or anything but I am a curious person at times.

 

[eppen]

Wow, thanks for all the research. Very cool.



I think that Israel and Turkey are regarded by the ISU as "European." They send skaters to the European Championships and not to Four Continents. That would make the ratio more like 19 European and 7 for everyone else. On the typical panel of 9 judges we would expect about 6 or 7 European judges and 2 or 3 non-European. About 3% of the time, the panel will be all European.

Each individual country, European and non-European alike, has about a 35% chance of being seated on any given 9-judge panel.

Armenia, Azerbajzan, Kazakstan, and Turkey have all bits in Europe and Asia, and KAZ chooses to compete in 4CC, the rest in Europeans. I went with geographic area, not other affiliations (coz then it gets even trickier).

But because I am curious, I did the calculations with the competition affiliations:

The judges for the 2017 Worlds were selected by the customary draw in 2016 (http://isu.org/communications/519-20...-2017-rev/file) and 26 countries participated (not necessarily to all the disciplines, btw): 19 European countries (AUT, BEL, BLR, CZE, FRA, GER, GBR, ISR, ITA, LAT, LTU, NOR, RUS, SLO, ESP, SWE, SUI, UKR, TUR), 4 Asian countries (CHN, JPN, KOR, UZB), 2 North American (CAN, USA) and Australia. 19-7 for Europe means that more European judges were likely to be drawn.

NB: here you have to consider that every country in the list did not necessarily participate in all the events, so it is not possible to estimate to likelihood of getting eg an all-European panel in any dispicline...

Men: Asia 8 countries (CHI, TAI, JPN, KAZ, MAL, PHI, KOR, UZB), Europe 16 (ARM, AZR, BEL, CRO, CZE, FIN, FRA, GEO, GER, ISR, ITA, LAT, RUS, ESP, SUI, UKR), North America 2 (CAN, USA) and Australia. 11:16 non-European:European (1:1,45).

The judges came from Asia with 3 judges (SP: UZB, CHI; FS: COR), Europe with 9 judges (SP: AUS, BEL, GER, GBR, ITA, ISR, ESP; FS: RUS, UKR). CAN had a judge in the FS. 4:9 non-European:European judges (1:2,25). In men the importance of new skating countries is quite clear - particularly Asia - and a better balance in the panels would be good. However, MAL and PHI do not have any ISU judges, TAI has them but did not participate in the draw for the judges.

Ladies: Asia 6 countries (CHN, TAI, JPN, KAZ, SGP, KOR), Europe 18 countries (ARM, AUT, BEL, CZE, EST, FIN, FRA, GER, HUN, ITA, LAT, NOR, RUS, SLK, SLO, SWE, SUI, UKR), Americas 3 countries (BRA, CAN, USA). 9:18 non-European:Europe (1:2).

The judges came from Asia 2 countries (JPN, KOR), Europe 8 countries (CZE, FRA, GBR, NOR, SUI; FS: LAT SLO, SWE), CAN and AUS. 4:8 between non-European and European. Considering the origins of the skaters a very good representation of judges countries (1:2).

Pairs: Asia 3 countries (CHN, KOR, JPN), Europe 13 countries (GER, RUS, FRA, ITA, CZE, AUS, CRO, LTU, BLR, HUN, FIN, GBR, SUI), North America 2 countries (CAN, USA) and Australia. 6:13 non-European:European (1:2,17).

The judges came from Asia 2 countries (CHN, JPN), Europe 7 countries (BLR, SUI, RUS, FRA, GER; FS: LTU, ITA), North America 2 countries (USA; FS: CAN) and Australia. 5:7 between between non-European and European (1:1,4). Here, Europe is slightly underrepresented based on skater origins.

Ice Dance: Asia 3 countries (CHN, KOR, JPN), Europe 19 countries (ARM, AZR, TUR, ISR, FRA, RUS, ITA, DEN, POL, UKR, GEO, ESP, GER, GBR, FIN, CZE, BLR, LAT, LTU), North America 2 countries (CAN, USA). 5:19 non-European:European (1:3,8).

The judges came from Asia 1 country (KOR), Europe 9 countries (ISR, TUR, RUS, FRA, CZE, ESP; FS: UKR, GER, GBR), North America 2 countries (USA, CAN). 3:9 non-European:European (1:3). I’d say a fairly good representation of the origins of the skaters.

So, no real big changes in the results in the end...

E

 

[OS]

Wow, thanks for all the research. Very cool.

I think that Israel and Turkey are regarded by the ISU as "European." They send skaters to the European Championships and not to Four Continents. That would make the ratio more like 19 European and 7 for everyone else. On the typical panel of 9 judges, we would expect about 6 or 7 European judges and 2 or 3 non-European. About 3% of the time, the panel will be all European.

Each individual country, European and non-European alike, has about a 35% chance of being seated on any given 9-judge panel.

These are in similar to what I had in mind accordance to proportionate presentation, but just out of curiosity Mathsman,

1. How do you calculate, and what is the probability of the same individual federation placed on the judging panel 4 times out 5 consecutive years in the final panel?

2. How do you calculate, and what is the probability of 8/9 panels are Europeans in 1 year, vs 2, 3, 4, 5 consecutive years?

Thanks.

 

[drivingmissdaisy]

This is a pretty inaccurate thing to say unless you are just mixing up Julia and Adelina.

http://www.isuresults.com/results/owg2014/index.htm

Not to mention there is no way to know which judge is which in this event.

Adelina's jumps were judged to be strong across the board in Sochi. Her 3T-3T in the SP, her 3Lo in the LP and her 2A-3T in the LP received +2 and +3 from every single judge. In the SP and/or LP, there were judges from KOR, RUS, ITA, USA, JPN, and CAN. The result was controversial, but agreement that those jumps were solid is unquestionable and in no way a reflection of the panel composition.

 

[zounger]

These are in similar to what I had in mind accordance to proportionate presentation, but just out of curiosity Mathsman,

1. How do you calculate, and what is the probability of the same individual federation placed on the judging panel 4 times out 5 consecutive years in the final panel?

2. How do you calculate, and what is the probability of 8/9 panels are Europeans in 1 year, vs 2, 3, 4, 5 consecutive years?

Thanks.

If my fading math knowledge is not fooling me. What you want to calculate is following the so-called Binomial Distrubution. So if the probability of having some individual federation in the panel is p, then the probability to have this federation 4 out 5 years is

P(4 out 5) =( 5!/(4! * (5-1)!) * p^4 * (1-p)^(5-4)


keep in mind that p is the actual fraction like 3/100 and not just 3.

! is the factorial sign. It's like 4!=1*2*3*4.

You can do the math now .

I hope Mathman approves the above positively.

 

[Sam-Skwantch]

Adelina's jumps were judged to be strong across the board in Sochi. Her 3T-3T in the SP, her 3Lo in the LP and her 2A-3T in the LP received +2 and +3 from every single judge. In the SP and/or LP, there were judges from KOR, RUS, ITA, USA, JPN, and CAN. The result was controversial, but agreement that those jumps were solid is unquestionable and in no way a reflection of the panel composition.

I have no issues with the results.

 

[OS]

If my fading math knowledge is not fooling me. What you want to calculate is following the so-called Binomial Distrubution. So if the probability of having some individual federation in the panel is p, then the probability to have this federation 4 out 5 years is

P(4 out 5) =( 5!/(4! * (5-1)!) * p^4 * (1-p)^(5-4)


keep in mind that p is the actual fraction like 3/100 and not just 3.

! is the factorial sign. It's like 4!=1*2*3*4.

You can do the math now .

I hope Mathman approves the above positively.

Goodness me, that went straight over my head lol... would some kind mathematical soul able to work these out?

 

[Mathman]

If my fading math knowledge is not fooling me. What you want to calculate is following the so-called Binomial Distrubution. So if the probability of having some individual federation in the panel is p, then the probability to have this federation 4 out 5 years is

P(4 out 5) =( 5!/(4! * (5-4)!) * p^4 * (1-p)^(5-4)

...I hope Mathman approves the above positively.

Now we compute p, by the hypergeometic distribution. :

We suppose that each time there are 26 nations participating in the draw, and that 9 judges are selected at random. The probability that a given country is selected is

P = 25!/8!x17! divided by 26!/9!/17! = .35

Check: The probability that you are NOT chosen 1st, 2nd, 3rd,…,,9th is

(25/26)x(24/25)x(23/24)x … x(18/19)x(17/18) = 17/26 = .65.

Plugging this p into zounger’s formula, the probability that a particular nation is chosen exactly 4 out of five times is

(5)x(.35)^4x(.65) = .05

(This is not the same as the probability that some nation will be selected 4 out of 5 times.)

Literary factoid. Professor Moriarty, Sherlock Holmes’ nemesis, wrote his doctoral thesis on the binomial distribution at Cambridge. (He was also collegate boxing champion.)

 

[Mathman]

Totally agree with this. it worked for the Ancient Olympic Games (only one case of unfair judgement in over 1000 years of history) it can as well work now, at least for Grand Prix series, World and Olympic Games.

I guess the exception is the games of 67 CE? The emperor Nero won gold medals in singing, acting, poetry and playing the lyre. The officials had to lock the gates to prevent the audience from leaving in the middle of his performances. He also won the chariot race, despite being thrown from the chariot. The other racers pulled up dutifully behind and waited for him to be helped back into the chariot. He finished the race and won.

Let this be a lesson to all figure skaters. If you fall, get back up and keep going. You might win!

 

[Mathman]

2. How do you calculate, and what is the probability of 8/9 panels are Europeans in 1 year, vs 2, 3, 4, 5 consecutive years?

Assume that each year there are 19 European countries in the pool and 7 others. Then the probability that exactly 8 European countries are chosen and 1 non-European country is:

"19 choose 8" x "7 choose 1" divided by "26 choose 9"

That is, 19x18x17x16x15x14x13x12 / 8x7x6x5x4x3x2x1 x 7 / 1, then divide by 26x25x24x23x22x21x20x19x18 / 9x8x7x6x5x4x3x2x1

= 75582 x 7 divided by 3124550

= .169 (About 17% of the time the judging panel will be 8 European, 1 non-Europen.)

Two years in a row: ,169x.169 = .0286

Three years in a row = .169x.169x.169 = .00483

Four years in a row: (.169)^4 = .000816

Five years in a row: .169)^5 = .000138

This (five years in a row) will happen randomly by chance alone, about 1 time out of 7254 5-year periods under review. If it happens significantly more often than 1 time out of 7254, then probably some other factor than just random chance is at play.

 

[lappo]

I guess the exception is the games of 67 CE? The emperor Nero won gold medals in singing, acting, poetry and playing the lyre. The officials had to lock the gates to prevent the audience from leaving in the middle of his performances. He also won the chariot race, despite being thrown from the chariot. The other racers pulled up dutifully behind and waited for him to be helped back into the chariot. He finished the race and won.

Let this be a lesson to all figure skaters. If you fall, get back up and keep going. You might win!

OT again. No, actually no one dared to question the "fairness" of Nero because even the Hellanodikai probably held in higher estime their lives than the Olympic Games (btw, he also had the Olympic Games held before, in order to coincide with his trip in Greece). The episode which was questioned happened in 372 b.C. when one of the Hellanodikai won two equestrian games at the Olympics...there was an investigation and it turned out that he corrupted the panel, so the Hellanodikai were banned from taking part in the competitions themselves.

 

[OS]

Assume that each year there are 19 European countries in the pool and 7 others. Then the probability that exactly 8 European countries are chosen and 1 non-European country is:

"19 choose 8" x "7 choose 1" divided by "26 choose 9"

That is, 19x18x17x16x15x14x13x12 / 8x7x6x5x4x3x2x1 x 7 / 1, then divide by 26x25x24x23x22x21x20x19x18 / 9x8x7x6x5x4x3x2x1

= 75582 x 7 divided by 3124550

= .169 (About 17% of the time the judging panel will be 8 European, 1 non-Europen.)
Two years in a row: ,169x.169 = .0286

Three years in a row = .169x.169x.169 = .00483

Four years in a row: (.169)^4 = .000816

Five years in a row: .169)^5 = .000138

This (five years in a row) will happen randomly by chance alone, about 1 time out of 7254 5-year periods under review. If it happens significantly more often than 1 time out of 7254, then probably some other factor than just random chance is at play.

Thanks for the breakdown Mathman.

So in summary of the probability of consecutive 8/9 European panels like we have experienced in these last 5 years for ladies FS WC final goes:

1st year happening = 17% probability
2nd year happening in a row = 3% probability
3rd year happening in a row= 0.5% probability
4th year happening in a row = 0.08% probability
5th year happening in a row = 0.01% probability

(bear in mind 2015 actually had 9/9 100% European panel, which has about 3% probability happening 1st time)

Seems like some amazing...uh... COP = cosmic onesided play streak happening here, just by some 'pure chance' of course, about 1 in 36+ Millenium chance (7254 x 5-year periods) . No wonder some ladies PCS inflation start resemble the realm of a bad Michael Bay science fiction (yes I am thinking PCS Transformers). Except they kept on making even more extreme sequels every year, they just turning into absurd dark comedies.

People tends to over think when the answer is simpler. Whenever you find there are artificial inflation, you just need to observe who stand to benefit, who has gained, who has the greatest motivation to inflate, and who has the power, the will, the know how to put together the right condition to make it happen. Did it happen. If they all line up, you have yourself a perfect storm.

I personally wouldn't equate the larger complexity of modern day European socio-economic-political climate with the figure skating world, which in itself is fairly antiquated, with its own exclusive niche sociosystem governed by its own rules, culture, isolated from the rest of the world or even just the sport world in general (just look at the programs, the lack of care on diversity representation (for last 5 years judges panel at least for the ladies), and the refusal to be held accountable and challenged in general. If all the sports in the world are to rank according to utilising sport tech as part of the transparent progressive measurer of sporting performances, figure skating would come dead last). It is governed like an 'old boys' club at their own discretion, practically run like an Oligarch with their clansmanship that holds majority power.

It will be quite interesting to see how the rest of the Olympic season goes for the selection of judges panel, now ISU have lined up the perfect storm for the ladies. If Maxim is as half talented as Hanyu with good consistency, I suspect the same thing will have happened to the mens as well. I do wish IOC can insist on proportional representation as a rule for fairplay, but I don't see this happening until the next management change.

 

[4everchan]

I agree with all the maths but one thing to consider... these numbers work in a real aleatory pool...

we know that this is not the case here

 

[Baron Vladimir]

What Mathman calculated is what is the probability of the same European countries be chosen again and again. But your question was about any European countries in the panel, where the point is just to be 8 European countries of 9 of them in the panel, not the same 8 European countries. So that is not right result for your question. And its just funny how you use some numbers which are totally different from the thing you trying to prove, to prove the same. But you can keep trying.. just use logic sometimes

 

[el henry]

I guess the exception is the games of 67 CE? The emperor Nero won gold medals in singing, acting, poetry and playing the lyre. The officials had to lock the gates to prevent the audience from leaving in the middle of his performances. He also won the chariot race, despite being thrown from the chariot. The other racers pulled up dutifully behind and waited for him to be helped back into the chariot. He finished the race and won.

Let this be a lesson to all figure skaters. If you fall, get back up and keep going. You might win!

And also our earliest examples of nationalism. The Romans were none too happy that Nero was such a Hellenist that he set up Olympic games in Rome; not to mention the Neronia And he had a horrible singing voice, but who was going to tell him?

At least I understand these examples, as opposed to the math:eeking: