Why can't judges just judge what they see?

Why does everything have to relate back to politics, voting/judging blocs, momentum?

I find it bizarre that figure skating is one of the only sports where reputation matters. It is infuriating.

"Why can't the judges just judge what they see?!"... or rather, "Why can't the judges just judge what I see?!"

Seems like a lot of people want the latter, so they assume the judges aren't doing the former. But YMMV.

Of course, there is reputation and politics at play, and the judges do get things wrong. But this wave of skepticism around the judges as a whole is rather amusing. We don't have to agree with the judges, but automatically dismissing their opinions as uneducated or "blind" or whatever pejorative adjective people use, unless their opinions align with ours isn't exactly productive either.

It's like certain people have gaslighted themselves (and going into this despondent, woe-is-me/the-sky-is-falling/I'm-done-with-this-sport mode) by thinking the judges are wrong no matter what. Except of course when a judge gives their favourite skater huge PCS with major error(s) or high GOE in spite of an imperfect element (and then suddenly... silence). I suppose it make sense - why would certain people condemn a judge for overscoring their fave?!

Ambivalent said:

I find it bizarre that figure skating is one of the only sports where reputation matters. It is infuriating.

Click to expand...

This is true in a ton of sports where the most popular players (e.g. LeBron James, Sidney Crosby) get favouritism from the referees because of their status and popularity.

It would be nice though, to put a paper bag over each skater and their flag, and let the judges decide for themselves who is the best regardless of these external variables. As much as many judges can get influenced by bias/reputation, there are also those who mark fairly. And even for the ones who don't, they only have so much sway under the current system. If a skater lands a quad lutz - they're getting that base value, no matter what the judge tries to do.

Oh, CSG, never cease to provide the laughs.

Rissa said:

Oh, CSG, never cease to provide the laughs.

Click to expand...

Why, thank you... I will not cease providing you laughs any time soon! Feel free to provide something to the discussion in return.

While I'm sympathetic towards the frustration you feel, I will disagree that figure skating is the only sport where this happens. Like CanadianSkaterGuy pointed out, other superstars get benefit-of-the-doubt calls all the time--fouls in basketball, penalties/cards in soccer, pitching in baseball.

It's simple psychology: Anchoring. New information is processed in relation to what you already know, or the first piece of information you get. So those with reputations for being incredibly artistic or technically proficient confirm that anchor with a good performance, or have their off-days quickly dismissed. It's so clear in the ladies because their senior debuts are almost make or break for a lot of them, whereas men seem to have more time to come into their own.

And not to mention, this happens elsewhere--when a new singer comes out with an album, how well it's received depends on their previous releases. A director's new movie may become an instant blockbuster because it was done by Christopher Nolan.

CanadianSkaterGuy said:

Why, thank you... I will not cease providing you laughs any time soon!

Click to expand...

Aw.

CanadianSkaterGuy said:

We don't have to agree with the judges, but automatically dismissing their opinions as uneducated or "blind" or whatever pejorative adjective people use, unless their opinions align with ours isn't exactly productive either.

Click to expand...

Yes it is productive, because that's exactly what most judges are, and it has been hurting the sport for ages. The scores are a mess and great skating is being lost, as a result of competitors/coaches knowing that superficial programs and lower quality elements can pull huge marks. The judges are not paragons of objectivity and knowledge and scoring skill. They are people appointed by individual skating federations, needing no extensive skating knowledge or judging capability outside of VERY basic requirements, and they are persuaded to judge in such a way that best benefits their country's skating federation (or their own pocketbook, via "gifts" received).

vitamintea said:

While I'm sympathetic towards the frustration you feel, I will disagree that figure skating is the only sport where this happens. Like CanadianSkaterGuy pointed out, other superstars get benefit-of-the-doubt calls all the time--fouls in basketball, penalties/cards in soccer, pitching in baseball.

It's simple psychology: Anchoring. New information is processed in relation to what you already know, or the first piece of information you get. So those with reputations for being incredibly artistic or technically proficient confirm that anchor with a good performance, or have their off-days quickly dismissed. It's so clear in the ladies because their senior debuts are almost make or break for a lot of them, whereas men seem to have more time to come into their own.

And not to mention, this happens elsewhere--when a new singer comes out with an album, how well it's received depends on their previous releases. A director's new movie may become an instant blockbuster because it was done by Christopher Nolan.

Click to expand...

Interesting post! And I didn't know the term Anchoring before but it completely makes sense. Judges start from a point of reference and then rarely sway from that. It's why we see 9's no matter how poorly certain top skaters skate. And it's not judges but fans too - based on previous scores/records/personal bests they are predisposed to thinking a skater should get or should not get a certain score and so they have an almost visceral reaction when a score isn't to their liking (their fave is scored lower than previously given or their rival is scored higher). For clean SPs the top guys for example have scored well above 45. So that means no matter how poorly they skate or how poorly constructed their future programs are, etc. their PCS score is unlikely to deviate from what the judge has in mind based on prior results. A top pairs team might get a +2 or +3 even with a touchdown on their throw because a judge starts with a +4 or +5 in mind and doesn't want to deviate too far from that.

CanadianSkaterGuy said:

Interesting post! And I didn't know the term Anchoring before but it completely makes sense. Judges start from a point of reference and then rarely sway from that. It's why we see 9's no matter how poorly certain top skaters skate. And it's not judges but fans too - based on previous scores/records/personal bests they are predisposed to thinking a skater should get or should not get a certain score and so they have an almost visceral reaction when a score isn't to their liking (their fave is scored lower than previously given or their rival is scored higher). For clean SPs the top guys for example have scored well above 45. So that means no matter how poorly they skate or how poorly constructed their future programs are, etc. their PCS score is unlikely to deviate from what the judge has in mind based on prior results. A top pairs team might get a +2 or +3 even with a touchdown on their throw because a judge starts with a +4 or +5 in mind and doesn't want to deviate too far from that.

Click to expand...

In general judges do judge what they see. Which is not always the same as what your seeing on the telly. It's amazing how different a performance can look in person without the tight camera angles.

Also people forget that judges do have stylistic preferences and there is disagreement over what technique is the "best".

tothepointe said:

In general judges do judge what they see. Which is not always the same as what your seeing on the telly. It's amazing how different a performance can look in person without the tight camera angles.

Click to expand...

This is very true.

Some things that are much more apparent live are the amount of ice that a skater can cover with each stroke, how efficiently they generate power/accelerate, what kinds of sounds their blades make or don't make, etc. The judges see (and hear) these things directly, whereas they are mediated and often some aspects enhanced or downplayed in translation to video.

And these aspects of skating skills may be among the most obvious and the most important aspects of the performance to many judges.

Fans who learned how to watch skating on video and haven't spent a lot of time in rinks watching up close might not be aware that these are considerations in the judging at all. Even very knowledgeable fans as well as experienced skaters and other judges watching on TV who do know to look (and if possible listen) for these skills may not perceive as big differences on video, or from the cheap seats, as they would live and up close.

Of course some judges are more knowledgeable than others, just as some fans are.

And yes,

judges do have stylistic preferences and there is disagreement over what technique is the "best".

Click to expand...

Sometimes unconscious expectations or preferences do affect what judges perceive. And yes, sometimes politics does consciously come into play.

But we each have expectations and preferences that affect what we see as well. And if we're watching on TV, what we see is shaped by production choices that may skew our perceptions one way or another, without our necessarily being consciously aware of how.

In general, if judges' scoring doesn't match how we each might have scored the same performances based on what we saw, especially if we were watching on video, the reasons are most likely that the judges saw the skating differently than we did and judged what they saw, not what we saw, and perhaps that they valued different aspects of the skating differently than we did.

Ever since I first started attending live skating competitions and studying the rulebooks, my impression has been that the more I learned, the more I understood why the judges scored the way they did. And that honest disagreements are par for the course in a sport as complex and qualitative as figure skating.

Of course judges are human beings and not immune to the same kinds of biases or imperfections that may affect our own viewing, sometimes in opposite directions. But my impression is that those effects are the exceptions. Most of the time the judges just judge what they see according to their best knowledge, which on average is higher than ours. When we agree with them (which will be true for the vast majority of decisions), we don't notice. When we disagree, it's more obvious, but why jump to assuming the worst to explain the reasons?

Besides everything that is said, that tastes differ, and one might think a jump satisfies 3 bullets, while the other one will think of only 1..besides all that obvious difference in opinions between all human beings, there are few things to consider:

1) Judges have to sit through all the groups, meaning sometimes hours and hours of watching figure skating, I would say this is a rather grueling and tiring work in general, and I wouldn't expect robot-like precision from any of them.
2) Decisions have to be made in split seconds, and you have to look up instantly after putting your mark, because there's no time for thinking them over. They give marks more on how they feel, rather than sitting and thinking through every single bullet.
3) Judges are punished if the results/marks that they give differ way too much from what other judges give. Although highest and lowest marks aren't taken into account anyways, if you're caught giving someone consistently much lower or much higher marks, you are very likely to end up punished.
Thus, judges sometimes go with the marks they expect others to give to the skater, and this is what we call to be "reputation" marks.

Besides, judging isn't a very well paid job, they have to travel a lot of time and have rather poor conditions in general, hence ISU constantly experiences the lack of judges.
Even when they catch someone cheating the results, they might punish the person, suspend him from judging international events for a few years, but they are not gonna fire that person, because ISU doesn't have this luxury of being able to throw out whoever they don't like. There's just not enough people.

nussnacker said:

Besides, judging isn't a very well paid job, they have to travel a lot of time and have rather poor conditions in general,

Click to expand...

It isn't a paid job at all. Judges are volunteers.

At best, they get their expenses paid to travel to the competition location (although they may not have any time for sightseeing or otherwise enjoying the location outside the rink) and get treated as VIPs while they're there.

At worst, they often have to travel to less-than-desirable locations and do their judging tasks in less-than-glamourous facilities for, as you say, long hours in the cold. There's a lot of putting in their dues in these kind of conditions before getting appointments and assignments to judge at elite levels.


To respond directly to the question in the original post:

The vast majority of decisions by the vast majority of judges constitute judging what they see.

A minority of decisions may be influenced by something other than what the judge just saw.

Of those confounding factors, many are due to unconscious psychological effects of the kinds of decisions the judges are asked to make and the conditions under which they make them.

And a few might be accurately described as politics and voting/judging blocs.

"Everything" doesn't relate back to politics, voting/judging blocs, momentum. Sometimes some decisions do, but most do not.

gkelly said:

It isn't a paid job at all. Judges are volunteers.

At best, they get their expenses paid to travel to the competition location (although they may not have any time for sightseeing or otherwise enjoying the location outside the rink) and get treated as VIPs while they're there.

At worst, they often have to travel to less-than-desirable locations and do their judging tasks in less-than-glamourous facilities for, as you say, long hours in the cold. There's a lot of putting in their dues in these kind of conditions before getting appointments and assignments to judge at elite levels.

Click to expand...

No, it is a paid job. You are probably talking about judging at Olympics being voluntary I guess, as well as some judging in countries on a national level.

ISU judges are paid for all the events they judge.

7.4 Officials Remuneration
The Referees, the Technical Controllers, the Technical Specialists and the Data & Replay Operators and
Judges will receive remuneration in accordance with ISU Communication No. 2096 (or any update of this
Communication). Only hard currencies are acceptable. The remuneration should be paid upon accreditation.

nussnacker said:

No, it is a paid job. You are probably talking about judging at Olympics being voluntary I guess, as well as some judging in countries on a national level.

ISU judges are paid for all the events they judge.

7.4 Officials Remuneration
The Referees, the Technical Controllers, the Technical Specialists and the Data & Replay Operators and
Judges will receive remuneration in accordance with ISU Communication No. 2096 (or any update of this
Communication). Only hard currencies are acceptable. The remuneration should be paid upon accreditation.

Click to expand...

They’re remunerated for the travel expenses, meals, hotel, etc— but they don’t actually get paid to judge. (Well, judges don’t. I don’t know whether other officials do.)

Thanks for the reference.

The amounts listed in Communication 2096 range from 200 to 450 Swiss Francs per event.

So we're talking about what might be considered stipends or honoraria, rather than wages or salaries. In most cases, the organizers would be paying out more money per judge separately on travel, on hotel accommodations, and on food than what actually makes it into each judge's pockets.

gkelly said:

Most of the time the judges just judge what they see according to their best knowledge, which on average is higher than ours. "Everything" doesn't relate back to politics, voting/judging blocs, momentum. Sometimes some decisions do, but most do not.

Click to expand...

This is categorically wrong. Nearly every decision made by most of the judges is informed by scoring the given skater "as they have been told to score them". It's very plain to see in the scores that there is little actual analysis going on by the judges as to each component, or even objective assessment of the technical elements. They do not have more knowledge about skating, much less the ability to accurately separate the competitors within the scores. Even if someone has great skating knowledge, it does little good if the actual scores given out do not reflect the precise differences between the competitors. If you are giving completely average spins +3 GOE and excellent spins +4, or only .25 or .5 higher on a component for a big difference in performance, then the quality is not being accurately marked.

Being a great judge is entire skill unto itself, and the ISU does not teach it nearly well enough, nor are judges selected based upon universal meritocracy. Calling it "Anchoring" or whatever you want, but it's bad judging.

nussnacker said:

Even when they catch someone cheating the results, they might punish the person, suspend him from judging international events for a few years, but they are not gonna fire that person, because ISU doesn't have this luxury of being able to throw out whoever they don't like. There's just not enough people.

Click to expand...

This is hilariously twisted logic. If someone is a cheater, then they are inherently a worse judge than other available options.

There's "not enough people" because of the ISU's own design. They don't train the judges well enough, only 1 judge per country allowed, and only the judges that country submits. Well, the hilarious part about this is how countries like Russia circumvent it. Russian judges just get planted in other European countries.

Blades of Passion said:

This is categorically wrong.

Nearly every decision made by most of the judges is informed by scoring the given skater "as they have been told to score them". It's very plain to see ...

Click to expand...

Saying something loud and saying it over and over doesn't make it true. Your assertions are quite clearly not plain to see. If they were "plain to see," there would not be different points of view on these threads.

(That's what I think, anyway. )

nussnacker said:

No, it is a paid job. You are probably talking about judging at Olympics being voluntary I guess, as well as some judging in countries on a national level.

ISU judges are paid for all the events they judge.

7.4 Officials Remuneration
The Referees, the Technical Controllers, the Technical Specialists and the Data & Replay Operators and
Judges will receive remuneration in accordance with ISU Communication No. 2096 (or any update of this
Communication). Only hard currencies are acceptable. The remuneration should be paid upon accreditation.

Click to expand...

Also in Senior B international events judges don't get paid as Communication 2096 refers to ISU Championships (Worlds, Euros, 4CCs, WTT) and Grand Prix series.
In ISU Championships and GPs they also get travel expenses covered.

In Senior B they get hotel and meals paid by organizers. But organizers are not obliged to cover travel expenses for judges (they do have to cover them for referee and technical panel) and in many cases they don't do it.

In other words, the net financial impact on an individual of being an international judge in most cases will be losing money rather than gaining income over the course of a career. Especially if they have a day job that they lose money from if they're not there -- which is why most judges probably have more flexible sources of actual income.

If a judge can get to the point of being an ISU judge instead of just an international judge, and especially if they come from a federation that will pay expenses that the ISU or the host federation doesn't pay, then they might come out ahead in the years they're at that level. But not by much.