Page 11 of 11 FirstFirst ... 2 3 4 5 6 7 8 9 10 11
Results 151 to 154 of 154

Thread: Judging math details

  1. #151
    On the Ice
    Join Date
    Dec 2016
    Posts
    252

    0 Not allowed!
    Quote Originally Posted by Mathman View Post
    The other problem is that I did not really address your (Miller's) actual question; instead I substituted an easier one that I happened to know the answer to. The actual question is not "How big an interval do we need to be 95% confident that the true value of Nathan's performance lies somewhere in the interval." The question is rather, "Did Nathan really beat Shoma, or is the contest "too close to call" using standard statistic analysis. The difference between these two questions is captured by this language: the first question requires an "independent sample test," while the latter calls for a "paired data test."

    What this means is that the analysis given above does not take into account the possibility that a particular judge might be tougher than another and might give lower than average scores to all the skaters, whether they skated well or not. If there are only two skaters in the contest, this can be addressed in a straightforward way, but with many skaters it starts to get murky. There is a cool way to handle this called a 2-way Analysis of variance (ANOVA), but I am afraid to try it because the conclusion might turn out to be that none of these numbers is worth squat and we have wasted our time.
    Of course this was the next question I was going to ask, LOL, but given that it was only 0.5 points out of 280 I would imagine it was the latter, plus of course there's always the random effect of things like deductions to take account of. According to the Eurosport commentators the only reason Shoma lost was that he had a new music edit that was longer than the 2 mins 50 seconds allowed for the SP and so he lost a point as a time violation. It would take a pretty good statistical system to take account of that!

    Re Moriel's point, maybe there is someone out there gathering the data and that will analyse the data one day. It would be interesting from an intellectual point of view certainly.

  2. #152
    On the Ice Mathman's Avatar
    Join Date
    Jun 2003
    Posts
    34,801
    Country: United States of America

    2 Not allowed!
    Quote Originally Posted by moriel View Post
    But for a paired test, we could go by the difference: calculate the differences between the individual scores first, and then average that...
    I thought of a crazy idea. We could match each skater against each of the others in this way. We would end up with a bunch of statements like, the probability that skater A really outperformed skater B is more than 95%, but we cannot say that the probability that skater C really outperformed skater D is more than 95%. Then we would have some sort of spreadsheet like in 6.0 ordinal judging, OBO, with only definite slays counting.

    The best of both worlds.

    For multiple skaters, we could do some sort of regression, with judge being one of the variables.

    But the thing is, it is VERY likely that very few of those numbers are worth anything, and we will come up with a result such as "in fact, 3 of those 6 skaters should have gotten [gold] medals"
    IMHO the conclusion (three skaters all won -- a sports decision) does not really follow from the premise (most of the numbers do not pass statistical muster -- a statistical decision). No, we cannot say that Nathan's performance would beat Shoma's performance 95% of the time -- but this time, with this panel of judges -- it did.
    Last edited by Mathman; 12-09-2017 at 05:28 PM.

  3. #153
    On the Ice Mathman's Avatar
    Join Date
    Jun 2003
    Posts
    34,801
    Country: United States of America

    0 Not allowed!
    Quote Originally Posted by Miller View Post
    Re Moriel's point, maybe there is someone out there gathering the data and that will analyse the data one day. It would be interesting from an intellectual point of view certainly.
    Publish or perish! I think we can be sure that many professors and researchers, and especially their graduate students, will jump on these questions for their next scholarly paper. There was a lull in such activity during the time of anonymous judging when it was hard to tease out the relevant data (not that people didn't try. ).

    For that matter, the ISU can hire statisticians just like anyone else can, and I have no doubt that they are constantly running statistical tests and analyses on the huge amount of data generated in a figure skating season.

    IMHO, though, the most relevant observation is that, yes, certainly it is "interesting from an intellectual view."
    But I think that the ISU (and the IOC) instead take a "sports point of view" and are guided by these principles:

    1. A sports contest should have a clear winner.

    2. Whoever scores the most points, wins.

    Is this an appropriate model for the sport of figure skating? Well, a few of us fought the good fight, but in the end that ship has sailed. The IJS is what we've got.
    Last edited by Mathman; 12-09-2017 at 06:08 PM.

  4. #154
    Bona Fide Member
    Join Date
    Mar 2015
    Posts
    3,320

    1 Not allowed!
    I personally would find interesting poking around the data. My main issue is really collecting it, as i am rather lazy.

    @Mathman, also, with enough data, we could probably have a collection of several competitions judges by a specific judge and so on, which would even add up.

    As for the paired tests for everybody, i think that would require some bonferroni correction, which would kill the significance and result in some pretty wide intervals.

Page 11 of 11 FirstFirst ... 2 3 4 5 6 7 8 9 10 11

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •