Ask Chuck Kennedy

[m_conners]

Chuck,

Do you think the ratings system will end up going "real-time" at some point in the future?

The ratings system is already a fantastic system, just wondering if you ever see the possibility of a real time ratings system...

Thanks man,

mc

[ck34]

Short answer is 'No.' I think you'll see improvements to the online unofficial ratings based on improvements for TDs to post results and improved "correctness" in terms of course layouts being entered properly. We are also working on a way to keep the unofficial round ratings up there until the official ones get processed so there's not a "dark" period inbetween.

Since TDs are allowed up to 30 days to get reports in and still some do not meet that deadline, this is the real time lag in the system we can't improve. If we process events any faster, the rating sequence of events will get more out of order than it already is. By spacing out the updates, there's time for Dave to get most of the reports for events hosted during a certain time period processed at the same time.

The process requires manual checking of properly filled out reports. There's no easy way to get around proper assigning of course layouts to each division who played them. We can sometimes guess but we really have to mostly rely on the TD to get that right. That's the Achilles heel of the process and the one most likely to generate wrong values.

We catch a lot of the mistakes before events are processed with our semi-automated process. If we automated the process further, more bad results would go out and we'd just have to go back and correct them after they were posted more than we do now.

One cool thing that a volunteer might write is a "What If" program either for the web or for someone to download that allows players to load in their new unofficial values and have it estimate their rating. Disc Golf United already has realtime updating of your handicap where you can enter your results as you get them. That's the closest you can get to seeing your rating or handicap after every round at the moment.

[sandalman]

would the calculation behind such a tool be simply to calc the SD, determine if any rounds should be dropped, determine which rounds get doubled, then do the calc with proper inclusions and weightings? anything i am missing?

[sandalman]

triple post removed

[sandalman]

double post removed

[AviarX]

Chuck, have you ever considered dropping late penalties and wrong score penalties from total scores for rounds only for the purpose of calculating round ratings (not to calculate total score relative to other competitors at an event)? Or have you considered dropping those penalties when calculating course SSA's? just curious.

[ck34]

I think this would be an option available after a person brings up their current Ratings Detail. They would click the 'What If' button and their current round ratings and dates would be listed. They could then enter as many round ratings and dates as they wanted. No events even need to be listed or the round order within a day.

The logic for the calc would first eliminate rounds 12 months older than last date entered unless there were fewer than 8. Then the number of rounds to be doubled would be determined and which ones. If only one of two on a date was to be doubled, choose the higher rated one. Then, the SD is calculated to see if any rounds are knocked out. (This is the way Roger does it now but I think the SD knock out should be done first before the doubling.)

Then calc the average. The primary thing that's slightly different is that the round ratings have already been rounded. The other item is that the number of props may increase with people renewing since the time an event posted unofficial ratings and when the official ones are done. This happens all of the time early in the year.

[ck34]

Chuck, have you ever considered dropping late penalties and wrong score penalties from total scores for rounds only for the purpose of calculating round ratings

Yes we have. The question we had was whether this might be a typical behavior for the player or not. If so, then it should be part of their rating because they might be prone to this error every so often. We weren't smart enough to divine one time situations versus a pattern so we've left it in there, let alone the hassle of TDs having to flag scores on TD reports. Remember that players who shoot more than 60 points below their rating are excluded from the SSA calc. So if someone is late by two holes, there's a good chance that score may not be used for the SSA even if it isn't low enough to get excluded from that player's rating average.

[AviarX]

Remember that players who shoot more than 60 points below their rating are excluded from the SSA calc. So if someone is late by two holes, there's a good chance that score may not be used for the SSA even if it isn't low enough to get excluded from that player's rating average.

i thought a round had to be 2.5 STD below the rating for the rounds used in determining the rating for it to be dropped? If i have a round 60 points or more less than my rating is it automatically dropped?

also is it 60 points less than the rating at the time the round was shot or 60 points less than the rating the round shot figures into?

[ck34]

You're missing the sequence here. The SSA for each round is temporarily calculated based on the scores and ratings of propagators. If a propagator shoots a round with an unofficial rating more than 60 points below their rating, then they are excluded from producing the SSA calculation, but not from getting a rating for that round.

Once the SSA is calculated from propagators with acceptable round ratings, everyone who played that layout will get a round rating from that SSA regardless whether they are a propagator or not, or a member or not.

[oceanjones]

Chuck, I know it's been brought up before, but...I would love to see the averages for a tourney on our player information page. I know its a pretty quick calculation to do ourselves, but it would be so much easier if there were just a TA (Tourney Average) column. I really like knowing those averages, they are a much more accurate indicator than finishing places. (Just because I place "well" at one tourney doesn't mean I will at the next, sometimes I don't notice all the high rated players and wonder..."Now why did I tank so badly...I didn't feel like I was tanking.")

[AviarX]

thanks for clarifying that you were talking about course SSA calculations. why doesn't the figure of 60 points slide higher for lower rated players and lower for higher rated players?

[ck34]

I'm not sure what averages you are referring to? Something like the average round ratings for the person taking last cash? The average score or SSA for the course layout?

The major problem is that linking this back to the player pages with unofficial results doesn't make sense when scores are first posted because TDs aren't held to any quality posting standards beyond getting the scores correct. In fact, we'd rather they spend their time getting the TD report done and sent in rather than do a complex course layout online for an event like the BG Ams for example.

Once the event is processed with official ratings, the SSAs for each round are posted with the Course Stats link on the tournament results page. We're working on getting those linked back to the Course Directory but I'm not sure what the status is on that.

[ck34]

why doesn't the figure of 60 points slide higher for lower rated players and lower for higher rated players?

It could but we don't retain each person's SD in the database. So, you would really be churning the server to process the SDs for each person for each event with minimal benefit versus a fixed figure to do the unofficial calculations. Likewise, doing the official calculations would require an SD recalculation for everyone as each event was processed which would dramatically increase the processing time. If we did anything along these lines, we might consider a sliding scale which would use a larger cutoff point the lower the prop average rating for a rating pool.

[AviarX]

okay, i think i waded in a little too deep. thanks for the answer but i am going to head back to the kiddie pool before i start sinking [img]/msgboard/images/graemlins/smirk.gif[/img] i am thinking it's safe to say it would be a little more accurate that way but that the drawbacks of doing it that way make it impractical...

[ck34]

This thread went somewhat adrift from your question, Mike. I'm not sure realtime is realistically in the future. However, I can see adding more regular updates than 5 per year, possibly in 2007.

[Parkntwoputt]

In fact, we'd rather they spend their time getting the TD report done and sent in rather than do a complex course layout online for an event like the BG Ams for example.

If the correct round ratings are not going to be used for BG Am's then all the ratings for this event should be thrown out.

The ratings are all for one course for each round. It basically shows that every player played the exact same course for all four rounds. Impossible since there were 8 courses used.

Sure this is confusing, but they got it right for the BG Open. If they cannot get it right for the BG Am's throw them out. I don't care if my round rating was high or low, but if it is not correct then it is not the truth about how I played.

Quality is better then quantity in this point.

[ck34]

BG Ams is being done properly for the official ratings. But it's more complicated to do the unofficial ratings properly with the online software than it is to do the TD report correctly.

[Alacrity]

Kris,

Have you done this before? You can define the courses for the "on-line" scoring and it will calculate differently for each Pool on each course, but it is cumbersome and occasionally odd things will occur. For instance, I have seen a proper setup for the on-line scoring and one pool did not get a rating for one round. The on-line scoring is nice, but it is only an estimate. I have a scoring spreadsheet that is being used by many TD's and it also calculates an estimated round rating, but once again, if it is not setup properly, the estimates can be off by quite a bit.

As long as the TD report is correct the rounds will be calculated correctly.

If the correct round ratings are not going to be used for BG Am's then all the ratings for this event should be thrown out.

The ratings are all for one course for each round. It basically shows that every player played the exact same course for all four rounds. Impossible since there were 8 courses used.

Sure this is confusing, but they got it right for the BG Open. If they cannot get it right for the BG Am's throw them out. I don't care if my round rating was high or low, but if it is not correct then it is not the truth about how I played.

Quality is better then quantity in this point.

[m_conners]

This thread went somewhat adrift from your question, Mike. I'm not sure realtime is realistically in the future. However, I can see adding more regular updates than 5 per year, possibly in 2007.

This thread is providing some good ratings talk, thanks Chuck! I now understand how real time ratings would tough to achieve.

[sandalman]

ok boyz'n'girlz, herre is something. it is basic, no frills, but pretty darn accurate.

ratings calculator

its an excel spreadsheet... just replace my data with your own and boom, out comes your anticipated rating.

ideas = welcome

[m_conners]

Very cool!

[AviarX]

wow Pat, that is awesome and user-friendly. you are a pretty cool guy for someone who likes the 2 meter rule and throws Discraft

i'll use it to see if my predictions using old math were correct... thanks again

[ck34]

Calculator looks good. Two tweaks needed are: (1) The 2.5SD is capped at 100 maximum. (2) you need to use the ROUNDUP function because 14 will be doubled with 53 rounds, not 13.

[Parkntwoputt]

BG Ams is being done properly for the official ratings. But it's more complicated to do the unofficial ratings properly with the online software than it is to do the TD report correctly.

ah.....what a mess..... Chuck needs a vacation.

"I see" said the blind man as he walks into the bar.

[sandalman]

i made the two changes chuck mentioned and posted a new sheet to the same link.

[AviarX]

i have a question Pat. the rating i get for the next update was lower based on your calculator's std. deviation figure than i expected (4 pts). does it calculate the std deviation of the rounds before the recent 25% are double weighted or after? i believe the proper way to do it would be to get the std. deviation with no double weightings first, and then double weight the rounds and calculate the rating. Chuck?

[ck34]

Roger does standard deviation calc based on non-doubled rounds but does the rejection after rounds have been doubled. So, it's unlikely but possible that a double weigthed round will get dropped.

[AviarX]

good. Pat could you please tweak your calculator so that the std deviation is calculated before doubling any rounds and then the rating will be calculated after doubling the rounds ...? i want those 4 extra points that will result from dropping one of my two sub-900 rounds

[sandalman]

SD is already calcd before doubling. SD is a derivative of the round ratings, not the doubled ratings.

there are any number of reasons the spreadsheet will differ from the actual rating. witness the changes that occur whenever round ratings go from unofficial to official. this ia an indicator only. that being said, i have been using the same algorithms for three years now and i only failed to correctly calculate my official rating one time, and i think that was due to some sort of correction they applied to help fix the high-vs-low rated pools problem.