Ask Chuck Kennedy

[Yeti]

Originally Posted by bruce_brakel

I didn't see a point in quoting back the whole post. It's up there.

Round by round ratings have always been like that. On the same day in the same conditions for the same score in different pools, Geoff Bennett got a 1030 and I got a 970 unofficially. When they averaged the pools together for official ratings the anomoly disappeared and we both had 997 or something.

Chuck, this really not about the Women's Global Event. Your points are correct there and almost everyone knew going into any Global event that the people who play the course and what course is played actually make all the difference in how well you do in the final results.

I am backing John and the 100's of others that we talk to across the nation that are frustrated with all of the anomolies of the ratings system.

You even spoke of the Krusty Brand that is represented with the current system: It's not Good, It's Good Enough.

Ratings are LOVED and DESIRED by the bulk of the membership. WE/THEY deserve a system that works VERY well and not just good enough were statistical manipulation and averaging are used to limit the inaccuracies.

Let's start playing the course for ratings and not the folks that are there. Don't bother posting the usual argument about the courses changing year to year, we have all heard it. The TD's know almost exactly what their tournament courses are as far as Par goes. There is a way to do an SSA for each course based on every hole is a Par 3 and some holes play 1 to even 3 strokes harder. You have enough stats to be able to add in factors of tightly wooded verses open course slopes as well as how much more difficult a course is in 12 mph verses 25 mph or in rain.

This will take a lot of work I am sure, but I am trying to provide you with some job security I am proud of the work you have done thus far. I just want our system to be a GREAT system. If the ratings are the number one reason members are members, it should be the best.

[John Hernlund]

Originally Posted by Yeti

...ratings are used to determine more now than ever before and the list growing. PDGA ratings at the Pro level are being used for seeding players in large scale tournaments, $1,000+ skins match involvement, sponsorship deals, sponsor bonuses, PDGA Player of Year Awards and now even Global event payout. WOW...

WOW, indeed!

Any top touring pros can be straightforwardly evaluated relative to any other top touring pro based on their heads-up scores in the same tournament rounds (since they play many of the same tournaments). This is easy, and so simple that nobody could really complain about the outcome. Just choose the time period for comparison, add up total throws in shared tournaments, et voila.

Just for the sake of example, I did this for Nikko Locastro and Nate Doss using data from the website. In 2012 Doss vs Locastro is at 876 vs 854 after 5 shared tournaments (Nikko is ahead by 22 throws). In 2011 Doss vs Locastro was 3136 vs 3159 (Nate ahead by 23 throws), and in 2010 it was 3278 vs 3193 (Nikko ahead by 85 throws). The ratings smooth over this time period...only now is Nate's rating higher than Nikko (1045 vs 1044), even though Nate is shooting slightly worse than Nikko so far this year and his better rounds against Nikko were in 2011.

The ratings in this example are out-of-phase with the data by about a year, as a consequence of ratings being averaged over many rounds in the past (including rounds both competitors did not play). This certainly raises questions about using ratings for player of the year or similar measures of a player's performance during a given time span. Anyways, I think it would be much cleaner and better-posed to use head-to-head throws over a given time period, instead of ratings.

[bruce_brakel]

Ratings being what they are, you should always consult the Vegas line before betting more than you can afford!

[cgkdisc]

Fixed course SSAs are a big FAIL for tournament quality ratings and just not workable. Ball golf wishes they could use a system like ours (actual words to me from their handicap committee 2001) but their clubs are too invested in the current financial dynamics of their handicap system to change.

Every time I do the ratings there are new and temp courses being played that would need to get added to the database. Where would those ratings come from? In order to add new courses to the database for future events, the TDs would have to become course designers: record the wind, calculate foliage density and measure hole lengths accurately.

Some TDs are still challenged even now with just entering course layouts, number of holes and divisions for the current system. Maybe only half enter lengths for each layout. It's apparent a fair percentage of those are guesses or worse when the short and long layouts are each entered as the same length.

I believe the current system is as objective as we can get. Every breath of wind at any time during the round is automatically accounted for via the scores thrown. There's no subjective judgment nor any judgment at all involved. Everyone's ratings are processed for every round using the same formulas for round ratings and everyone's ratings update is calculated the same way. The only judgment comes in for the few events when there aren't enough propagators and we have to make some judgment calls using objective information.

The new league program is further proof that the current dynamic system is the fairest way to do ratings. We have reports from several leagues that course SSAs are coming in 2-3 shots lower than events run on the same course. However, the average ratings aren't any lower because the system doesn't allow it. Several people were concerned that ratings would get inflated from league play but it's not happening and actually can't happen.

The important thing to note is we're getting confirmation that the nature of the event itself seems to impact the SSA of the course that day. Top players are always more impressed with say a 47 rated 1070 shot at an NT or Major than a 47 shot during a C-tier or league. Well, what we're discovering is that the ratings show that score during a Major should be more impressive because that 47 in league might only be rated 1052, not too shabby, but not 1070. Fixed SSAs could never reflect that difference in round quality. The current ratings system does and SHOULD have that capability.

[NASCARVW]

Originally Posted by cgkdisc

Since the next ratings update is after your June event, you need to contact the TD to switch divisions to Intermediate. The TD also may see the problem before the event and contact you about switching.

Just seems strange that a tourney in early March isn't included in the May updates...

[jconnell]

Originally Posted by NASCARVW

Just seems strange that a tourney in early March isn't included in the May updates...

It appears the tournament results haven't been officially submitted to the PDGA offices, as the event results still show up as "unofficial". That would be the reason they haven't been included in the ratings updates...they can't include what they don't yet have. I'd contact the TD and find out what's up. Yours seems a good case for having a beef with them not doing their duties in a timely manner.

[ERicJ]

Originally Posted by Yeti

The TD's know almost exactly what their tournament courses are as far as Par goes.

While that might be true for NT's, as you work your way down to C-Tiers I doubt it. Out of the 1300+ 2011 events how many TD's are also course scoring experts?

Does the sport even have a widely accepted quantitative definition of "Par"?

Originally Posted by Yeti

There is a way to do an SSA for each course based on every hole is a Par 3 and some holes play 1 to even 3 strokes harder. You have enough stats to be able to add in factors of tightly wooded verses open course slopes as well as how much more difficult a course is in 12 mph verses 25 mph or in rain.

There is so much subjectivity required there.

How is a TD in AZ who has never played in MI or NC supposed to accurately judge his course's woodedness?

What about rounds that start with calm wind, have a storm come through in the middle with heavy gusting winds, and then end calm again?

The examples are endless....

[JenniferB]

Originally Posted by steveganz

I'll take a look at men showing up in the search for women's statistics. Thanks for the heads up on that!

Currently, the statistics search only displays players who have competed during the current calendar year in any given age bracket that you are searching within. As it stands now, if a pro woman player plays in any amateur division they show up when searching for amateur women. I've been trying to get a consensus on this and am leaning more and more towards simply not including pros that play in am divisions and ams who play in pro divisions in the search results.

Like Chuck alluded to, there is a lot that is wrong with the statistics search. I've spent some time tweaking it recently and expect that work to continue over the next few weeks.

I will definitely add clarification on how the statistics search works right there on the page so at least people will know what to expect when they perform a search.

Thanks Steve. I'm sure you have other priorities right now, but if it would be possible to add in additional search filter options for 40+, 50+, etc. I'd really appreciate that. You don't necessarily need to do away with the the existing age filters of under 40, 40-49, etc. Just adding the additional ones would be appreciated.

[JenniferB]

Originally Posted by cgkdisc

...
"The 95% error range is directly affected by how many props shoot scores on a layout, not how good the ratings are of the props."
...
If I had my preference, Global events would be at least four rounds so the statistical fluctuations on all of the sites have a chance to converge toward the "true" numbers whatever those may be. We went with three rounds the first Global to see players' acceptance of the format as an option. The Women chose two rounds not because it was good enough statistically but because they felt this would be more likely to increase participation which was a more important goal for them.
...
If you look at the final rankings, there were very few surprises in the rankings compared with their initial ratings. Only the unrated non-PDGA players in some of the lower divisions surprised by not knowing where their skills currently were in comparison to the division they entered.

Would it be possible to develop a confidence metric for a set of ratings on a layout based on how many propagators played, and then correct or adjust an entire set of low confidence ratings in a global event by comparison to sets of ratings having higher confidence metrics and the ratings of players therein?

I'm trying to think of how it might work.

I recall at one point you said players are statistically improbable to shoot more then 50 points (or something like that) above or below their rating, and that the statistical improbability is greater with higher rated players.

If a certain percentage of player's ratings on a layout are higher or lower than expected, and the number of propagators is low enough, and the rating of a player is high enough, could it trigger a thresholding effect to normalize the ratings on that layout for all players on that layout, or adjust them upwards or downwards?

Or could you determine the course SSAs for each round at each event, adjust raw scores of players in a weighted fashion (based on number of propagators for that SSA, and how high above or below a player's own rating they shot) to put them all on the same "virtual layout," and refigure the ratings for everyone participating in the global event?

If you had an expected SSA range for each course layout, could you use that to detect a need to adjust ratings on that layout? If so, maybe you could develop expected SSA ranges for commonly recorded course layouts in the ratings database, and require that global events having less than a given number of competing propagators be played on such a known layout.

[cgkdisc]

Requiring higher minimum number of propagators for events to be included and playing at least 3 rounds would be better ways to improve the quality of the ratings generated for global events. If certain locations pay into the global purse and don't meet the criteria, you just rebate their entry fees for their global not their local competition. That way the ratings for every site and player are calculated the same way and you can handle still new courses or temp courses where there's no data to adjust things as you propose.

[bruce_brakel]

Originally Posted by bruce_brakel

Ratings being what they are, you should always consult the Vegas line before betting more than you can afford!

This past weekend at the tournament I played, you got a 25 point boost if you played in the Advanced division at The Ponds versus the Intermediate Division. (Compare MA1 Rd 3 to MA2 Rd 1). The weather conditions were the same and the PDGA will erase that difference when the rating go official by calculating one averaged rating for the three tournament rounds played on that layout.

However, none of the MA1 played The Ponds short tees. Whatever ratings detriment the MA2 and MA3 divisions incurred during that round will not get corrected by averaging with some other MA1 round.

Once a year, when they let 945 rated players into USADGC, I'd rather have those 25 points. 20 times a year, I'm fine with having the choice to play MA1, MA2, MM1, MG1, MPG and MPM.

I have been contemplating trying to intentionally jigger my rating up by only playing at tournaments where there are different layouts for pro and advanced so that my rating does not suffer the MA2/MA3 mathmatical suppression effect. And then I think, why do all that to get into USADGC? Why not just practice your putting a little?

[JenniferB]

Originally Posted by cgkdisc

Requiring higher minimum number of propagators for events to be included and playing at least 3 rounds would be better ways to improve the quality of the ratings generated for global events. If certain locations pay into the global purse and don't meet the criteria, you just rebate their entry fees for their global not their local competition. That way the ratings for every site and player are calculated the same way and you can handle still new courses or temp courses where there's no data to adjust things as you propose.

Do global event TDs already have the option of imposing a requirement for a higher number of minimum propagators? If so, all that remains to do is start asking for the higher minimum number.

[ERicJ]

Chuck (or maybe Steve?),

The new tourney results listings are now showing some player ratings in bold font.
Is that to indicate propagators?

E.g.

[steveganz]

That is correct. Also, if you hover over the player rating it will display a tooltip that says "propagator".

[bruce_brakel]

The Coolness Committee officially approves. Can you also make it so that if a player is registered for a division that he is not eligible to play in, his name appears in red bold type. The tool tip could say, "bagger."

[cgkdisc]

Steve also made it so you can mouse over a player's name to see their hometown and state/country. Nice additions.

[ERicJ]

Originally Posted by steveganz

That is correct. Also, if you hover over the player rating it will display a tooltip that says "propagator".

Excellent!

On a related note... any progress towards locking the display of those player ratings to the players' ratings at the time the event was played? That way when you go back and look at old tournaments it doesn't show 950-rated players in REC divisions.

[steveganz]

I'll put it on my list of things to look into to.

[gvan]

Originally Posted by ERicJ

Excellent!

On a related note... any progress towards locking the display of those player ratings to the players' ratings at the time the event was played? That way when you go back and look at old tournaments it doesn't show 950-rated players in REC divisions.

I'm not sure I'd like this. When there is a ratings update, I can go to a local tournament and see how all my friends ratings improved (hopefully). I like that better than looking people up one by one.

[ERicJ]

Originally Posted by gvan

I'm not sure I'd like this. When there is a ratings update, I can go to a local tournament and see how all my friends ratings improved (hopefully). I like that better than looking people up one by one.

The other indirect benefit the present system has is that you can look up the current player rating of any player, even if that player isn't current on their PDGA membership.

E.g. Casey Guthrie #28668 isn't a current member (as of this writing):
"This player's PDGA membership is not current. Individual statistics, including player ratings and tournament history, are a benefit of PDGA membership."
--http://www.pdga.com/player_stats/44091

But a simple Google search will return 50+ pages of tournament results that all show Casey's current rating is 964.



However, for accurate historical data it makes no sense to show current player ratings for old tournaments. Look at the REC division results below from a tournament three years ago... 15 of the top 25 finishers are currently rated over 900. How stupid does that look now? It'd be more informative if those results showed the sub-900 player ratings that those players had at that time.

[wsfaplau]

Chuck my quest to better understand the ratings continues.

I understand why in some tourneys the ratings for different rounds might be averaged so a 51 in a morning round might match the rating of a 51 in an afternoon round on the same course..

Take a look at these results.

http://www.pdga.com/tournament_results/79035

This was a 2 day event. Round 1 was a 21 hole course in Pueblo's City Park. For round 2 about 10-12 holes were moved to longer harder positions. Round 3 was at Widefield, a completely different course in a different city.

Shooting a par round of 63 gets a rating of 984 regardless of which layout or course was played.

I don't think so. Can you please explain why the ratings for this event were done this way? They are in no way an accurate representation of what happened at this event.

Thx

[cgkdisc]

The ratings can only be as accurate as what the TDs provide us for who played what course layout. If they are indicated incorrectly on the TD report, we don't know how to do them correctly without feedback from players. If you or anyone else thinks the course layouts were done incorrectly, contact Andrew at PDGA HQ to check it out.

[askmifo]

After finishing the first part of our league,
(http://www.pdga.com/tournament_resul...6340285996#MPO)

we have almost only positive feedback from the players. However, there is one question that I really cant answer:

-Is the rating system designed so that what one players gains in rating, another looses? That is, assuming that all players only participate in the local league and nowhere else, is there a grand total of ratings to play for and divide among the players? Some people are scared that it becomes like living in an isolated pond, it is hard to get higher ratings unless some other players really underachieves?

[cgkdisc]

Yes, that's true for propagators. Those newer players (non-propagators) who don't have a rating based on at least 8 rounds yet are not part of the fixed ratings pool and can shoot as high of a rating as possible without taking points from others.

This fixed pool of propagator ratings at a league or tournament has been a fundamental basis of the system from the beginning. Think about an isolated island of players. If the only people they ever play are each other, then it doesn't matter if some players are taking rating points from other players because the system simply shows how much better some players are getting in comparison to the other players on the island. It doesn't matter how their ratings compare with players located anywhere else in the world since they aren't playing them.

Now if some of those players play in events outside the island, they will either gain or lose points from players who are part of the much bigger "island" and their ratings will eventually adjust. This is what happened when the players outside the U.S. came to play events like Worlds and the USDGC in the first years the ratings system started a decade ago. They played in the large island of the U.S. and their ratings gradually got adjusted. These ratings were brought back to Europe, Japan and other countries and helped adjust the ratings of those smaller disc golf islands relative to the U.S.

So it's true that leagues provide the opportunity to improve your rating only relative to the other players on your island. If you want to improve it relative to bigger or other islands, you need to also play in those other locations and player pools.

[Patrick P]

@Chuck, my understanding is there can only be one 1st place winner in a division. However, looking at the Sylmar Open on 4/1/12, both in the Masters and Novice division there are two 1st place winners. When looking at these four players individual stats they are all listed as 1st place. You can probably tell in the Masters Division who was the winner, one player earned $340 and the other player $215. In that case the 'Place' should be corrected.

The main reason I brought this up is that I'm calculating league points using this event.

[cgkdisc]

Perhaps the date of the event has something to do with it?

Contact asweeton at pdga.com to get that resolved.

[xterramatt]

Hey Chuck, how can we get this issue fixed? I have sent a note to the TD.

2012 Buckhorn Open
http://www.pdga.com/tournament_results/78761

Intermediate division seems to be listed as playing a harder layout than the rest, look at those ratings. I'm not sure how this happened...

I shot 8 strokes worse than the Int division winner, and I averaged 959.75. The int winner averaged 998.75

All divisions played 1SHORT 2LONG 3SHORT 4LONG - EXCEPT the intermediates who shot from the shorts all 4 rounds.

This pretty much compromises everybody's rating average for the event, but it seems they are listed as playing a HARDER layout as their ratings are oddly higher than the same rounds in other divisions.
I'm thinking the layout fields got screwed up with the Intermediates getting Long Short Long Short and the rest of the field getting Short Short Short Short. As opposed to the REST of the field playing Short Long Short Long and Ints playing Short Short Short Short.

I'm not sure, perhaps you can look into it and contact Scott Anderson to correct it.

[cgkdisc]

The TD report shows the Intermediates playing the long tees all four rounds and everyone else playing LSLS. I'm guessing that was not the case. I presume they played the shorter layout all four rounds?

I sent a note to Sweeton to check with TD to confirm. It will be corrected in the July 3rd update.

[xterramatt]

Thanks Chuck. I wonder how much this messed up player ratings for new intermediates... let alone hurt those of us who shot better...

[xterramatt]

well here's your answer...
Paul Rohrbaugh #35004
Player Info

Location: Cary, NC, USA

Classification: Amateur

Current Rating: 954 (as of 15-May-2012)

Player Statistics | Ratings Detail | Ratings History
Ratings History
Effective As Of Rating Rounds Used
15-May-2012 954 6
23-Mar-2012 877 2
25-Nov-2008 890 10
9-Sep-2008 865 7
29-Apr-2008 860 3

Wow. Big jump, ehh?