Old May 15 2012, 12:50 PM   #3121
JenniferB
PDGA Member
 
Join Date: Feb 2011
Posts: 231
Default

My rating also changed but my ratings detail doesn't include any rounds after January, and the 5 tournaments I played in in Feb-apr all say the results are official, but there are no options to show round ratings. What's going on?
JenniferB is offline   Reply With Quote
Old May 15 2012, 01:13 PM   #3122
bruce_brakel
PDGA Member
 
bruce_brakel's Avatar
 
Join Date: Nov 2003
Location: Dying More Discs
Posts: 5,571
Default

This is just a hunch, JenniferB, but I would guess that right now they are deciding whose job it is to announce that the complete ratings update is going to be delayed until next week.
__________________
Circuit court staff attorney - 25 years. My judge must retire. Looking for employment.
bruce_brakel is offline   Reply With Quote
Old May 15 2012, 01:18 PM   #3123
John Hernlund
PDGA Member
 
John Hernlund's Avatar
 
Join Date: Feb 2011
Posts: 156
Default

Chuck, a few questions, building upon some discussions I've been having with others:

1) Have you ever thought about publishing error bars along with the ratings? Based on a high (e.g., 95%) confidence interval, and letting the chips fall where they may? This seems like a more honest way of doing things, especially given that the number of propagators is insufficient for accuracy.

2) Another interesting project would be to perform a network analysis of players and events, and to explore the topology in detail. Have you or anyone else looked at this in the past? It would also be interesting to do this with ball golf, and try to understand any differences or similarities.

3) Given that the ratings are a coarse measure, only, perhaps you or the PDGA can make a statement that event TDs should never use ratings as a qualifier for their tournaments? If people stopped misusing ratings in this way, then the Pros would never care about their ratings, which would be a good thing. (Anyways, ratings are only useful for roughly preventing sand-bagging, which isn't an issue for Pros.)

4) I always submit my own work for peer-review, which allows others to assess the robustness of the conclusions drawn from any data sets that are employed. Has the PDGA ratings system ever been subjected to anything like an external peer-review process?
John Hernlund is offline   Reply With Quote
Old May 15 2012, 01:29 PM   #3124
steveganz
Administrator
 
steveganz's Avatar
 
Join Date: Nov 2003
Location: NoVa
Posts: 425
Default

The official ratings are still being processed for publishing. Please check back later this afternoon.
steveganz is offline   Reply With Quote
Old May 15 2012, 01:30 PM   #3125
JenniferB
PDGA Member
 
Join Date: Feb 2011
Posts: 231
Default

Quote:
Originally Posted by bruce_brakel View Post
This is just a hunch, JenniferB, but I would guess that right now they are deciding whose job it is to announce that the complete ratings update is going to be delayed until next week.
To be clear, my ratings page does indicate that I played those tournaments, it's just that they aren't showing up in my ratings detail. Is it just the case that an update needs to be made to the ratings detail, but the new ratings are correct?
JenniferB is offline   Reply With Quote
Old May 15 2012, 03:35 PM   #3126
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

They are still uploading the backend part of the ratings details. The individual ratings got updated first. They are having as tough of a time getting the files uploaded as we all are trying to access the site.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 15 2012, 04:14 PM   #3127
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

JH: 1) Have you ever thought about publishing error bars along with the ratings? Based on a high (e.g., 95%) confidence interval, and letting the chips fall where they may? This seems like a more honest way of doing things, especially given that the number of propagators is insufficient for accuracy.

CK: The PDGA decides want they want to publish. We (Roger and Chuck - contractors) have a lot more detail that could be published but there hasn't been the will and more importantly resources to do more, at least yet.

JH: 2) Another interesting project would be to perform a network analysis of players and events, and to explore the topology in detail. Have you or anyone else looked at this in the past? It would also be interesting to do this with ball golf, and try to understand any differences or similarities.

CK: Several studies of various potential issues that players have brought up over the past 10 years have been made with several of them published. There are only a few individuals who produce enough data in a year to drill down to the individual level with much reliability.

JH: 3) Given that the ratings are a coarse measure, only, perhaps you or the PDGA can make a statement that event TDs should never use ratings as a qualifier for their tournaments? If people stopped misusing ratings in this way, then the Pros would never care about their ratings, which would be a good thing. (Anyways, ratings are only useful for roughly preventing sand-bagging, which isn't an issue for Pros.)

CK: Coarse is a matter of opinion in relation to what might be a better measure of past tournament performance - average finish rank, money won, points, player height, years as a PDGA member, etc. Sponsors are satisfied that ratings are useful enough to use as one of their elements to consider sponsorship. And, TDs certainly have a choice to use them as qualifications if they want to use qualifications. I'm not sure there's a better parameter to use.

JH: 4) I always submit my own work for peer-review, which allows others to assess the robustness of the conclusions drawn from any data sets that are employed. Has the PDGA ratings system ever been subjected to anything like an external peer-review process?

CK: The ratings system isn't a theoretical math project. It was primarily created as a functional mechanism for the PDGA to more fairly slot amateur players into divisions. One thought in the beginning was to simply keep the numbers hidden and simply indicate the lowest division an amateur could enter. But the Board decided ratings were "cool" and wanted us to figure out ways to provide them to everyone. So the system has incorporated a few mathematical compromises to deliver that promise to members. Stats professors would wince. But then, no one is yet getting paid based on their rating, only tournament results. A few Boards since 2000 have reviewed the system more in depth including last year and they were satisfied that it was delivering what the PDGA wants to deliver as a member benefit.

When you consider the flukiness of things like cut-thrus, inappropriately punitive OB setups and rollaways in the game, these random elements produce more volatility in an invidual player's round rating than the SSA calculation even with just a half a dozen propagators or averaging these individual performances to produce a player's overall rating.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 15 2012, 06:28 PM   #3128
JenniferB
PDGA Member
 
Join Date: Feb 2011
Posts: 231
Default

Quote:
Originally Posted by cgkdisc View Post
They are still uploading the backend part of the ratings details. The individual ratings got updated first. They are having as tough of a time getting the files uploaded as we all are trying to access the site.
Thanks for explaining that. :-)
JenniferB is offline   Reply With Quote
Old May 15 2012, 11:57 PM   #3129
ERicJ
PDGA Member
 
ERicJ's Avatar
 
Join Date: Jan 2008
Location: Mo.City, TX
Posts: 413
Question 2012 AM World Doubles missing

Chuck: why isn't 2012 PDGA AM World Doubles showing up in anyone's tournament history? I.e. for points purposes.

http://www.pdga.com/tournament_results/77578
__________________
HFDS #756, PLDGA #10, DGCR #519
ERicJ is offline   Reply With Quote
Old May 16 2012, 02:41 AM   #3130
John Hernlund
PDGA Member
 
John Hernlund's Avatar
 
Join Date: Feb 2011
Posts: 156
Default

Quote:
Originally Posted by cgkdisc View Post
JH: 1) Have you ever thought about publishing error bars along with the ratings? ...

CK: The PDGA decides want they want to publish. We (Roger and Chuck - contractors) have a lot more detail that could be published but there hasn't been the will and more importantly resources to do more, at least yet.
Error bars are a way of answering the question: how good is the number you're giving people?

I sense that you'd like to publish more of this kind of data, so we should lobby the board.

I'm not sure what additional resources are needed. You have a database. I assume you can write codes to produce all sorts of detailed results and statistical measures. It would run in 2 seconds on my laptop.

Quote:
Originally Posted by cgkdisc View Post
JH: 2) Another interesting project would be to perform a network analysis...

CK: Several studies of various potential issues that players have brought up over the past 10 years have been made with several of them published. There are only a few individuals who produce enough data in a year to drill down to the individual level with much reliability.
Can you share any of these references?

Quote:
Originally Posted by cgkdisc View Post
JH: 3) Given that the ratings are a coarse measure, only, perhaps you or the PDGA can make a statement that event TDs should never use ratings as a qualifier for their tournaments? ...

CK: Coarse is a matter of opinion in relation to what might be a better measure of past tournament performance - average finish rank, money won, points, player height, years as a PDGA member, etc. Sponsors are satisfied that ratings are useful enough to use as one of their elements to consider sponsorship. And, TDs certainly have a choice to use them as qualifications if they want to use qualifications. I'm not sure there's a better parameter to use.
"Coarse" is indeed in the eye of the beholder, but if an error bar is published then we would at least have an objective measure of coarseness. Maybe if TDs have a quantitative estimate of how good the ratings are, then they can make better-informed decisions about the use of ratings qualifications.

Quote:
Originally Posted by cgkdisc View Post
JH: 4) I always submit my own work for peer-review, which allows others to assess the robustness of the conclusions drawn from any data sets that are employed. Has the PDGA ratings system ever been subjected to anything like an external peer-review process?

CK: The ratings system isn't a theoretical math project. It was primarily created as a functional mechanism for the PDGA to more fairly slot amateur players into divisions. One thought in the beginning was to simply keep the numbers hidden and simply indicate the lowest division an amateur could enter. But the Board decided ratings were "cool" and wanted us to figure out ways to provide them to everyone. So the system has incorporated a few mathematical compromises to deliver that promise to members. Stats professors would wince. But then, no one is yet getting paid based on their rating, only tournament results. A few Boards since 2000 have reviewed the system more in depth including last year and they were satisfied that it was delivering what the PDGA wants to deliver as a member benefit.
I agree that it isn't a theoretical math project. However, subjecting to peer review is a way of holding any kind of work to a higher and more objective standard. And since the entirety of the data aren't publicly available, it is good to examine the genie behind the curtain in some other way.

Quote:
Originally Posted by cgkdisc View Post
When you consider the flukiness of things like cut-thrus, inappropriately punitive OB setups and rollaways in the game, these random elements produce more volatility in an invidual player's round rating than the SSA calculation even with just a half a dozen propagators or averaging these individual performances to produce a player's overall rating.
That is a hypothesis that can be tested, with the right data. Maybe it's true, maybe not. But would be nice to know!
John Hernlund is offline   Reply With Quote
Old May 16 2012, 09:11 AM   #3131
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

I've said what I can at this point. We still don't have the new website up and running which holds up any more info being developed and posted. If you search on "ratings" in the Search window, you can find some stories that have been posted in addition to the docs posted on the menu Ratings link.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 16 2012, 01:04 PM   #3132
John Hernlund
PDGA Member
 
John Hernlund's Avatar
 
Join Date: Feb 2011
Posts: 156
Default

Quote:
Originally Posted by cgkdisc View Post
I've said what I can at this point...
What you've said is:

1) You have additional statistical measures, but the PDGA Board does not wish to publish any information regarding the quality of the rating.

2) The website is inadequate for publishing further details, error bars, or other measures reflecting upon the quality of a player's rating.

3) You believe the ratings are good, but you decline to share any data or other information that would support your claims.

4) Despite the withholding of information that would allow us to better evaluate the ratings system, and with a widespread belief among the membership that the quality of these ratings are poor, you still advocate using these ratings above every other criteria, and to continue expanding its use well beyond the purposes for which it was originally intended (to prevent sandbagging in Am divisions).

If PDGA views ratings as a "product" for the membership, then it would make sense to also give members, TDs, etc., the tools they need to evaluate the quality of the ratings. The membership is concerned about the use and misuse of ratings, and therefore it seems appropriate to also subject the ratings to some sort of external review process. It isn't that we don't trust you, Chuck, it is simply a matter of being thorough and developing greater confidence in a number that you are promoting as the be-all end-all of assessing player quality.
John Hernlund is offline   Reply With Quote
Old May 16 2012, 01:20 PM   #3133
steveganz
Administrator
 
steveganz's Avatar
 
Join Date: Nov 2003
Location: NoVa
Posts: 425
Default

Quote:
Originally Posted by John Hernlund View Post
2) The website is inadequate for publishing further details, error bars, or other measures reflecting upon the quality of a player's rating.
That is not the case. The current website is more than capable of publishing all of the above, but it would take a development effort that we don't have the resources for at this time. We are focusing on launching a new platform while optimizing the current platform as much as possible. The new platform will enable us to introduce new functionality much more easily, but keep in mind that any new functionality needs to be prioritized appropriately based on the impact it will have for our members.
steveganz is offline   Reply With Quote
Old May 16 2012, 02:45 PM   #3134
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

John, I've probably made 1000 posts here and elsewhere specifically answering ratings questions for members and providing calculation information including precision. But the reality is Roger and my direct customer is the PDGA Staff and Board. They determine the amount of ratings information they wish to publish.

The elected Board represents the members. They have seen much more detailed analysis at various times when requested over the past 12 years on how the ratings system works, reliability, accuracy, etc. Those meetings have even been open to members. They seem to be satisfied with the choices made for how the system is implemented and how it's presented.

Consider that the ratings are simply the transformation of a player's scores. They aren't some mystical judgment of a player's performance. The only numbers used in the calculation to produce SSA and round ratings are the scores and current ratings of propagators. There are no subjective elements in there. A score of 50 on a course getting a 50 SSA has remained at 1000 rating since the beginning.

The 95% error range is directly affected by how many props shoot scores on a layout, not how good the ratings are of the props. For a course around 50 SSA, the 95% range for 5 props is +/- 2.7 in SSA which is 5.3% potential error. For 60 props, the 95% range is +/- 0.8 or 1.5% error. Those are the ranges for playing the same course under the same conditions which is never exactly the same even going from morning to afternoon with no wind either round.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 16 2012, 03:43 PM   #3135
Spinthrift
PDGA Member
 
Join Date: Sep 2007
Location: Charlotte, NC
Posts: 45
Default Ratings search accurate?

Chuck - I did a search in "Player Statistics" for Amateur Senior Grandmasters by ratings. What it gives me is an incomplete listing, and the only reason I know it's not right is two players who should be ranked in the top 10 are not listed (Ryalls-Clephane/953, Hornsby/933). When I search the states they are from, they don't show up there, either. It may be missing others, as well. Am I doing something wrong in the search, or can the sytem not produce the desired results?
Thanks, Jim
Spinthrift is offline   Reply With Quote
Old May 16 2012, 03:48 PM   #3136
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

For some reason that system hasn't worked well in a long time. It appears to the user to be random whether a player who should be included is seen by the system to be part of the selection criteria you specify. I know Ganz collected some info on this to provide to the website contractor but it hasn't moved up the priority list to be addressed yet.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 16 2012, 04:03 PM   #3137
steveganz
Administrator
 
steveganz's Avatar
 
Join Date: Nov 2003
Location: NoVa
Posts: 425
Default

The Player Statistics search is based on the current calendar year. The reason you don't see Ryalls-Clephane or Hornsby is because neither of them have any rated rounds for the 2012 season.

I'll take a look at modifying this search to pull in all current members regardless of whether or not they've played a competitive round in the current calendar year.
steveganz is offline   Reply With Quote
Old May 16 2012, 05:51 PM   #3138
wsfaplau
PDGA Member
 
wsfaplau's Avatar
 
Join Date: Feb 2005
Location: Boulder, CO
Posts: 457
Default

Steve, I think you are doing a good job. Keep up the good work. Thanks
__________________
Pete Kenny

Learn a new shot every year
wsfaplau is offline   Reply With Quote
Old May 16 2012, 11:25 PM   #3139
John Hernlund
PDGA Member
 
John Hernlund's Avatar
 
Join Date: Feb 2011
Posts: 156
Default

I think all of these guys are doing a great job. Chuck Kennedy has devoted an enormous amount of time, effort, and love to the sport of disc golf, and I respect him enormously. Guys like Chuck are my heroes. Just to be clear, I don't mean to be critical in any antagonistic sense, I am only interested in useful critiques, to offer ideas, or at least spark discussions that might eventually lead to improvements. Let me also be clear that I'm willing to offer help in whatever way I am able, including a willingness to work with the PDGA ratings committee to help address member concerns about the ratings.

I hope that my primary critique is clear: It is not good to offer a number without also offering a quantitative estimate of the errors involved. When we throw out a number that is supposed to measure something, without knowing how accurate that number is, then it is easy to misuse it.

What does it mean to parse differences between a 1000 and a 999 rated player? It happens all the time, but really the rating system is incapable of resolving that level of difference in player quality. Is there a significant difference between a 1030 and 970 rated player? Almost certainly, there is a significant difference. But where is the middle ground of uncertainty? 995 vs 1005? 990 vs 1010? Who knows? That's the essential issue we need to resolve.

And there are a lot of other tools to evaluate the ratings system, such as network topology. It can tell you whether or not there are sources of skewness or poor coupling that will affect the robustness of various schemes for transforming player scores into ratings. They can also serve as a test of basic assumptions, such as normally distributed errors, etc.. For example, consider that tournament ratings in one region are only coupled to tournament ratings in another region by propagators traveling between the regions. What kind of players travel furthest, and are most likely to play the role of propagators that couple different regions together? Of course, these are typically better players shooting lower scores. Mediocre players are more likely to only play tournaments in their home region. In this way, inter-regional coupling can become highly sensitive to the movements and fluctuations in performances of the traveling players (who play one tournament while visiting, and then go somewhere else, so good/bad rounds that happen will adversely affect the ratings coupling). We don't know how much this influences the outcome of the ratings. But everybody suspects that it does present an adverse effect.

Anyways, looking forward to continuing this discussion.
John Hernlund is offline   Reply With Quote
Old May 17 2012, 08:30 AM   #3140
JenniferB
PDGA Member
 
Join Date: Feb 2011
Posts: 231
Default

Regarding the player statistics, I noticed that pro masters women are getting pulled into results when searching on amateur master women. Also, when you search amateur women, there is a guy that shows up near the top of the rankings. It also doesn't make any sense to me to pull up only women age 40-50 when searching on amateur master women. Grandmaster age and older women are eligible to play that division, and often do, but they don't show up as competition for that division. A local paper did a feature on me and apparently used the statistics search to report that I am the 3rd highest rated am master woman in my state, and 18th in the world, when I think there are older women who deserve that recognition. Similarly, Des Reading is now Master's age, and she doesn't show up when searching open women anymore. I just think it would make more sense to have one category for all pro women and all open women, and let the pro masters women be a filter that only removes pro women inelegible to play masters. IDK, I guess that's just me.

Last edited by JenniferB; May 17 2012 at 08:36 AM.
JenniferB is offline   Reply With Quote
Old May 17 2012, 08:54 AM   #3141
steveganz
Administrator
 
steveganz's Avatar
 
Join Date: Nov 2003
Location: NoVa
Posts: 425
Default

I'll take a look at men showing up in the search for women's statistics. Thanks for the heads up on that!

Currently, the statistics search only displays players who have competed during the current calendar year in any given age bracket that you are searching within. As it stands now, if a pro woman player plays in any amateur division they show up when searching for amateur women. I've been trying to get a consensus on this and am leaning more and more towards simply not including pros that play in am divisions and ams who play in pro divisions in the search results.

Like Chuck alluded to, there is a lot that is wrong with the statistics search. I've spent some time tweaking it recently and expect that work to continue over the next few weeks.

I will definitely add clarification on how the statistics search works right there on the page so at least people will know what to expect when they perform a search.
steveganz is offline   Reply With Quote
Old May 17 2012, 09:49 AM   #3142
ERicJ
PDGA Member
 
ERicJ's Avatar
 
Join Date: Jan 2008
Location: Mo.City, TX
Posts: 413
Default

Quote:
Originally Posted by ERicJ View Post
Chuck: why isn't 2012 PDGA AM World Doubles showing up in anyone's tournament history? I.e. for points purposes.

http://www.pdga.com/tournament_results/77578
__________________
HFDS #756, PLDGA #10, DGCR #519
ERicJ is offline   Reply With Quote
Old May 17 2012, 10:23 AM   #3143
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

Contact PDGA HQ.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 17 2012, 02:01 PM   #3144
Yeti
PDGA Member
 
Yeti's Avatar
 
Join Date: Nov 2003
Location: The Great Outdoors
Posts: 560
Default "Something About Mary" or at least something about her ratings

Congrats on the Women's Global Event to everyone involved. Truly an awesome showing for the women of disc golf.

That being said I would like to explore the unique opportunity the WGE presents using ratings for event ranking and even payout. John H has made some good points that have been asked over the years. I particularly would like to address one of CK's responses that has not been true over the last three years or so as ratings are used to determine more now than ever before and the list growing. PDGA ratings at the Pro level are being used for seeding players in large scale tournaments, $1,000+ skins match involvement, sponsorship deals, sponsor bonuses, PDGA Player of Year Awards and now even Global event payout. WOW, that's a huge list of potential money directly tied to PDGA Ratings. Ratings are a great thing for the PDGA base membership, don't get me wrong. I am very appreciative of the hard work by CK, Roger and others. The idea of having a rating system that doesn't allow the players to get rated against the course they are playing, but instead sets a course SSA against the quality of players playing that course on that day or round is what is not right.

CK: The ratings system isn't a theoretical math project. It was primarily created as a functional mechanism for the PDGA to more fairly slot amateur players into divisions. One thought in the beginning was to simply keep the numbers hidden and simply indicate the lowest division an amateur could enter. But the Board decided ratings were "cool" and wanted us to figure out ways to provide them to everyone. So the system has incorporated a few mathematical compromises to deliver that promise to members. Stats professors would wince. But then, no one is yet getting paid based on their rating, only tournament results.

Let's take a quick peak at the Women's Global Event. A good case because the women played the exact same layout for two rounds on the same day. A number of ratings anomalies pop up with just a quick review of the Pro Women. No in depth analysis here, I am not a big stats guy. I just know when stats don't feel or seem right.

WGE Winner Paige Pierce shot (3) strokes better in second round, but only received (8) points increase in her round rating.
(57 at a 983 and a 54 at a 991)

Sarah (Stanhope) Cunningham shot the hot rounds both rounds at 53 for a 956 and a 52 for a 967. One stroke better for an eleven point increase. That is not that strange unless you compare that to other events. In Sarah's event 64% shot the same or better for round rating increase of 1 point. In Paige's event 63% shot same or better and rounds decreased 22 points for a same score.

In the Bartseville event with Liz Lopez, 44% of the field shot same or better and ratings went up between 14-17 points for the exact same score from round one and two.

Des Reading shot one stroke better in round two (55) than round one (56). For shooting that one stroke better, her round rating went down 15 points. The weird thing with this one was that she played first round with Pro/Adv propagators and second round with Int/Rec propagators. This supports the long standing belief that the better the propagators, the higher the ratings. If her same 55 would have been with the same Pro/Adv propagators in round two, her round rating would have been 949 instead of 924 for a 25 point difference.


Last edited by Yeti; May 17 2012 at 03:33 PM. Reason: scores reversed
Yeti is offline   Reply With Quote
Old May 17 2012, 02:46 PM   #3145
John Hernlund
PDGA Member
 
John Hernlund's Avatar
 
Join Date: Feb 2011
Posts: 156
Default

I figured I would explain my comments about studying the network topology of ratings, starting with a simple example. It all begins with constructing a graph. For starters, if you were to list all the players who've played in a PDGA sanctioned tournament in the past several years, and order them in a list as:
{1,2,3,4,...}
And then you were to also make a list with each entry for any two players that played in the same tournament round as:
{{1,54},{1,54},{1,54},{32,78},{32,78},...}
where in this case players 1 and 54 played in the same tournament (in which there were 3 rounds), 32 and 78 played in the same tournament (in which there were 2 rounds), and so on (there could also be multiple entries if a pair play in the same tournament more than once). These lists form a graph, with individual players as vertices, and tournament rounds in which pairs of players competed as edges connecting the vertices. This is the beginning of a network analysis. From here, you can use free software (many available for Python, such as Graph-tool, NetworkX) and compute various diagnostics. You can also generate graphical images showing the network of connections between various players. And by adding properties to players such as rating and regions where they play most often, you can also begin to address the issue of coupling between different classes of players, inter-diffusion between regions, etc., etc.. You can also compare the graph to random graphs with the same number of vertices and edges.

Companies like Google use this kind of analysis (though with many more layers of complexity added) to assign ranks for web searches, and it has obviously been effective. Biologists use this kind of analysis to link together the chemical reactions between different components in organisms to understand how everything is coupled together. Epidemiologists use it. Etc. I think this seems like the obvious direction for PDGA to go with ratings...
John Hernlund is offline   Reply With Quote
Old May 17 2012, 03:25 PM   #3146
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

Jay, you're not posting anything that hasn't been known for years regarding fluctuations in ratings on the same course on the same day under what appear to be the same conditions with the same pool of props or a different pool of props like Des experienced. Those fluctuations are not flaws but expected with a statistical system. I posted this above (#3134) and it's worth posting again:

"The 95% error range is directly affected by how many props shoot scores on a layout, not how good the ratings are of the props. For a course around 50 SSA, the 95% range for 5 props is +/- 2.7 in SSA which is 5.3% potential error. For 60 props, the 95% range is +/- 0.8 or 1.5% error. Those are the ranges for playing the same course under the same conditions which is never exactly the same even going from morning to afternoon with no wind either round."

Based on these parameters, we would expect the ratings for 4 of the 82 rounds played (5%) by the women in the 41 Global events to fall outside these ranges perhaps 2 above and 2 below. It has no bearing on the average ratings of the props. It just happens naturally with this or any other statistical system. The good news is that the other round at each event likely fell within the range where the average of the two rounds produced a better representation of the true SSA for the course that day which can't be known by any other means.

If I had my preference, Global events would be at least four rounds so the statistical fluctuations on all of the sites have a chance to converge toward the "true" numbers whatever those may be. We went with three rounds the first Global to see players' acceptance of the format as an option. The Women chose two rounds not because it was good enough statistically but because they felt this would be more likely to increase participation which was a more important goal for them.

The reason Worlds is at least 7 or 8 rounds is to reduce the impact of lucky aspects that are part of our sport. Ratings have been shown to be pretty accurate in that 93% of the final ranking at Worlds can be predicted by the initial rank of the players' ratings before it starts.

Think of a basketball game. Now imagine instead of determining the winner after 4 quarters, they decided to play two 2-quarter games a night due to the shortened NBA season. If we agree that only a 4 quarter game truly determines a winner in that sport, then determining standings based on a bunch of 2-quarter games is a lesser approximation statistically in the same way using 2 rounds of DG has lesser statistical quality than 4 rounds to determine winners. Why don't they have single round playoff matches in B-ball or hockey and instead play up to 7 games? They've determined that a single game even at a neutral venue is not statistically good enough to determine who is better.

Here's an example directly relevant to disc golf where using ratings would have been better than using actual scores for rankings. At the Charlotte Worlds, the Open A pool played two wide open courses on the first day and B pool played the two mostly wooded courses. The next day they switched courses and the winds were wailing. This was in 1997 before ratings, but it's estimated the open courses played 4-6 shots tougher on the second day.

The B pool players got the shaft in this format. At the time, the A & B pools were evenly seeded with top players in both pools unlike the current procedure to stack at least the top 18 players in the A pool. Now had ratings been used for the competition, obviously a radical concept, the B pool players would have gotten ratings equivalent to the A pool players on those courses and still been in the competition rather than being torched using the conventional scores system. While individual round ratings are less accurate than many ratings averaged together, they still would have been superior to straight scores in that scenario.

What it boils down to is what is a good enough format for players to be willing to enter and compete. Each format has a different level of statistical "quality." We know players are willing to play single round leagues even with a weaker version of handicapping all the way up to Worlds currently with 7.5 rounds.

Should I be surprised to see the highest rated local player enter a one round league night where he has the lowest probability of winning? The more rounds played in a competition, the more likely he will win from a statistics standpoint. I shouldn't be surprised because the player has chosen to compete under that format knowing ahead of time it's not the best format statistically for him.

I'm guessing less than half of the 646 women participating in the Global knew how it all worked but accepted that it was good enough. If you look at the final rankings, there were very few surprises in the rankings compared with their initial ratings. Only the unrated non-PDGA players in some of the lower divisions surprised by not knowing where their skills currently were in comparison to the division they entered.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Old May 17 2012, 04:46 PM   #3147
ERicJ
PDGA Member
 
ERicJ's Avatar
 
Join Date: Jan 2008
Location: Mo.City, TX
Posts: 413
Default

Quote:
Originally Posted by cgkdisc View Post
Think of a basketball game. Now imagine instead of determining the winner after 4 quarters, they decided to play two 2-quarter games a night due to the shortened NBA season. If we agree that only a 4 quarter game truly determines a winner in that sport, then determining standings based on a bunch of 2-quarter games is a lesser approximation statistically in the same way using 2 rounds of DG has lesser statistical quality than 4 rounds to determine winners.
Everyone knows that only the 4th quarter matters.

Quote:
Originally Posted by cgkdisc View Post
Why don't they have single round playoff matches in B-ball or hockey and instead play up to 7 games?
This one is easy: they can sell 4-7x the tickets, concessions, and TV commercials which is 4-7x the $$$$.

__________________
HFDS #756, PLDGA #10, DGCR #519
ERicJ is offline   Reply With Quote
Old May 18 2012, 10:29 AM   #3148
bruce_brakel
PDGA Member
 
bruce_brakel's Avatar
 
Join Date: Nov 2003
Location: Dying More Discs
Posts: 5,571
Default

Quote:
Originally Posted by Yeti View Post
Congrats on the Women's Global Event to everyone involved. Truly an awesome showing for the women of disc golf.
I didn't see a point in quoting back the whole post. It's up there.

Round by round ratings have always been like that. On the same day in the same conditions for the same score in different pools, Geoff Bennett got a 1030 and I got a 970 unofficially. When they averaged the pools together for official ratings the anomoly disappeared and we both had 997 or something.

Speaking as a player who actually plays in the ratings defined divisions and whose rating goes back and forth across the MA1/MA2 line, I think our rating system does an excellent job of sorting the MA1s, 2s, 3s and 4s. At the tournaments I play, there is always a strong correlation between rating and score.

If many people cared about their rating so much that they only played tournaments that jiggered their rating up or down, then there might be a problem with ratings. The fact that a small pool of players can produce ratings that vary by 25 or 30 points from morning to afternoon is not a problem because no one is going to be changing divisions mid-tournament based on a noontime ratings update.

And one other point: The rating for par changes from morning to afternoon because a different pool of players plays it in the afternoon. The morning pool on the average is sleepier, colder, stiffer and more hung over than the afternoon pool. The afternoon pool has all played the course recently, they are warmed up, their meds have had a chance to kick in and the DTs have subsided. The same player never plays a good course twice, because after she plays it once she isn't the same player she was the first time.
__________________
Circuit court staff attorney - 25 years. My judge must retire. Looking for employment.
bruce_brakel is offline   Reply With Quote
Old May 18 2012, 11:23 AM   #3149
NASCARVW
PDGA Member
 
Join Date: May 2012
Posts: 3
Default NEWBIE Ratings questions...

Recently joining the PDGA, I am not up to date on how this whole process works...
Obviously I am a new player and enjoyed playing my first tournament in February as a REC/NOV.
Placed 6th. - it was rated at a 904.
I played another tournament in March, but since the ratings had not been updated, i entered REC/NOV.
Placed 5th - those rounds average ratings is 874.
I never really had a chance at a win in either tournament, but I'm not complaining about that. Those guys just shot better.
My dilemma is, that I recently entered a tournament last week presuming that my 904 rating would fall below the 900 cutoff for REC/NOV before the ratings update May 15.
For some reason the March tournament is not counted in my ratings, which keeps it at a 904. So now what do I do regarding this tournament I pre-registered for upcoming at the end of June? Do I get in trouble for playing in REC/NOV? do i need to contact the tournament director to get him to switch my entry to the INT division?
Sorry for the newbie questions. Bare with me.
NASCARVW is offline   Reply With Quote
Old May 18 2012, 11:45 AM   #3150
cgkdisc
PDGA Member
 
cgkdisc's Avatar
 
Join Date: Mar 2008
Location: Twin Cities, Minnesota
Posts: 6,220
Default

Since the next ratings update is after your June event, you need to contact the TD to switch divisions to Intermediate. The TD also may see the problem before the event and contact you about switching.
__________________
Rater of the tossed arc.
cgkdisc is offline   Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump


All times are GMT -4. The time now is 12:39 AM.