A simple, novel and accurate method to estimate track record: a new “P” value
How much should past performance contribute to picking winners in grant competitions?
The weight that “track record” (TR) should contribute to the assessment of worth is a simple concept that is difficult to estimate. A synthesised value of TR is used to assess research grant applications. But what percentage of the overall variance does TR account for in determining success? In most human endeavours, assessors synthesise many variables to produce a single metric for TR that then becomes part of an overall scoring matrix. In assessing grant applications, the objective of this ranking is to minimise the risk that the grant given will not be productive.
In order to determine the true impact of TR, I used betting on horse races to determine the win–loss ratio when the shortest-odds favourite is backed (Pilowsky’s “P” for TR). Horse races were chosen to simulate grant panels, as horses, like grants, are “marked” to some extent on the basis of their TR. A nominal “bet” of $1000 was placed on each horse at an off-track totaliser betting facility between 16 December and 20 December 2013. A total of 125 bets were placed. Losses were 83 and wins 42 (34% of total). The simulation suggests that the pooled synthesis of available information by those with an interest in estimating the best outcome (a win) gives a value of between 23% and 46% on each day (n = 5 days;…