This is going to be an exploratory post, if you'll indulge me. The truth is that you can't trust most breeding statistics. The basic statistics, such as percentage of stakes winners, do not provide simple answers. You can't sort from highest to lowest and trust that the stallions really rate in the order they come up. For broodmare sire percentage of stakes winners, for instance, we have the following confounds.
*Which mares are going to stud? The A. P. Indy mares are virtually all being bred; the Lear Fan mares aren't. That means that the Lear Fan mares that are being bred had to do more on the track than the A. P. Indy mares, all things being equal. Then again, A. P. Indy is a better sire than Lear Fan, so the beginning odds are that the A. P. Indy mares did more on the track. And broodmare sire stats might not be reflecting A. P. Indy's ability to elevate his mares at all, but the fact that the A. P. Indy mares are out of better mares than the Lear Fan mares. You see how teasing out the broodmare sire role goes on and on?
*What quality of stallions are the daughters of the stallions being bred to? At least we can get a good idea of this from Comparable Index, while the selection effect mentioned in point 1 would require extensive research to pinpoint.
*Over what years have the broodmares' foals been running? Percentage of stakes winner norms clearly have evolved for stallions over the years, and have to be evolving for broodmare sires as well. What we want is more rankings year by year, and then an aggregate, rather than just the aggregate. Is Seattle Slew's career percentage of stakes winners from foals as a broodmare sire of 6.8 really better than Seeeking the Gold's of 6.3, when Seeking the Gold is 11 years younger, and compiling his numbers in a tougher era? Probbaly not.
All of these problems present themselves even before we get to "sample size." A lot of people don't understand the nature of the sample size problem. In a nutshell, it's that rates like percentage of stakes winners never just represent ability, but luck as well. So, pretending for a moment that there are no confounds, Storm Cat's 5.8% of stakes winners as a broodmare sire is higher than A. P. Indy's 5.2% percentage of stakes winnes as a broodmare sire. But we cannot necessarily conclude that Storm Car is really the better stakes winning producing broodmare sire. He may just be the luckier one. Unfortunately, if the sample sizes for the two stallions aren't big enough to interpret the difference, there is no way of telling whether it indicates better luck or better ability.
The formula for standard error for percentages is fun and simple, at least if a person has done some statistical reading. It's the square root of (p*(1-p))/n, where p is any percentage. A 95% confidence interval requires more than a standard error, however, but 1.96 standard errors.
I ran confidence intervals for a bunch of different broodmares sires. A rule of thumb is that a broodmare sire whose daughters have been at it for a while has +- of about 1% on his percentage. Sadler's Wells, with a gigantic 4294 foals as a broodmare sire, has a 95% confidence interval on his % of stakes winners of +- 0.7; A. P. Indy, with 1,377 foals as a broodmare sire, has a 95% confidence interval on his % of stakes winners of +-1.2.
These error ranges might sound small, but they're really not. Use +- 1%, and you can see that we can't separate Storm Cat and A. P. Indy; Storm Cat may really be at 5%, and A. P. Indy at 6%, while on the statistics to date, it's been basically the reverse. Deputy Minister at 7.2% with 2735 foals is probably actually really good.
I've seen enough surprisingly talented runners out of Dixieland Band mares, for instance, that my gut tells me the role of the broodmare sire is important, that if we make too big a deal of it, that's only by a modest margin. But the skeptic in me looks at the rather narrow range of the broodmare sire percentage of stakes winners and wonders if the broodmare sire is playing any role at all. We could just be looking at confounds and luck, my head tells me. I think maybe a Dixieland Band or a Carson City broodmare stands out mostly in that they can get a freak once in a while, more than they make a difference horse by horse.
If you believe in the broodmare sire, Conquistador Cielo and Devil's Bag may be your best argument. They were born three years apart. Both were memorable champions of even greater ability than accomplishment, and both were trained by Woody Stephens. Both were dissatisfying stallions, although they couldn't be called out and out bad. Yet Conquistador Cielo's % of stakes winners as a broodmare sire is 5.8 from 2195 foals. Devil's Bag's is 3.8 from 2001 foals. Within 95% confidence, Conquistador Cielo could be as low as 4.8%, and Devil's Bag as high as 4.6%. Those figures don't overlap. The Comparable Index of the stallions bred to Conquistador Cielo mares is 1.41, and the Comparable Index of the stallions bred to Devil's Bag mares, 1.45, reinforcing that they are a pretty fair comparison.
But some of my statistics teachers would have said, "David, well of course you can cherry pick one or two examples of broodmare sire performance differing from expectation. You didn't make a blind test. The idea of the 95% confidence interval is bogus in this context." And my teachers would be right. But the Conquistador Cielo, Devil's Bag comparison is still interesting.
There's another problem. The confidence intervals underestimate the true size of the range. Here's why. Conquistador Cielo has had 2195 foals as a broodmare sire. But he hasn't had 2195 daughters producing the 2195 foals. He's probably had more like 500 or 750. And that changes things. Broodmare sire numbers, and many breeding counts in general, are what is known as multi-layered data. Blue hens can skew the data; not all Conquistador Cielo daughters are equally good producers, and that renders the 2195 count inaccurate.
No comments:
Post a Comment