944Spec - No BRI numbers?

[pdy]

Bill:

Are you refering to this?
http://www.pcasdr.org/porsche_events/tr ... /index.php

That site might have good info, and/or maybe needs
to be updated/formatted.

[Carl Scragg]

As one of the developers of the BRI, it’s always amusing to listen to others speculate on what the intent was. Simply put, my intent was to use historical data to highlight “exceptional” performances by a driver, regardless of car class. No conscious attempt was made to favor any particular class or to favor stock over prepared or street stock. And in fact, over the years the BRI has been won by the fastest cars (AR class) and the slowest cars (AS/S) and just about everything in between.

The system is based upon the rather flawed assumption that we have enough historical data from many different drivers to average out the effect of the drivers and to isolate the car’s potential. A good finish in the BRI at any event was intended to indicate that the driver’s performance was statistically very good relative to the historical performance of other top drivers in well-prepared cars of that class.

Now the data used to develop the indices was from 1998 to 2001. The index for your car is based upon how fast the winner in your class was driving during those years. If your class was very competitive back then, your index suffers today. If your class was less competitive, the opposite applies. In the case of IS -- the class is highly competitive today with several well-driven and well-prepared SC’s fighting it out. But back in 1998-2001, that class was almost dead.

And why the recent predominance of S, over both SS and P? My guess is that this has more to do with rule changes than anything else. Recent rule changes have made SS more restrictive (i.e. slower). And a number of cars that used to be in S have been bumped up to P by other recent rule changes (making them slow P-cars rather than fast S-cars ). But the few tweeks to the BRI over the years have been modest and haven’t kept up with the rule changes.

Should the BRI be revised in light of class changes and rule changes? Most definitely! The data base is much bigger and more readily available today than in the past, so all we need is a volunteer with a modest knowledge of statistics and ample spare time – and a thick skin – because only the BRI winner really believes that the indices are any good.

Carl

[pdy]

so all we need is a volunteer with a modest knowledge of statistics and ample spare time – and a thick skin – because only the BRI winner really believes that the indices are any good.

Carl

He meant a knowledge of Sadistics....

[Irksome]

Great post, Carl.

Question: I was told once that the BRI was computed based on data from all types of track events, not just A/X. Can you shed some light on that? What data was used to come up with the BRI adjustment values?

Also, it would seem possible to make the BRI adjustments adjust continuously as more data comes in. Since all the results are stored nicely in a database, it looks like if you have formulas to compute them, they could theoretically be re-calculated automatically each time more data came in. It should even be possible to run it separately for A/X vs. TT. And this would also (in theory) allow for less abuse of those who create the values, since it will be purely statistical calculation...

I'm not sure I've talked myself into volunteering yet, but I know there are some good programmers in our midst *cough* Mark Garriott *cough*.

[Don Middleton]

Chuck,

Our club's typical AX track layout does seem to slightly favor the lower SS and S classes. But, every so often, we have a faster, more agressive layout that changes the BRI top ten considerably.

Get the AX chairs to put up a P-class track layout at the next event and you'll see the advantage swing in your direction.

I think everyone in IS will back you in getting a P-class layout. We like fast as much as you.

[PShipman]

As the most junior member of the AX chairs (the one everyone else sits on ????) I have observed that track design/outcome is in large part based upon the deteriorating conditions of Qualcomm's parkinglot(s). Many of the twisting turns on the first half of the 26th's course were added at the last minute to bypass holes/cracks/etc. in the pavement. One of the reasons we got started a little late was due to our twice having to revise the first quarter of the course just before the track walk.

That said, I just LOVE a fast track, even though the 944 Turbos just run away from me there!

[ttweed]

Question: I was told once that the BRI was computed based on data from all types of track events, not just A/X. Can you shed some light on that? What data was used to come up with the BRI adjustment values?

Have you read all the info on the page that Paul linked above about the BRI?
http://www.pcasdr.org/porsche_events/track_events/bench_racers/index.php

Since I collected most of the original data that Carl used in the BRI development, I can tell you that it began as an Autox-only index. It was only expanded a few years later, through the efforts of Otto Obrist, to include TT data, with changes to the indexes for potential big track performance. That effort resulted in the "TT BRI" indexes that are listed in the second table on that page. No single index could cover both types of events, and when people attempted to apply the original Autox BRI to TT events, the shortcomings became obvious. Otto attempted to rectify this by developing a different set of numbers, with some success.

As to having the BRI update automatically by feeding more ongoing data into a mathematical model--that idea makes my head hurt, but have a go at it if you'd like! It may be more trouble than it is worth, though, since the BRI is an unofficial, off-the-cuff system anyway, and no one has taken it seriously enough to spend that kind of time on development and maintenance.

TT

[PShipman]

As to having the BRI update automatically by feeding more ongoing data into a mathematical model--that idea makes my head hurt, but have a go at it if you'd like! It may be more trouble than it is worth, though, since the BRI is an unofficial, off-the-cuff system anyway, and no one has taken it seriously enough to spend that kind of time on development and maintenance.

Tom ... what if we built just an Excel spreadsheet with:

1) some sort of changeable algorythm sheet;

2) a data sheet into which we dump data out of the timing database each year at years end; and

3) a "results" sheet which could be used as a basis to "re-evaluate" the BRI algorythm for the next year.

I would volunteer to write such a thing as part of a team putting it together.

[ttweed]

Recent rule changes have made SS more restrictive (i.e. slower). And a number of cars that used to be in S have been bumped up to P by other recent rule changes (making them slow P-cars rather than fast S-cars.

Good points, Carl, and I would add that Stock class is now allowed 8 points for improvements, while it was only 6 points back then. This alone is enough justification for increasing all the S class indexes, which has never been done. 2 points can allow you to go from RA1s to V710s or Hoosier A6 tires, which is good for at least 1-2 seconds per lap, for instance.

Beside being restricted to 2 points of improvements instead of 4, the SS class has been saddled with a host of exclusions which has definitely made it slower, overall, so all their indexes should probably be adjusted slightly downward as well.

TT

[ttweed]

Tom ... I would volunteer to write such a thing as part of a team putting it together.

I am all for an annual update of the BRI, Perry, especially taking into consideration the annual rule changes in Zone 8, but I am dubious about a purely mathematical model obtaining perfect results. You should talk to Carl about the original development. As he said, the difference in driving abilities across the club, and the differences in development levels of cars within the same class, make it extremely difficult to just compare the winning times for each class, average them and come up with a perfect index. We simply do not have enough data to normalize the samples in a way that predicts the car's performance vs. the driver's. It took some "artful" analysis for him to eliminate anomalies and come up with the original numbers. Perhaps this could be done mathematically, but I haven't seen a spreadsheet yet that had good judgement built in.

The PAX index used in SCCA has the benefit of regional tour events all across the country that attract 300+ drivers each, and a national event that brings 1200 drivers together, mostly of high caliber and with optimized equipment for the class. It is easier to synthesize that amount of data into meaningful results.

TT

[ttweed]

No conscious attempt was made to favor any particular class or to favor stock over prepared or street stock.

Strictly speaking, this is not true, Carl. The BRI must, and obviously does, favor the lower classes, with less overall speed potential, by assigning them a lower index, in order to "level the field" and compare driving performances in one class vs. another.

I think what you mean is that you did not intentionally build any bias into the indexes other than the potential performance for a given class of cars. It is a very carefully structured and (hopefully) unbiased "favoritism", designed to give the driver of the slowest car at the event a shot at beating the driver of the fastest car, on "adjusted time" rather than "raw time", but it is favoritism nonetheless, in its most egalitarian form.

TT

[Carl Scragg]

It took some "artful" analysis for him to eliminate anomalies and come up with the original numbers.

Tom makes an important point here. A straightforward statistical analysis of a finite data set can lead to some glaringly bad results. For example, it might tell you that IS cars are faster than IP cars

We need to mix the statistical analysis with some common sense about the relative performance levels. I suspect that a more hands-on approach will again be required.

Carl

[pdy]

.... designed to give the driver of the slowest car at the event a shot at beating the driver of the fastest car, on "adjusted time" rather than "raw time" ....

TT

Tom, I think you mean slowest class. I could
have a Gemballa Turbo with tall gears and
crappy tires. It might wind up in class OI,
but compared to the class, would be a pretty
slow car at an autocross.

The class should be more important than the
historical data, but the data will have some
importance across base classes.

For example, the historical data might show that
the MS class is traditionally faster than NS cars
at autocross, but not at Time Trials. This might
make the autocross BRI handicap for MS higher.

OTOH, this logic cannot be applied to class
progression. If it is found that historically
IS cars are faster than IP cars, it isn't fair
(IMHO) to make the handicap more severe for IS
compared to IP. Maybe the difference between
the BRI number for IS and IP is not as large
as other progressed classes, but never should
there be a situation where the SAME car with
potentially more performance has a lower BRI
handicap.

I think the existing BRI numbers are overall
close to correct. I would agree now that with
an allowance of 8 points in stock vs 6, all
Stock classes should probably bear a slightly
higher BRI number.

[ttweed]

Tom, I think you mean slowest class.

Yah, you're right, Paul. I definitely meant "the driver of a well-prepared car in the slowest class vs. a driver in a well-prepared car in the fastest class."

Here I was trying to correct the precision of Carl's language in his post and committed a similar error myself. It just goes to show you how difficult it can be to express things precisely and accurately in this written medium. It's important to try, though, to avoid misunderstandings, misinterpretations, and ultimately flame wars over misspoken remarks!

I totally agree with you about the logic of progression through the class being preserved (i.e., ISS-IS-IP-IM), etc. With more improvements allowed, there is no logical reason for a progressed car to be slower, other than it not being driven up to its potential, or the improvements being poorly executed.

I also think we should strive to preserve the "hierarchy of speed" represented by the A-Z class progression. If cars in the M class, for instance, were consistently faster than the N class, at BOTH autox and TT events, when driven equally well, then instead of changing the indexes, we should change the classification of the cars.

Thx,
TT

[bibbetson]

Per the 8 vs. 6 points in stock classes changing the index, initially the points were expanded to compensate for two new rules that effected stock class cars, LSD and adjustable shocks. Now that the adjustable shock rule has been relaxed, I see this as somewhat of a wash depending on if an LSD is a big improvement for your car. Certainly it was a wash last year when you got dinged for LSD and shocks, this year some cars, namely 911's that don't need LSD as much, will gain an advantage with the extra two points. This is all in comparison to the 2005 and prior stock class rules.

That was a lot of rambling. What I'm trying to say is that the stock class points have expanded, but so have the penalties for standard stock class improvements, thus I think we should be very careful adjusting the indexes.