How much do Stats tell us about driver performance?

[Stradivarius]

The way I see it, it's not about knowledge at all. It's all about feelings and nothing more. If your feelings tell you that one driver is better than another, you will automatically, probably both consciously and unconsciously, try to justify your feeling and look for information that support it, while you will be less likely to accept information that points in the oposite direction.

I agree to a extent.

If two drivers who are teammates are very close on points, I will examine other stats. As its is basic knowledge when dealing with statistics to not present them as fact if they are not significant, as a small variable can distort the data. Its not a case of looking to support your driver if they are on the losing end, its just a basic need if you want to see if its a significant reflection (which is rare when points stats are so close) In these cases theoretical analysis that is indeed subjective usually forms a opinion, like with Prost/Senna or Lewis/Jenson.

For example, if two drivers are pretty equal but one has more external faults like mechanical failures or team pit stop errors which leads to the other driver having a small advantage in the points total, then the gap will not be significant enough to support that one driver has outperformed the other. But on the other hand, you can never prove a driver who has not outscored a rival is definitively better, again as there is no way you can prove its significant.

In other cases like I mentioned like say Jules/Chilton and Webber/Vettel or if you want to go back Schumi vs any of his teammates pre Rosberg, the stats are so one sided that the degree of probability tell us they are significant and almost a definitive sign that one driver is outperforming another based on pure merit. In that case the stats do indeed mean something.

Sometimes it is not a case of trying to justify anything, its just statistical analysis. For example, if we are to take the Williams partnership this year, if it was presented to you that Pastor has scored and Bottas has not by 1-0, would you take that as significant?

Maybe I was a bit unclear earlier. I didn't mean to say that the points (which I have used as an example of stats) are always telling the whole truth. Obviously, it is possible for a driver to have bad luck and be the victim of circumstances beyond his control on several occasions in the same season. But this is typically a short-term phenomenon. If a driver consistently is unlucky throughout his career, it is most likely not a question of luck and you should try to find other explainations.

With regards to what I wrote about Senna and Prost, you have a valid point in that Senna had 4 car failures in 1989, while Prost only had 1. But my point was that in those 12 races where Senna's car didn't fail, he only scored points in 7 of them. In those races where Prost's car didn't fail, he brought the car home in the points 13 times out of 15. So if you want to try and look at this objectively, you have to consider that 13/15 is a significantly higher ratio than 7/12. You can actually argue that it was 13/14 for Prost, since he chose not to race in the final race, when the championship was already decided. Then you also have to consider that Senna was quicker, of course, but I don't agree with you when you decide to give Senna's speed more credit than Prost's consistency when Prost won the title.

As I tried to explain earlier, an inconsistent driver is always more vulnerable to technical failures than a consistent driver. In 1988, McLaren had very good reliability and this played into Senna's hands. In 1989, McLaren were not that reliable and then it played into Prost's hands, and that was not because Prost only had one retirement due to techincal problems. Prost could have afforded at least two more retirements and still won. Probably he could have afforded three additional retirements, since he would probably have chosen to run in the final race if he needed the points. If McLaren had been bulletproof in 1989, Senna would probably have beaten Prost to the title. But by finishing less races, Senna had less margin. I could also make the argument that if McLaren had been less reliable in 1988, Prost would probably have beaten Senna that year, since he scored in more races. Each additional race without points would have cost Senna dearly since he only had 4 points to spare, while Prost had 18 points to spare. Prost actually had two retirements in 1988 due to technical problems, while Senna to my knowledge had only one. Despite this, Prost scored in more races.

But in the end, I agree that the difference would have to be large before you can consider any stats to be significant and speak clearly. Especially if you compare driver over just one season. If you compare drivers like Schumacher and Barrichello, who were team mates for 6 years and raced more than 100 races for Ferrari, the stats are more reliable, even if the actual difference is quite small. But in the case of Senna and Prost, the stats won't allow us to conclude one way or the other because it's too close. In my opinion, as I said before, any conclusion beyond the stats is all down to feelings. And then you can of course look for stats that support your feelings, but even if feelings can be disguised as stats, by selectively choose the stats that support your feelings, they are still feelings. Stats usually need to be complete (or randomly selected) in order to be objective. This is something many amateur interpreters of stats aren't aware of and that is probably the reason why so many people think that stats lie all the time.

[sennafan24]

With regards to what I wrote about Senna and Prost, you have a valid point in that Senna had 4 car failures in 1989, while Prost only had 1. But my point was that in those 12 races where Senna's car didn't fail, he only scored points in 7 of them. In those races where Prost's car didn't fail, he brought the car home in the points 13 times out of 15. So if you want to try and look at this objectively, you have to consider that 13/15 is a significantly higher ratio than 7/12. You can actually argue that it was 13/14 for Prost, since he chose not to race in the final race, when the championship was already decided. Then you also have to consider that Senna was quicker, of course, but I don't agree with you when you decide to give Senna's speed more credit than Prost's consistency when Prost won the title.

But in the end, I agree that the difference would have to be large before you can consider any stats to be significant and speak clearly. Especially if you compare driver over just one season. If you compare drivers like Schumacher and Barrichello, who were team mates for 6 years and raced more than 100 races for Ferrari, the stats are more reliable, even if the actual difference is quite small. But in the case of Senna and Prost, the stats won't allow us to conclude one way or the other because it's too close. In my opinion, as I said before, any conclusion beyond the stats is all down to feelings. And then you can of course look for stats that support your feelings, but even if feelings can be disguised as stats, by selectively choose the stats that support your feelings, they are still feelings. Stats usually need to be complete (or randomly selected) in order to be objective. This is something many amateur interpreters of stats aren't aware of and that is probably the reason why so many people think that stats lie all the time.

The thing is I do not think Prost was more consistent than Senna in 1989, given that even when Senna' problems were not car related, it was not Senna's fault. Berger chopping of in Brazil, Mansell colliding with him in Portugal, and a car going into the back of him in the last race. I will leave what happened in Japan, as it is a played out subject.

Prost did drive well that year, and as you said he was always consistent enough to capatalise on Senna's bad luck. Amazingly, Senna did make a silly mistake with a backmarker in 1990 a year later, so his silly mistakes were not out of his system until around 1991.I am open to fans taking Prost's side in the debate, as you said the stats are too close to dismiss either side of the argument, and does come down to "feelings" or what I describe as "subjective opinions"

To clarify, in Brazil 1988 Senna had a host of mechanical problems on the grid, that ruined his race before he was black flagged, he also had differential problems in two races that allowed Prost to tally the scored from 7-4 to 7-6. To be fair though, Prost did have gearbox problems in Japan that helped Senna make that legendary comeback to win the title. Senna did make more driver errors in 1988, I can agree on that.

I agree on Schumi/Rubens that if a small gap in the stats is repeated several times over it could be seen as a significant reflection that one driver is greater than the other. I would consider Mika to be a better driver than D.C, but the stats maybe very close if I was to research them, due to Mika having that dreadful last year before he retired and D.C outscoring Mika in 1997.

[Richard]

Any adjective is subjective. So objective stats become meaningless when we try to use them to justify subjective opinion, or in extreme cases to post-rationalise emotion.

For example the most successful driver on the grid must be Vettel because he has 3 WDC. Hang on a second, Alonso has the most wins. Ah, but Vettel is only 2 races behind Alonso in a much shorter career. That's only because Vettel has been lucky to have a good car. etc etc

So we end up with a lot of subjective words. Real statistics discussion should look like this:



So let's not pretend that lots of words discussing who is best is anything but opinion.

[GitanesBlondes]

As I tried to explain earlier, an inconsistent driver is always more vulnerable to technical failures than a consistent driver. In 1988, McLaren had very good reliability and this played into Senna's hands. In 1989, McLaren were not that reliable and then it played into Prost's hands, and that was not because Prost only had one retirement due to techincal problems. Prost could have afforded at least two more retirements and still won. Probably he could have afforded three additional retirements, since he would probably have chosen to run in the final race if he needed the points. If McLaren had been bulletproof in 1989, Senna would probably have beaten Prost to the title. But by finishing less races, Senna had less margin. I could also make the argument that if McLaren had been less reliable in 1988, Prost would probably have beaten Senna that year, since he scored in more races. Each additional race without points would have cost Senna dearly since he only had 4 points to spare, while Prost had 18 points to spare. Prost actually had two retirements in 1988 due to technical problems, while Senna to my knowledge had only one. Despite this, Prost scored in more races.

This isn't actually an argument though.

You didn't address when I pointed out that there is no way you can prove some of the mechanical failures Senna suffered in 1989 were down to him being "inconsistent".

Unless you can prove that, you're just making a nicely worded subjective argument that has no actual objective merit.

[sennafan24]

So we end up with a lot of subjective words. Real statistics discussion should look like this:

http://laussy.org/elena/thesis/img1436.png

SPSS simplifies things these days

Agreed on your general point though.

[sennafan24]

This isn't actually an argument though.

You didn't address when I pointed out that there is no way you can prove some of the mechanical failures Senna suffered in 1989 were down to him being "inconsistent".

Unless you can prove that, you're just making a nicely worded subjective argument that has no actual objective merit.

He did try to be fair.

He pointed out that when mechanical failures did not affect Senna in 1989, he only finished 7/12. I deployed a counter argument above to show my argument that even then it did not prove any sort of inconsistency, due to the nature of those non-mechanical retirements.

[bdr529]

Any adjective is subjective. So objective stats become meaningless when we try to use them to justify subjective opinion, or in extreme cases to post-rationalise emotion.

So we end up with a lot of subjective words. Real statistics discussion should look like this:

http://laussy.org/elena/thesis/img1436.png

So let's not pretend that lots of words discussing who is best is anything but opinion.

You must be the offspring of my college professor He'd say this everyday "Math is a language on its own, it doesn't need words, the answer is always numbers"

I stumbled across this yesterday it was completed in 2006,

Who is the Best Formula 1 Driver?... An Econometric Analysis
Master Thesis
David Stadelmann
University of Fribourg, Switzerland
http://david.stadelmann-online.com/pdf/ ... rmula1.pdf

His top 3 Schumacher, Prost, Senna.

[GitanesBlondes]

I appreciate their effort to try and quantify everything based on numbers alone. It's not a bad effort at all.

However, where they fail completely is with regards to politics in Formula 1. It's not something that can be quantified into a pure number. I did not see one mention of the word politic in the entire document in spite of the fact that sport is governed by it.

For example, how do you quantify what happened at Austria 2002 when Barrichello was told to move aside to let Schumacher through?

[sennafan24]

Another thing that in my opinion is impossible to determine is how good the machinery is at a drivers disposal. The link I posted on the other page which tries to determine this found some interesting results, like the Honda in 2004 was equal to the Ferrari, but it was the drivers who made the difference. If you value that system (which I do) it does speak volumes for Schumi/Rubens. I will be honest that before I read that argument, I assumed the Ferrari was the best car by leaps and bounds in 2004.

I think the difficulty in determining who's car is better than who's makes it impossible to present a stats based answer as to who is better than who.

[bdr529]

Another thing that in my opinion is impossible to determine is how good the machinery is at a drivers disposal. The link I posted on the other page which tries to determine this found some interesting results, like the Honda in 2004 was equal to the Ferrari, but it was the drivers who made the difference. If you value that system (which I do) it does speak volumes for Schumi/Rubens. I will be honest that before I read that argument, I assumed the Ferrari was the best car by leaps and bounds in 2004.

I think the difficulty in determining who's car is better than who's makes it impossible to present a stats based answer as to who is better than who.

Why not read it before dismissing it all together

[bdr529]

I appreciate their effort to try and quantify everything based on numbers alone. It's not a bad effort at all.

However, where they fail completely is with regards to politics in Formula 1. It's not something that can be quantified into a pure number. I did not see one mention of the word politic in the entire document in spite of the fact that sport is governed by it.

For example, how do you quantify what happened at Austria 2002 when Barrichello was told to move aside to let Schumacher through?

It maybe about F1 but I have to admit it's not the most exciting thing I've read and I'm only half way done.
As for Austria 2002 I don't know if he makes adjustments for this.
I guess in this case 1 win less win for Schumacher doesn't really alter his totals much, and did he not give that back by accident at USGP

[sennafan24]

Another thing that in my opinion is impossible to determine is how good the machinery is at a drivers disposal. The link I posted on the other page which tries to determine this found some interesting results, like the Honda in 2004 was equal to the Ferrari, but it was the drivers who made the difference. If you value that system (which I do) it does speak volumes for Schumi/Rubens. I will be honest that before I read that argument, I assumed the Ferrari was the best car by leaps and bounds in 2004.

I think the difficulty in determining who's car is better than who's makes it impossible to present a stats based answer as to who is better than who.

Why not read it before dismissing it all together

I did not dismiss, I found it very interesting and well structured paper (I like how they admitted limitations to their study and approaches)

I was just adding a point to something posted above, it was not anything in relation to the paper. I was talking about a similar critic I posted on the previous page.

[Stradivarius]

This isn't actually an argument though.

You didn't address when I pointed out that there is no way you can prove some of the mechanical failures Senna suffered in 1989 were down to him being "inconsistent".

Unless you can prove that, you're just making a nicely worded subjective argument that has no actual objective merit.

I never said that Senna was inconsistent because his car failed more often than Prost's in 1989. And I never claimed that it was Senna's fault when his car failed. The reason why I say that Senna was more incosistent than Prost is that in those 12 races where his car didn't fail, he only scored points in 7 of them. Prost, on the other hand, scored points in 13 of the 14 races in which he participated where his car didn't fail. The only race he didn't finish in the points was the one in Japan. I tend to agree with sennafan24 that it is probably not a great idea to discuss that particular race here, but until this infamous incident, Prost had suffered one car problem and he had finished all the other races in the points. Senna, however, had only scored in half of the races until then and that wasn't only because of car problems.

If I interpret sennafan24 correctly, he feels that it wasn't Senna's fault when he was involved in collisions. But my opinion is that a driver can actually influence the risk of these things happening. I don't believe it is pure coincidence that Senna collided more than Prost while driving for McLaren (or in general for that matter). I think I see a connection between the driver's reputation as brilliant overtakers and the rate of collisions. I am now talking out of an impression I have without having actually checked the stats, but Hamilton and Senna both have a reputation for overtaking, but also a lot of collisions. I think the stats will confirm this if you check it out. Button and Prost are more modest when it comes to overtaking and I think they have lost more wheel-to-wheel duels, but they also have a lower collision rate. I also think it makes perfect sense that in order to pull off these brave overtaking manouvers, you need to take high risks. I think most people would say that Senna was better wheel-to-wheel than Prost and that Hamilton is better wheel-to-wheel than Button. But from another point of view, Prost seemed better at avoiding collisions and I think the same could be said about Button, at least when the car was competitive. In some situations you can either yield and make sure you don't crash, or you can keep fighting and run the risk of colliding. (It is natural that you increase the risk when you are far down the field, as you have more to gain and less to loose.) It occurs to me that Senna never yielded.

[sennafan24]

If I interpret sennafan24 correctly, he feels that it wasn't Senna's fault when he was involved in collisions. But my opinion is that a driver can actually influence the risk of these things happening. I don't believe it is pure coincidence that Senna collided more than Prost while driving for McLaren (or in general for that matter). I think I see a connection between the driver's reputation as brilliant overtakers and the rate of collisions. I am now talking out of an impression I have without having actually checked the stats, but Hamilton and Senna both have a reputation for overtaking, but also a lot of collisions. I think the stats will confirm this if you check it out. Button and Prost are more modest when it comes to overtaking and I think they have lost more wheel-to-wheel duels, but they also have a lower collision rate. I also think it makes perfect sense that in order to pull off these brave overtaking manouvers, you need to take high risks. I think most people would say that Senna was better wheel-to-wheel than Prost and that Hamilton is better wheel-to-wheel than Button. But from another point of view, Prost seemed better at avoiding collisions and I think the same could be said about Button, at least when the car was competitive. In some situations you can either yield and make sure you don't crash, or you can keep fighting and run the risk of colliding. (It is natural that you increase the risk when you are far down the field, as you have more to gain and less to loose.) It occurs to me that Senna never yielded.

You do interpret me right in relation to 1989, as Australia and the incidents with Mansell and Berger were not Senna's fault. I again will leave the Prost/Japan saga. All this cost Senna serious points.

I do agree with your general point about Senna getting caught up more not being coincidental. I referenced above that Senna up until around 1991 could make silly mistakes with backmarkers, like Monza 1988 and Brazil 1990, races he should have won. Prost never made mistakes like that to my knowledge.

[Stradivarius]

You do interpret me right in relation to 1989, as Australia and the incidents with Mansell and Berger were not Senna's fault. I again will leave the Prost/Japan saga. All this cost Senna serious points.

Don't you think it would have been possible for Senna to avoid these collisions?

In Brazil, Senna was in a difficult position, as he had Patrese to the left and Berger to the right. And when Patrese started to turn, Senna squeezed Berger towards the edge of the track and they touched. You could say in Senna's defense that he had no where to go. But that is simply because you exclude the possibility of lifting slightly and let Patrese and Berger take the fight between themselves. This is speculation, but I actually suspect that this is what Prost would have done. Patrese was already ahead and Berger was all the way up on the inside with higher speed, so keeping fighting was always going to be both difficult and risky. It's a long race and it's not all decided in the first corner, unless you crash.

In Portugal, when Senna and Mansell collided, I also feel it would have been very possible for Senna to avoid a collision. Mansell came up on the inside and Senna turned in, giving him very little space. We all know that this is an efficient way of defending a position. But we also know that it is always risky to do so, as the guy on the inside will have very little space and he is likely to come outwards at some point. Again I speculate, but if this had been Prost in Senna's place, I suspect he would have given Mansell a bit more room and avoided a collision.

In Australia the visibility was low and Senna hit Brundle from behind. You can say that it wasn't Senna's fault, but you could also say that given the condition, he could maybe have been a bit more careful. After all, he was driving blindly, he couldn't see what was in front of him. When you do that, there will be a risk that you hit something that you don't see in time.

I am not putting the blame on Senna for any of these incidents. But that doesn't change the fact that what happened was driver related and it wasn't all the other driver's fault. I would say it was typical racing incidents where both drivers could have avoided a collision by backing off, but none of them did. And this has been my point all the time. Senna rarely backed off, Prost rarely crashed.