Intro to Logistic Regression: Predicting NFL Game Win Probabilities

How much does the QB matter?

The quarterback is the most important position in football. No one individual player has a more consistent impact on the game than them. This short read looks to show an application of logistic regression by taking a QBs rating scraped from footballreference.com to predict the probability the QBs team won the matchup for the corresponding QB rating.

Logistic regression uses a sigmoid function in order to produce a single value output in the form of a probability. Although multiple variables (or features) can be used, I am using a simple one variable input (QB rating) to predict the probability of a win or loss. The data consists of the career game logs for every starting quarterback from the 2019 season. I filtered to only games where the QB started the game. The end result produced a data set with 2.7 thousand games.

The distribution of the games shows what we might expect: the higher the QB rating the more likely the team is to win!

In fact, when a QB has a rating of 100-149 their team wins 80 percent of the time, 150 or above the team never lost!

I split the data into training and testing. Normally, I would randomly pick training and testing, but since I really want to see how our model performs on todays game we are making the most recent season that just occurred the test set. You could also perform cross validation but that is for another read.

The model results showed that not only was QB rating significant but it allowed me to see how much probability would increase for every QB rating point increase. Using the coefficient for rating, our winning probability increases by .04 for every rating value increase of 1.

Below the summary is the mean probability for QB ratings in groups that match what was showed a bit earlier. Notice above 150 rating doesn’t give us a perfect 100 percent like real life. This is good as we can never know for certain. 100-149 averages a win probability 95 percent while 50-99 averages 47 percent.

Call:
glm(formula = WL ~ Rate, family = "binomial", data = qb_train)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.3240  -1.0159   0.5135   0.8976   2.5919  

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -3.323630   0.194022  -17.13   <2e-16 ***
Rate         0.040207   0.002092   19.22   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 3113.1  on 2292  degrees of freedom
Residual deviance: 2600.4  on 2291  degrees of freedom
AIC: 2604.4

Number of Fisher Scoring iterations: 4

How do we know what probability is reasonable to say a team wins given their QB rating. The default for logistic is .5, but here I tested the threshold using validation sets. I started from probability greater than .1 to a probability of greater than .9, each stepping by .1. The graph below told me that .6 was the best percent to use in terms of accuracy error.

The final two tables show other metrics commonly used to evaluate models, this particular model was 69 percent accurate which is pretty decent but could be improved. Precision, recall, and F1 are all decent but will be discussed in more detail another day. The final table has the probability of winning the game in the first column, the QB in the second, passer rating in the game, a W or L for a win or loss in the actual game, and finally a W or L for a win or loss predicted by the logistic model.

Hope you enjoyed this slight intro into the application of Logistic Regression! If you want to learn more or see the code email me!