The LendingClub data set contains information about 96736 funded loans for home improvement, debt consolidation, or small business purposes.
We have data on each borrower’s credit sub-grade, loan amount, interest rate, income, and more. We also know whether each loan was fully paid or charged off (defaulted).
Let’s see if we can predict whether a loan will be fully paid based on the borrower’s credit score (a 0–100 scale derived from LendingClub’s sub-grade system).
The Y variable here is categorical (two levels—a loan is either fully paid or it isn’t), so regular linear regression won’t work here.
The idea behind logistic regression
Instead of predicting whether a loan is fully paid, let’s predict the probability that it is fully paid.
In logistic regression, one level of Y is always called “success” and the other “failure.” Since Y=1 for fully paid loans, we designate fully paid as “success.”
Logistic regression predicts the probability of success p as a function of X indirectly, by modeling the log-odds of success as a linear function of X.
Odds
When something has “even (1/1) odds,” the probability of success is 1/2
When something has “2/1 odds,” the probability of success is 2/3
When something has “3/2 odds,” the probability of success is 3/5
In general, the odds of something happening are p/(1-p)
…and so the probability of something happening is \text{odds}/(1+\text{odds})
Odds
Before Texas played Gonzaga in the 2026 NCAA Tournament Second Round, DraftKings listed Texas at +200—meaning a $100 bet on Texas winning would pay out $200.
That is, they were giving Texas odds to win of \$100/\$200 = 1/2 (ignoring house edge).
What’s their implied probability that Texas would win?
\frac{p}{1-p} = \frac{1}{2} \Rightarrow p = \frac{1/2}{1+1/2} \approx 0.333
Odds
Before Texas played Purdue in the 2026 NCAA Tournament Sweet 16, DraftKings listed Texas at +290—meaning a $100 bet on Texas winning would pay out $290.
That is, they were giving Texas odds to win of \$100/\$290 \approx 0.345. What’s their implied probability that Texas would win?
\frac{p}{1-p} = \frac{1}{2} \Rightarrow p = \frac{0.345}{1+0.345} \approx 0.256
Probability vs odds vs log odds
Odds and log odds are just different ways of expressing probabilities:
Probability
Odds
Log odds
p
p/(1-p)
\log(p/(1-p))
Between 0 and 1
Between 0 and \infty
Between -\infty and \infty
0
0
-\infty
0.25
0.33
-1.10
0.5
1
0
0.75
3
1.10
0.8
4
1.39
0.9
9
2.20
0.95
19
2.94
1
\infty
\infty
The logistic regression model
Logistic regression predicts the log odds of success p as a linear function of X:
\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X
This fits an S-shaped curve to the data (we’ll see what it looks like shortly).
Let’s try it
# Fit logistic regression: predict P(fully paid) from credit scoremodel <-glm(fully.paid ~ credit_score, data = loans,family = binomial)summary(model)
Call:
glm(formula = fully.paid ~ credit_score, family = binomial, data = loans)
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -5.86756 0.08163 -71.9 <2e-16 ***
credit_score 0.10464 0.00123 85.2 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 107169 on 96735 degrees of freedom
Residual deviance: 99212 on 96734 degrees of freedom
AIC: 99216
Number of Fisher Scoring iterations: 4
How to interpret the curve?
The regression output tells us that our prediction is
\begin{aligned}
\log(\text{odds}) &= \log\left(\frac{P(\text{fully paid})}{1-P(\text{fully paid})}\right) \\
&= -5.868 + 0.105 \cdot\text{credit\_score}.
\end{aligned}
We give our interepretations in odds, since log-odds are less intuitive.
This works just like a linear regression for \log(Y)!
Interpreting the coefficients
Rewriting the prediction equation as:
\text{Predicted odds of full repayment} = e^{-5.868 + 0.105 \cdot\text{credit\_score}}
Predicted odds when credit score = 0: e^{-5.868} = 0.003.
Minimum credit score in the data is 45, so not meaningful on its own here.
Interpreting the coefficients
Rewriting the prediction equation as:
\text{Predicted odds of full repayment} = e^{-5.868 + 0.105 \cdot\text{credit\_score}}
Increasing credit score by 1 point multiplies the odds by e^{0.105}=1.11; i.e., increases the odds by 100(e^{0.105} - 1) = 11%.
Increasing credit score by 2 points multiplies the odds by e^{2 \cdot 0.105}=1.23; i.e., increases the odds by 100(e^{2\cdot 0.105} - 1) = 23%.
Logistic regression with 2+ predictors
Adding another predictor: can we do better?
Just like with a linear regression model, we can add additional predictors to the model.
Our interpretation of the coefficients in multiple logistic regression is similar to multiple linear regression: each coefficient represents the predicted effect of one X on Y, holding the other X variables constant.
Does loan purpose predict repayment?
# Proportion fully paid within each loan purposextabs(~ purpose + fully.paid, data = loans) %>%prop.table(margin =1) %>%round(2)
For a debt consolidation loan with credit score 0, the predicted log odds are -5.850 (the intercept).
Among loans with the same purpose, each additional credit score point corresponds to a 11% increase in predicted odds of full repayment (multiplied by e^{0.104} = 1.11).
Among loans with the same credit score, home improvement loans have 15% higher predicted odds of full repayment vs. debt consolidation (multiplied by e^{0.138} = 1.15).
Among loans with the same credit score, small business loans have 7% lower predicted odds of full repayment vs. debt consolidation (multiplied by e^{-0.068} = 0.93).
Visualizing the model
Which curve is which?
Visualizing the model
Which curve is which?
Visualizing the model
Evaluating the model
How good is our model?
R^2 and residual standard error metrics aren’t available for logistic regression.
There are many “pseudo-R^2” metrics that indicate model fit.
But most of these are difficult to interpret, so we’ll focus on other metrics
How many cases did we accurately predict?
We could classify each loan using a 0.5 probability threshold: