Logistic Regression

Goal

By the end of this lesson, you can fit a binary logistic-regression classifier, interpret its positive-class scores, and explain how a decision threshold turns those scores into class predictions.

Why a yes/no model often needs more than a yes/no output

Suppose two emails are both classified as spam.

One receives a positive-class score of 0.51; the other receives 0.99.

The final label is the same, but the model is treating the cases very differently.

Logistic regression gives us a continuous value between 0 and 1 before the final class decision. That lets us reason about thresholds and tradeoffs rather than treating every positive prediction as identical.

From a weighted sum to a 0–1 value

Like linear regression, logistic regression first forms a weighted sum:

z = w1*x1 + w2*x2 + ... + b

Then it passes z through the logistic function:

1 / (1 + exp(-z))

The transformation has a useful shape:

• very negative z maps near 0;
• z = 0 maps to 0.5;
• very positive z maps near 1.

Under the model's assumptions, this output is used as a class probability estimate. It is still a model estimate, not a guarantee that “0.8” will always correspond perfectly to eight positives out of ten similar cases.

The threshold is a decision rule, not part of the probability itself

Suppose the model outputs:

0.20, 0.55, 0.90

With threshold 0.5, the class decisions are:

negative, positive, positive

With threshold 0.8, they become:

negative, negative, positive

The model scores did not change. The decision policy changed.

That distinction matters whenever false positives and false negatives have different costs.

Inspect scores and decisions in the Lab

The Lab fits logistic regression to one study-hours feature and binary pass labels.

1. Click Run.
2. Read positive probabilities: [0.087895, 0.499997, 0.912103] for the probe students who studied 2.5, 4.5, and 6.5 hours.
3. Look closely at the middle value. A student who studied 4.5 hours sits exactly between the failing (1–4 hours) and passing (5–8 hours) examples, so the model gives 0.499997—just under 0.5. That is why labels at threshold 0.5: shows [0, 0, 1].
4. Find decision_threshold = 0.5. Change only 0.5 to 0.4.
5. Before running, predict: will the three probabilities change? Will any label change?
6. Click Run. The probabilities are identical, but the labels become [0, 1, 1]. The model did not change; only the rule that turns a score into a decision changed.
7. Press Reset afterward.

Loading lab…

If only the threshold changed, refitting is not necessary: you are changing the rule that maps existing scores to actions.

Probability-looking values still need evaluation

A common mistake is to treat every number between 0 and 1 as automatically trustworthy.

Probability estimates can be poorly calibrated, especially with small datasets, model mismatch, or distribution shift.

A model can rank examples usefully while its numerical probabilities are too confident or not confident enough.

Later work may evaluate calibration in addition to classification metrics.

Why is it called “regression” if it classifies?

The name can be confusing.

Logistic regression uses a linear weighted score inside the model, but the logistic transformation and classification training objective make the final model suitable for binary classification.

The important thing is the behavior, not the historical name.

Quick Check

1. What range does the logistic function map scores into?

Values strictly between 0 and 1Any positive real number, because the logistic function exponentiates its inputNegative infinity to positive infinity

2. What can change a final class without refitting the model?

The decision threshold that turns each score into a classThe optimizer learning rate used during the completed training runThe regularization strength used during the completed training run

3. Why should probability outputs be evaluated?

Any value between 0 and 1 is calibrated by definitionOnly final class labels matter; probability quality cannot affect threshold decisionsThey can be poorly calibrated or shift on new data

0 of 3 questions answered.

Key Takeaways

• Logistic regression is a binary classifier despite its name.
• It maps a weighted linear score into the 0–1 range.
• A decision threshold turns model scores into class labels.
• Changing a threshold can change decisions without refitting the model.
• Probability estimates should be evaluated rather than trusted because of their format.

Next Lesson

Next, you will examine the border in feature space where a classifier changes from one predicted class to the other.

References

• scikit-learn, LogisticRegression.