Difference Between Linear Regression and Logistic Regression
Both Linear Regression and Logistic Regression are supervised machine learning algorithms, but they are used for different types of problems.
Quick Interview Definition
Linear Regression
Used to predict continuous numerical values.
Example:
-
Predict house price
-
Predict salary
-
Predict temperature
Logistic Regression
Used to predict categories/classes (mainly binary classification).
Example:
-
Spam or Not Spam
-
Pass or Fail
-
Fraud or Not Fraud
Core Difference
| Feature | Linear Regression | Logistic Regression |
|---|---|---|
| Purpose | Predict numeric values | Predict categories/classes |
| Output Type | Continuous number | Probability (0 to 1) |
| Problem Type | Regression | Classification |
| Output Example | ₹45,000 salary | 90% chance of spam |
| Formula Output | Any value | Between 0 and 1 |
| Main Function | Straight line | Sigmoid/Logistic function |
| Evaluation Metrics | MAE, MSE, RMSE, R² | Accuracy, Precision, Recall, F1 |
| Algorithm Type | Regression algorithm | Classification algorithm |
Linear Regression
Linear regression tries to fit a straight line between input and output.
The equation is:
Where:
-
( y ) = predicted output
-
( m ) = slope
-
( x ) = input
-
( b ) = intercept
Simple Example (Linear Regression)
Suppose:
| Experience | Salary |
|---|---|
| 1 year | 30k |
| 2 years | 40k |
| 3 years | 50k |
The model predicts salary based on experience.
If someone has 4 years experience → predicted salary may be 60k.
Output:
A numeric value.
Logistic Regression
Logistic regression predicts probability using the sigmoid function.
The sigmoid equation:
This converts values into a range between 0 and 1.
Simple Example (Logistic Regression)
Suppose we predict whether a student passes:
| Study Hours | Result |
|---|---|
| 1 | Fail |
| 2 | Fail |
| 5 | Pass |
| 7 | Pass |
The model predicts probability:
-
0.9 → Pass
-
0.2 → Fail
Usually:
-
Probability > 0.5 → Class 1
-
Probability < 0.5 → Class 0
Output:
Category/Class.
Graph Difference
Linear Regression
Produces a straight line.
Logistic Regression
Produces an S-shaped sigmoid curve.
Important Interview Point
Why not use Linear Regression for Classification?
Linear regression can produce values like:
-
-2
-
1.8
-
5
But classification probabilities must stay between 0 and 1.
Logistic regression solves this using the sigmoid function.
Loss Function Difference
| Algorithm | Loss Function |
|---|---|
| Linear Regression | Mean Squared Error (MSE) |
| Logistic Regression | Log Loss / Cross Entropy |
Real-World Examples
Linear Regression
-
House price prediction
-
Stock price estimation
-
Sales forecasting
Logistic Regression
-
Disease prediction
-
Email spam detection
-
Customer churn prediction
-
Fraud detection
Advantages
Linear Regression
-
Simple and easy to interpret
-
Fast training
-
Works well for linear relationships
Logistic Regression
-
Good for binary classification
-
Outputs probabilities
-
Easy to understand and implement
Limitations
Linear Regression
-
Not suitable for classification
-
Sensitive to outliers
-
Assumes linear relationship
Logistic Regression
-
Works poorly with highly complex relationships
-
Assumes linear boundary between classes
Common Interview Questions
1. Is Logistic Regression actually a regression algorithm?
Technically, it is named regression because it estimates probabilities mathematically, but it is mainly used for classification tasks.
2. Can Logistic Regression handle multiclass classification?
Yes.
Using:
-
One-vs-Rest (OvR)
-
Softmax Regression
3. Why use sigmoid in Logistic Regression?
Because sigmoid converts output into probabilities between 0 and 1.
4. Which is easier to interpret?
Both are interpretable, but linear regression coefficients are usually more directly understandable.
Short Interview Answer
Linear Regression is used for predicting continuous numeric values like salary or house price, while Logistic Regression is used for classification problems like spam detection or pass/fail prediction.
Linear Regression outputs any numeric value and uses a straight-line equation, whereas Logistic Regression uses a sigmoid function to produce probabilities between 0 and 1.
Linear Regression is evaluated using metrics like MSE and RMSE, while Logistic Regression uses accuracy, precision, recall, and log loss.