BANA Learning Hub

Lecture ModeSlide 0 / 1

Week 2 • Advanced Regression

Keep These in Mind

By the end of tonight, you should be able to answer all five.

What is an interaction effect? When do we need it?

How do we capture a U-shaped or inverted U-shaped relationship?

How do we interpret log-log coefficients?

What is an exponential (log-linear) model and how do we back-transform?

What is cross-validation and why is it better than R-squared?

Phase I • Interaction Effects

What Is an Interaction Effect?

When the effect of one variable depends on another.

🤔

The Motivating Question

You are an HR analyst predicting starting salaries. You know an MIS concentration helps, and a Statistics minor helps.

Does having both create extra value beyond having each one separately?

No Interaction

The effect of MIS is always the same, regardless of Statistics.

MIS adds 5K. Stats adds 3K. Both add 8K.

With Interaction

The effect of MIS depends on Statistics. The combo is worth more than the sum.

MIS adds 5K. Stats adds 3K. Both add 11.5K with a synergy bonus.

❗

Definition

An interaction effect occurs when the effect of one independent variable on Y depends on the level of another independent variable.

Phase I • Interaction Effects

The Math Behind Interactions

We multiply the two variables together and add the product as a new term.

Without Interaction

Salary = β₀ + β₁GPA + β₂MIS + β₃Statistics + ε

Effect of MIS is always β₂.

With Interaction

Salary = β₀ + β₁GPA + β₂MIS + β₃Statistics + β₄(MIS × Statistics) + ε

Effect of MIS when Statistics = 0:

β₂

Effect of MIS when Statistics = 1:

β₂ + β₄

β₄ is the synergy bonus — the extra value from having both.

Quick Check

In the model Salary = β₀ + β₁GPA + β₂MIS + β₃Stats + β₄(MIS × Stats), if β₂ = 5000 and β₄ = 3500, what is the salary boost for someone with BOTH an MIS concentration AND a Statistics minor (compared to someone with neither)?

Phase I • Interaction Effects

When Do We Need Interactions?

Three clues that you should add an interaction term.

🧠

Theoretical Reason

You believe variables modify each other's effects. Training should reduce the impact of inexperience.

📊

Different Slopes in Plots

The relationship between X and Y is steeper for one group than another.

💼

Common Business Scenarios

Gender × Experience

Training × Experience

Marketing × Region

Two Types of Interactions

Dummy × Dummy

MIS (0/1) × Statistics (0/1)

Continuous × Continuous

Experience × Training

Phase II • R Lab

R Lab: Six Examples

We'll build models together in RStudio. Here's tonight's roadmap.

🎓

Ex 1: MIS Salary

Dummy × Dummy

📦

Ex 2: Pick Errors

Continuous × Continuous

👫

Ex 3: Gender Gap

Dummy × Continuous

💰

Ex 4: Wages

Quadratic model

🏠

Ex 5: Rent

Log transformations

🏥

Ex 6: Health

Cross-validation

💡

Follow Along

Open RStudio now. We'll write code together — each example builds on the last.

Phase II • R Lab

Example 1: MIS Salary — Without Interaction

Does MIS or a Statistics minor boost starting salary?

🎓

Salary Prediction (MIS.xlsx)

Y = Salary, X1 = GPA, X2 = MIS, X3 = Statistics

Model 1: No Interaction

MIS <- read_excel( "Data/MIS.xlsx")

Model1 <- lm( Salary ~ GPA + MIS + Statistics, data = MIS)

summary(Model1)

Two Predictions: Same GPA, Different Qualifications

# No MIS, No Stats minor, GPA = 3.5

Salary1 <- predict(Model1, data.frame(GPA = 3.5, MIS = 0, Statistics = 0))

# With MIS AND Stats minor, GPA = 3.5

Salary2 <- predict(Model1, data.frame(GPA = 3.5, MIS = 1, Statistics = 1))

In this model, the combined boost is simply β₂ + β₃. The model assumes no synergy.

❗

The Limitation

Model 1 assumes the value of MIS is the same whether or not you have a Statistics minor. Is that realistic? Let's add the interaction and find out.

Phase II • R Lab

Example 1: MIS Salary — With Interaction

Adding the interaction term reveals the synergy effect.

Model 2: With Interaction

Model2 <- lm( Salary ~ GPA + MIS + Statistics + MIS*Statistics, data = MIS)

summary(Model2)

MIS*Statistics automatically includes MIS + Statistics + MIS:Statistics

The MIS:Statistics interaction coefficient

≈ +3,492

All else being equal, the value of an MIS concentration is enhanced by about 3,492 when accompanied by a Statistics minor.

Predictions with Model 2 (GPA = 3.5)

Salary00 <- predict(Model2, data.frame(GPA = 3.5, MIS = 0, Statistics = 0))

Salary21 <- predict(Model2, data.frame(GPA = 3.5, MIS = 1, Statistics = 0))

Salary11 <- predict(Model2, data.frame(GPA = 3.5, MIS = 1, Statistics = 1))

Compare Salary11 vs Salary21: the difference is more than just the Statistics coefficient — it includes the interaction bonus.

💡

Business Insight

Complementary skills create extra value. Encourage both technical skills (MIS) and quantitative foundations (Statistics) — the combination is worth more than the sum.

Phase II • R Lab

Example 2: Pick Errors — Continuous × Continuous

Does training reduce the impact of inexperience on warehouse errors?

📦

Warehouse Pick Errors (Pick_errors.xlsx)

Y = Errors, X1 = Experience, X2 = Training

Compare Models

myData <- read_excel( "Pick_errors.xlsx")

# Model 1: No interaction

Model1 <- lm( Errors ~ Exper + Train, data = myData)

# Model 2: With interaction

Model2 <- lm( Errors ~ Exper + Train + Exper*Train, data = myData)

summary(Model2)

How to Interpret Continuous × Continuous

The effect of one more year of experience depends on training, and the effect of one more hour of training depends on experience. The interaction coefficient tells you how the slope of one changes with the other.

Phase II • R Lab

Example 3: The Gender Pay Gap

Does experience affect salaries differently for men and women?

👫

Gender Wage Analysis (Gender_Gap.xlsx)

Y = Salary, X = Experience, Female (0/1)

Build the Interaction Model

myData <- read_excel( "Data/Gender_Gap.xlsx")

Model2 <- lm( Salary ~ Experience + Female + Female*Experience, data = myData)

summary(Model2)

The Regression Equation

Salary = 38.6 + 4.53·Exp + 4.77·Female − 1.9·Exp×Female

✏️

Your Turn: Write the Equations

Using the equation above, write out:

The salary equation for males (Female = 0)

The salary equation for females (Female = 1)

What does the interaction term tell us about the gender gap over time?

Take 2 minutes, then we'll debrief.

Phase II • R Lab

Example 3: Gender Gap — Revealed

The interaction reveals dynamics a simple model would miss entirely.

Males (Female = 0)

Salary = 38.6 + 4.53·Exp

Start at 38.6K, gain 4.53K per year

Females (Female = 1)

Salary = 43.37 + 2.63·Exp

Start at 43.37K, gain 2.63K per year

The Crossover Point

Women start higher but grow slower. When do men overtake women?

38.6 + 4.53X = 43.37 + 2.63X → 1.9X = 4.77 → X ≈ 2.5 years

After about 2.5 years, men earn more than women. The gap widens with experience.

Quick Check

A simple model (without interaction) showed Female coefficient = +4.77. Why did that model mislead us?

Phase III • Nonlinear Models

When Straight Lines Aren't Enough

Many real-world relationships are curved, not straight.

💰

Wages and Age

Wages rise with age, then decline near retirement

Inverted U-shape

📺

Advertising

Each extra dollar of ad spend generates less return

Diminishing returns

🏭

Production Costs

Costs decrease, then increase with scale

U-shape

A linear model draws a straight line through a curved relationship — it is wrong everywhere.

We need tools to capture curves: quadratic terms and log transformations.

Phase III • Nonlinear Models

The Quadratic Model

Add X squared to capture U-shaped or inverted U-shaped relationships.

Y = β₀ + β₁X + β₂X² + ε

If β₂ < 0

Inverted U

If β₂ > 0

U-shape

The Turning Point

X* = −β₁ / (2β₂)

This is where the curve peaks (inverted U) or bottoms out (U-shape).

In R: -coef(model)[2] / (2 * coef(model)[3])

❗

R Syntax Alert

Use I(X^2) in your formula. The I() wrapper tells R to square the variable.

Phase III • Nonlinear Models

Logarithmic Transformations

Four specifications — each tells a different story.

Model	Equation	Interpretation of β₁
Linear	Y = β₀ + β₁X	1-unit ↑ in X → β₁ unit ↑ in Y
Log-Linear	ln(Y) = β₀ + β₁X	1-unit ↑ in X → (β₁ × 100)% ↑ in Y
Linear-Log	Y = β₀ + β₁ ln(X)	1% ↑ in X → β₁/100 unit ↑ in Y
Log-Log	ln(Y) = β₀ + β₁ ln(X)	1% ↑ in X → β₁% ↑ in Y

Why Log-Log Matters for Business

The coefficient β₁ is an elasticity — a unit-free measure of responsiveness.

Example: 1% increase in size → 0.65% increase in rent.

❗

Critical Warning

You cannot directly compare R-squared between models with different dependent variables (Y vs ln(Y)). We'll learn the fix in the Rent example.

Phase IV • R Lab: Nonlinear

Example 4: Wages and Age

At what age do wages peak? A linear model can't tell us — but a quadratic can.

Step 1: Visualize the Relationship

myData <- read_excel( "Data/Wages.xlsx")

ggplot(myData, aes(x = Age, y = Wage)) +

geom_point() +

stat_smooth(method = "lm", col = "red", se = F) +

stat_smooth(method = "lm", col = "blue", se = F, formula = y ~ poly(x, 2))

Red line = linear fit. Blue line = quadratic fit.

Step 2: Fit the Quadratic Model

Quadratic_Model <- lm( Wage ~ Graduate + Age + I(Age^2), data = myData)

Coeff <- coef(Quadratic_Model)

# Turning point

-Coeff[3] / (2 * Coeff[4])

Interpretation

Wages peak around age 50, then decline. The quadratic model captures the rise and fall.

Phase IV • R Lab: Nonlinear

Example 5: Rent Models

Compare linear, log-linear, linear-log, and log-log models.

Build the Models

Rent <- read_excel( "Rent.xlsx")

Model1 <- lm( Rent ~ Beds + Baths + Sqft, data = Rent)

Model2 <- lm( Rent ~ Beds + Baths + log(Sqft), data = Rent)

Model3 <- lm( log(Rent) ~ Beds + Baths + Sqft, data = Rent)

Model4 <- lm( log(Rent) ~ Beds + Baths + log(Sqft), data = Rent)

❗

Reminder

You cannot compare R-squared between models with different dependent variables. We must back- transform predictions to the original scale first.

Phase IV • R Lab: Nonlinear

Back-Transforming Log Predictions

Convert ln(Rent) predictions back to rent in dollars.

Back-Transformation Correction

plog4 <- predict(Model4, data.frame(Beds = 3, Baths = 2, Sqft = 1600))

se4 <- sigma(Model4)

exp(plog4 + se4^2 / 2)

Interpretation Check

In a log-log model, a coefficient of 0.65 on log(Sqft) means: a 1% increase in square footage is associated with a 0.65% increase in rent.

Phase IV • R Lab: Nonlinear

Guided Practice: Rent Models

Answer these based on the four Rent models you just estimated.

Which model would you use to predict rent for a 3-bed, 2-bath, 1,600 sqft apartment and why?

In Model 4 (log-log), interpret the coefficient on Beds.

Why can’t we directly compare the R-squared of Model 1 with Model 3?

0 / 60 characters

Phase V • Cross-Validation

The Problem with R-squared

R-squared always rises when you add variables — even junk ones.

A model can fit the training data perfectly and still fail on new data. That is overfitting.

❗

The Fix

Use cross-validation to evaluate performance on data the model has never seen.

Phase V • Cross-Validation

The Holdout Method

Split data into training (75%) and validation (25%).

Training Set

75%

Validation Set

25%

Train here

Test here

Phase V • Cross-Validation

The Weakness of Holdout

Results depend on which observations land in each set.

A different random split could lead to a different winner. We need a more robust method.

Phase V • Cross-Validation

K-Fold Cross-Validation

Divide data into K parts. Each part takes a turn as validation. Average the RMSEs.

Fold 1

Validation

Train on the rest

Fold 2

Validation

Train on the rest

Fold 3

Validation

Train on the rest

Fold 4

Validation

Train on the rest

💡

Rule of Thumb

Common choices are K = 5 or K = 10. The model with the lowest average RMSE wins.

Phase V • Cross-Validation

Cross-Validation Takeaways

Cross-validation evaluates prediction, not fit.

RMSE is the standard comparison metric.

K-fold is more robust than a single holdout.

Choose the model with the lowest average RMSE.

Phase VI • R Lab: Validation

Example 6: Holdout Validation

Compare RMSE across two competing models.

Holdout Method

myData <- read_excel( "Health_Factors.xlsx")

TData <- myData[1:90, ]

VData <- myData[91:120, ]

Model1 <- lm( Health ~ Social + Income + College, data = TData)

Model2 <- lm(Health ~ Social + Income + College + SocialIncome + SocialCollege, data = TData)

Pred1 <- predict(Model1, VData)

RMSE1a <- sqrt( mean((VData$Health - Pred1)^2))

💡

Decision Rule

Lower RMSE means better predictions on unseen data.

Phase VI • R Lab: Validation

K-Fold RMSE Comparison

Repeat holdout splits and average RMSE across folds.

K = 4 Example

# Fold 1, 2, 3, 4 ... compute RMSE1a..RMSE1d

c(RMSE1a, RMSE1b, RMSE1c, RMSE1d)

mean(c(RMSE1a, RMSE1b, RMSE1c, RMSE1d))

❗

Conclusion

Choose the model with the lowest average RMSE. A single split can mislead you.

Wrap-Up • Focus Questions

Answering Our Focus Questions

Interaction effect: the effect of one variable depends on another.

U-shaped or inverted U-shaped: add a quadratic term X² and use the turning point.

Log-log coefficients are elasticities: 1% change in X → β₁% change in Y.

Log-linear model: ln(Y) = β₀ + β₁X, interpret β₁ as percent change in Y.

Cross-validation tests predictive accuracy on unseen data, usually via RMSE.

Wrap-Up • Summary

Key Takeaways

Technique	When to Use	Key Metric
Interaction Terms	Effect of X1 depends on X2	Interaction coefficient
Quadratic Model	Curved relationship	Turning point
Log Transformations	Diminishing returns	Elasticity
Cross-Validation	Model comparison	Average RMSE

Topic

Lecture Outline

Currently viewing:

Slide 0 of 1

Use ← → to navigate

More Topics in Regression Analysis

Tonight's Toolkit Expansion

Interaction Terms

Nonlinear Models

Cross-Validation

Keep These in Mind

What Is an Interaction Effect?

The Motivating Question

No Interaction

With Interaction

The Math Behind Interactions

Without Interaction

With Interaction

In the model Salary = β₀ + β₁GPA + β₂MIS + β₃Stats + β₄(MIS × Stats), if β₂ = 5000 and β₄ = 3500, what is the salary boost for someone with BOTH an MIS concentration AND a Statistics minor (compared to someone with neither)?

When Do We Need Interactions?

Theoretical Reason

Different Slopes in Plots

Common Business Scenarios

Two Types of Interactions

R Lab: Six Examples

Example 1: MIS Salary — Without Interaction

Salary Prediction (MIS.xlsx)

Model 1: No Interaction

Two Predictions: Same GPA, Different Qualifications

Example 1: MIS Salary — With Interaction

Model 2: With Interaction

Predictions with Model 2 (GPA = 3.5)

Example 2: Pick Errors — Continuous × Continuous

Warehouse Pick Errors (Pick_errors.xlsx)

Compare Models

How to Interpret Continuous × Continuous

Example 3: The Gender Pay Gap

Gender Wage Analysis (Gender_Gap.xlsx)

Build the Interaction Model

The Regression Equation

Your Turn: Write the Equations

Example 3: Gender Gap — Revealed

Males (Female = 0)

Females (Female = 1)

The Crossover Point

A simple model (without interaction) showed Female coefficient = +4.77. Why did that model mislead us?

When Straight Lines Aren't Enough

Wages and Age

Advertising

Production Costs

The Quadratic Model

The Turning Point

Logarithmic Transformations

Why Log-Log Matters for Business

Example 4: Wages and Age

Step 1: Visualize the Relationship

Step 2: Fit the Quadratic Model

Example 5: Rent Models

Build the Models

Back-Transforming Log Predictions

Back-Transformation Correction

Interpretation Check

Guided Practice: Rent Models

The Problem with R-squared

The Holdout Method

The Weakness of Holdout

K-Fold Cross-Validation

Cross-Validation Takeaways

Example 6: Holdout Validation

Holdout Method

K-Fold RMSE Comparison

K = 4 Example

Answering Our Focus Questions

Key Takeaways