When I first started learning PyTorch, one concept felt surprisingly confusing:

The training loop — especially forward pass, loss, backpropagation, and optimizer steps.

Over time, I realized something important:

👉 The problem wasn’t the code. 👉 It was the mental model.

So I built an intuitive analogy:

Think of a machine learning model as a car learning how to drive.

In this post, I’ll walk you through both training and testing loops using this metaphor—while keeping everything technically accurate.

Big Picture

The Training Loop (Learning Phase)

This is where the car learns how to drive better.

1. Forward Pass — The Car Drives

y_pred = model(X_train)

The model processes input data and produces predictions.

🚗 Analogy: You let the car drive on a road using its current setup.

• No correction yet

• Just observing behavior

2. Loss Calculation — Measure the Mistake

loss = loss_fn(y_pred, y_train)

We compare predictions with the actual values.

🚗 Analogy: You measure how far the car is from the correct path.

Important: We are NOT comparing two cars. We compare car vs ideal path.

3. Zero Gradients — Clear Old Notes

optimizer.zero_grad()

PyTorch accumulates gradients by default, so we reset them.

🚗 Analogy: The mechanic clears the notebook before analyzing a new trip.

Otherwise:

• Old mistakes mix with new ones

• Learning becomes incorrect

4. Backpropagation — Analyze the Journey

loss.backward()

Gradients are computed using the chain rule.

🚗 Analogy: The mechanic reviews the trip and asks:

• What caused the drift?

• Steering?

• Engine?

• Balance?

⚠️ This is NOT the car going backward. It’s analysis, not movement.

5. Optimizer Step — Fix the Car

optimizer.step()

Weights are updated to reduce error.

🚗 Analogy: The mechanic adjusts the car slightly:

• Fine-tunes steering

• Improves balance

• Optimizes performance

Small steps = Gradient Descent

Full Training Loop Code

for epoch in range(epochs):

    model.train()

    # Forward pass
    y_pred = model(X_train)

    # Loss calculation
    loss = loss_fn(y_pred, y_train)

    # Reset gradients
    optimizer.zero_grad()

    # Backpropagation
    loss.backward()

    # Update weights
    optimizer.step()

The Testing Loop (Evaluation Phase)

Now the key idea:

No learning happens here!

1. Evaluation Mode

model.eval()

🚗 Analogy: The car is now being tested — no modifications allowed.

2. Forward Pass Only

with torch.inference_mode():
    test_pred = model(X_test)

🚗 Analogy: The car drives on a new road.

• No mechanic

• No adjustments

• Just performance

3. Measure Performance

test_loss = loss_fn(test_pred, y_test)

🚗 Analogy: You observe:

• How well did the car perform?

• Did it follow the road correctly?

But you DO NOT fix anything.

❌ What’s Missing (Intentionally)

In testing, we DO NOT do:

optimizer.zero_grad()
loss.backward()
optimizer.step()

🚗 Because:

The mechanic is not allowed to touch the car.

Full Training + Testing Loop

for epoch in range(epochs):

    # TRAINING
    model.train()
    y_pred = model(X_train)
    loss = loss_fn(y_pred, y_train)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # TESTING
    model.eval()
    with torch.inference_mode():
        test_pred = model(X_test)
        test_loss = loss_fn(test_pred, y_test)

    if epoch % 10 == 0:
        print(f"Epoch: {epoch} | Train Loss: {loss.item():.4f} | Test Loss: {test_loss.item():.4f}")

Final Mental Model

Training:

Drive → Analyze → Fix → Repeat

Testing:

Drive → Observe → No Changes

Key Insights

• You don’t compare two runs → you compare prediction vs truth

• Backpropagation is analysis, not movement

• Gradients must be cleared to avoid accumulation

• Testing is strictly evaluation

Conclusion

Understanding PyTorch loops isn’t about memorizing code—it’s about building the right intuition.

Once you see it this way:

A model isn’t just math. It’s a system learning from its mistakes—like a car being refined by a mechanic.

And that’s when everything starts to click.

If this analogy helped you, try extending it:

• What would batching look like?

• What about overfitting?

• What if the road changes?

That’s where deeper understanding begins.