Logging

Record what your software does as it runs, so you can understand its behavior after the fact.

Understand This First

• Observability – the capability that logging helps you achieve.
• Side Effect – logging is itself a side effect, and it records others.

Context

Your code runs. Something happens — maybe the right thing, maybe not. The moment passes and the state that produced the outcome vanishes. You need a record.

This is a tactical practice at the foundation of runtime understanding. Tests verify behavior before code ships; logging captures behavior while code runs. Tests ask “does it work?” Logs ask “what did it do?”

Problem

Software doesn’t come with a flight recorder. When a function returns the wrong result, when a background job stops processing, when a user reports something that works on your machine but not on theirs, your first question is always the same: what happened? Without a record, you’re guessing. You reconstruct state from memory, from reading code, from “I think it probably went down this path.” Guessing is slow, unreliable, and scales badly.

How do you give yourself a reliable account of what your software did without drowning in noise or leaking sensitive information?

Forces

• You need enough detail to diagnose problems, but too much output buries the signal.
• Log entries are useful only when they carry context: which request, which user, which step.
• Sensitive data (passwords, personal information, API keys) must never appear in logs.
• Logging costs CPU cycles, disk writes, and network bandwidth. In hot paths, that cost adds up.
• Logs must serve both humans and machines. Free-form sentences are easy to write and hard to search.

Solution

Instrument your code to emit structured records of significant events as they happen. Every record should answer three questions: what happened, when, and in what context.

Structured logging means each entry is a set of named fields, not a prose sentence. Instead of "User placed order successfully", emit {event: "order_placed", user_id: 42, order_id: 789, total: 34.50, duration_ms: 230}. Structured entries are searchable, filterable, and parseable by automated systems. JSON is the default format because every major log aggregation platform consumes it natively.

Severity levels separate routine events from problems. The standard progression is DEBUG, INFO, WARN, ERROR, and FATAL. Use them consistently:

• DEBUG — details useful during development but noisy in production: variable values, branch decisions, cache hits.
• INFO — normal operation worth recording: a request served, a job completed, a connection established.
• WARN — recoverable anomalies: a retry that succeeded, a deprecated endpoint called, a configuration that fell back to a default.
• ERROR — failures that need attention: a request that couldn’t be fulfilled, a connection that dropped, a payment declined.
• FATAL — failures that stop the process: out of memory, missing required configuration, corrupted state.

Context propagation ties related entries together. When a web request generates log entries across five functions and two services, each entry should carry the same request ID. That ID lets you pull every entry for a single request in order, reconstructing the full story.

Log at boundaries. The highest-value log points are where your code crosses a boundary: incoming HTTP requests, outgoing database queries, calls to external services. These are the junctions where failures surface and latency accumulates. Logging at boundaries gives you a skeleton of every operation without instrumenting every internal function.

The key discipline is knowing what not to log. Record decisions, outcomes, and errors. Skip variable assignments and loop iterations. A good log reads like a concise narrative of what the system did, not a line-by-line transcript of how it did it.

How It Plays Out

A payment processing service handles thousands of transactions per hour. Each transaction logs its start (INFO: payment_initiated), the authorization result (INFO: payment_authorized or WARN: payment_declined), and completion (INFO: payment_settled). Every entry carries the transaction ID, customer ID, and amount. When a customer reports an unrecognized charge, a support engineer searches by customer ID and pulls the full event sequence for that day. The investigation takes two minutes instead of two hours.

A team building a REST API adds structured logging to every endpoint. Three weeks later, they notice WARN entries for /search spiking every afternoon. The logs reveal a third-party geocoding service timing out during peak hours. They add a local cache and the warnings vanish. Without logging, they’d have discovered the problem only when users complained about slow searches, with no data pointing to the geocoding service as the cause.

In agentic coding, logging is how you understand what an agent did and why. The agent reads files, runs tests, edits code, runs tests again. Its session log records each tool call, each model decision, each test result. When the agent produces unexpected output, you trace its reasoning through the log. Did it misread the test output? Edit the wrong file? That log is your only window into the agent’s process. Without it, debugging means re-running the entire session and hoping to spot the mistake on a second pass.

Consequences

Benefits:

• Problems get diagnosed faster because you have a factual record instead of guesses.
• Patterns emerge from log data that you’d never spot from individual incidents: a slow dependency affecting only certain regions, an error that correlates with a specific client version.
• On-call engineers can investigate incidents without the original developer’s knowledge of the code.
• Automated monitoring systems can consume structured logs and detect anomalies without human attention.

Liabilities:

• Log storage costs money. High-throughput services generate gigabytes per day.
• Poorly designed logging creates noise that buries real signals.
• Sensitive data in logs creates security and compliance risks. Review log contents as carefully as any other output.
• Synchronous log writes add latency. In hot paths, asynchronous logging or sampling may be necessary.
• Stale log statements referencing removed features or renamed fields become misleading. Logging code needs maintenance like any other code.

Related Patterns

• Enables: Observability – logging is the primary mechanism for achieving runtime observability.
• Prevents: Silent Failure – logged events make failures visible that would otherwise go unnoticed.
• Supports investigation of: Failure Mode – logs are the primary evidence when diagnosing which failure mode occurred.
• Supports investigation of: Regression – when a regression reaches production, logs help identify when it started and what changed.
• Complements: Test – tests verify behavior before deployment; logging captures behavior after.
• Specialized by: Progress Log – a progress log is a structured log designed for agentic session journals.
• Used by: Deprecation — per-call logging of deprecated usage, with caller identity, is the minimum instrumentation.
• Related: Printf Debugging — logging is the permanent, structured version of what printf debugging does temporarily. If a printf investigation keeps recurring, convert it to a log.

Sources

• The term “log” comes from nautical tradition, where a ship’s log recorded speed, weather, and events during a voyage. System operators have kept logs of machine behavior since the earliest mainframe installations.
• Ceki Gulcu created Apache Log4j in 2001, establishing the severity level convention (DEBUG through FATAL) that nearly every logging framework since has followed across languages and platforms.
• The shift from free-form text to structured logging accelerated in the 2010s as log aggregation platforms (Splunk, Elasticsearch, Datadog) made machine-parseable formats a practical necessity at scale.