Logging #

Every program running in production needs good logging. Without logs, you can’t diagnose bugs that only appear on servers, trace wrong execution flows, or monitor application performance. Python provides a very complete logging module — far better than print() for almost every real-world case. Understanding how it works will save you a lot of debugging time later.

Why Not `print()` #

# ANTI-PATTERN: debugging with print()
def proses_pembayaran(order_id, jumlah):
    print(f"Processing payment for order {order_id}")   # no timestamp
    print(f"Amount: {jumlah}")                         # no severity level
    # how do you turn all these prints off in production?
    # how do you send this log to a file?
    # how do you filter only errors?

# CORRECT: use logging
import logging

logger = logging.getLogger(__name__)

def proses_pembayaran(order_id, jumlah):
    logger.info("Processing payment for order %s", order_id)
    logger.debug("Amount detail: %s", jumlah)
    # controllable level, sendable to files, formatable, filterable

Log Levels #

Python defines five standard log levels, from lowest to highest:

DEBUG    (10) -- detailed information for debugging, usually turned off in production
INFO     (20) -- confirmation that things are working as expected
WARNING  (30) -- something unexpected happened, but the program keeps running
ERROR    (40) -- an error caused a function to fail, but the program keeps running
CRITICAL (50) -- a serious error that might stop the program

import logging

logging.basicConfig(level=logging.DEBUG)

logging.debug("Database query finished in 12ms")
logging.info("User login successful: user_id=42")
logging.warning("Database connection slow, response time > 500ms")
logging.error("Failed to send email to [email protected]")
logging.critical("Database unreachable, all requests failing")

Output:

DEBUG:root:Database query finished in 12ms
INFO:root:User login successful: user_id=42
WARNING:root:Database connection slow, response time > 500ms
ERROR:root:Failed to send email to [email protected]
CRITICAL:root:Database unreachable, all requests failing

basicConfig — Quick Setup #

logging.basicConfig() is the fastest way to configure logging. It’s enough for simple scripts or prototyping.

import logging

# Log to the console with a full format
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)

# Log to a file
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    filename="app.log",
    filemode="a",   # "a" for append, "w" to overwrite each run
    encoding="utf-8",
)

basicConfig() only takes effect if the root logger doesn’t have a handler yet. If you call logging.info() before basicConfig(), the root logger is already configured with defaults and subsequent basicConfig() calls have no effect. Always call basicConfig() at the start of the program before any logging.

Loggers, Handlers, and Formatters #

For more complex applications, you need to understand the three main components of Python’s logging system:

flowchart TD
    Logger["Logger\n(entry point you use in code: getLogger)"] --> Handler["Handler\n(determines where logs go: file, console, email, etc.)"]
    Handler --> Formatter["Formatter\n(determines how the log message is formatted)"]

Loggers #

import logging

# Always use __name__ as the logger name
# This makes the logger name follow the module hierarchy: "myapp.services.payment"
logger = logging.getLogger(__name__)

# Don't use the root logger directly in library/application modules
# ANTI-PATTERN:
logging.info("pesan")   # uses the root logger -- hard to control

# CORRECT:
logger = logging.getLogger(__name__)
logger.info("pesan")    # uses a named logger

Handlers #

A handler determines the output destination of logs. One logger can have several handlers at once.

import logging

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)   # minimum level processed by this logger

# StreamHandler -- log to the console (stdout/stderr)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)   # filter: only INFO and above

# FileHandler -- log to a file
file_handler = logging.FileHandler("app.log", encoding="utf-8")
file_handler.setLevel(logging.DEBUG)     # filter: all levels to the file

# Add handlers to the logger
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Now: DEBUG and above go to the file, INFO and above also go to the console
logger.debug("debug detail")    # file only
logger.info("important info")   # file AND console
logger.error("an error!")       # file AND console

Formatters #

import logging

# Commonly used format strings
FORMAT_SEDERHANA = "%(levelname)s: %(message)s"
FORMAT_LENGKAP = "%(asctime)s [%(levelname)-8s] %(name)s:%(lineno)d - %(message)s"
FORMAT_JSON_LIKE = '{"time": "%(asctime)s", "level": "%(levelname)s", "msg": "%(message)s"}'

# Attributes available in the format string:
# %(asctime)s    -- when the log was created
# %(name)s       -- the logger name
# %(levelname)s  -- the level as a string (DEBUG, INFO, etc.)
# %(levelno)d    -- the level as a number (10, 20, etc.)
# %(message)s    -- the log message
# %(filename)s   -- the file name
# %(lineno)d     -- the line number
# %(funcName)s   -- the function name
# %(process)d    -- the process ID
# %(thread)d     -- the thread ID

formatter = logging.Formatter(
    fmt="%(asctime)s [%(levelname)-8s] %(name)s:%(lineno)d - %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S"
)

handler = logging.StreamHandler()
handler.setFormatter(formatter)

Full Application Configuration #

Here’s a common configuration pattern for production applications — logging to the console and a file simultaneously, with different levels.

import logging
import logging.handlers
from pathlib import Path

def setup_logging(log_level: str = "INFO", log_file: str = "app.log"):
    """Logging configuration for the application."""

    # Create the log directory if it doesn't exist
    Path(log_file).parent.mkdir(parents=True, exist_ok=True)

    # Format
    fmt = "%(asctime)s [%(levelname)-8s] %(name)s - %(message)s"
    formatter = logging.Formatter(fmt=fmt, datefmt="%Y-%m-%d %H:%M:%S")

    # Root logger
    root_logger = logging.getLogger()
    root_logger.setLevel(logging.DEBUG)   # catch everything, filter in handlers

    # Handler 1: console -- INFO and above only
    console = logging.StreamHandler()
    console.setLevel(getattr(logging, log_level.upper()))
    console.setFormatter(formatter)

    # Handler 2: rotating file -- all levels
    # RotatingFileHandler: rotates when the file reaches a certain size
    file_handler = logging.handlers.RotatingFileHandler(
        log_file,
        maxBytes=10 * 1024 * 1024,   # 10 MB per file
        backupCount=5,                # keep 5 old files
        encoding="utf-8",
    )
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)

    root_logger.addHandler(console)
    root_logger.addHandler(file_handler)


# Call once at the application entry point
setup_logging(log_level="INFO", log_file="logs/app.log")

RotatingFileHandler vs TimedRotatingFileHandler #

import logging.handlers

# RotatingFileHandler -- rotation based on file size
rotating = logging.handlers.RotatingFileHandler(
    "app.log",
    maxBytes=10_000_000,   # 10 MB
    backupCount=5,          # keep app.log.1 through app.log.5
)

# TimedRotatingFileHandler -- rotation based on time
timed = logging.handlers.TimedRotatingFileHandler(
    "app.log",
    when="midnight",   # rotate every midnight
    interval=1,        # every 1 day
    backupCount=30,    # keep the last 30 days
    encoding="utf-8",
)
# Old files: app.log.2024-01-15, app.log.2024-01-14, etc.

Logger Hierarchy #

Loggers in Python follow a dot-separated hierarchy based on their names. This allows per-module or per-subsystem logging control.

import logging

# Logger hierarchy:
# root
#   └── myapp
#         ├── myapp.services
#         │     └── myapp.services.payment
#         └── myapp.api

# Logs from children propagate to the parent by default (propagate=True)
logger_payment = logging.getLogger("myapp.services.payment")
logger_api = logging.getLogger("myapp.api")
logger_app = logging.getLogger("myapp")

# Set different levels per subsystem
logging.getLogger("myapp").setLevel(logging.INFO)
logging.getLogger("myapp.services.payment").setLevel(logging.DEBUG)
# -- the payment logger records DEBUG, but other subsystems only INFO

# Turn off propagation if you don't want logs forwarded to the parent
logger_terlalu_verbose = logging.getLogger("library.noisy")
logger_terlalu_verbose.propagate = False

Logging Exceptions #

Include the exception traceback in the log to make debugging easier.

import logging

logger = logging.getLogger(__name__)

def bagi(a, b):
    try:
        return a / b
    except ZeroDivisionError:
        # ANTI-PATTERN: logging without a traceback
        logger.error("A division error occurred")

        # CORRECT: include the traceback with exc_info=True
        logger.error("Failed to divide %s by %s", a, b, exc_info=True)

        # Or use logger.exception() -- automatically includes the traceback
        logger.exception("Failed to divide %s by %s", a, b)
        return None

bagi(10, 0)

Output with logger.exception():

ERROR:__main__:Failed to divide 10 by 0
Traceback (most recent call last):
  File "app.py", line 6, in bagi
    return a / b
ZeroDivisionError: division by zero

Extra and LoggerAdapter #

Add extra context to every log message — useful for tracking request IDs, user IDs, or session information.

import logging

logger = logging.getLogger(__name__)

# extra -- add fields once per call
logger.info(
    "Payment successful",
    extra={"user_id": 42, "order_id": "ORD-001", "amount": 150000}
)

# LoggerAdapter -- add the same context to all messages
class RequestLogger(logging.LoggerAdapter):
    def process(self, msg, kwargs):
        return f"[req:{self.extra['request_id']}] {msg}", kwargs

# Create an adapter with the request context
request_logger = RequestLogger(logger, {"request_id": "abc-123"})
request_logger.info("Request received")
request_logger.info("Input validation done")
request_logger.error("Processing failed")
# All messages automatically include [req:abc-123]

Silencing Noisy Third-Party Libraries #

Libraries like urllib3, boto3, or sqlalchemy often produce too many logs. How to quiet them:

import logging

# Raise the level to WARNING so only errors show up
logging.getLogger("urllib3").setLevel(logging.WARNING)
logging.getLogger("boto3").setLevel(logging.WARNING)
logging.getLogger("botocore").setLevel(logging.WARNING)
logging.getLogger("sqlalchemy.engine").setLevel(logging.WARNING)

# Or turn them off completely
logging.getLogger("library.yang.berisik").disabled = True

Configuration via `dictConfig` #

For larger applications, separate the logging configuration into a dictionary or YAML/JSON file so it’s easy to change without touching code.

import logging
import logging.config

LOGGING_CONFIG = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "standard": {
            "format": "%(asctime)s [%(levelname)-8s] %(name)s - %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S",
        },
        "simple": {
            "format": "%(levelname)s: %(message)s",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "INFO",
            "formatter": "simple",
            "stream": "ext://sys.stdout",
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "DEBUG",
            "formatter": "standard",
            "filename": "logs/app.log",
            "maxBytes": 10485760,
            "backupCount": 5,
            "encoding": "utf-8",
        },
    },
    "loggers": {
        "myapp": {
            "level": "DEBUG",
            "handlers": ["console", "file"],
            "propagate": False,
        },
        "myapp.services.payment": {
            "level": "DEBUG",
            "handlers": ["file"],
            "propagate": True,
        },
    },
    "root": {
        "level": "WARNING",
        "handlers": ["console"],
    },
}

logging.config.dictConfig(LOGGING_CONFIG)

logger = logging.getLogger("myapp")
logger.info("Application started")

Summary #

Don’t use print() for logging — no levels, no timestamps, not controllable, can’t be sent to files.
Always use logging.getLogger(__name__) in every module — this makes the logger name follow the module hierarchy and is easy to control per subsystem.
Set levels on loggers and handlers separately — the logger determines the minimum level processed, the handler determines the minimum level sent to its destination.
Use logger.exception() inside an except block — it automatically includes the traceback without needing exc_info=True.
RotatingFileHandler for size-based rotation; TimedRotatingFileHandler for daily/weekly rotation.
Quiet overly verbose third-party libraries with logging.getLogger("lib_name").setLevel(logging.WARNING).
Use dictConfig for complex logging configurations so they’re easy to change and manage.

← Previous: Collections Next: Random →