File I/O, Encodings & JSON Serialization

1. The "with open()" Context Manager & File Descriptors

Calling f = open("data.txt") without a context manager leaves the operating system file descriptor open if an unhandled exception occurs before f.close(). Under high-throughput servers, this causes fatal "Too many open files" (EMFILE) OS crashes.

The with statement guarantees that f.close() is executed under all conditions:

# ALWAYS specify encoding="utf-8" to prevent Windows CP1252 character corruption!
with open("audit_log.txt", mode="w", encoding="utf-8") as f:
    f.write("2026-10-15T14:30:00Z - System initialized successfully.\n")
    f.write("User: Alice authenticated via OAuth2.\n")

# File is automatically closed here, even if an exception was raised above!

2. Memory-Efficient Streaming for Large Files

Never use f.read() or f.readlines() on large datasets because it loads the entire file into RAM at once. Instead, iterate over the file handle directly—Python streams chunks on demand through an internal buffer:

# Memory usage remains constant O(1) regardless of whether the file is 1MB or 50GB!
error_count = 0

with open("server_access.log", mode="r", encoding="utf-8") as file:
    for line in file:  # Lazy generator stream!
        if "[ERROR]" in line:
            error_count += 1

print(f"Total error events detected: ${error_count}")

3. JSON Serialization & Deserialization

Python ships with the built-in json module to translate between Python dictionaries/lists and JSON text:

• json.dump(obj, f): Serializes Python object directly into an open file stream.
• json.load(f): Deserializes an open file stream into native Python objects.
• json.dumps(obj): Serializes object to an in-memory JSON string.
• json.loads(str): Parses a JSON string into Python objects.

import json

server_state = {
    "cluster_id": "us-east-4a",
    "nodes": 16,
    "active": True,
    "services": ["auth", "billing", "gateway"]
}

# Write formatted JSON to disk
with open("state.json", mode="w", encoding="utf-8") as f:
    json.dump(server_state, f, indent=2)

# Read and parse JSON from disk
with open("state.json", mode="r", encoding="utf-8") as f:
    loaded_state = json.load(f)
    print("Cluster Active Status:", loaded_state["active"])

4. Object-Oriented Paths with pathlib

Legacy Python used brittle string concatenations with os.path.join(). Modern code uses the object-oriented pathlib module:

from pathlib import Path

# Create path object with intuitive / operator
config_dir = Path.home() / ".config" / "hifiapp"
config_dir.mkdir(parents=True, exist_ok=True)  # Creates parent directories if missing

settings_file = config_dir / "settings.json"

# Write & read text directly without manual with open boilerplate!
settings_file.write_text('{"version": "2.1.0"}', encoding="utf-8")
print("Settings content:", settings_file.read_text(encoding="utf-8"))
print("File exists?", settings_file.exists())
print("File size:", settings_file.stat().st_size, "bytes")