Python shelve Module — Persistent Dict-Like Storage

The shelve module gives you a persistent dictionary: open a file, assign db["key"] = value, and read it back the next time your script runs. It is built on pickle and dbm, so you get Python-native storage without SQL — useful for caches, small app state, and local tooling.

The examples below follow the official shelve documentation; run them locally with Python 3.12+ to confirm behavior on your machine.

Tested on: Python 3.12+ (standard library); examples match the official shelve documentation.

Quick answer: store and read data with shelve

import shelve

with shelve.open("my_shelf") as db:
    db["name"] = "John"
    db["age"] = 30

with shelve.open("my_shelf") as db:
    print(db["name"], db["age"])

This creates shelf files in the current working directory (typically my_shelf.db plus index files, depending on the dbm backend). Keys must be strings.

shelve vs JSON, CSV, pickle, and SQLite

Use shelve when you want dictionary ergonomics on a single machine. Move to SQLite when you need SQL, integrity constraints, or safer multi-process access.

Open a shelf and perform basic operations

A shelf behaves like a Python dictionary for most day-to-day use. Pick a path the same way you would for write to file in Python — use a dedicated folder if the shelf will grow.

import shelve

with shelve.open("my_shelf") as db:
    db["name"] = "John"
    db["age"] = 30
    db["active"] = True

Read values with subscript or get():

with shelve.open("my_shelf") as db:
    name = db["name"]
    age = db.get("age", 0)
    missing = db.get("missing_key", "default")

List keys, values, or items when you need to inspect the store:

with shelve.open("my_shelf") as db:
    print(list(db.keys()))
    print(list(db.values()))
    print(list(db.items()))

Keys, values, and custom objects

Keys must be strings. Integer keys raise TypeError:

with shelve.open("my_shelf") as db:
    db["ok_key"] = 123

with shelve.open("my_shelf") as db:
    db[123] = "fails"  # TypeError: keys must be strings

Values can be lists, nested dicts, or custom objects (picklable):

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

with shelve.open("my_shelf") as db:
    db["people"] = [{"name": "John", "age": 30}]
    db["john"] = Person("John", 30)

Access flags: read-only, create, and exclusive create

Read-only example:

import shelve

with shelve.open("my_shelf", flag="r") as db:
    print(db.get("name"))

Exclusive create for a fresh database file:

with shelve.open("new_shelf", flag="n") as db:
    db["first"] = "run only"

If the shelf path does not exist and you pass flag='r', shelve.open raises an error — which is what you want for read-only tooling.

writeback, sync(), and the mutable-value gotcha

By default, in-place mutation of a value does not persist:

with shelve.open("my_shelf") as db:
    db["tags"] = ["a", "b"]
    db["tags"].append("c")  # change is lost when the shelf closes

Either reassign the key or enable writeback=True:

with shelve.open("my_shelf", writeback=True) as db:
    if "tags" not in db:
        db["tags"] = []
    db["tags"].append("c")
    db.sync()  # flush writeback cache to disk

sync() writes cached data without closing the shelf. close() (or exiting a with block) also flushes. With writeback=True, shelve keeps accessed entries in memory — fine for small shelves, costly for huge ones.

Thread safety and concurrent access

shelve is not thread-safe and does not support safe concurrent writes across processes. For threads, guard access with a threading.Lock:

import shelve
import threading

lock = threading.Lock()

def write_key(key, value):
    with lock:
        with shelve.open("my_shelf") as db:
            db[key] = value

def read_key(key, default=None):
    with lock:
        with shelve.open("my_shelf") as db:
            return db.get(key, default)

For multi-process or high-concurrency workloads, use SQLite, Redis, or another database instead of shelve.

Security and error handling

Wrap I/O in try / except in Python for permission errors, missing files, and corrupted databases:

import shelve

try:
    with shelve.open("my_shelf") as db:
        db["key"] = "value"
except OSError as exc:
    print(f"Could not access shelf: {exc}")

Validate keys (strings only) before write to avoid TypeError in long-running jobs.

Advanced: subclassing Shelf

You can extend shelve.Shelf for custom behavior — for example, storing expiry metadata alongside values. Keep the implementation picklable and document that subclasses are for advanced use only.

import shelve
import time

class TimestampShelf(shelve.Shelf):
    def __setitem__(self, key, value):
        super().__setitem__(key, (time.time(), value))

    def __getitem__(self, key):
        _ts, value = super().__getitem__(key)
        return value

Most apps never need this — flags, writeback, and sync() cover typical scripts.

When to choose something else

Reach for JSON or CSV when humans or other languages must read the file. Use SQLite when you need:

• Concurrent readers/writers with transactions
• Indexed lookup and SQL reporting
• Safer handling of partially trusted inputs (still parameterize queries)

shelve remains a strong choice for single-user tools, local caches, and prototypes where a dict API is enough.

Summary

shelve.open() exposes a dict-like store backed by disk files. Use string keys, pick access flags deliberately, and understand writeback before mutating lists or dicts in place. Do not use shelve for security-sensitive or highly concurrent production paths — use SQLite or a dedicated database instead.

References

• Python shelve — official documentation
• pickle — object serialization
• sqlite3 — built-in SQL engine