Skip to content

Object Pool

Intent

Pay the cost of creating an expensive object once, then lend that object to one borrower at a time from a bounded set. The caller asks the pool for an object, uses it and gives it back; the pool decides whether that means reusing an idle one, creating one within the cap, or making the caller wait.

When to use and when not to

Use it when

  • Creation dominates use. A database connection costs a TCP handshake, a TLS handshake and authentication: three round trips or more, 3 x 500 µs ~ 1.5 ms inside one datacenter. Taking an idle one from a queue is an uncontended lock, ~17 ns, about 10^5 times cheaper.
  • The objects are interchangeable: any healthy connection, any free lane, any compact car in the fleet.
  • You need a cap. A pool of 20 connections is also a bulkhead: the twenty-first borrower waits or fails fast instead of opening a connection the database will refuse.
  • The objects have a lifecycle: they go stale, keep state between uses or must be closed at shutdown.

Leave it out when

  • Creation is cheap. A Python object is an allocation, about one memory reference (~100 ns).
  • The objects have identity. If the caller needs that car, by licence plate, you want a repository with a status field.
  • You want a cache: keyed, allowed to miss, evicting. A pool is unkeyed, waits instead of missing and keeps its objects.

Structure

Four roles: the generic pool, the expensive object it lends, the factory that builds one, and a subclass that binds the lifecycle hooks.

classDiagram
    class ObjectPool~T~ {
        -_factory: Callable
        -_idle: Queue~T~
        -_lent: Dict~int,T~
        -_lock: Lock
        +max_size: int
        +size: int
        +in_use: int
        +available: int
        +acquire(timeout: float) T
        +release(obj: T) None
        +discard(obj: T) None
        +lease(timeout: float) ContextManager~T~
        +close() None
    }
    class ConnectionPool {
        +ConnectionPool(factory: ConnectionFactory, max_size: int, timeout: float)
    }
    class ConnectionFactory {
        +opened: int
        +__call__() Connection
    }
    class Connection {
        +conn_id: str
        +healthy: bool
        +in_transaction: bool
        +closed: bool
        +query(sql: str) str
        +is_healthy() bool
        +reset() None
        +close() None
    }
    class PoolExhaustedError {
        <<exception>>
    }
    ObjectPool <|-- ConnectionPool
    ConnectionPool ..> ConnectionFactory : creates through
    ConnectionPool o-- "0..max_size" Connection : idle or lent
    ObjectPool ..> PoolExhaustedError : raises after the timeout

ObjectPool knows nothing about connections: it takes a factory and three optional callables. ConnectionPool only passes Connection.is_healthy, Connection.reset and Connection.close as those hooks. lease() is the API callers should see; acquire and release exist for code that cannot use a with block.

Canonical example in Python

The object and its factory come first (code/patterns/object_pool.py, tested by code/patterns/tests/test_object_pool.py); the connection is a stand-in that counts queries, remembers an open transaction and can be marked unhealthy.

code/patterns/object_pool.py — the expensive object and its factory
@dataclass(slots=True)
class Connection:
    """Stands in for a real connection: slow to open, cheap to reuse, able to go stale."""

    conn_id: str
    healthy: bool = True
    queries: int = 0
    in_transaction: bool = False
    closed: bool = False

    def query(self, sql: str) -> str:
        if self.closed or not self.healthy:
            raise InvalidStateError(f"{self.conn_id} is not usable")
        self.queries += 1
        self.in_transaction = True
        return f"{self.conn_id} ran {sql!r}"

    def is_healthy(self) -> bool:
        return self.healthy and not self.closed

    def reset(self) -> None:
        """Rollback on release, so the next borrower starts from a clean state."""
        self.in_transaction = False

    def close(self) -> None:
        self.closed = True


class ConnectionFactory:
    """Opening a connection is the expensive step the pool exists to amortise.

    ``_lock`` protects ``_opened``, the count that lets tests and the demo prove reuse.
    """

    def __init__(self, ids: IdGenerator | None = None) -> None:
        self._ids = ids or SequentialIdGenerator("conn")
        self._opened = 0
        self._lock = threading.Lock()

    @property
    def opened(self) -> int:
        with self._lock:
            return self._opened

    def __call__(self) -> Connection:
        with self._lock:
            self._opened += 1
        return Connection(self._ids.next_id())

The pool's public surface:

code/patterns/object_pool.py — the pool: acquire, release, lease, close
class ObjectPool[T]:
    """A bounded, thread-safe pool of interchangeable objects.

    Locking: the ``queue.Queue`` of idle objects is its own lock and supplies the
    blocking wait; ``_lock`` protects ``_created``, ``_lent`` and ``_closed``.
    Objects are created lazily, so a pool with ``max_size`` 8 that only ever
    needs two opens two. The three hooks are optional: ``validate`` runs on
    acquire (a stale object is destroyed and replaced), ``reset`` on release,
    ``destroy`` whenever the pool lets go of an object. Every wait is bounded:
    a pool that can block forever turns one leaked object into a hung service.
    """

    def __init__(
        self,
        factory: Callable[[], T],
        max_size: int,
        *,
        timeout: float = DEFAULT_TIMEOUT_SECONDS,
        validate: Callable[[T], bool] | None = None,
        reset: Callable[[T], None] | None = None,
        destroy: Callable[[T], None] | None = None,
    ) -> None:
        if max_size < 1:
            raise ValidationError("max_size must be at least 1")
        if timeout <= 0:
            raise ValidationError("timeout must be positive")
        self._factory = factory
        self._max_size = max_size
        self._timeout = timeout
        self._validate = validate
        self._reset = reset
        self._destroy = destroy
        self._idle: queue.Queue[T] = queue.Queue(maxsize=max_size)
        self._lock = threading.Lock()
        self._created = 0
        self._lent: dict[int, T] = {}  # id -> object; holding it keeps the id unique
        self._closed = False

    @property
    def max_size(self) -> int:
        return self._max_size

    @property
    def size(self) -> int:
        """Objects alive, idle or lent."""
        with self._lock:
            return self._created

    @property
    def in_use(self) -> int:
        with self._lock:
            return len(self._lent)

    @property
    def available(self) -> int:
        return self._idle.qsize()

    def acquire(self, timeout: float | None = None) -> T:
        """Take an idle object, create one if the cap allows, else wait (bounded) for a release."""
        with self._lock:
            if self._closed:
                raise InvalidStateError("pool is closed")
        obj = self._take_idle()
        if obj is None:
            obj = self._create_if_room()
        if obj is None:
            obj = self._wait_for_object(self._timeout if timeout is None else timeout)
        if self._validate is not None and not self._validate(obj):
            obj = self._replace(obj)
        with self._lock:
            self._lent[id(obj)] = obj
        return obj

    def release(self, obj: T) -> None:
        """Give an object back; refuses objects this pool did not lend (double release)."""
        with self._lock:
            if self._lent.pop(id(obj), None) is None:
                raise ValidationError("object was not acquired from this pool, or was already released")
        if self._reset is not None:
            try:
                self._reset(obj)
            except Exception:
                self._dispose(obj)
                raise
        with self._lock:
            closed = self._closed
            if not closed:
                self._idle.put_nowait(obj)  # never full: at most max_size objects exist
        if closed:
            self._dispose(obj)

    def discard(self, obj: T) -> None:
        """For a borrower who knows the object is broken: drop it and free its slot."""
        with self._lock:
            if self._lent.pop(id(obj), None) is None:
                raise ValidationError("object was not acquired from this pool")
        self._dispose(obj)

    @contextmanager
    def lease(self, timeout: float | None = None) -> Iterator[T]:
        """``with pool.lease() as conn:`` releases on every exit path, exceptions included."""
        obj = self.acquire(timeout)
        try:
            yield obj
        finally:
            self.release(obj)

    def close(self) -> None:
        """Destroy idle objects now and lent ones as they come back; new borrowers are refused."""
        with self._lock:
            self._closed = True
            idle: list[T] = []
            while True:
                try:
                    idle.append(self._idle.get_nowait())
                except queue.Empty:
                    break
        for obj in idle:
            self._dispose(obj)

Four decisions to say out loud:

  • queue.Queue is the pool. A thread-safe FIFO with a condition variable inside, so the blocking wait and the wake-up on release come for free; your own lock guards only _created, _lent and _closed.
  • Every wait is bounded. acquire uses the pool-wide timeout unless the caller passes one, then raises PoolExhaustedError: a leaked object becomes an error you can alarm on, not a hung service. A release wakes a waiter through the queue; the waiter also re-checks the cap each slice of the deadline, because discard frees a slot without queueing an object.
  • Health check on the way out, reset on the way in. A connection dropped while idle fails validate on the next acquire and is replaced in the same slot; reset rolls back what the last borrower left open. Both are injected callables, so the pool never imports a driver.
  • A double release is refused. _lent maps id(obj) to the object itself, so the id cannot be recycled while it is out, and a second release raises instead of queueing one connection for two borrowers.

The slot accounting is where the thread-safety lives, and ConnectionPool binds the hooks:

code/patterns/object_pool.py — slot accounting, and the ConnectionPool that binds the hooks
    def _take_idle(self) -> T | None:
        try:
            return self._idle.get_nowait()
        except queue.Empty:
            return None

    def _wait_for_object(self, wait: float) -> T:
        """Wait up to ``wait`` seconds for an object, then give up with a typed error.

        A ``release`` wakes this immediately through the queue's condition variable.
        The deadline is sliced because ``discard`` and a failed ``reset`` free a *slot*
        without queueing an object: without the re-check a borrower already waiting
        would block for the whole timeout while the pool sat empty.
        """
        deadline = time.monotonic() + wait
        while True:
            remaining = deadline - time.monotonic()
            if remaining <= 0:
                raise PoolExhaustedError(
                    f"all {self._max_size} objects are in use (waited {wait} s)"
                )
            try:
                return self._idle.get(timeout=min(remaining, RECHECK_SECONDS))
            except queue.Empty:
                obj = self._create_if_room()
                if obj is not None:
                    return obj

    def _create_if_room(self) -> T | None:
        """Reserve a slot under the lock, then run the slow factory outside it."""
        with self._lock:
            if self._closed or self._created >= self._max_size:
                return None
            self._created += 1
        try:
            return self._factory()
        except Exception:
            with self._lock:
                self._created -= 1  # the slot goes back, so the next borrower may try again
            raise

    def _replace(self, stale: T) -> T:
        """Destroy a stale object and build a fresh one in the same slot."""
        if self._destroy is not None:
            self._destroy(stale)
        try:
            return self._factory()
        except Exception:
            with self._lock:
                self._created -= 1
            raise

    def _dispose(self, obj: T) -> None:
        if self._destroy is not None:
            self._destroy(obj)
        with self._lock:
            self._created -= 1


class ConnectionPool(ObjectPool[Connection]):
    """The pool interviews ask for: the generic pool wired to Connection's lifecycle."""

    def __init__(
        self,
        factory: ConnectionFactory,
        max_size: int = 4,
        timeout: float = DEFAULT_TIMEOUT_SECONDS,
    ) -> None:
        super().__init__(
            factory,
            max_size,
            timeout=timeout,
            validate=Connection.is_healthy,
            reset=Connection.reset,
            destroy=Connection.close,
        )

_create_if_room reserves the slot under the lock and calls the factory outside it: two threads cannot both win the last slot, a slow handshake stalls no release, and a factory that raises gives the slot back. It also refuses to build for a closed pool, so _wait_for_object cannot open a connection nobody will take back. close drains the idle queue under the same lock, so a racing release either returns its object first or sees the flag and destroys it.

Running python -m patterns.object_pool prints:

--- 6 sequential borrowers, pool of 3 ---
  borrower 1: conn-1 ran 'SELECT 1'
  borrower 2: conn-1 ran 'SELECT 2'
  borrower 3: conn-1 ran 'SELECT 3'
  borrower 4: conn-1 ran 'SELECT 4'
  borrower 5: conn-1 ran 'SELECT 5'
  borrower 6: conn-1 ran 'SELECT 6'
  connections opened: 1 (one is enough when nobody overlaps)
--- 3 borrowers inside the pool at once, then 8 threads x 25 leases ---
  held at the same time: ['conn-1', 'conn-2', 'conn-3']; opened: 3
  leases: 200, opened: 3 (the cap held), in use afterwards: 0
--- a stale idle connection is replaced on acquire (pool of 1) ---
  db-1 failed the health check; got db-2, closed old: True
  pool size still 1, opened 2
--- bounded wait: both connections lent out, a third borrower gives up after 50 ms ---
  PoolExhaustedError: all 2 objects are in use (waited 0.05 s)
  after one release: got c-1 immediately
--- a double release is refused ---
  ValidationError: object was not acquired from this pool, or was already released
--- close(): idle connections are closed, new borrowers are refused ---
  InvalidStateError: pool is closed; size now 0

Pythonic variant

When objects never go stale and the count is known up front, the queue alone is the pool:

code/patterns/object_pool.py — a prefilled queue and a borrow context manager
def prefilled_pool(factory: Callable[[], Connection], size: int) -> queue.Queue[Connection]:
    """Eager and fixed-size: the queue is the pool, and a blocking ``get`` is the wait."""
    idle: queue.Queue[Connection] = queue.Queue(maxsize=size)
    for _ in range(size):
        idle.put(factory())
    return idle


@contextmanager
def borrow(idle: queue.Queue[Connection], timeout: float | None = None) -> Iterator[Connection]:
    conn = idle.get(timeout=timeout)  # blocks until a peer puts one back
    try:
        yield conn
    finally:
        idle.put(conn)
  • Eager, not lazy. prefilled_pool opens every object at start-up: right for a worker that will use them all, wrong for a CLI that may use none.
  • queue.Empty is the whole error handling, and a borrow body that raises still puts the object back.
  • What you give up is validation, reset, metrics and shutdown: a broken connection goes straight back into rotation.
Reach for When
No pool Creation is cheap, or the call is one-off
queue.Queue plus borrow Fixed count, objects never go bad, pool lifetime equals process lifetime
threading.BoundedSemaphore(n) You want the cap but not the objects: at most n concurrent calls to a vendor API
ObjectPool with hooks Lazy creation, health checks, reset between uses, close(), counters to export
A library pool Production: SQLAlchemy, urllib3 and the drivers have met the reconnect edge cases

Draw the class diagram, then say "in Python I would start from a queue.Queue behind a context manager and grow it into a class the day I need a health check or lazy creation under a cap".

Real-world usage

  • concurrent.futures.ThreadPoolExecutor and multiprocessing.Pool pool threads and child processes, the most expensive objects a process owns; submit is acquire with the work attached.
  • urllib3 HTTPConnectionPool (under requests.Session) keeps keep-alive sockets per host: maxsize is the cap, block=True the bounded wait.
  • SQLAlchemy QueuePool is this module with production edges: max_overflow for bursts, pool_timeout (30 s by default), pool_pre_ping as the health check, pool_recycle to retire old connections, rollback on return as the reset.
  • Database-side poolers (PgBouncer, RDS Proxy) move the pool out of the process: 50 app servers x 20 connections need not become 1,000 server-side connections.
Looks like Object Pool How to tell them apart
Singleton One shared instance versus N interchangeable ones lent exclusively. The pool is usually one per process: build it in the composition root and inject it rather than reach for ConnectionPool.instance().
Flyweight Immutable and shared by everyone at once; a pooled object is mutable and owned by one borrower at a time, hence release and reset.
Cache Keyed, may miss, evicts under pressure. A pool has no key, waits instead of missing, and destroys objects only when they break or at close().
Producer-consumer Same queue.Queue, opposite direction: a consumed item is gone, a borrowed one comes back.

Where it appears in LLD problems

  • Design a car rental system — a fleet is a pool per category: a reservation is acquire over a date range, a return is release, the inspection on return is the health check.
  • Design a bowling alley — lanes are a small fixed pool; a waiting group is a bounded acquire, a lane under maintenance is a discard.
  • Design a parking lot — spots per size are a pool in spirit, except that an allocation strategy picks which idle spot.

Interview tips

Interview tip

Lead with the three numbers the pool manages: max_size (the cap), timeout (the bounded wait) and what "idle" means (a health check). Name the two hooks, validate on acquire and reset on release, and say that with pool.lease() as conn: is the only API application code sees.

Common mistake

An unbounded wait. queue.get() with no timeout looks safe until one code path forgets to release; then the pool drains, every thread blocks forever and the service is up but does nothing. Give the pool a default timeout, raise a typed error, release in a finally. Runner-up: returning a broken object without reset or discard, so the next borrower inherits the fault.