Question 1

A function def add_transaction(item, ledger=[]) produces unexpected cross-request data leaks where user transactions from previous invocations persist in subsequent function calls. Explain how Python evaluates default arguments at function definition time and provide the idiomatic None sentinel pattern fix.

Accepted Answer

Python evaluates default argument expressions exactly once, when the `def` statement runs, not on every call. So `ledger=[]` creates one list object and binds it to the function's `__defaults__` tuple; every call that doesn't pass its own `ledger` reuses that same list object. Since lists are mutable, any append inside the function sticks around for the next call that also relies on the default.

You can see it directly: `add_transaction('a'); add_transaction('b')` — the second call's ledger already contains `'a'` even though it looks like a fresh call.

The fix is the None-sentinel pattern: default to `None`, and create the mutable object fresh inside the function body.

```python
def add_transaction(item, ledger=None):
    if ledger is None:
        ledger = []
    ledger.append(item)
    return ledger
```

This applies to any mutable default — lists, dicts, sets, or custom mutable objects — and is one of the most common real production bugs in Python because it doesn't error, it just silently accumulates state.

Question 2

A list of lambdas created inside a loop funcs = [lambda x: x * i for i in range(5)] returns 4 * x for all generated functions when evaluated later. Explain Python's late-binding lexical scoping behavior and how to bind the loop variable immediately using default argument evaluation.

Accepted Answer

Closures in Python capture variables by reference to the enclosing scope, not by their value at the time the closure was created. `i` here isn't a fresh variable per loop iteration — it's the same loop variable in the enclosing function/module scope, and each lambda just holds a reference to it. By the time any lambda actually gets called, the loop has finished and `i` is sitting at its final value, `4`, so every lambda sees `4`.

The fix is to force early binding by capturing the current value as a default argument, since default arguments ARE evaluated immediately at function-definition time (the same mechanism that causes the mutable-default-argument bug above, used here on purpose).

```python
funcs = [lambda x, i=i: x * i for i in range(5)]
funcs[0](10)  # 0
funcs[3](10)  # 30
```

Each lambda now has its own `i` parameter defaulted to the value `i` held at that specific point in the loop. The same problem and fix apply to closures defined with `def` inside a loop, not just lambdas.

Question 3

A data processing script parallelizes image matrix convolutions using Python's threading module, but execution time increases by 40% compared to single-threaded execution. Explain how Global Interpreter Lock (GIL) lock-contention degrades CPU-bound tasks and how to redesign the architecture using multiprocessing with shared memory buffers.

Accepted Answer

CPython's GIL allows only one thread to execute Python bytecode at a time, regardless of how many CPU cores are available. For CPU-bound work like matrix convolutions, threads spend almost all their time holding the GIL doing computation, so they never actually run in parallel — they just take turns. Worse, the interpreter periodically forces a thread to release the GIL (every ~5ms by default, via the "switch interval") so other threads get a chance, which adds context-switching and cache-invalidation overhead on top of getting zero parallelism benefit. That's exactly why you see it get slower, not faster.

Threading only helps here if the work is I/O-bound (network calls, disk reads) where threads release the GIL while blocked. For CPU-bound work, you need `multiprocessing`, which sidesteps the GIL entirely by running separate OS processes, each with its own interpreter and its own GIL.

The catch with plain multiprocessing is that passing large arrays between processes means pickling and copying them through IPC, which is expensive. The fix is shared memory:

```python
from multiprocessing import shared_memory, Process
import numpy as np

shm = shared_memory.SharedMemory(create=True, size=arr.nbytes)
shared_arr = np.ndarray(arr.shape, dtype=arr.dtype, buffer=shm.buf)
shared_arr[:] = arr[:]
# worker processes attach to shm.name and read/write the same buffer directly
```

Each worker process attaches to the same underlying memory block by name instead of receiving a serialized copy, so large convolution buffers are shared, not duplicated — giving you real multi-core parallelism without the IPC copy cost.

Question 4

You are building a pipeline processing multi-gigabyte log streams. How do you construct a pipeline of generator functions using yield from that supports bidirectional communication via .send(), and how do you handle exception propagation with .throw() and graceful generator teardown?

Accepted Answer

`yield from` delegates a generator's iteration, sending, and exception handling to a sub-generator, so you can chain generator stages without manually forwarding every value and every `.send()`/`.throw()`/`.close()` call yourself. This is what lets you build a multi-stage pipeline (parse → filter → aggregate) where each stage is a generator and the outer generator just does `yield from next_stage`.

```python
def parser(lines):
    for line in lines:
        received = yield parse(line)  # value sent back in via .send()
        if received == 'skip':
            continue

def pipeline(lines):
    yield from parser(lines)
```

With `.send(value)`, a caller can push data back into a paused generator — the `yield` expression evaluates to whatever was sent, letting a downstream stage adjust upstream behavior (e.g., backpressure or filtering hints).

For exceptions, `.throw(exc)` raises `exc` at the point the generator is currently suspended, and with `yield from` that exception is automatically propagated down into the active sub-generator first, giving it a chance to handle it via try/except around its `yield`, before bubbling up. For teardown, `.close()` raises `GeneratorExit` inside the generator; a well-behaved generator catches `GeneratorExit` only to release resources (close a file handle, flush a buffer) and then re-raises or simply returns — it must not `yield` again after that, or Python raises `RuntimeError: generator ignored GeneratorExit`. For gigabyte log streams, wrapping the pipeline in `try/finally` at each stage guarantees file handles and buffers are released even if the consumer stops early.

Question 5

An internal framework requires all child classes to validate that specific class attributes are defined with valid type annotations upon class definition. Compare implementing this via a custom metaclass (__new__/__init__) versus using Python 3.6+ __init_subclass__ in terms of maintainability and inheritance conflicts.

Accepted Answer

A metaclass intercepts the class-creation process itself: `type.__new__`/`__init__` run before the class object even exists, giving you the namespace dict to inspect/mutate before the class is built. That's powerful — you can rewrite attributes, inject methods, or reject the class outright — but it comes with a real cost: every class that needs this validation must have your metaclass as its metaclass (directly or via inheritance), and Python only allows one metaclass per class hierarchy. If a child class also needs to inherit from something with a different, incompatible metaclass (e.g. an ABC or a Pydantic/SQLAlchemy base), you get `TypeError: metaclass conflict`.

```python
class ValidatingMeta(type):
    def __new__(mcls, name, bases, ns):
        cls = super().__new__(mcls, name, bases, ns)
        for attr, ann in cls.__annotations__.items():
            if ann not in (int, str, float):
                raise TypeError(f"{attr} has unsupported type {ann}")
        return cls
```

`__init_subclass__`, added in 3.6, solves the common case far more simply: it's a regular classmethod hook that Python automatically calls on every direct or indirect subclass, with no metaclass required.

```python
class Base:
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        for attr, ann in cls.__dict__.get('__annotations__', {}).items():
            if ann not in (int, str, float):
                raise TypeError(f"{attr} has unsupported type {ann}")
```

Because it's just a regular classmethod, it composes cleanly with multiple inheritance and other base classes' own metaclasses — there's no conflict to resolve. For maintainability, `__init_subclass__` should be your default: reach for a metaclass only when you need to control the class *before* it exists (e.g., modifying `bases` or the namespace itself), which validation-on-subclassing doesn't need.

Question 6

A doubly linked list node implementation in Python causes memory to grow continuously when nodes are unlinked but retain parent/child references. Explain how reference counting immediately reclaims acyclic objects, why circular references require the generational cyclic garbage collector (gc module), and what causes collection latency.

Accepted Answer

CPython's primary memory management is reference counting: every object carries a count of how many references point to it, incremented on assignment and decremented when a reference goes out of scope or is reassigned. The instant that count hits zero, the object is deallocated immediately — no waiting, no background sweep. This is why most Python objects (a string, a plain list with no self-references) are freed the moment they become unreachable.

A doubly linked list is the classic counter-example: node A holds `next` pointing to node B, and B holds `prev` pointing back to A. Even after you remove both from the list and drop your own external reference, A and B still reference each other — their refcounts never reach zero, so pure reference counting can never free them.

This is exactly what the generational cyclic garbage collector (the `gc` module) exists for. It periodically scans container objects (things that can hold references to other objects — lists, dicts, class instances, but not strings or ints) looking for groups that are only reachable from within the group itself, and frees those cycles even though their individual refcounts are nonzero.

Collection latency comes from it being generational and threshold-triggered, not continuous: objects are grouped into 3 generations (gen0, gen1, gen2), and a collection of a generation only runs after enough allocations have accumulated (`gc.get_threshold()`, default `(700, 10, 10)`). So a cyclic node pair can sit in memory for a while after becoming unreachable, until enough other allocations happen to trigger a gen0 (and eventually gen2) sweep — you can force it early with `gc.collect()`, or break the cycle manually by nulling `prev`/`next` on unlink so refcounting reclaims it immediately instead of waiting on the collector.

Question 7

Instantiating 5 million coordinate point objects causes an application to exceed a 2GB RAM budget due to per-instance __dict__ overhead. How does defining __slots__ optimize memory allocation, and what are its behavioral implications regarding dynamic attribute assignment and class inheritance?

Accepted Answer

By default, every instance of a Python class carries its own `__dict__` to store instance attributes — a full hash table, which has significant fixed overhead (roughly 100+ bytes minimum) completely independent of how few attributes the object actually holds. For a `Point(x, y)` class with only two floats, that dict overhead dwarfs the actual data, and at 5 million instances it adds up to gigabytes of pure bookkeeping.

`__slots__` tells Python to skip creating a per-instance `__dict__` entirely and instead allocate a fixed-size, dict-free structure (similar to a C struct) with slots only for the named attributes:

```python
class Point:
    __slots__ = ('x', 'y')
    def __init__(self, x, y):
        self.x = x
        self.y = y
```

This typically cuts per-instance memory by more than half for small objects, since you drop both the dict overhead and the dict's own load-factor overallocation.

The trade-offs: instances can no longer accept arbitrary new attributes at runtime (`p.z = 5` raises `AttributeError`) unless `'__dict__'` is explicitly added to `__slots__`, which defeats the purpose. And with inheritance, if a subclass doesn't also define `__slots__`, it silently gets a `__dict__` anyway, negating the savings for that subclass's instances — every class in the hierarchy that should be memory-optimized needs its own `__slots__` declaration. Multiple inheritance from two classes that each define non-empty, non-identical `__slots__` also raises `TypeError: multiple bases have instance lay-out conflict`, since CPython can't merge two incompatible fixed memory layouts.

Question 8

A Flask web service leaks memory on every error response because caught exceptions are stored in a module-level dictionary alongside sys.exc_info(). Walk through how storing exception tracebacks creates reference cycles involving stack frames, local variables, and closures, preventing timely garbage collection.

Accepted Answer

A traceback object holds a chain of frame objects, one per stack level the exception passed through. Each frame object, in turn, holds a reference to all of its local variables at the time of the exception — including, very often, the exception object itself (e.g. inside an `except Exception as e:` block, `e` is a local variable of that frame). So the chain becomes: traceback → frame → locals dict → exception object → traceback (via `exception.__traceback__`). That's a genuine reference cycle.

Normally this isn't a big deal — the cyclic GC eventually reclaims it. The problem in this scenario is that the code stores `sys.exc_info()` (which includes the traceback) into a **module-level, long-lived dictionary**. A module-level dict lives for the entire process lifetime, so anything referenced from it — including this entire cycle, and everything each frame's locals were holding (request objects, DB connections, large buffers pulled in via closures) — is kept artificially alive far longer than the collector's normal scheduling would ever leave it, since it's still technically reachable, just circularly.

The fix is to never keep a raw traceback (or full `exc_info()`) around past the point you actually need it. Extract only what you need immediately: `traceback.format_exc()` gives you the string representation without holding live frame references, or `repr(exception)` for the message. If you must keep structured data, explicitly break the cycle with `exc_info[2] = None` / delete the traceback reference, or wrap the stored reference in a `weakref` so it doesn't keep the frames' locals pinned in memory.

Question 9

After loading and then deleting a 10GB dataset in Python using del data followed by gc.collect(), the OS system monitor still reports the process consuming several gigabytes of RSS memory. Explain how the pymalloc small-object allocator organizes memory into arenas/pools and why memory is not returned to the OS.

Accepted Answer

CPython's `pymalloc` allocator handles all objects smaller than 512 bytes (most Python objects — small ints, short strings, list/dict internal nodes) using its own layered allocator rather than calling `malloc`/`free` directly for each one. Memory is requested from the OS in large chunks called **arenas** (256KB each in modern CPython), each arena is subdivided into **pools** (4KB each), and each pool is subdivided into fixed-size **blocks** matching one of several small-object size classes.

When you `del data` and even force `gc.collect()`, Python does free the individual blocks those objects occupied — but a pool is only returned to its arena's free list once *every* block in that pool is empty, and an arena is only released back to the OS once *every* pool in it is empty. If your 10GB dataset's objects were scattered across many arenas, and even a handful of unrelated small objects (interned strings, cached small ints, leftover references from other parts of the app) still occupy even one block in each arena, none of those arenas can be fully freed — the RSS stays high even though the logical Python-level memory was released.

This is fragmentation, not a leak: the memory is available for Python to reuse for new small-object allocations, it's just not given back to the OS. For very large arrays or datasets, this is one reason to prefer objects that bypass pymalloc entirely — NumPy arrays and other buffer-protocol objects allocate directly via `malloc`/`free` (or their own allocator) in large contiguous blocks, which the OS can reclaim as soon as the array is freed. If RSS reduction genuinely matters (long-running services), the practical mitigations are: process recycling after big batch jobs, using `numpy`/`array.array` for bulk numeric data instead of Python lists of objects, or explicitly calling platform-specific `malloc_trim` (not exposed cleanly from pure Python) — there's no supported pure-Python API to force pymalloc to return partially-used arenas to the OS.

Question 10

Design an in-memory cache for expensive image rendering objects that reclaims cached values automatically when no other parts of the application hold references to them, using weakref.WeakValueDictionary and weakref.finalize without causing broken reference errors.

Accepted Answer

`weakref.WeakValueDictionary` stores values via weak references instead of strong ones, so an entry doesn't keep its value alive — the moment nothing *else* in the application holds a strong reference to that rendered image, its refcount drops to zero, it's deallocated, and the dictionary automatically removes the now-dead entry on its own.

```python
import weakref

_cache = weakref.WeakValueDictionary()

def get_rendered_image(key, render_fn):
    img = _cache.get(key)
    if img is None:
        img = render_fn(key)
        _cache[key] = img
    return img
```

The classic broken-reference pitfall: if `render_fn` returns something with no other strong reference held anywhere (e.g. the caller doesn't keep the returned `img` around), it can be garbage collected between the `_cache[key] = img` line and the caller actually using it, or the entry can vanish between a `key in _cache` check and a subsequent `_cache[key]` access (a TOCTOU race). The fix is to always retrieve via `.get()` (returns `None` cleanly instead of raising) rather than checking membership then indexing separately, and to make sure the caller holds its own strong reference to the object for as long as it needs it — the cache should never be the *only* thing keeping an object alive if you still expect to use it after the lookup.

`weakref.finalize(obj, callback, *args)` complements this by registering a callback that fires exactly once, right when `obj` is about to be garbage collected — useful here for releasing an associated GPU texture handle or temp file the rendered image was backed by:

```python
weakref.finalize(img, release_gpu_texture, img.texture_id)
```

Unlike `__del__`, `finalize` callbacks don't participate in reference cycles and are guaranteed to run even during interpreter shutdown, which makes them the safer choice for cleanup tied to cache eviction.

Question 11

A complex multi-inheritance hierarchy raises TypeError: Cannot create a consistent method resolution order (MRO). Walk through how the C3 Linearization algorithm computes method resolution order in Python and how to refactor multiple inheritance classes to maintain cooperative super() dispatch.

Accepted Answer

C3 linearization computes a class's MRO by merging the MROs of its parent classes together with the list of parents itself, following two rules: a class always appears before its own parents (local precedence), and the relative order of parents as declared in the class statement is preserved across the whole hierarchy (monotonicity). Concretely, `L[C] = C + merge(L[B1], L[B2], ..., [B1, B2, ...])`, where `merge` repeatedly takes the head of the first list that doesn't appear in the tail of any other list, and removes it from all lists.

The `TypeError` happens when no valid ordering can satisfy both rules simultaneously — typically because two classes declare their shared parents in conflicting orders. For example:

```python
class A: pass
class B: pass
class X(A, B): pass
class Y(B, A): pass
class Z(X, Y): pass  # TypeError: cannot create a consistent MRO
```

`X` says A should precede B; `Y` says B should precede A. `Z` inherits from both, and C3 can't satisfy both orderings at once — there's no linear order consistent with both declared preferences.

The fix isn't a language trick, it's a design fix: the base classes need to agree on a single relative ordering everywhere they're used together. In practice this means standardizing your mixin ordering convention project-wide (e.g., always most-specific-to-most-general, left to right) so every class that combines the same mixins declares them in the same relative order. For cooperative multiple inheritance to actually work, every class in the chain — including mixins — needs `super().__init__(*args, **kwargs)` calls (not calling the parent class directly by name), so control flows through the *computed* MRO chain rather than a hardcoded parent, letting each class in the diamond contribute its behavior exactly once.

Question 12

Implement a reusable data validation descriptor that validates whether assigned class fields are positive integers. Explain the mechanics of __get__, __set__, and __set_name__, and explain why storing values directly on descriptor instances causes cross-instance data collisions.

Accepted Answer

A descriptor is any object defining `__get__`/`__set__` (or `__delete__`) that's assigned as a *class* attribute; Python's attribute lookup machinery detects this and routes `instance.attr` access through the descriptor's methods instead of a plain dict lookup. `__set_name__(self, owner, name)` is called automatically once, at class-creation time, telling the descriptor what attribute name it was assigned to on the owning class — useful for knowing where to actually store the per-instance value.

```python
class PositiveInt:
    def __set_name__(self, owner, name):
        self._name = '_' + name

def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return getattr(obj, self._name)

def __set__(self, obj, value):
        if not isinstance(value, int) or value <= 0:
            raise ValueError(f"{self._name[1:]} must be a positive integer")
        setattr(obj, self._name, value)

class Order:
    quantity = PositiveInt()
```

The critical bug to avoid: the descriptor object itself (`PositiveInt()`) is created **once**, at class-body execution time, and shared as a single class attribute across every instance of `Order`. If `__set__` stored the value as `self.value = value` (on the descriptor instance) instead of on the *owner instance* (`obj`), every `Order` instance would be reading and writing the exact same shared value — setting `order1.quantity = 5` would make `order2.quantity` also read `5`, because there's only one descriptor object backing the attribute for the whole class. That's why `__set__` must store into `obj.__dict__` (via `setattr(obj, self._name, value)` here) — keyed per-instance — rather than onto `self` (the descriptor).

Question 13

You need to build a custom database transaction context manager class. How do you implement __enter__ and __exit__ to handle auto-commit on clean completion, automatic rollback on exception, and intentional suppression of specific expected database errors by returning a boolean flag?

Accepted Answer

`__enter__(self)` runs when the `with` block is entered and its return value is what gets bound to the `as` variable — typically the transaction/connection object itself. `__exit__(self, exc_type, exc_value, traceback)` runs unconditionally when the block exits, whether it exited normally or via an exception; the three arguments are all `None` on a clean exit, and populated with the exception details otherwise.

```python
class Transaction:
    def __init__(self, conn):
        self.conn = conn

def __enter__(self):
        self.conn.begin()
        return self.conn

def __exit__(self, exc_type, exc_value, tb):
        if exc_type is None:
            self.conn.commit()
            return False
        self.conn.rollback()
        if issubclass(exc_type, KnownRecoverableError):
            return True  # suppress: exception is swallowed, `with` block exits normally
        return False  # re-raise: propagate everything else
```

The key mechanic is `__exit__`'s return value: returning a truthy value tells Python to **suppress** the exception — it won't propagate past the `with` block, as if it never happened — while returning `False` (or `None`, the implicit default) lets it propagate normally after rollback still runs. You only want to suppress errors you've specifically identified as expected and safely handled (e.g. a known, already-retried transient DB error) — suppressing broadly (like `return True` unconditionally) is dangerous because it silently swallows real bugs. Rollback should always run on any exception, suppressed or not, since the whole point is leaving the DB in a consistent state regardless of what happens to the exception afterward.

Question 14

You want a custom data structure to pass isinstance(obj, collections.abc.Mapping) checks without inheriting from Mapping directly. How do you use Python's abc.ABCMeta.register or implement the __subclasshook__ classmethod to establish virtual subclass relationships?

Accepted Answer

Normally `isinstance`/`issubclass` check the actual inheritance chain (real, nominal subtyping). ABCs built on `ABCMeta` support a second mechanism — **virtual subclasses** — where a class can be registered as "counting as" a subclass of an ABC without ever inheriting from it, so `isinstance` returns `True` while `MRO`/`super()` behavior stays completely untouched.

The simplest approach is `register()`:

```python
from collections.abc import Mapping

class MyCustomDict:
    def __getitem__(self, key): ...
    def __iter__(self): ...
    def __len__(self): ...

Mapping.register(MyCustomDict)
isinstance(MyCustomDict(), Mapping)  # True
```

This is a manual, one-time opt-in — you (or a third party) explicitly declare the relationship, and it doesn't verify the class actually implements the right interface; it's just a promise.

`__subclasshook__` goes further and makes the check *structural/automatic* — it's a classmethod on the ABC that Python calls during every `issubclass()`/`isinstance()` check against that ABC, letting you inspect the candidate class and decide programmatically whether it qualifies (this is exactly how `collections.abc.Hashable`, `Iterable`, etc. work internally — pure duck-typing checks):

```python
class MyMapping(ABC):
    @classmethod
    def __subclasshook__(cls, C):
        if cls is MyMapping:
            required = ('__getitem__', '__iter__', '__len__')
            if all(any(m in B.__dict__ for B in C.__mro__) for m in required):
                return True
        return NotImplemented
```

Use `register()` for a quick, explicit one-off relationship; use `__subclasshook__` when you want *any* class satisfying a structural contract to automatically pass the check without each one having to opt in individually — the trade-off is `__subclasshook__` runs on every check, so it should stay cheap (attribute presence checks, not deep validation).

Question 15

You are writing an authentication decorator for API endpoints. How do you use functools.wraps to preserve the original function's name, docstring, type annotations, and introspection signatures, and how do you inspect endpoint parameters using the inspect module?

Accepted Answer

A plain decorator replaces the original function object with the wrapper function it returns, so without help, things like `func.__name__`, `func.__doc__`, and `func.__module__` all end up pointing to the generic wrapper (`"wrapper"`, `None`, etc.) instead of the real endpoint — breaking API docs generators, debuggers, and anything that introspects the function.

```python
import functools, inspect

def require_auth(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        if not is_authenticated():
            raise PermissionError("Unauthorized")
        return func(*args, **kwargs)
    return wrapper

@require_auth
def get_user_profile(user_id: int) -> dict:
    """Fetch a user's profile."""
    ...
```

`functools.wraps(func)` copies over `__name__`, `__doc__`, `__module__`, `__dict__`, and `__annotations__` from the original function onto the wrapper, and additionally sets `wrapper.__wrapped__ = func` — a pointer back to the original. That last part matters: `inspect.signature(get_user_profile)` (and tools like FastAPI/Flask route introspection that rely on it) follows `__wrapped__` automatically, so it reports `(user_id: int) -> dict` — the *original* signature — rather than the wrapper's generic `(*args, **kwargs)`.

`inspect.signature()` and `inspect.getfullargspec()` are what frameworks use under the hood to validate request payloads against endpoint parameters, generate OpenAPI docs, or do dependency injection by parameter name — none of that works correctly if `functools.wraps` is missing, since without `__wrapped__` those tools see only the decorator's opaque `*args, **kwargs` signature instead of the real one.

Question 16

A machine learning data ingestion script fails intermittently after running for 3 hours on an unhandled exception. How do you configure sys.excepthook or use pdb.post_mortem() to automatically drop into an interactive debug shell with preserved local variable frames at the exact point of failure?

Accepted Answer

Normally, an unhandled exception just prints a traceback and the process exits — by the time you see the error, the stack (and every local variable at each frame) is already gone, which is brutal for a 3-hour job that fails intermittently and is expensive to reproduce.

The simplest fix is running the whole script under `python -m pdb -c continue script.py`: pdb catches the uncaught exception and drops you into an interactive shell sitting exactly at the frame where it died, with every local variable still alive and inspectable (`p variable_name`, `up`/`down` to walk the stack).

For unattended runs (or when you can't easily re-invoke the whole process under pdb), override `sys.excepthook`, which Python calls with `(type, value, traceback)` whenever an exception escapes all the way to the top uncaught:

```python
import sys, pdb, traceback

def debug_hook(type_, value, tb):
    traceback.print_exception(type_, value, tb)
    pdb.post_mortem(tb)

sys.excepthook = debug_hook
```

`pdb.post_mortem(tb)` takes an existing traceback object and starts an interactive debugging session at the deepest frame of that traceback — without needing to re-raise or re-run anything — giving you the exact same locals-inspection capability the exception had at the moment it was raised. For a long-running ingestion job where the failure is intermittent and expensive to reproduce, this is far more useful than a static stack trace, since you can actually poke at the malformed record or corrupted state that triggered the crash, live, instead of guessing from the printed traceback alone.

Question 17

A background worker process leaks memory under sustained load. Walk through how you use the standard library tracemalloc module to take baseline and peak snapshots, filter memory traces by domain, and diff top allocations to locate the specific line of Python source allocating leaked blocks.

Accepted Answer

`tracemalloc` is a standard-library memory profiler that records, per allocation, the Python traceback (file + line number) responsible for it — so instead of just knowing memory grew, you can find exactly which line is allocating the objects that never get freed.

```python
import tracemalloc

tracemalloc.start()
baseline = tracemalloc.take_snapshot()

# ... run the worker under sustained load for a while ...

peak = tracemalloc.take_snapshot()
diff = peak.compare_to(baseline, 'lineno')
for stat in diff[:10]:
    print(stat)
```

Taking a snapshot before load and one after lets `compare_to()` compute the delta per allocation site, sorted by size or count depending on the grouping key (`'lineno'` groups by exact file+line, `'traceback'` gives the full call stack, `'filename'` is coarser). The top entries in that diff are your leak suspects — lines whose allocated-and-still-live byte count grew the most between the two snapshots, which is exactly what a steady leak looks like (one-off allocations show up in both snapshots and cancel out in the diff; leaking ones keep growing).

For domain filtering — separating your own code's allocations from library/interpreter internals — `tracemalloc.take_snapshot().filter_traces(filters)` accepts `Filter` objects that include/exclude by filename pattern, so you can restrict the diff to just your application's source tree and cut out noise from third-party dependencies or the standard library itself. Once you've narrowed to a specific line, the fix is usually one of the classic culprits: an ever-growing module-level cache/dict with no eviction, a registered callback/listener that's never unregistered, or a reference cycle involving `__del__` that the collector can't safely break automatically.

Question 18

An internal batch-processing script is taking too long to execute. How do you profile the script using cProfile, export stats to pstats, and distinguish between cumulative time spent inside a function versus high per-call overhead across millions of small helper function invocations?

Accepted Answer

`cProfile` is Python's built-in deterministic profiler — it instruments every function call/return and records exact timing and call counts, unlike sampling profilers which estimate. You can run it from the command line or wrap it around code directly:

```python
import cProfile, pstats

profiler = cProfile.Profile()
profiler.enable()
run_batch_job()
profiler.disable()

stats = pstats.Stats(profiler)
stats.sort_stats('cumulative').print_stats(20)
```

`pstats.Stats` gives you two key columns to distinguish very different problems: **tottime** (total time spent inside that function's own code, excluding calls it made to other functions) and **cumtime** (total time including everything it called into). A function with high `cumtime` but low `tottime` is just a thin wrapper around expensive sub-calls — the real cost is further down the call tree, so sort by `cumulative` to find the top of the slow call chain. A function with high `tottime` and a huge `ncalls` count (millions of invocations of a small helper) tells a different story — the function itself is cheap per-call, but call *overhead* (Python function-call machinery, argument unpacking, attribute lookups) dominates because it's called so many times; here the fix is usually inlining the helper, vectorizing with NumPy, or restructuring the loop to avoid the repeated call entirely rather than micro-optimizing the helper's body.

Sorting by `'tottime'` instead of `'cumulative'` surfaces the second category directly — functions that are individually hot regardless of what they call — which is exactly what you want when millions of small helper calls are suspected as the bottleneck rather than one big expensive operation.

Question 19

An asyncio microservice experiences severe latency spikes where all concurrent tasks freeze for several seconds. How do you enable asyncio debug mode (PYTHONASYNCIODEBUG=1), detect synchronous blocking I/O calls starving the event loop, and offload CPU-bound tasks using loop.run_in_executor?

Accepted Answer

asyncio's concurrency model is cooperative and single-threaded: all tasks share one event loop, and only one coroutine actually runs at a time — concurrency comes from tasks voluntarily yielding control at each `await`. If any single task calls something synchronous and blocking (a plain `requests.get()`, `time.sleep()`, a synchronous DB driver call, or heavy CPU-bound computation with no `await` inside it), it never yields, and the event loop is frozen for that entire duration — every other task, including ones that would otherwise be instantly ready, simply stops making progress. That's exactly the "all tasks freeze for several seconds" symptom: one blocking call is starving the whole loop, not each task independently misbehaving.

Setting `PYTHONASYNCIODEBUG=1` (or `asyncio.get_event_loop().set_debug(True)`) turns on asyncio's built-in diagnostics: it logs a warning whenever a callback or task takes longer than a threshold (default 100ms) to complete, which is exactly the signature of a blocking call hiding inside a coroutine, and it also enables extra checks like flagging coroutines that were never awaited.

```python
import asyncio, functools

loop = asyncio.get_running_loop()
result = await loop.run_in_executor(
    None,  # default ThreadPoolExecutor
    functools.partial(cpu_heavy_or_blocking_call, arg)
)
```

`loop.run_in_executor(None, func)` offloads the blocking call to a background thread pool (or `ProcessPoolExecutor` for CPU-bound work, sidestepping the GIL too), returning an awaitable immediately so the event loop stays free to run other tasks while the blocking work happens off the main thread. The fix, once debug mode has pointed at the offending call, is either replacing it with an async-native equivalent (`aiohttp` instead of `requests`, an async DB driver) or wrapping it in `run_in_executor` if no async version exists.

Question 20

A production Python daemon hangs in an unresponsive state, but attaching GDB provides unhelpful C-level traces. How do you utilize a non-invasive sampling profiler like py-spy to inspect Python thread stacks, detect lock acquisitions, and trace active line numbers without stopping or restarting the running process?

Accepted Answer

GDB shows you the CPython interpreter's own C call stack (`PyEval_EvalFrameEx`, `_PyEval_EvalFrame`, etc.), which is technically accurate but useless for diagnosing application-level logic — you can't tell which Python function or line the process is actually stuck in without manually walking C-level frame pointers.

`py-spy` solves this by reading the target process's memory directly from outside it (using platform APIs like `process_vm_readv` on Linux or equivalent on macOS/Windows) and reconstructing the Python-level call stack from CPython's own frame objects in memory — it never injects code into the process, sends signals, or pauses execution, so it's safe to run against a live production daemon without any risk of making the hang worse or crashing it.

```bash
py-spy dump --pid 12345          # one-shot snapshot of every thread's Python stack
py-spy top --pid 12345           # live, continuously-refreshing view, like `top` for Python calls
```

`py-spy dump` prints each thread's current Python-level stack (file, line number, function name) at that instant — for a hang, you typically run this a few times a couple seconds apart; if the same line shows up every time across all threads, that's your deadlock or infinite-wait point, most often a `threading.Lock.acquire()` or a blocking call on a resource nothing is ever going to release. `py-spy top` is closer to a live CPU profiler view, useful when the daemon isn't fully hung but is pathologically slow, showing which functions are consuming sampled time across threads in real time. Because it's purely a passive reader of process memory, it's one of the few debugging tools safe to reach for on a live production process without needing a restart, a debug build, or code changes to add instrumentation.

Question 21

What is the difference between a list and a tuple in Python, and when would you choose one over the other?

Accepted Answer

Both are ordered sequences that can hold mixed types, but lists are mutable (`append`, `remove`, item assignment all work) while tuples are immutable — once created, you can't change, add, or remove elements. That immutability has real consequences beyond just "can't edit it": tuples are hashable (as long as their contents are), so they can be used as dictionary keys or set members, while plain lists can't.

```python
coords = (10, 20)          # tuple: fixed pair, safe as a dict key
cache = {coords: 'cached_result'}

scores = [90, 85, 70]      # list: expected to grow/shrink
scores.append(60)
```

Use a tuple when the data represents a fixed, small structure that shouldn't change after creation — a coordinate pair, an RGB triple, a function returning multiple values (`return min_val, max_val`). Use a list when the collection's size or contents are expected to change over its lifetime. Tuples are also slightly more memory-efficient and marginally faster to construct than lists of the same size, since Python doesn't need to over-allocate for future growth — but that performance difference is rarely the deciding factor in practice; the mutability/hashability distinction is.

Question 22

What are *args and **kwargs in Python, and how would you use both together in a single function signature?

Accepted Answer

`*args` collects any extra positional arguments into a tuple, and `**kwargs` collects any extra keyword arguments into a dict. The names `args`/`kwargs` are just convention — it's the `*`/`**` unpacking that matters.

```python
def log_call(name, *args, **kwargs):
    print(f"{name} called with args={args}, kwargs={kwargs}")

log_call("fetch_user", 42, retries=3, timeout=5.0)
# fetch_user called with args=(42,), kwargs={'retries': 3, 'timeout': 5.0}
```

In a signature, ordering is fixed: regular positional params, then `*args`, then keyword-only params, then `**kwargs`. This pattern is most commonly used for pass-through wrappers — decorators, proxy functions, or a subclass method that wants to accept whatever its parent accepts without repeating the whole signature:

```python
def wrapper(*args, **kwargs):
    return real_function(*args, **kwargs)
```

The `*`/`**` also work in reverse, at the call site, to *unpack* an existing sequence/dict into separate arguments: `func(*my_list, **my_dict)` spreads `my_list`'s items as positional args and `my_dict`'s items as keyword args, which is how you forward a captured call or build dynamic argument lists.

Question 23

What's the difference between == and is in Python, and why does using is with numbers or strings sometimes give surprising results?

Accepted Answer

`==` calls `__eq__` and checks value equality — do these two objects represent the same value? `is` checks identity — are these two names pointing at the exact same object in memory (same `id()`)? Two different list objects with identical contents are `==` but not `is`.

```python
a = [1, 2, 3]
b = [1, 2, 3]
a == b   # True: same contents
a is b   # False: different objects
```

The surprising part is that small integers (-5 to 256) and some short strings are automatically **interned/cached** by CPython as a performance optimization, so two separately-created references to the same small int or literal string can end up pointing at the *same* cached object, making `is` return `True` even though you never explicitly asked for that:

```python
x = 100; y = 100
x is y   # True — small int caching

x = 1000; y = 1000
x is y   # False (usually) — outside the cached range, separate objects
```

This is exactly why `is` should never be used for value comparisons on ints, strings, or any general object — the behavior depends on an internal CPython optimization detail, not on the language guaranteeing it, and it can differ between CPython versions or other implementations entirely. The one place `is` is genuinely correct and idiomatic is comparing against the singletons `None`, `True`, and `False` (`if x is None:`), since there's only ever one of each in a running process.

Question 24

What's the difference between f-strings, str.format(), and the % operator for string formatting in Python, and which should you use?

Accepted Answer

All three interpolate values into a string template, but they differ in syntax, when the expression is evaluated, and readability. `%` formatting (`"%s is %d" % (name, age)`) is the oldest, C-style, positional-only in its simplest form, and generally considered legacy for new code.

`str.format()` (`"{} is {}".format(name, age)`) improved on that with named/indexed placeholders and richer format specs, but it's verbose for anything beyond simple substitution — you're referencing variables by name or position separately from where they're used.

f-strings (`f"{name} is {age}"`, Python 3.6+) let you embed the actual expression directly inside the string literal, which is both the most readable and, because the interpolation is compiled at parse time rather than resolved via method calls at runtime, also the fastest of the three.

```python
name, age = "Asha", 29
f"{name} is {age} years old, next year: {age + 1}"
# f-strings support arbitrary expressions, not just variable names
f"{value:.2f}"          # format specs work the same as str.format()
f"{value=}"              # debug specifier (3.8+): prints 'value=<repr>'
```

For new code, f-strings are the default choice. `str.format()` still shows up where the template needs to be a separate string built ahead of time (e.g., loaded from a config file or translation resource) since f-strings can't be deferred — they're evaluated immediately at the point they're written.

Question 25

What's the difference between a shallow copy and a deep copy in Python, and when does a shallow copy cause a real bug?

Accepted Answer

A shallow copy (`copy.copy()`, `list(x)`, `x[:]`, `dict(x)`) creates a new outer container but fills it with references to the *same* nested objects the original contained. A deep copy (`copy.deepcopy()`) recursively copies every nested object too, so the result shares nothing mutable with the original at any depth.

```python
import copy

original = {'scores': [90, 85]}
shallow = copy.copy(original)
shallow['scores'].append(70)
print(original['scores'])   # [90, 85, 70] -- mutated! same inner list object

deep = copy.deepcopy(original)
deep['scores'].append(60)
print(original['scores'])   # unaffected
```

The bug shows up specifically with nested mutable structures — a dict of lists, a list of custom objects, a config object holding sub-configs. Code that does `new_config = dict(old_config)` assuming it got an independent copy, then mutates a nested list inside it, silently corrupts the "original" too, because both dicts still point at the exact same inner list object. For flat structures with only immutable values (ints, strings, tuples of immutables), shallow and deep copies behave identically since there's nothing nested to alias — which is exactly why this bug tends to appear later, once someone adds a nested mutable field to a previously-flat structure and the existing shallow-copy call sites silently stop being safe.

Question 26

What's the difference between @staticmethod, @classmethod, and a regular instance method in Python?

Accepted Answer

A regular instance method implicitly receives `self` — the specific instance it was called on — and is meant to read/modify that instance's state. A `@classmethod` receives `cls` — the class itself, not an instance — and is used when the logic needs to know which class it's operating on but doesn't need a specific instance, most commonly for alternative constructors. A `@staticmethod` receives neither `self` nor `cls` — it's just a regular function that happens to live inside the class's namespace for organizational purposes.

```python
class Pizza:
    def __init__(self, toppings):
        self.toppings = toppings

def add_topping(self, topping):        # instance method: mutates this pizza
        self.toppings.append(topping)

@classmethod
    def margherita(cls):                    # classmethod: alternative constructor
        return cls(['mozzarella', 'basil'])

@staticmethod
    def is_valid_topping(name):             # staticmethod: no instance/class state needed
        return name in ALLOWED_TOPPINGS
```

The classmethod pattern matters especially with inheritance: `cls(...)` inside `margherita` resolves to whatever subclass actually called it (`Pizza.margherita()` builds a `Pizza`, `DeepDish.margherita()` builds a `DeepDish`, since `cls` is the subclass, not hardcoded to `Pizza`), whereas a staticmethod has no way to know or respect the calling subclass at all — it behaves identically no matter what class or instance you call it through.

Question 27

What is the memory and performance trade-off between a list comprehension and a generator expression in Python?

Accepted Answer

A list comprehension `[x*2 for x in range(1_000_000)]` builds the *entire* list in memory immediately, all million elements at once. A generator expression `(x*2 for x in range(1_000_000))` builds nothing up front — it returns a lazy iterator that computes each value one at a time, on demand, as something actually asks for the next item.

```python
total = sum(x**2 for x in range(10_000_000))   # generator: O(1) extra memory
squares = [x**2 for x in range(10_000_000)]     # list: allocates ~80MB+ up front
```

For anything that just gets consumed once in a loop or fed into a function like `sum()`/`max()`/`any()`, the generator expression uses effectively constant memory regardless of how many elements it produces, since only one element exists in memory at a time. The list comprehension is the right choice when you need to: index into the result, iterate over it more than once, know its length (`len()`), or need it materialized because a downstream API requires an actual list (not every iterable). Generator expressions also can't be re-iterated — once exhausted, they're done, whereas a list can be looped over repeatedly. Performance-wise, list comprehensions are usually marginally faster per-element for small-to-medium sizes since there's no generator-frame overhead per `next()` call, but that's dwarfed by the memory difference once the data set is large.

Question 28

What does the @property decorator do in Python, and when would you add a setter alongside it?

Accepted Answer

`@property` turns a method into something that's accessed like a plain attribute — no parentheses — while still running actual code behind the scenes. This lets you start with simple public attributes and later add validation or computed logic without breaking every call site that already does `obj.value` instead of `obj.value()`.

```python
class Temperature:
    def __init__(self, celsius):
        self._celsius = celsius

@property
    def fahrenheit(self):
        return self._celsius * 9/5 + 32

@fahrenheit.setter
    def fahrenheit(self, value):
        self._celsius = (value - 32) * 5/9
```

`fahrenheit` here is a fully computed value — there's no stored `_fahrenheit` field, it's derived from `_celsius` every time it's read. Adding `@fahrenheit.setter` lets `temp.fahrenheit = 100` work too, converting back and storing into `_celsius`, so both directions stay in sync automatically and there's exactly one source of truth (`_celsius`).

You'd add a setter specifically when you need validation on assignment (reject a negative age, clamp a percentage to 0–100) or when setting one "virtual" attribute needs to update other internal state consistently, as above. A property with only a getter (no setter) effectively makes that attribute read-only from outside the class — assigning to it raises `AttributeError: can't set attribute`, which is a clean way to expose a computed or protected value without extra boilerplate getter/setter method calls at every use site.

Question 29

How do you write a Python decorator that itself accepts arguments, like @retry(times=3)?

Accepted Answer

A plain decorator is a function taking one argument (the function being decorated) and returning a replacement. A decorator *factory* like `@retry(times=3)` needs an extra layer: the outer function takes the decorator's own arguments and returns the actual decorator, which then takes the function and returns the wrapper.

```python
import functools, time

def retry(times=3, delay=1):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            last_exc = None
            for attempt in range(times):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    last_exc = e
                    time.sleep(delay)
            raise last_exc
        return wrapper
    return decorator

@retry(times=5, delay=0.5)
def call_flaky_api():
    ...
```

`@retry(times=5, delay=0.5)` evaluates `retry(times=5, delay=0.5)` first — that call runs immediately at decoration time and returns `decorator`, a closure that has captured `times` and `delay`. *Then* Python applies `decorator` to `call_flaky_api`, exactly like a normal decorator, producing `wrapper`. `functools.wraps(func)` inside `decorator` is still needed at the innermost level to preserve the original function's `__name__`/`__doc__`/signature, same as with any other decorator. The three-function nesting (factory → decorator → wrapper) is the standard shape for any parametrized decorator, whether the parameters are retry counts, permission names, or cache TTLs.

Question 30

How do you write a custom context manager using contextlib.contextmanager instead of a class with __enter__/__exit__?

Accepted Answer

`@contextlib.contextmanager` lets you write a context manager as a single generator function instead of a class with two separate dunder methods — everything before the `yield` is the `__enter__` logic, the yielded value is what `as` binds to, and everything after the `yield` is the `__exit__` logic, including a `try/finally` (or `try/except`) around the `yield` to handle exceptions raised inside the `with` block.

```python
from contextlib import contextmanager
import time

@contextmanager
def timed_block(label):
    start = time.perf_counter()
    try:
        yield
    finally:
        elapsed = time.perf_counter() - start
        print(f"{label}: {elapsed:.3f}s")

with timed_block("db_query"):
    run_query()
```

If the `with` block raises an exception, it's re-raised at the `yield` statement inside the generator — so the `finally` block still runs (guaranteeing cleanup, exactly like a real `__exit__`), and if you want to suppress specific exceptions the way a class-based `__exit__` can by returning `True`, you wrap the `yield` in `try/except SpecificError: pass` instead.

This approach is the right choice for simple, linear setup/teardown logic (the common case) — it's shorter and more readable than a class. Reach for a full class-based context manager instead when you need the object to hold reusable state across multiple `__enter__`/`__exit__` cycles (re-entrant or reusable managers), or when the logic genuinely needs separate named methods rather than fitting linearly around one `yield`.

Question 31

How do you manage Python dependencies and isolate project environments — what's the role of venv, pip, requirements.txt, and pyproject.toml?

Accepted Answer

`venv` (built into the standard library) creates an isolated Python environment with its own `site-packages` directory, so installing a package for one project doesn't affect another project or the system Python — critical since different projects often need conflicting versions of the same library.

```bash
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install requests
```

`pip` is the package installer that pulls packages from PyPI into whatever environment is currently active. `requirements.txt` is the traditional way to pin and share a project's dependencies (`pip freeze > requirements.txt`, `pip install -r requirements.txt` to reproduce it elsewhere) — it's simple but flat, with no distinction between direct and transitive dependencies, and no built-in dependency resolution beyond what pip does at install time.

`pyproject.toml` is the modern, standardized project-metadata file (PEP 518/621) that's largely superseding `setup.py`/`requirements.txt` for defining a package's metadata, dependencies, and build system in one place — tools like `poetry`, `pdm`, and `uv` all read/write it, and it supports separating core dependencies from optional/dev dependency groups declaratively. For a simple script or small project, `venv` + `requirements.txt` is still perfectly fine; for anything you intend to package, publish, or that has non-trivial dependency groups (dev/test/docs extras), `pyproject.toml` with a modern tool is the current best practice.

Question 32

How do you write unit tests with pytest and mock an external API call using unittest.mock?

Accepted Answer

pytest tests are plain functions prefixed with `test_`, using bare `assert` statements — no special assertion methods needed, since pytest rewrites assertions to give detailed failure output automatically.

```python
# app.py
import requests

def get_user_name(user_id):
    resp = requests.get(f"https://api.example.com/users/{user_id}")
    return resp.json()['name']
```

```python
# test_app.py
from unittest.mock import patch, Mock
from app import get_user_name

def test_get_user_name():
    with patch('app.requests.get') as mock_get:
        mock_get.return_value = Mock(json=lambda: {'name': 'Asha'})
        assert get_user_name(42) == 'Asha'
        mock_get.assert_called_once_with("https://api.example.com/users/42")
```

The key detail with `patch` is *where* you patch: you patch `'app.requests.get'` — the name as it's looked up in the module under test — not `'requests.get'` directly, because `app.py` imported `requests` into its own namespace, and patching has to replace the reference the code under test actually uses. `mock_get.return_value` configures what the mock returns when called; `mock_get.assert_called_once_with(...)` verifies the call happened with the expected arguments, which is how you test that your code calls the external API correctly without ever making a real network request. pytest `fixtures` (`@pytest.fixture`) are the standard way to share setup code (like a configured mock or test database) across many tests, and `pytest.raises(SomeError)` is the idiom for asserting a function raises the expected exception.

Question 33

What does raise ... from ... do in Python, and why is exception chaining useful for debugging?

Accepted Answer

When you catch one exception and raise a different, more meaningful one in its `except` block, Python by default still shows both in the traceback — the original exception, labeled "During handling of the above exception, another exception occurred," followed by the new one. `raise NewError(...) from original_error` makes that relationship explicit and intentional rather than incidental, setting `NewError.__cause__ = original_error`.

```python
def load_config(path):
    try:
        with open(path) as f:
            return json.load(f)
    except FileNotFoundError as e:
        raise ConfigError(f"Missing config file: {path}") from e
```

Without `from e`, Python still chains it implicitly via `__context__` (any exception raised inside an `except` block gets this automatically), but the traceback phrasing differs and it doesn't signal that the chaining was deliberate versus accidental (e.g. a bug in your own except-block code that happened to raise while handling the first error). `raise ... from e` explicitly says "this new error is a direct, intentional consequence of that original one" which is clearer for anyone reading the traceback later.

The debugging value is that you get the *real* root cause (the low-level `FileNotFoundError`, with the actual missing path) preserved right alongside the higher-level, more meaningful error your API surfaces (`ConfigError`), instead of losing the original context entirely — which is what happens if you `raise ConfigError(...)` with no chaining at all, or worse, silently `raise ConfigError(...) from None` (which explicitly suppresses the chain, appropriate only when the original exception is genuinely irrelevant noise).

Question 34

What's the difference between threading, multiprocessing, and asyncio in Python, and how do you decide which one to use for a given workload?

Accepted Answer

All three give you concurrency, but they solve different bottlenecks. `threading` runs multiple threads within one process, sharing memory — but because of the GIL, only one thread executes Python bytecode at a time, so threading only actually speeds things up when threads spend most of their time *waiting* on I/O (network calls, disk, DB queries), during which the GIL is released and another thread can run.

`multiprocessing` runs separate OS processes, each with its own interpreter and its own GIL, giving genuine parallel execution across CPU cores — the right tool for CPU-bound work (heavy computation, image processing, data crunching) where threading gives no benefit. The cost is heavier process startup and the need to serialize (pickle) data crossing process boundaries, unless you use shared memory.

`asyncio` is single-threaded, cooperative concurrency: one thread runs many coroutines, switching between them only at explicit `await` points. It's the best fit for high-volume I/O-bound workloads (thousands of concurrent network connections) because it avoids both the GIL contention of threading and the per-unit overhead of spawning OS threads or processes — a single event loop can juggle far more concurrent I/O-bound tasks than an equivalent thread pool, with lower memory and context-switch cost, provided every I/O call in the code path is actually async-native (a single blocking call anywhere freezes the whole loop, as covered in the asyncio-debugging question above).

**Decision rule**: CPU-bound → multiprocessing. I/O-bound with a small/moderate number of concurrent operations, or integrating with blocking libraries → threading. I/O-bound at high concurrency, with async-native libraries available end to end → asyncio.

Question 35

How does Python's dict maintain insertion order since 3.7, and what's the underlying implementation that makes both fast lookups and ordering work?

Accepted Answer

Before 3.6/3.7, dicts made no ordering guarantee at all — iteration order was an implementation detail that could change. Since 3.7 (officially guaranteed; 3.6 had it as a CPython implementation detail), dicts preserve insertion order as a language guarantee, and it's not a performance trade-off — dict lookups remain O(1) average case.

The implementation change that made this possible: CPython's dict is actually two structures combined. A sparse array of indices (hash table, sized larger than the number of items, mostly holding "empty" markers) maps a key's hash to a slot, and that slot stores an *index* into a second, separate compact array holding the actual key/value/hash entries in the order they were inserted. Lookup still works the classic hash-table way — hash the key, probe the sparse index array, follow the index into the dense array, compare keys — so it's still O(1) average. Iteration just walks the dense array directly, which is naturally in insertion order since new entries are always appended to the end of it (deletions leave a hole that's compacted on resize, not immediately).

This "compact dict" design (from PEP 468/the Raymond Hettinger–proposed layout that shipped in 3.6) also reduced dict memory usage significantly compared to the older implementation, since the sparse index array only needs to store small integers rather than full hash/key/value triples — a nice side effect of the change, not the primary motivation, which was mainly the memory win with ordering as a bonus that Python then made an official guarantee.

Question 36

How do you sort a list of dictionaries by a specific key in Python, and what does it mean that Python's sort is 'stable'?

Accepted Answer

`sorted()` and `list.sort()` both accept a `key` function that extracts the value to compare on, rather than requiring you to implement a custom comparator:

```python
people = [{'name': 'Bob', 'age': 25}, {'name': 'Amy', 'age': 30}, {'name': 'Cal', 'age': 25}]
sorted(people, key=lambda p: p['age'])
# by age ascending; ties (Bob/Cal both 25) keep their original relative order

sorted(people, key=lambda p: p['age'], reverse=True)
sorted(people, key=lambda p: (p['age'], p['name']))   # sort by age, then name as tiebreaker
```

`operator.itemgetter('age')` is a common, slightly faster alternative to a lambda for this exact pattern (`sorted(people, key=operator.itemgetter('age'))`), since it avoids the overhead of a Python-level function call per comparison.

"Stable" means that when two elements compare equal under the key function, their relative order from the *original* input is preserved in the output — Bob and Cal, both age 25, stay in the order they appeared in the input list rather than being arbitrarily swapped. This matters in practice for multi-key sorting done in separate passes: sorting first by `name` and then, stably, by `age`, correctly produces a result grouped by age with names still sorted within each age group — an unstable sort couldn't guarantee that. Python's built-in sort (Timsort) has been stable since it was introduced, and that guarantee is part of the documented language behavior, not an implementation detail that could change.

Question 37

Write a Python solution for the Two Sum problem: given a list of integers and a target, return the indices of the two numbers that add up to the target.

Accepted Answer

The naive approach checks every pair, which is O(n²). The efficient approach uses a hash map to record each number's index as you scan once, checking at each step whether the *complement* needed to reach the target has already been seen — that's O(n) time and O(n) space.

```python
def two_sum(nums, target):
    seen = {}  # value -> index
    for i, num in enumerate(nums):
        complement = target - num
        if complement in seen:
            return [seen[complement], i]
        seen[num] = i
    return None  # no solution

two_sum([2, 7, 11, 15], 9)   # [0, 1] -- nums[0] + nums[1] == 9
```

The trick is checking for the complement *before* inserting the current number into `seen` — this correctly handles the case where a number would need to pair with itself (e.g., target 8 with `[4, 4]`) without falsely matching a number against its own not-yet-seen occurrence, while still matching correctly once the second `4` is scanned and finds the first one already in `seen`. Dict lookups are O(1) average case, which is what gets the whole algorithm down to a single O(n) pass instead of the O(n²) nested-loop approach.

Question 38

Write Python code to reverse a singly linked list, both iteratively and using recursion.

Accepted Answer

Reversing a linked list means flipping every node's `next` pointer to point backward instead of forward, then returning the old tail as the new head.

Iterative (O(n) time, O(1) extra space — generally preferred in interviews and in production, since it avoids stack growth):

```python
class Node:
    def __init__(self, val, next=None):
        self.val = val
        self.next = next

def reverse_iterative(head):
    prev = None
    curr = head
    while curr:
        next_node = curr.next   # save before overwriting
        curr.next = prev        # flip the pointer
        prev = curr
        curr = next_node
    return prev   # prev is the new head
```

Recursive (O(n) time, O(n) space due to the call stack — elegant but risks a stack overflow on very long lists, which the iterative version doesn't):

```python
def reverse_recursive(head):
    if head is None or head.next is None:
        return head   # base case: empty or single node is already "reversed"
    new_head = reverse_recursive(head.next)
    head.next.next = head   # make the next node point back to this one
    head.next = None         # this node becomes the new tail
    return new_head
```

The recursive version's key insight is that it fully reverses everything *after* the current node first, then fixes up just the one link between the current node and what used to be its neighbor — `head.next.next = head` is what actually flips that pointer. For linked lists that could be very long (tens of thousands of nodes), Python's default recursion limit (~1000) makes the iterative version the only safe choice in practice.

Question 39

Write a Python function to merge overlapping intervals, e.g. [[1,3],[2,6],[8,10],[15,18]] becomes [[1,6],[8,10],[15,18]].

Accepted Answer

Sort the intervals by start time first — once sorted, any interval that overlaps with the current merged interval must come immediately after it, so a single linear pass suffices; no need to compare every pair.

```python
def merge_intervals(intervals):
    if not intervals:
        return []
    intervals.sort(key=lambda pair: pair[0])
    merged = [intervals[0]]
    for start, end in intervals[1:]:
        last_end = merged[-1][1]
        if start <= last_end:              # overlaps (or touches) the last merged interval
            merged[-1][1] = max(last_end, end)
        else:
            merged.append([start, end])
    return merged

merge_intervals([[1,3],[2,6],[8,10],[15,18]])
# [[1, 6], [8, 10], [15, 18]]
```

The overlap condition is `start <= last_end` — using `<=` rather than `<` also merges intervals that just touch (e.g. `[1,3]` and `[3,5]` become `[1,5]`), which is usually the desired behavior for calendar/scheduling-style merging; using strict `<` would leave touching-but-not-overlapping intervals separate, which is a common off-by-one bug in this problem. Sorting dominates the complexity: O(n log n) for the sort, O(n) for the single merge pass, so O(n log n) overall — a very common building block for scheduling problems (meeting room conflicts, calendar merging, free/busy time calculations).

Question 40

Implement an LRU (Least Recently Used) cache in Python from scratch with O(1) get and put operations.

Accepted Answer

The O(1) requirement for both operations rules out a plain list (finding/removing the least-recently-used item would be O(n)). The standard solution combines a dict for O(1) key lookup with `collections.OrderedDict`, which tracks insertion/access order and supports O(1) move-to-end and pop-from-front:

```python
from collections import OrderedDict

class LRUCache:
    def __init__(self, capacity):
        self.capacity = capacity
        self.cache = OrderedDict()

def get(self, key):
        if key not in self.cache:
            return -1
        self.cache.move_to_end(key)   # mark as recently used
        return self.cache[key]

def put(self, key, value):
        if key in self.cache:
            self.cache.move_to_end(key)
        self.cache[key] = value
        if len(self.cache) > self.capacity:
            self.cache.popitem(last=False)   # evict least-recently-used (front)
```

`OrderedDict` internally maintains a doubly linked list alongside its hash table specifically so that `move_to_end()` and `popitem(last=False)` (pop from the front) are both O(1) — exactly the two operations an LRU cache needs on every access. Every `get` or `put` moves the touched key to the end, so the *front* of the dict is always the least-recently-used entry, ready to evict the moment capacity is exceeded.

In real production Python code you'd almost never hand-roll this — `functools.lru_cache(maxsize=...)` is a built-in decorator doing exactly this for memoizing function results — but implementing it manually (as here) is a very common interview exercise precisely because it tests whether you know *why* `OrderedDict` (not a plain dict plus manual bookkeeping) gets you true O(1) on both ends.

Question 41

How do you implement binary search in Python, and what does the bisect module give you for free?

Accepted Answer

Binary search finds a target in a *sorted* sequence in O(log n) by repeatedly halving the search range: compare the target to the middle element, and discard the half that can't contain it.

```python
def binary_search(arr, target):
    lo, hi = 0, len(arr) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            lo = mid + 1
        else:
            hi = mid - 1
    return -1   # not found
```

A classic off-by-one bug is `mid = (lo + hi) // 2` when `lo + hi` could overflow in languages with fixed-width integers — not a concern in Python (arbitrary-precision ints), but the loop bound `lo <= hi` (not `<`) and the `+1`/`-1` adjustments are still easy to get wrong under interview pressure.

The standard library's `bisect` module gives you a correct, tested implementation for the most common use case — finding an *insertion point* to keep a list sorted, rather than just a yes/no "is it in there":

```python
import bisect

arr = [1, 3, 4, 7, 9]
bisect.bisect_left(arr, 4)    # 2 -- index of the leftmost position to insert 4
bisect.bisect_right(arr, 4)   # 3 -- rightmost position (after existing 4s)
bisect.insort(arr, 5)         # inserts 5 in sorted position: [1, 3, 4, 5, 7, 9]
```

`bisect_left`/`bisect_right` are also directly useful for exact-match search (`bisect_left(arr, x)` gives an index you then check `arr[i] == x`) so in real code you'd almost always reach for `bisect` rather than hand-rolling binary search — writing it from scratch is mainly an interview exercise to demonstrate understanding of the technique.

Question 42

Write a Python function to check whether a string of brackets — (), [], {} — is balanced/valid.

Accepted Answer

A stack is the natural fit: push every opening bracket, and on every closing bracket, check that it matches whatever's currently on top of the stack — if it doesn't match (or the stack is empty), the string is invalid.

```python
def is_balanced(s):
    pairs = {')': '(', ']': '[', '}': '{'}
    stack = []
    for char in s:
        if char in pairs.values():
            stack.append(char)
        elif char in pairs:
            if not stack or stack.pop() != pairs[char]:
                return False
    return not stack   # must be empty -- no unmatched openers left

is_balanced("{[()()]}")   # True
is_balanced("{[(])}")     # False -- wrong nesting order
is_balanced("(()")        # False -- unmatched opener
```

Three distinct failure modes all fall out of this same logic naturally: a closing bracket with nothing on the stack to match (`stack` empty when we hit a closer), a closing bracket that doesn't match the most recent opener (`stack.pop() != pairs[char]`), and leftover unmatched openers at the end (`not stack` check after the loop — `"(()"` never triggers a mismatch mid-scan, but leaves `'('` on the stack). This is a genuinely common building block beyond the interview-classic phrasing — it's the same core logic used in expression parsers, XML/HTML tag validators, and syntax highlighters checking for matched delimiters.

Question 43

Implement quicksort in Python and explain its average-case versus worst-case time complexity.

Accepted Answer

Quicksort picks a pivot, partitions the array into elements less than and greater than the pivot, and recursively sorts each partition.

```python
def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[len(arr) // 2]
    less = [x for x in arr if x < pivot]
    equal = [x for x in arr if x == pivot]
    greater = [x for x in arr if x > pivot]
    return quicksort(less) + equal + quicksort(greater)
```

This version is simple and readable but not in-place (it allocates new lists at every level) — a true in-place version uses index-swapping partitioning (Lomuto or Hoyle partition schemes) to sort within the original array using O(log n) auxiliary space for the recursion stack instead of O(n) for new lists at every level.

Average case is O(n log n): a reasonably balanced pivot choice splits the array roughly in half each time, giving log n levels of recursion, each doing O(n) work to partition — same shape as merge sort's recurrence. Worst case is O(n²), which happens when the pivot choice is consistently bad — most notably, picking the first or last element as pivot on an *already-sorted* (or reverse-sorted) array produces maximally unbalanced partitions (one side empty, one side n-1 elements) at every level, degenerating into n levels of O(n) work each. This is exactly why production sort implementations either randomize the pivot choice, use median-of-three (comparing first/middle/last and picking the median as pivot), or — like Python's actual built-in `sorted()`/`list.sort()` — don't use quicksort at all, using Timsort instead (a hybrid of merge sort and insertion sort, guaranteed O(n log n) worst case and specifically optimized to exploit already-sorted runs in real-world data).

Question 44

What's the difference between WSGI and ASGI, and why does that distinction matter when choosing between Flask and FastAPI?

Accepted Answer

WSGI (Web Server Gateway Interface) is the long-standing standard interface between Python web servers and web applications/frameworks — it defines a simple synchronous contract: the server calls your app with a request environment, your app returns a response, one request fully blocks that worker until it's done. Flask, Django (classically), and most older Python web frameworks are built on WSGI.

ASGI (Asynchronous Server Gateway Interface) is the newer standard that adds native support for `async`/`await`, WebSockets, and long-lived connections — an ASGI app can `await` I/O (a DB query, an external API call) without blocking the entire worker process, letting a single worker handle many concurrent in-flight requests cooperatively, the same way asyncio does in general.

FastAPI is built on ASGI (via Starlette) from the ground up, so route handlers can be defined as `async def` and use `await` for I/O-bound work, giving much higher concurrency per worker for I/O-heavy APIs (lots of concurrent, mostly-waiting requests) compared to WSGI's one-blocking-request-per-worker-thread model. Flask historically was WSGI-only and scaled concurrency by adding more worker processes/threads (each one still blocks per request); newer Flask versions do support async view functions, but the underlying WSGI server still processes them within the same blocking-per-worker model unless paired with an ASGI-compatible setup, so it doesn't get ASGI's full concurrency benefit natively.

The practical decision: for CPU-bound or simple, low-concurrency APIs, WSGI/Flask is perfectly fine and has a simpler mental model. For high-concurrency, I/O-heavy APIs (many simultaneous slow upstream calls, WebSockets, streaming), ASGI/FastAPI's async-native model handles far more concurrent connections per process without needing to scale worker count linearly with concurrent requests.

Question 45

What is the N+1 query problem in an ORM like SQLAlchemy or Django ORM, and how do you fix it?

Accepted Answer

The N+1 problem happens when code fetches a list of N parent objects with one query, then — for each of those N objects — triggers a *separate* query to fetch a related object or collection, because the ORM's default lazy-loading only fetches related data the moment it's actually accessed.

```python
# Django ORM example
authors = Author.objects.all()          # 1 query
for author in authors:
    print(author.book_set.count())      # N more queries -- one per author!
```

That's 1 query to get the authors plus N more queries, one per author, for a total of N+1 — instead of the 2 queries (or even 1, with a join) it should actually take. This scales terribly: with 1,000 authors, that's 1,001 round trips to the database instead of 1 or 2.

The fix is telling the ORM up front that you'll need the related data, so it fetches everything in one batched query (or one JOIN) instead of lazily, one at a time. In Django, `select_related()` (for foreign-key/one-to-one, generates a SQL JOIN) and `prefetch_related()` (for many-to-many/reverse-FK, runs a second batched query and joins in Python) solve this:

```python
authors = Author.objects.prefetch_related('book_set').all()   # 2 queries total, not N+1
```

SQLAlchemy has the equivalent concept via `joinedload()`/`selectinload()` in `.options()`. This is one of the most common real-world Python backend performance bugs precisely because it's invisible in development with small test datasets (10 authors, 11 queries, feels fine) and only becomes a visible latency/DB-load problem once the dataset scales to production size — catching it usually means turning on SQL query logging (Django Debug Toolbar, SQLAlchemy's `echo=True`) and noticing a suspicious query count that scales linearly with result set size.

Question 46

What is monkeypatching in Python, and what are the risks of using it in production code?

Accepted Answer

Monkeypatching means modifying or replacing a class, function, or module attribute at runtime, from outside its original definition — Python allows this freely because almost everything (classes, modules, even built-in-adjacent objects in some cases) is a mutable object whose attributes can be reassigned.

```python
import requests

def fake_get(url, **kwargs):
    return MockResponse({'status': 'ok'})

requests.get = fake_get   # monkeypatch: replaces the real function globally
```

This is legitimate and common in tests (`unittest.mock.patch` is essentially a scoped, auto-reverting monkeypatch), and occasionally for hotfixing a bug in a third-party library without forking it, or adding instrumentation to code you don't control.

The risks in production code are real: a monkeypatch is invisible at the call site — anyone reading `requests.get(...)` elsewhere in the codebase has no way to tell it's actually `fake_get` without knowing the patch exists somewhere else entirely, which makes debugging and onboarding much harder. It's also global and order-dependent — if two different parts of the codebase patch the same thing differently, whichever runs last wins, silently, and the bug only shows up depending on import order. And patches applied against a specific library version can silently break (or silently stop taking effect) on a library upgrade, since you're relying on internal implementation details that aren't part of the library's public contract. For production code, the safer alternatives are usually dependency injection (pass in the behavior you want as a parameter) or subclassing/wrapping rather than mutating shared global state at runtime — monkeypatching is best reserved for tests, where `unittest.mock.patch`'s automatic teardown (via context manager or decorator) keeps the blast radius scoped to a single test.

Question 47

What is functools.singledispatch, and how does it let you implement function overloading based on argument type in Python?

Accepted Answer

Python doesn't have built-in method overloading by parameter type the way Java or C++ does — defining two functions with the same name just makes the second one silently replace the first. `functools.singledispatch` gives you a clean way to dispatch to different implementations based on the type of the *first* argument, registered separately per type.

```python
from functools import singledispatch

@singledispatch
def render(value):
    raise NotImplementedError(f"no renderer for {type(value)}")

@render.register
def _(value: int):
    return f"Integer: {value}"

@render.register
def _(value: list):
    return f"List of {len(value)} items"

@render.register
def _(value: str):
    return f"String: {value!r}"

render(42)          # "Integer: 42"
render([1, 2, 3])   # "List of 3 items"
```

The base function (decorated with `@singledispatch`) acts as both the default/fallback implementation and the dispatcher — calling `render(x)` looks at `type(x)`, finds the most specific registered implementation for it (walking the MRO if there's no exact match, so a registered `Sequence` handler would catch a `list` if no more specific `list` handler existed), and calls that instead. This is genuinely useful for things like serializers, visitors over heterogeneous data (AST nodes, JSON-like structures), or pretty-printers where the logic naturally varies by input type — without it, you'd end up hand-writing a chain of `isinstance()` checks in one big function, which gets unwieldy and isn't easily extensible by other modules the way `@render.register` is (a plugin can register a handler for its own custom type without touching the original function at all).

Question 48

Why can't you always use a mutable object as a dictionary key or set member in Python, and what's the actual contract between __hash__ and __eq__?

Accepted Answer

Dicts and sets rely on hashing to achieve O(1) average lookup — the hash of a key determines which bucket it goes in, and equality (`__eq__`) resolves collisions within a bucket. The contract Python requires is: if two objects are equal (`a == b`), they *must* have the same hash (`hash(a) == hash(b)`); the reverse isn't required (different objects can share a hash — that's just a collision, handled fine by `__eq__`).

This is exactly why mutable built-in types (`list`, `dict`, `set`) are explicitly *unhashable* — `hash([1,2,3])` raises `TypeError`. If lists were hashable and you mutated one after inserting it into a dict as a key, its hash would change, but the dict's internal bucket placement (computed from the *old* hash) wouldn't move it — the entry would become permanently unfindable via normal lookup, silently breaking the dict's core invariant. Tuples are hashable specifically because they're immutable — `hash((1,2,3))` is stable for the tuple's whole lifetime, provided its contents are also hashable (a tuple containing a list is still unhashable, since the contract has to hold recursively).

```python
class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y
    def __eq__(self, other):
        return (self.x, self.y) == (other.x, other.y)
    def __hash__(self):
        return hash((self.x, self.y))
```

A subtle real bug: if you define `__eq__` on a custom class but *don't* define `__hash__`, Python automatically makes the class unhashable (sets `__hash__ = None`) — because the default identity-based hash would now violate the contract (two equal-by-value objects with different default hashes). So any class overriding `__eq__` for value-based equality must also implement a matching `__hash__` (typically hashing the same fields used in `__eq__`) if you ever want instances usable as dict keys or set members.

Question 49

What is PEP 703 (the free-threaded / no-GIL CPython build), and what does it actually change architecturally compared to the GIL model?

Accepted Answer

PEP 703, accepted for phased rollout starting with CPython 3.13 as an optional build (`python3.13t`), removes the Global Interpreter Lock as a single global mutex and replaces reference counting and internal data-structure access with fine-grained, per-object locking and biased/deferred reference counting instead. The goal is letting multiple threads execute Python bytecode *truly simultaneously* on separate cores, something the standard GIL build has never allowed for CPU-bound code, without requiring `multiprocessing`'s process-per-worker overhead and IPC serialization costs.

Architecturally, this is a much bigger change than it sounds: the GIL wasn't just protecting the interpreter loop, it was implicitly protecting *every* mutable object's internal state (a dict's hash table, a list's backing array, an object's refcount) from concurrent mutation, essentially for free. Removing it means CPython needs new mechanisms to keep those structures consistent under real concurrent access — this includes per-object locks for container mutation, a modified reference-counting scheme (since naive atomic increment/decrement on every refcount change would be a massive performance regression on single-threaded code), and changes to how the memory allocator and garbage collector coordinate across threads.

The practical implications as of the current phased rollout: single-threaded performance takes a real hit compared to the standard GIL build (early benchmarks showed a meaningful percentage slowdown, though this keeps improving release over release), C extensions that assumed GIL-protected access to Python objects (which is most existing C extensions, including large chunks of the numeric/scientific Python ecosystem) need to be explicitly updated and marked thread-safe to work correctly under free-threading, and it remains an opt-in, non-default build for now rather than replacing the standard interpreter. It's the first serious, officially-sanctioned attempt to solve the GIL's CPU-bound threading limitation directly, rather than working around it via multiprocessing or asyncio, but it's still an actively-evolving feature, not yet the default production choice.

Question 50

What are contextvars in Python, and why are they needed for propagating request-scoped context (like a request ID) across asyncio tasks, where thread-locals don't work?

Accepted Answer

`threading.local()` gives each *thread* its own isolated storage — great for synchronous, thread-per-request servers, since a value set in one thread is invisible to others. But asyncio runs many concurrent tasks (coroutines) on a *single* thread, all sharing one event loop, so `threading.local()` is useless there: every task on that thread would see the exact same thread-local storage, with no way to distinguish "this value belongs to request A's task" from "this value belongs to request B's task" running concurrently on the same thread.

`contextvars.ContextVar` solves this by tracking context per *logical task/coroutine chain* rather than per-thread. Each `asyncio.Task` automatically gets a copy of the context that was active when it was created, and that copy propagates correctly through `await` points and into any tasks/coroutines spawned from within it — without threads being involved at all.

```python
import contextvars

request_id = contextvars.ContextVar('request_id', default=None)

async def handle_request(rid):
    request_id.set(rid)
    await process()   # request_id is still correctly 'rid' inside process(), even across await

async def process():
    print(request_id.get())   # correctly isolated per concurrent request/task
```

Even with many `handle_request` coroutines running "concurrently" (interleaved on the single event-loop thread), each one's `contextvars` state stays isolated to its own task's context — a value set in one request's task doesn't leak into another's, which is exactly the isolation you need for things like request IDs, tracing spans, or auth context in an async web framework. This is precisely how libraries like `structlog`'s contextual logging or OpenTelemetry's async tracing correctly attribute log lines and spans to the right request even when thousands of requests are being processed concurrently on one thread — `contextvars` (not thread-locals) is the standard-library mechanism that makes it possible.

SDE (Python) Interview Questions & Answers

Ready to practice these live?