Memory optimization

Memory optimization

Memory is spent in three ways a CPU profile hides: how each object is laid out, how many you allocate, and how much you hold at once. A program can be fast and still get OOM-killed because it buffers a whole file it could have streamed. Cut footprint by attacking layout, allocation rate, and retention, in that order.

Method

1. Measure the heap before touching layout. Get a real allocation profile: pprof for Go, tracemalloc or memory_profiler for Python, a heap snapshot in Chrome DevTools, jmap/jcmd for the JVM. Find which type owns the most live bytes and which call site allocates most. Optimize the top one, not a guess.
2. Shrink the hot object's layout. Reorder struct fields so same-size members sit together and padding collapses; a naive Go or C struct can waste near a third to alignment. Use a narrower type where the range allows (int32 over int64), and in Python give hot classes __slots__ to drop the per-instance __dict__.
3. Store many-of-a-thing column-wise. A million small objects each carry header and pointer overhead. A struct-of-arrays layout (parallel typed arrays, a NumPy array, an Arrow column) cuts per-element overhead to near zero and packs the working set into fewer cache lines.
4. Pool and reuse instead of churning. For short-lived objects allocated in a hot loop, reuse buffers: a sync.Pool in Go, a slice you reslice to [:0] and refill, a reused bytearray. Fewer allocations means fewer collections, and GC time often falls faster than the byte count.
5. Stream instead of buffering the whole payload. Reading a file or response into one string peaks at its full size; iterate line by line or in fixed chunks so peak memory is a window, not the total. Use generators, io readers, or a SAX-style parser rather than loading a full document into a DOM.
6. Cap what you retain. Bound caches and queues with an eviction policy (LRU with a max size) so retained memory has a ceiling. A cache with no bound is a slow leak that a heap profile eventually blames on the wrong code.

Checks

• Does peak resident memory stay flat as input grows, or track its size?
• Did the dominant type in the heap profile actually shrink after the change?
• Under load, has GC time or pause frequency dropped, not just live bytes?
• Does every long-lived cache or buffer carry an explicit size bound?

Boundaries

This reduces intended footprint by design. Hunting an unbounded climb from unintended retention is memory-leaks; reading the profile that finds the offender is profiling-memory. Collector pause tuning past allocation rate belongs to gc-tuning.