Hardware ceilings, not hotspots, should drive your optimization strategy

Casey Muratori challenges the conventional approach to software optimization. He argues that most engineers rely on profiling to find hotspots, which only leads to local improvements. The correct method starts with understanding the theoretical maximum throughput of the hardware and then measuring the delta from that ceiling. This bottom-up approach reveals systematic inefficiencies that profiling

cannot catch. A key architectural sin Muratori identifies is the creation of false serial dependency chains. When code is structured as a series of dependent network calls or blocking operations, no amount of hotspot optimization can fix it. The performance ceiling is determined by this longest serial chain, and fixing it often requires a complete

rewrite. Teams must design architectures upfront that allow for future optimization, or they are "rolling the dice." Muratori emphasizes that reading assembly language is essential, not for writing it, but for understanding what the CPU actually executes. This knowledge empowers engineers to make informed architectural decisions about data layout, access patterns, and language choice. He

cites the example of Python's overhead for a simple A+B operation, which requires hundreds of instructions compared to a single 'add' in C, explaining why library calls or compiled extensions are necessary for performance. Finally, Muratori draws a parallel between the rise of accessible game engines and the current state of AI coding tools. While