How Pagination Optimization Birthed Addictive Infinite Scroll

Offset-based pagination forces databases to scan and discard many rows, causing increasingly slow queries as users navigate deeper pages. This also requires a costly count query to calculate page numbers, which users rarely need beyond the first page.

Cursor-based pagination avoids this by using the last seen record's ID and a 'greater than' clause, leveraging indexes for fast retrieval. It eliminates the need for count queries and maintains consistent performance regardless of depth.

This optimization was meant for bounded content like friends' posts, but social platforms applied it to algorithmic feeds. They now cache multiple pages ahead, learn from early interactions, and tune subsequent queries to keep users scrolling indefinitely.

The lesson for engineering leaders: pagination design has profound performance and UX implications. Cursor pagination is efficient but must be paired with mindful product choices to avoid unintended addictive patterns.