Two Hard Things: caching bugs · Part 5
Your cache never caches "not found"
Postgres returns null, so you cache nothing, and a scraper walking millions of IDs that don't exist goes straight to your database. Fix: cache the absence with a short TTL, and put a bloom filter in front (about 1.2 MB for a million users) to say "definitely not here" first.
Comment below: Do you cache your 404s?
Two Hard Things: a series on the caching bugs you already ship, and how to fix them. "There are only two hard things in computer science: cache invalidation and naming things." (Phil Karlton) All code is real TypeScript on Node, typechecked before it went on screen, and every claim was reproduced against real Redis and Postgres.
Transcript
Your cache has a hole, and it's shaped like a scraper. getUser checks Redis. Miss. Postgres returns null. Nothing to cache, so you cache nothing. Now a scraper asks for user 1, 2, 3, all the way to 10 million. 9 million IDs that don't exist. Every single request misses. Every single one hits the database. Your hit rate is perfect for real users and zero for the attack. Fix. Cache the absence. Store not found as a value with a short TTL and delete it when that user is created. That stops repeats. To stop the walk, add a Bloom filter in front. About 1.2 megabytes for a million users. And it can say definitely not here before Redis or Postgres ever hear about it. Best practice, not found is an answer. Cache it. Next, the cache bug that shows one user another user's data. Part six. Do you cache your 404s?
