GitHub describes how it optimized case folding in Blackbird, its code search engine indexing more than 180 million repositories and over 480 TB of source code. Case folding is performed before n-gram extraction and is also needed when locating potential query matches. The surprising result is that the ASCII fast path became faster after removing an optimization: instead of stopping at the first non-ASCII byte, a branchless scan sweeps the entire buffer. The post uses this case study to examine how branch behavior and memory throughput can dominate seemingly simple text-processing operations at very large scale.
No heat snapshots are available in the last 24 hours.