How do you fit a 250kB dictionary in 64kB of RAM and still perform fast lookups? For reference, even with modern compression techniques like gzip -9, you can't compress this file below 85kB. In the 1970s, Douglas McIlroy faced this exact challenge while implementing the spell checker for Unix at AT&T. The constraints of the PDP-11 computer meant the entire dictionary needed to fit in just 64kB of
// Load a text file. let mut text = ropey::Rope::from_reader( File::open("my_great_book.txt")? )?; // Print the 516th line (zero-indexed). println!("{}", text.line(515)); // Get the start/end char indices of the line. let start_idx = text.line_to_char(515); let end_idx = text.line_to_char(516); // Remove the line... text.remove(start_idx..end_idx); // ...and replace it with something better. text.
I found out about pool allocators some time ago, and I really liked its simplicity and high performance, so I decided to write my own. This article was initially inspired by Dmitry Soshnikov’s article. It’s also worth mentioning that this article was discussed in Hacker News. Similarly to malloc, a pool allocator allows the user to allocate memory at run time. The pool allocator, however, is much
1 Introduction1.1 Problem statement1.2 Motivation1.3 Recommended reading1.4 Binary search and Eytzinger layout1.5 Hugepages1.6 A note on benchmarking1.7 Cache lines1.8 S-trees and B-trees2 Optimizing find2.1 Linear2.2 Auto-vectorization2.3 Trailing zeros2.4 Popcount2.5 Manual SIMD3 Optimizing the search3.1 Batching3.2 Prefetching3.3 Pointer arithmetic3.3.1 Up-front splat3.3.2 Byte-based pointers3.
This years’ Advent of Code has loads of 2D grids, and makes you traverse them all to find paths of various kinds. At some point I had to implement Dijkstra’s algorithm, in Haskell. In trying to make my implementation reusable for the following days, I realized that Dijkstra’s algorithm is actually way more general than I remembered (or was taught)! In short, weights don’t have to be real-valued! I
2002年から約10年 IIJ技術研究所長. 年を取ってからは古い計算機や昔の計算法に興味が増し, シミュレーターを作ってそのプログラムを書いたり. 近頃はKnuthのTAOCPにあった問題のプログラムなどに挑戦したりしている. 【IIJ 2024 TECHアドベントカレンダー 12/3の記事です】 人生の長い間に亘ってプログラムを書いてきた. 最初は大学院生の時, 記憶装置のない加減乗算だけの計算機を作り, 紙テープから数値と計算の命令を読み入れ, 中間結果は一旦紙テープに書き出し, それをまた読み入れて計算を続けるという代物だったが, 初期のプログラミングの楽しさはあった. やがて研究室で, 今から見るとままごとのような小規模の計算機を組立てると, 私はライブラリプログラムの整備に熱中した. 大学で講義する身分になると, 次々と多くの計算機, 多くのプログラミング言語と付き合うことにな
Nov 19, 2024 BM25, or Best Match 25, is a widely used algorithm for full text search. It is the default in Lucene/Elasticsearch and SQLite, among others. Recently, it has become common to combine full text search and vector similarity search into "hybrid search". I wanted to understand how full text search works, and specifically BM25, so here is my attempt at understanding by re-explaining. Motiv
Hash functions are incredibly neat mathematical objects. They can map arbitrary data to a small fixed-size output domain such that the mapping is deterministic, yet appears to be random. This “deterministic randomness” is incredibly useful for a variety of purposes, such as hash tables, checksums, monte carlo algorithms, communication-less distributed algorithms, etc, the list goes on. In this art
I recently released papaya, a fast and feature-complete concurrent hash table for Rust. In this post I want to dive into the design and research that went into creating it, as well as why you might consider using it over existing solutions. If you're looking for an overview of papaya, you might find it more useful to consult the documentation. Philosophy Concurrent hash tables are a well explored
(This is part of a series on the design of a language. See the list of posts here.) Many 'scripting' languages use a hashmap for their default associative data-structure (javascript objects, python dicts, etc). Hashtables have a lot of annoying properties: Vulnerable to hash flooding. If protected against hash flooding by random seeds, the iteration order becomes non-deterministic which is annoyin
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く