Why this project exists
Kimbundu is one of the major Bantu languages of Angola, spoken by millions of people. Despite that, it's barely documented in digital form. The best lexical references exist as scanned books or old PDFs that no search engine can read and no language tool can use.
I wanted to take one of the most complete surviving Kimbundu–Portuguese dictionaries, digitise it properly, and publish it as a structured, searchable public resource.
This is also personal. I'm of Angolan heritage, and Kimbundu matters to my family and community. As large language models increasingly shape how people find knowledge, languages that are not digitised risk becoming invisible in the systems that will define future access to culture and history.
The problem
The source material is a historical printed dictionary, hundreds of scanned pages in a dense two-column layout. Getting clean structured data out of it is hard:
- Two-column page layouts that need accurate segmentation before OCR can even start
- OCR noise from aged print, uneven scanning, and non-Latin diacritics
- Dense abbreviations, noun-class markers, and grammatical annotations crammed into compact entries
- Line, column, and page boundary issues that break naive text extraction
- Every entry needs to carry provenance back to its source page and position
The approach
I designed the pipeline as a series of explicit, auditable stages. Each step produces inspectable outputs, which made it possible to debug OCR issues, reconstruction errors, and editorial decisions without hiding uncertainty inside a single opaque process.
- 1Historical PDF
- 2↓Page extraction
- 3↓Column segmentation
- 4↓OCR capture
- 5↓Deterministic parsing
- 6↓Corpus reconstruction
- 7↓Cleanup
- 8↓Conservative LLM audit
- 9↓Editorial merge
- 10↓Public website dataset
What I built
The initial pipeline produced a structured lexical corpus: headwords with grammatical metadata, source-derived definitions, cross-references, and provenance tracking back to the original page, column, and line. A conservative AI auditing layer (Ollama and OpenAI) proposed corrections without overwriting source data, with all suggestions tracked and reviewable.
Since launching the site, I have continued expanding and modernising the corpus:
Orthography modernisation. The source dictionary uses a historical transcription that differs from the modern ILN standard adopted by Kimbundu translators today. I applied orthographic transformations informed by the modern standard: glide re-spelling (u→w and i→y) and Ri→Di prefix correction (over 600 entries, since Kimbundu has no /r/ phoneme). Over 1,500 entries now carry modernised spellings alongside their historical forms, with over 1,100 redirects so old URLs resolve to the new canonical pages.
Internationalisation. Every entry now carries Portuguese, English, and French definitions, over 10,600 entries across all three languages. The site UI, orthography page, and static content are all localised.
Search infrastructure. I extracted the search engine into its own HTTP service with a warm index, built an as-you-type smart search box with quality telemetry, and redesigned the homepage around discovery rather than browsing. Search handles orthographic variant expansion (w↔u, y↔i, c↔k) so both historical and modern spellings resolve correctly.
Corpus expansion. I built pipelines to acquire, OCR, and extract vocabulary from new source materials: Chatelain's 1889 Grammar and 1894 Folktales (50 tales with interlinear Kimbundu–English), the KmbPtMT curated dataset (over 18,000 pairs), and Oscar Ribas's Ilundo (2009).
Self-hosted deployment. The site moved from Vercel to a self-hosted Coolify instance on Hetzner, with Docker images built in GitHub Actions, pushed to GHCR, and auto-deployed on push to main. PostHog analytics track search failures and content gaps.
Results
Why it matters
This isn't just engineering for the sake of it. Languages that haven't been digitised don't show up in the tools people actually use to find information. AI trains on what's already digital. What isn't there won't be reflected.
High-quality digital cultural libraries are infrastructure. They give learners, researchers, and future language tools something reliable to build on, rather than forcing underdocumented languages to remain absent from the systems that increasingly mediate knowledge.
Kimbundu.org is a small but real contribution to that.
What comes next
The corpus is live and growing. Near-term priorities:
- Extract the remaining Chatelain vocabulary. The 1889 Grammar has a full numerals chapter (cardinals 1–10 with noun-class agreement tables, tens, hundreds, ordinals) ready for structured extraction, and the 1894 Folktales contain natural vocabulary in context
- Build morphological decomposition so a word page shows its internal structure: root, noun-class prefix, singular and plural forms, and agreement patterns. The noun class data is already in the corpus. The missing piece is the graph layer that surfaces these relationships in the UI
- Mine translated content (KmbPtMT and Chatelain folktales) for usage examples, so entries show real sentences alongside definitions
- Build a Kimbundu → English machine translation model using the aligned parallel texts as training data
- Add grammar resources: verb paradigms, noun-class reference, and usage patterns from the Chatelain grammar
- A modern Portuguese gloss layer, showing contemporary translations alongside the original early-20th-century definitions