Recompose
A native macOS OCR app that rebuilds the document, not just the text inside it.
- 88 tests
- 64 MoE experts · Top-6
- text · tables · diagrams
- no Python runtime
What it is
A macOS app that turns a scan or a photo of a document back into a document, not a text file. Tables stay tables, headings stay headings, and diagrams are carried over as image crops placed where they belong. The result exports to DOCX, to a rebuilt PDF, and to a searchable PDF.
Why it exists
Ordinary OCR solves recognition and stops there: what you get is a stream of lines. Then a person sits down and rebuilds by hand what the original already had — columns, tables, the order of blocks. On a document like a specification with tables and a diagram, that work takes longer than reading the document itself.
Recompose removes exactly that step. The model recognises page structure alongside characters, and the app assembles a file you can open and edit.
What I did
The full Unlimited-OCR inference in Swift and MLX, from scratch:
- SAM Vision Encoder — parsing the page image;
- CLIP-like Vision Encoder — the second visual path;
- Vision Fusion — merging both representations;
- MoE decoder — 64 experts, Top-6 routing.
Plus everything around the model: assembling the document from predictions, export to three formats, two surfaces — CLI and GUI — and 88 tests over the pipeline.
What’s technically interesting
The model runs inside the app on a native Metal runtime. There is no Python in the bundle: no interpreter, no torch, no separate local server the app talks to over a socket. That changes what the product is — you download an app instead of provisioning an environment.
The 64-expert MoE decoder with Top-6 routing was the hardest part of the port: token routing has to match the reference implementation, otherwise the drift accumulates through generation and breaks the document layout, not just individual characters.
What came out of it
An image of a document becomes an editable file: DOCX for editing, a rebuilt PDF for printing, a searchable PDF when the original look has to be preserved with a text layer added. Diagrams the model cannot express as text are inserted as image crops.
Release in preparation.