Summary:
I write a question on my reMarkable 2, tap the page three times, and a few seconds later the answer appears right underneath it — in handwriting, in the language I wrote in. It runs on the tablet itself and uses Claude to read the page. The starting point was an open-source project, Smart Remarkable (a fork of awwaiid's ghostwriter). It didn't work on the newest tablet software, so I fixed it and reshaped it with Claude until it worked the way I think.
The ask
A paper-like notebook that can answer back. No laptop, no app, no copying text: just write a question with a question mark, give a small signal, and get an answer on the same page. It had to fit how I actually write — messy handwriting, Dutch and English mixed, notes all over the page — and it should never scribble over my own notes.
How it was built
I found the project on GitHub and asked Claude what it was and whether it could install it. From there it was a conversation: I tested on the tablet, sent back what I saw, and Claude changed the code. These are the steps that mattered:
- Building it without the usual tools. The project expects Docker to compile for the tablet's ARM processor. Claude built it a different way (with the Zig cross-compiler, targeting an older system library so it runs on the tablet) and packed it with a one-click install script for my Mac. The API key stays on the tablet only.
- The answer was always "Could not read the selection". The app reads the screen straight from the tablet's memory, and on software 3.28 that memory had moved. I ran a test that saved what the app actually "saw": a band of noise on top and the page shifted halfway. From that one screenshot Claude calculated exactly how far the image had moved — 2,064,384 bytes, precisely 504 memory pages — and patched it. After that the screen came through perfectly.
- Three taps instead of five. The original asks you to tap a corner, then two corners around the question, then two corners for the answer. I wanted it simpler: tap three times next to the question. The app now tells Claude where you tapped, Claude finds the nearest handwritten question, returns where it sits on the page, and the answer is written directly below it.
- Palm rejection. The first version also fired while I was just writing — a resting hand touches the screen constantly. Now only three short, single-finger taps in quick succession count.
- Re-measuring the toolbar. The first handwritten answers came out huge, half in highlighter, and over my own notes. The toolbar had moved in the new software, so the app sometimes picked the wrong pen; and it enlarged the handwriting 2.5×. Claude measured the new toolbar positions from a screenshot, stopped the upscaling and aligned answers to the left under the question.
- Answer in my language. It answered in English at first. Now it detects the language of the handwritten question and answers in that — Dutch when it can't tell.
- Kept on GitHub. The changes live in my own private repository on top of the original code, so the original author stays credited and I can roll back any step.
The result
A notebook that answers. I write "how many days does February have?", tap three times, and the answer appears below in handwriting I can move or erase like my own. It's still a hack — after a reMarkable update it needs one command to switch back on — but it changed how I use the tablet: questions I'd normally look up later now get answered on the page, while I'm still thinking.


0 Comments