RE: LeoThread 2025-10-22 02-12

[theycallmejp]

When asked to read a paragraph of non-contiguous words, such as a bitcoin mnemonic, every word must be tokenized to be processed.

Comments

[theycallmejp]

During skim-reading, eyes move in saccades and the brain tokenizes chunks of words together; speed-reading lowers recall accuracy while preserving general comprehension.

[theycallmejp]

This shifts processing from "word tokens" to "idea tokens" — the source of the roughly 10x speed-up

[theycallmejp]

DeepSeek-OCR is presented as an engineering achievement.
It’s been suspected that VLM/OCR models can be much smaller; pre-VLM state-of-the-art OCR (e.g., Google Cloud OCR) was likely under ~100M parameters.

[theycallmejp]

Recently, small open-weight models have matched or surpassed closed-source options on many benchmarks.
Notably, dots.ocr’s 1.7B model often outperforms OpenAI/Anthropic and sometimes Gemini, at a fraction of the cost.

[theycallmejp]

OCR is largely a pattern-recognition task that requires little internal memory, which explains why a lightweight 12-layer architecture can work well.

[theycallmejp]

DeepSeek-OCR advances two areas: a "small" mixture-of-experts approach with only ~500M active parameters to process large batches, and aggressive encoding with semantic pooling.

[theycallmejp]

This resembles tokenizerless models where the encoder composes signals into higher-level units, speeding up inference alongside other optimizations.

[theycallmejp]

However, the model is unlikely to radically shift OCR research yet: the training pipeline used synthetic/emulation data with limited diversity, especially for graph rendering.

[theycallmejp]

More technical development will be needed, but DeepSeek-OCR could serve as a foundational OCR model that nails the inference/performance trade-off while still requiring custom annotation and processes for large-scale use