PixelRAG: A New Way to Search the Web
PixelRAG takes a simple idea and turns it into a interesting new shift in retrieval. Instead of parsing a page into text, it renders the page as screenshots and searches the images directly. This keeps tables, charts, diagrams, layout and visual structure intact. Traditional RAG pipelines lose these details when they strip a page down to text.
PixelRAG uses a vision‑language embedding model fine‑tuned on screenshots. It breaks a page into tiles, embeds each tile and builds a visual index. You can query the hosted index of more than eight million Wikipedia pages with text or images. The system returns the tiles that contain the answer, even when the answer is inside a table or chart.
PixelRAG also ships a screenshot tool called pixelshot. Claude can use it through the pixelbrowse plugin to read pages visually. Instead of fetching HTML, Claude looks at the rendered page and interprets the content the way a person would.
![]()
The project includes tools for rendering, embedding, indexing and serving your own visual search pipeline. You can build a local index, serve it through a FastAPI service or use Qdrant for large‑scale vector storage.
PixelRAG shows what happens when retrieval stops relying on text and starts using the full visual structure of a document. It is a practical step toward search systems that understand pages the way humans see them.
![]()
Author: David G. Smith • Source: https://davidgsmith.net/thoughts/pixelrag-a-new-way-to-search-the-web.html
Related Thoughts
Perspectives sharing related architectures, models, and domain context.
Why Vector Search Breaks Down When You Need Real Reasoning
There is a growing realization in the AI engineering world that vector search is hitting a ceiling. It still works well...
GeoAI Imagery Data Pipelines
The Earth observation community is living through a golden age of AI. Today, it's easier than ever to download a...
Why the Idea of a "Knowledge Graph Lakehouse" Has Me Thinking About Environmental Data
We've been working toward overhauling and reinventing our enterprise data platform, and this has me researching lots of...