Dip — Give LLMs Eyes
A tool that gives LLMs real-time visual context of your screen, so you never have to describe what's on it.
PythonGoogle Gemini APIScreen Capture
Dip streams real-time screen context into an LLM: it captures what's on your screen and feeds it to the model, so describing on-screen content by hand becomes unnecessary. A lightweight client captures the screen and a Python backend pipes it into Gemini for pair-programming and debugging workflows.
- ▪1st Place, Google DeepMind Gemini Track
- ▪400+ participants at the Cursor AI Hackathon Hamburg
- ▪Real-time screen capture as live LLM context
Learned how to feed multimodal models with real-time image data while keeping latency low, working with screen-capture APIs and the Gemini Vision API.