Darkgrade wants your camera to understand the world

There's a strange inversion sitting in millions of camera bags.

A $4,000 mirrorless camera has a sensor and lens that embarrass any phone. But the phone is programmable. Thousands of apps can open its camera, read frames, adjust exposure, and build products on top of it. The dedicated camera, the one with the genuinely great glass, is a sealed box. Software mostly can't touch it. If you've ever tried to control a Canon or Sony body from code, you know the drill: per-vendor SDKs, decades-old protocols, and community wrappers that work until they don't.

That gap is what Darkgrade is attacking. The company, founded by Kevin Schaich and part of Y Combinator's Fall 2024 batch, describes its mission in one line: a protocol for large language models to directly interface with image sensors. Its tagline is shorter still. Let your camera understand the world.

That's the destination. The first shipped product is deliberately more grounded: an open-source TypeScript SDK that connects to and controls Canon EOS, Sony Alpha, and Nikon Z cameras in two lines of code. Before an LLM can drive a camera, ordinary software has to be able to. Today, mostly, it can't.

The protocol everyone uses and nobody loves

Nearly every camera maker has shipped the same control protocol for about twenty years: PTP, the Picture Transfer Protocol, standardized as ISO 15740. It's how a camera talks to a computer over USB, and in extended form over Wi-Fi. In principle, PTP already lets software do most of what Darkgrade promises: change settings, trigger captures, pull files, stream live view.

In practice, PTP has been one of the least approachable corners of consumer hardware. The base standard covers only part of what any modern camera does, so each vendor piles proprietary extensions on top, and those extensions differ for Canon, Sony, and Nikon. The tooling that exists reflects that history. Camera control has traditionally meant wrestling with vendor desktop SDKs or C libraries born in the Linux world, far from where most application developers now work. The result is that an entire category of excellent hardware sits outside the modern software stack. Camera makers build for photographers pressing physical buttons, and the developer who wants to treat a camera as a programmable device has been on their own.

Darkgrade's SDK, published as @darkgrade/link, is a full TypeScript implementation of the PTP standard plus the vendor extensions that matter. The pitch to a developer is the whole product philosophy in miniature:

import { Camera } from '@darkgrade/link'

const camera = new Camera()
await camera.connect()
await camera.setIso('800')
const { data } = await camera.captureImage()

Connect, set ISO, capture. The library detects the brand automatically and loads the right vendor support, so the same code runs against a Sony body or a Canon one.

What the SDK actually covers

For an alpha-stage library, the hardware coverage is unusually concrete, because every claim is tied to specific bodies the code has been validated on.

On Sony Alpha, the SDK supports live view, video recording, and Sony's SDIO operations, tested against the α6700, α7 IV, and α7 V. On Nikon Z it handles live view and Nikon's extended properties, tested on the Z6 III. On Canon EOS R it covers remote control and event polling, tested on the EOS R6 Mark III. Cameras outside those lines fall back to a generic PTP implementation, which Darkgrade has exercised on hardware as unglamorous as a 2012 Olympus E-PL5.

The transport story matters as much as the brand list. USB is the default. There's also a PTP/IP transport, so a camera on Wi-Fi can be driven over the network from Node. And the library runs in the browser, which changes what a camera integration even is: a web page that connects to the mirrorless camera on your desk, with no installer and no driver disc.

The engineering choices read like a team that expects this code to be embedded everywhere. The whole library bundles to around 55 kB and is tree-shakable. Everything is typed. Controls are capability-aware, meaning the SDK only exposes the values a connected camera actually advertises, instead of letting you set an ISO the body doesn't have. Schaich is candid that the project is in alpha and the API will shift. The README reads less like marketing than like lab notes, down to documented limitations per camera and per mode.

Development happens in the open, and the openness is already paying off. The repository's recent pull requests include Sony camera-body button control and on-screen display capture, contributed from outside the company and verified live on an α6700 before merging. A hardware SDK where outside contributors show up with cameras and test against real bodies is a healthy early sign. This is the kind of project that gets better with every strange camera somebody plugs in.

TypeScript is the strategic choice, not a detail

It would have been natural to write this in C and ship bindings. Darkgrade wrote it in TypeScript, and that decision says a lot about who the SDK is for.

The people building AI applications today live overwhelmingly in the JavaScript and Python ecosystems. The agent frameworks, the LLM SDKs, the quick product experiments: a huge share of that work happens in TypeScript running on Node or in a browser. A camera library in that language drops into the same codebase as the model calls, with no native compilation step and no FFI layer to maintain.

Browser support compounds the effect. A capability-aware, 55 kB library that can reach a camera from a web page lowers the cost of trying something to nearly zero. Demos become links. The historical audience for camera SDKs was a small guild of imaging specialists. Darkgrade's implicit bet is that the next audience is every developer building software that needs eyes, and that audience should not have to learn 2004-era USB programming first.

The actual thesis: models should drive sensors, not just receive files

The SDK is useful on its own. The mission statement points somewhere more ambitious.

Today, when a language model "sees," it's handed an image someone else captured. The model has no say in exposure, focus, framing, or timing. It sits at the far end of a pipeline, consuming whatever JPEG arrives. Darkgrade's stated goal, a protocol for LLMs to interface with image sensors directly, inverts that relationship. A model that can set ISO, pull a live-view frame, evaluate it, adjust, and shoot again stops being a captioning service and starts being an agent in control of its own perception.

You can see the outline of what that unlocks. An inspection system that notices a frame is underexposed and fixes the exposure itself instead of flagging the image as unusable. A product photography rig where the operator describes the shot in plain language and software handles the bracketing. A camera that records video when something it understands, rather than a dumb motion heuristic, says the moment matters. In each case, the missing piece isn't model intelligence. Vision models are already good enough for most of these jobs. The missing piece is a clean, standard path between the model and the sensor.

There's a precedent for how this plays out. Language models became dramatically more useful when the Model Context Protocol gave them a standard way to reach tools and data, and an ecosystem formed around that standard almost immediately. Nothing comparable exists for cameras. PTP is the wire protocol nearly every dedicated camera of the past two decades already speaks, and until now there's been no implementation of it that modern AI software could simply import. Whoever builds that layer well sits in an interesting position as agents start to want eyes.

A solo founder with infrastructure habits

Darkgrade is one person, and the background fits the problem.

Kevin Schaich studied computer science at Cornell, then spent five years at Palantir between 2017 and 2022, working as an engineer and project lead deploying software into large, messy enterprise environments. After that came founding-engineer stints at early-stage startups. Palantir is a particular kind of training: the work is making serious software function inside systems that were never designed to cooperate with it. Reimplementing a two-decade-old ISO standard, vendor quirks included, and validating it body by body against real hardware is the same discipline aimed at a consumer protocol.

The structure of the company reflects the stage. Darkgrade is based in Denver, the SDK is open source, and the community gathers in a Discord where camera support grows roughly at the pace people show up with hardware. It's early. The library is in alpha, some Wi-Fi pairing flows are still unfinished, and the LLM protocol remains a mission statement rather than a spec. But the sequencing is right. The companies that end up owning a protocol layer usually start by shipping the unglamorous implementation everyone else gets to build on.

Cameras went two decades as appendages of the computer: storage devices with lenses, mounted like disks, polled for files. Darkgrade is working on the version where they become participants instead, hardware that software, and eventually models, can genuinely operate. The first step was never going to be a keynote. It was going to be somebody finally making camera.connect() just work.