Ember Robotics tells you why your robot's camera went dark

Ryan Bednar8 min read
Ember Robotics tells you why your robot's camera went dark

Ember Robotics tells you why your robot's camera went dark

Somewhere right now, a warehouse robot is driving around half blind and nobody knows it yet.

Its camera didn't die. That would be easy. Instead it's dropping every fifth frame, or its timestamps have drifted out of sync with the LiDAR, or a connector has rattled loose just enough to corrupt the stream intermittently. The perception model keeps running. The robot keeps moving. The failures show up later, downstream, as a mispicked item or a near miss, and the engineer assigned to figure out why gets a ticket that says the robot "acted weird" on Tuesday.

What happens next is the part the robotics industry doesn't put in its demo videos. Someone SSHes into the robot, if it's reachable. They pull logs by hand and try to reconstruct Tuesday from whatever the robot happened to keep. If the machine is in a warehouse three states away with bad Wi-Fi, someone may end up on a plane with a laptop. Teams that run web servers solved this class of problem fifteen years ago. Teams that run robots are mostly still living it.

Ember Robotics exists to close that gap. The San Francisco startup, founded in 2024 and part of Y Combinator's Summer 2024 batch, describes itself as Datadog for robots: an observability platform that watches the health of a robot's sensors and systems from inside the robot itself, built by two engineers who came out of Tesla Autopilot.

Why Datadog doesn't work on a robot

The obvious question is why robotics teams don't simply use the monitoring stack the rest of software uses. Datadog, Grafana, and their peers are mature, battle tested, and everywhere. The answer is that the standard observability playbook quietly assumes things that are true in a data center and false on a robot.

It assumes the machine has a reliable network connection, because the whole model is built on streaming metrics to the cloud in near real time. A robot on warehouse Wi-Fi, or on a cellular link in a field, loses connectivity routinely, and it tends to lose it in exactly the moments and places where failures happen. A monitoring tool that only works when the robot can phone home misses the data that matters most.

It assumes telemetry is small. Server monitoring ships numbers: request counts, latencies, memory curves. A robot's most important signal is its sensor data, and sensor data is enormous. Multiple camera streams, LiDAR point clouds, and IMU readings add up fast. Shipping all of it to the cloud over a spotty link is a non-starter, so most teams ship almost none of it, and then have nothing to look at when something breaks.

And it assumes someone else already built the integrations. Datadog works out of the box because decades of server software standardized around a handful of interfaces. Robotics has ROS, a long tail of sensor vendors, and a lot of duct tape. Every robotics company ends up assigning engineers to build an internal logging-and-diagnostics system, which is a distraction from the actual robot and is usually about as good as you'd expect a side project to be.

So the state of the art at many robotics companies, including well funded ones, is some mix of rsync, grep, and airplane tickets.

Start with the camera

Ember's launch wedge is deliberately narrow: camera diagnostics. The company's launch pitch was a framework that ties together real-time data, logs, and reports to contextualize a robot's vision health, with a stated goal of reducing vision failures at the source.

Cameras are a shrewd place to begin, because vision is where modern robots are most dependent and most fragile at the same time. The current generation of autonomy is overwhelmingly camera driven, and a camera is a physical object living a hard life: vibration, heat, dust, cables that flex thousands of times a day. Its failure modes are rarely clean. Disconnections, dropped frames, and time-sync drift are the canonical three, and Ember tracks all of them out of the box.

What makes these failures expensive is that they're silent. A perception model fed a degraded stream doesn't raise an exception. It produces slightly worse answers, and the robot's behavior decays in ways that look like a software bug, a model regression, or operator error. Teams can burn weeks hunting a "planning issue" that was actually a camera quietly losing frames. Catching the sensor fault at the moment it happens, with the context attached, collapses that whole search.

The platform has already grown past its wedge. Ember lists real-time health analytics for cameras, IMU, LiDAR, CPU, and memory, with off-the-shelf support for common sensor makers like Intel RealSense, Luxonis, and Orbbec, plus ROS integrations and support for the compute platforms robots actually run on, Jetson and Raspberry Pi among them.

An agent that lives on the robot

The architecture is where Ember's answer to the connectivity problem shows up, and it's the inverse of the cloud-first model.

Teams install an on-device agent, packaged as a plain Linux .deb, on each robot. The agent records telemetry, sensor streams, and failure events locally, on the machine, where the data is generated. It doesn't need a network connection to do its job. When connectivity returns, it syncs to a cloud dashboard where the team gets fleet-wide visibility and real-time alerts when something is wrong. Because the agent keeps a history of each sensor's behavior over time, the same data supports predictive maintenance: a camera that has started dropping an occasional frame this week is a repair ticket, not a mystery, before it becomes next month's field failure.

The local-first design sounds like a detail. It's the product. It means the robot that lost Wi-Fi at 2pm and started dropping frames at 2:03 still has a complete record of what happened, rather than a gap in a cloud chart exactly where the incident lives. Smart caching keeps the on-robot footprint sane, which matters on embedded compute where every CPU cycle is contested.

It also means the economics work. Instead of streaming raw sensor data continuously, the system can be selective about what leaves the robot and when. That's the difference between observability that scales across a fleet and a bandwidth bill that doesn't. Ember's own framing is that the platform grows reliably from 1 robot to 10,000 with consistent, structured data collection, which is the Datadog promise restated for machines that move.

Built by the people who had the pager

Ember was founded by Shivani Mouleeswaran and Ritika Shrivastava, and their backgrounds read like a job description for this exact company.

Both worked on Tesla Autopilot, about as demanding an environment for camera-based autonomy as exists. Mouleeswaran, the CEO, also built software at Simbe Robotics, whose inventory robots roam grocery store aisles all day. Companies like that live and die by whether machines keep working in messy real-world buildings, far from the people who wrote the code. Shrivastava, the CTO, holds a B.S. and an M.S. in EECS from UC Berkeley, where she taught computer vision and robotics before working as a computer vision engineer on Autopilot.

That history explains the product's shape. Engineers who have worked on deployed vision systems learn that the failure usually isn't the model. It's the frame that never arrived, the timestamp that lied, the sensor that degraded for a week before anyone noticed. Ember is built camera-first because cameras are where its founders spent their careers watching things go wrong.

The two went through YC as a two-person team and raised a pre-seed round backed by Y Combinator and Lombardstreet Ventures. It's early. The bet is that early is exactly the right time, and the reason is what's happening around them.

The deployment decade

Robotics is in the middle of a shift from demos to fleets. Warehouse automation is now ordinary. Delivery robots, inspection robots, agricultural robots, and humanoid pilots are moving from videos into purchase orders. Every one of those programs hits the same wall at the same point: the robot that worked in the lab has to keep working, unattended, at a customer site, and someone has to be accountable for its uptime.

That transition is precisely when observability markets get built. Datadog wasn't founded in the mainframe era; it rode the cloud wave, because the cloud created millions of machines that someone had to monitor and nobody wanted to monitor by hand. If robot fleets follow the same curve, the infrastructure for keeping them observable gets built now, by whoever understands the problem best, and sold to everyone else who would rather build robots than build logging.

The alternative future is the status quo: every robotics company maintains its own half-finished internal diagnostics stack, forever. Having watched how that movie ends in web infrastructure, it's hard to bet on it.

A robot fleet nobody can observe is a pilot program that hasn't failed somewhere inconvenient yet. Ember Robotics is building for the companies that have already learned that, usually at 2am, usually three states from the robot.

Related Posts