“Stainless has been an incredible partner every step of the way.”
How Replicate turned their SDKs from a liability into a DX advantage
Stainless' tooling is intuitive, well-documented, and automated, which is essential when you're managing SDKs across multiple languages.
Replicate lets you run AI models with a cloud API, without having to understand machine learning or manage your own infrastructure. You can run state-of-the-art commercial models from companies like Google, OpenAI, Anthropic, Runway, and Ideogram, as well as open-source models created by the community. Replicate also lets you use open-source tools, train custom models, or build and publish custom models from scratch. And all of this is exposed through a fast-evolving HTTP API.
Over time, Replicate's SDKs began to drift from their underlying HTTP API. As new features shipped, the client libraries had to be updated manually, leading to gaps like missing methods, inconsistent behaviors, or incomplete support for key abstractions like the replicate.run() method, which wraps complex prediction logic into a simple interface.
For a company where the API is the product, this was becoming a liability for providing an exceptional developer experience. New users encountered friction when getting started. Docs and SDKs were out of sync and issues were piling up in open-source repos, creating a perception that Replicate wasn’t actively maintaining its tooling.
Replicate’s team wanted to reverse course. Their goals:
- Keep SDKs reliably in sync with the HTTP API
- Reduce friction and time-to-value for users integrating the SDKs
- Use OpenAPI as the single source of truth across implementation, testing, documentation, and client generation
- Make maintenance easy and intuitive for new engineers joining the team
“We wanted our engineers to spend time building new APIs to help other businesses.”
Replicate already had SDKs for a handful of languages. Some were written in-house, some community-maintained. But over time, these client libraries fell out of sync with the underlying API for the following reasons:
- No dedicated owner for each language
- Bandwidth issues made it difficult to address feedback from the open-source community
- Changes to the underlying HTTP API required manual changes to all client libraries with zero automation
The team was painfully aware that unmaintained libraries were leaving their users in the cold. Replicate needed a reliable way to generate, test, and ship SDKs that would stay aligned with their API without pulling more engineering resources away from core product work.
To solve this, they kicked off a head-to-head evaluation of Stainless and Fern. Zeke Sikelianos, Replicate’s founding designer and the co-creator of Swagger (now known as OpenAPI), led the efforts.
Because Replicate’s SDKs and docs are generated from their OpenAPI spec, Zeke brought incredible depth to the evaluation. He understands how API structure shapes downstream tooling: how clients are generated and typed, how changes are diffed, tested, and versioned. His involvement pushed the evaluation beyond surface-level DX into the details of codegen pipelines, CI integration, and long-term maintainability. The final decision carried the weight of someone who didn’t just use the spec but helped define it.
Zeke used the following criteria to evaluate each platform:
- What is it like to collaborate with the provider?
- What is the developer experience like for Replicate users consuming the client libraries?
- What is the developer experience like for the Replicate team maintaining the libraries?
- How well is the tool documented?
- How easy is it to migrate from current SDKs to better ones?
- What features and automation does the tool offer?
The table below is the exact evaluation Zeke used to score both providers. The only edits were removing expired or private links, which are now shown in blue.
"Stainless has been incredibly responsive and thoughtful—they showed up with a full team on day one and helped us move fast.”
Zeke’s summary of what made Stainless stand out:
- Collaboration: “Within the first week, eight different team members joined Zoom calls to help unblock development. They’ve also been highly engaged on Slack.”
- Stainless Studio: The web UI is “intuitive, powerful, and well documented.”
- First-class support for TypeScript, Python, Go, and other languages.
- Diagnostics tooling: Effective for pinpointing issues in the API spec.
- JSON documentation output: Every build generates structured JSON docs and code samples, which can be used for custom front-end rather than relying on off-the-shelf docs generators.
- Automated workflows: Everything is automated from PR nudges when reviews are pending to changelog and release publishing.
- CLI generation: “They’re building a CLI generator on top of the Go SDK, which means we could easily revive a
replicateCLI.” - MCP server support: Each NPM release can automatically include a matching MCP server.
- Terraform provider generation: Built-in.
- Ecosystem alignment: “Stainless maintains SDKs for companies like OpenAI and Anthropic. We benefit from any improvements made upstream.”
Zeke shared additional notes on things that stood out about Stainless’ dev practices:
- Scripts to rule them all: Each repo has a
scripts/directory with setup scripts for bootstrapping any project, regardless of language. This lowers the barrier to entry for new contributors. - Conventional commits: They follow the spec for semantically meaningful commit messages (e.g.
feat,fix). This enables automated semantic versioning and changelog generation. - Safe, automated releases: Release PRs accumulate changes and require human review before publishing.
Beyond SDKs, Replicate is also working with Stainless to ship a more efficient MCP server.
One challenge they’re addressing is how to reduce the size of large API responses so they fit within LLM context windows. Instead of modifying the public API, the teams collaborated on a dynamic filtering layer—using jq expressions—to automatically slim down response payloads inside the MCP server.
This lets the LLM request only the fields it needs, like name, owner, or latest_version.id, without requiring changes to Replicate's underlying HTTP API. Replicate is already testing this approach in production and planning to make it the default experience for AI agents calling their MCP tools. It’s a promising pattern for other teams looking to expose real-world APIs to LLMs without bloating context or compromising on compatibility.
Replicate chose Stainless for the responsive support they received, intuitive tooling, and strong automation around CI/CD. With Stainless Studio, their team can efficiently iterate on their OpenAPI spec, catch issues earlier, and ship updated SDKs faster with far less manual effort.
What used to be a liability now helps Replicate and their users move faster—from evaluating models to building full creative workflows, all powered by a stable and up-to-date SDK layer.
If you think Stainless can help you with your SDK maintenance, you can give it a try by uploading an OpenAPI spec.
"Once we started fixing the OpenAPI spec, we uncovered 14 different bugs in our API. It’s funny—until you tighten up your docs, you don’t always notice the quirks in your API design. Treating the spec as a source of truth became a forcing function to clean house.”