How Replicate turned their SDKs from a liability into a DX advantage
TL;DR: Replicate’s first client libraries were written by hand, but became a maintenance burden as their HTTP API grew and evolved, requiring manual updates and cumbersome release processes. The team wanted to transition to using an SDK generator, so they ran a thorough bake-off of Stainless against another provider.
This is a summarized recording Zeke shared internally, following his evaluation of Stainless and Fern.
Replicate lets you run AI models with a cloud API, without having to understand machine learning or manage your own infrastructure. You can run state-of-the-art commercial models from companies like Google, OpenAI, Anthropic, Runway, and Ideogram, as well as open-source models created by the community. Replicate also lets you use open-source tools, train custom models, or build and publish custom models from scratch. And all of this is exposed through a fast-evolving HTTP API.Over time, Replicate's SDKs began to drift from their underlying HTTP API. As new features shipped, the client libraries had to be updated manually, leading to gaps like missing methods, inconsistent behaviors, or incomplete support for key abstractions like the replicate.run() method, which wraps complex prediction logic into a simple interface.For a company where the API is the product, this was becoming a liability for providing an exceptional developer experience. New users encountered friction when getting started. Docs and SDKs were out of sync and issues were piling up in open-source repos, creating a perception that Replicate wasn’t actively maintaining its tooling.
Replicate’s team wanted to reverse course. Their goals:
Keep SDKs reliably in sync with the HTTP API
Reduce friction and time-to-value for users integrating the SDKs
Use OpenAPI as the single source of truth across implementation, testing, documentation, and client generation
Make maintenance easy and intuitive for new engineers joining the team
The challenge
“We wanted our engineers to spend time building new APIs to help other businesses.”
Replicate already had SDKs for a handful of languages. Some were written in-house, some community-maintained. But over time, these client libraries fell out of sync with the underlying API for the following reasons:
No dedicated owner for each language
Bandwidth issues made it difficult to address feedback from the open-source community
Changes to the underlying HTTP API required manual changes to all client libraries with zero automation
The team was painfully aware that unmaintained libraries were leaving their users in the cold. Replicate needed a reliable way to generate, test, and ship SDKs that would stay aligned with their API without pulling more engineering resources away from core product work.
To solve this, they kicked off a head-to-head evaluation of Stainless and Fern. Zeke Sikelianos, Replicate’s founding designer and the co-creator of Swagger (now known as OpenAPI), led the efforts.
Because Replicate’s SDKs and docs are generated from their OpenAPI spec, Zeke brought incredible depth to the evaluation. He understands how API structure shapes downstream tooling: how clients are generated and typed, how changes are diffed, tested, and versioned. His involvement pushed the evaluation beyond surface-level DX into the details of codegen pipelines, CI integration, and long-term maintainability. The final decision carried the weight of someone who didn’t just use the spec but helped define it.
The “bake-off” between Stainless and Fern
Zeke used the following criteria to evaluate each platform:
What is it like to collaborate with the provider?
What is the developer experience like for Replicate users consuming the client libraries?
What is the developer experience like for the Replicate team maintaining the libraries?
How well is the tool documented?How easy is it to migrate from current SDKs to better ones?
What features and automation does the tool offer?
The table below is the exact evaluation Zeke used to score both providers. The only edits were removing expired or private links, which are now shown in orange.
Feature | Fern | Stainless |
|---|---|---|
Low-level API - for making custom requests e.g. to undocumented APIs | 🔴 Not supported by default. They implemented a custom wrapper around native | 🟢 Supported by default for all clients. See Python. |
Mid-level API - generated methods that map to OpenAPI operations like replicate.models.get | 🟢 Supported. See TypeScript. | 🟢 Supported. See TypeScript. |
High-level API (replicate.run) | 🟢 POC | 🟢 POC |
Custom code | 🟢 Explicit and flexible: | 🟡 Convention-based and slightly magical: Put custom code in a |
Tests/ CI | 🟢 Actions workflows | 🟢 Actions workflows and Stainless Studio automation |
README | https://github.com/fern-demo/replicate-typescript | https://github.com/stainless-sdks/replicate-client-python |
Fetching OpenAPI updates | GitHub Action | Automatic hourly pulls via Stainless Studio. They also have a GitHub Action if we want more control. |
Docs GUI - useful but not really needed since we want to render our own docs (see JSON data) | 🟢 Very fancy | |
JSON docs | 🟡 Not available | 🟢 Yep! They generate a doctored-up version of the OpenAPI schema on every build, which includes They're also working on some open-source React components for rendering these docs, but that's still WIP. |
Example code snippets | They have an API for fetching snippets, and a way to package pre-baked snippets in an npm package | Included on every build. Example |
Versioning | 🔴 Choose your own adventure | 🟢 Conventional Commits™, SemVer, and Release Please™ Trick: You can use labels on PRs to override or influence versioning. |
Releases | 🟡 Choose your own adventure. | 🟢 Semantic PRs with Release Please - changes accumulate on a self-versioning release branch with changelog, etc. Humans press a button when we're ready to cut a release. Example. |
Pagination | Paid feature: https://buildwithfern.com/learn/sdks/capabilities/auto-pagination | List methods are paginated automatically. https://github.com/replicate/replicate-python-stainless?tab=readme-ov-file#pagination |
Staging clients | Build locally | |
Error handling | https://github.com/fern-demo/replicate-typescript?tab=readme-ov-file#error-handling There's not yet a way to inspect request bodies when errors occur. | https://github.com/stainless-sdks/replicate-client-python?tab=readme-ov-file#handling-errors |
MCP server generation | https://www.stainless.com/docs/guides/generate-mcp-server-from-openapi | |
CLI generation | Not supported today, but it's on the roadmap. | WIP - depends on a Golang SDK binary. |
llms.txt generation | Gotta buy Fern Docs product…? | |
Intellisense for individual models | POC: https://github.com/fern-demo/replicate-run-demo | |
Terraform/OpenTOFU provider | "coming soon" | https://app.stainless.com/docs/guides/getting-started-with-terraform |
Why Stainless won
"Stainless has been incredibly responsive and thoughtful—they showed up with a full team on day one and helped us move fast.”
Zeke’s summary of what made Stainless stand out:
Collaboration: “Within the first week, eight different team members joined Zoom calls to help unblock development. They’ve also been highly engaged on Slack.”
Stainless Studio: The web UI is “intuitive, powerful, and well documented.”
First-class support for TypeScript, Python, Go, and other languages.
Diagnostics tooling: Effective for pinpointing issues in the API spec.
JSON documentation output: Every build generates structured JSON docs and code samples, which can be used for custom front-end rather than relying on off-the-shelf docs generators.
Automated workflows: Everything is automated from PR nudges when reviews are pending to changelog and release publishing.
CLI generation: “They’re building a CLI generator on top of the Go SDK, which means we could easily revive a
replicateCLI.”MCP server support: Each npm release can automatically include a matching MCP server.
Terraform provider generation: Built-in.
Ecosystem alignment: “Stainless maintains SDKs for companies like OpenAI and Anthropic. We benefit from any improvements made upstream.”
Zeke shared additional notes on things that stood out about Stainless’ dev practices:
Scripts to rule them all: Each repo has a
scripts/directory with setup scripts for bootstrapping any project, regardless of language. This lowers the barrier to entry for new contributors.Conventional commits: They follow the spec for semantically meaningful commit messages (e.g.
feat,fix). This enables automated semantic versioning and changelog generation.Safe, automated releases: Release PRs accumulate changes and require human review before publishing.
Optimizing MCP for LLM context limits
Beyond SDKs, Replicate also worked with Stainless to ship a more efficient MCP server.
One challenge they addressed is reducing the size of large API responses so they fit within LLM context windows. Instead of modifying the public API, the teams collaborated on a dynamic filtering layer—using jq expressions—to automatically slim down response payloads inside the MCP server.
This lets the LLM request only the fields it needs, like name, owner, or latest_version.id, without requiring changes to Replicate's underlying HTTP API. It’s a promising pattern for other teams looking to expose real-world APIs to LLMs without bloating context or compromising on compatibility.
The next chapter for Replicate
Replicate chose Stainless for the responsive support they received, intuitive tooling, and strong automation around CI/CD.
With Stainless Studio, their team can efficiently iterate on their OpenAPI spec, catch issues earlier, and ship updated SDKs faster with far less manual effort.What used to be a liability now helps Replicate and their users move faster—from evaluating models to building full creative workflows, all powered by a stable and up-to-date SDK layer.
Their TypeScript and Python SDKs are now available as alpha.
If you think Stainless can help you with your SDK maintenance, you can give it a try by uploading an OpenAPI spec.