Generate a Terraform Provider

Guides

This guide describes how to generate a Terraform provider for your API. Terraform is a declarative language for configuring or provisioning resources, commonly used by cloud infrastructure providers. You can also use it for any type of API where you want to configure static resources and check the configuration into source control.

Example use cases:

Cloud infrastructure provisioning
Logging and monitoring products
Cloud automation
Billing rules
Security workflows
Web services

Considerations

Terraform providers require more maintenance than our other SDKs and impose some additional constraints on your API. The following section can help you determine whether setting up a Terraform provider for your API makes sense.

A CRUD-like API

The Terraform provider works best for APIs where the endpoints on a given resource are uniform — the create, update, and read requests and responses all have the same shape and field names, and conceptually “set” all the properties in a straightforward way.

For example, a CRUD-like API for a resource called Product in an e-commerce application might look like this:

post /products Create a product, and return the product (with ID) in the response
get /products List the products (potentially filtering with query params)
put /products/{product_id} Update the product fields and return the product
get /products/{product_id} Return the product given the ID
delete /products/{product_id} Delete the product given the ID

In OpenAPI terms, it might look something like this:

paths:
  /products:
    get:
      responses:
        '200':
          $ref: '#/components/schemas/ProductListResponse'

    post:
      requestBody:
        $ref: '#/components/requestBodies/ProductRequest'
      responses:
        '200':
          $ref: '#/components/schemas/ProductResponse'

  /products/{product_id}:
    get:
      parameters:
        - $ref: '#/components/parameters/ProductID'
      responses:
        '200':
          $ref: '#/components/schemas/ProductResponse'

    put:
      parameters:
        - $ref: '#/components/parameters/ProductID'
      requestBody:
        $ref: '#/components/requestBodies/ProductRequest'
      responses:
        '200':
          $ref: '#/components/schemas/ProductResponse'

    delete:
      parameters:
        - $ref: '#/components/parameters/ProductID'

components:
  parameters:
    ProductID:
      name: product_id
      in: path
      required: true
      schema:
        type: string
  requestBodies:
    ProductRequest:
      required: true
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Product'
  responses:
    ProductResponse:
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Product'
    ProductArrayResponse:
      content:
        application/json:
          schema:
            type: array
            items:
              $ref: '#/components/schemas/Product'
  schemas:
    Product:
      properties:
        product_id:
          type: string
          readonly: true
        name:
          type: string
        description:
          type: string
        image_url:
          type: string
        price:
          type: integer

An example resource for the above schema:

resource "ecommerce_product" "the_cube" {
  name = "The Cube"
  description = "A large stainless steel cube"
  image_url = "https://images.squarespace-cdn.com/content/v1/5488f21fe4b055c9cb360909/1427135193771-IM8B5KC0OUVQCRG7YCQD/2013-10+Steel+Cubes+-2.jpg?format=750w"
  price = 1000
}

Your API doesn’t have to be perfectly regular, but the more CRUD-like it is, the less configuration it needs.

If needed, you can conform your API to have more CRUD-like behavior using the Stainless config, OpenAPI spec, or custom code.

Acceptance tests against real infrastructure

Because not all server behavior is captured in the OpenAPI spec, we recommend testing your Terraform provider against real infrastructure to make sure that it behaves correctly.

You can write acceptance tests using Hashicorp’s built-in testing framework. They execute against your provider and can run against your real infrastructure. You’ll want to have a suitable demo or dev environment such that you can create and delete resources safely.

To ensure that data correctly round-trips to your server and back, we recommend writing at least one acceptance test per resource and data source.

State migrations for breaking changes

Unlike our other SDKs, Terraform providers have a concept of “state”. Every time a user creates or edits resources, the state of that resource is saved into a file with the extension *.tfstate (or saved to the cloud). This state is used for future runs to preserve mappings of IDs and version information and to determine which resources already exist.

This means that if you make a breaking change to your API, you need to write a state migration to automatically upgrade the user’s state to the new version.

A state migration is a Go function within the provider codebase that describes how to go from one schema version to the next for a given resource, much like a database migration for Terraform state.

Breaking changes in Terraform are similar to breaking changes in your API or other SDKs. Some examples of breaking changes include:

Renaming a Terraform resource
Renaming an attribute of the resource
Removing an attribute

See Hashicorp’s documentation for the full list and guidance on how to version properly.

To write state migrations, implement deprecations, and do other customization, edit your Terraform repository using Stainless’s custom code feature — we’ll preserve the changes.

Getting started

Enable Go and Terraform

The Terraform provider uses the generated Go SDK to make requests to your backend when fulfilling Terraform requests. This means you must enable the Go SDK for the Terraform SDK to work. Any custom logic in your provider can also use the generated Go SDK.

In the Studio, add go and terraform to your project:

targets:
  go:
    package_name: <your-go-package-name>
    production_repo: null
  terraform:
    provider_name: <your-provider-name>
    production_repo: null

Choose Resources to Enable

You can control which Terraform resources and data sources are generated by editing the Stainless config. The infer_all_services option in the terraform target enables generation of all Terraform resources in your project.

terraform:
  provider_name: <your-provider-name>
  production_repo: null
  options:
    infer_all_services: true

This option is set to true for all new projects. If it’s false or absent, we don’t generate any Terraform resources or data sources by default.

To enable or disable Terraform for a specific resource, you can annotate the resource with terraform: true or terraform: false.

resources:
  accounts:
    terraform: true
    methods:
      create: post /accounts
      edit: put /accounts/{account_id}
      get: get /accounts/{account_id}
      delete: delete /accounts/{account_id}

For more fine-grained control, you can specify Terraform configuration options for the resource:

resources:
  accounts:
    terraform:
      name: my_cool_resource
      resource: true # do generate Terraform resource
      data_source: false # don't generate Terraform data sources for this resource
    methods:
      create: post /accounts
      edit: put /accounts/{account_id}
      get: get /accounts/{account_id}
      delete: delete /accounts/{account_id}

See the Stainless config reference for the full list of supported options.

Address diagnostics

When you switch to the Terraform tab in Stainless Studio, you may see some diagnostics that relate to the resources you enabled. View our diagnostics reference for instructions on how to resolve them. You don’t have to address them all right away, but consider addressing any Error diagnostics to maximize the chances that the provider will work while testing.

View your generated provider

Once the provider is generating, take a look at the repo to see the generated output.

View examples and documentation

You can find generated documentation and examples in the ./docs/resources and ./docs/data-sources folders.

You can use Terraform’s documentation preview tool to see a formatted version of each resource and confirm that they make sense.

Install the generated provider

Install Terraform on your machine.
Clone the generated Terraform repository from github, and cd into it.
Run ./scripts/bootstrap to install go and dependencies.
Run go build -o terraform-provider-<providername> to build the binary.
Edit (or create) your ~.terraformrc file and configure it to point to the directory where you put the Terraform provider. This ensures that when you try out the provider yourself, it looks locally instead of at the registry:
```
 provider_installation {
  dev_overrides {
    "<your-org>/<providername>" = "/path/to/local/terraform/directory"
  }
  direct {}
}
```

Copy an example resource from the ./examples directory of your generated Terraform repo to a file called main.tf, which should look similar to:

terraform {
  required_providers {
    <your-org> = {
      source  = "<your-org>/<providername>"
      version = "~> 1.0.0"
    }
  }
}

provider "<providername>" {
  api_key = "<dev api key>"
}

# copied from examples/resources
resource "<providername>_<resourcename>" "example" {
  name = "The Cube"
  description = "A large stainless steel cube"
  image_url = "https://images.squarespace-cdn.com/content/v1/5488f21fe4b055c9cb360909/1427135193771-IM8B5KC0OUVQCRG7YCQD/2013-10+Steel+Cubes+-2.jpg?format=750w"
  price = 1000
}

Run terraform apply to see the resource get created.

Write an acceptance test

Now that you have tested some resources manually, write automated tests that cover all relevant cases and ensure that the provider is ready to ship.

Create a test for a resource

Follow Hashicorp’s acceptance test guide to run a test against the resource. A test typically looks like:

package example

// example.Widget represents a concrete Go type that represents an API resource
func TestAccExampleWidget_basic(t *testing.T) {
  var widgetBefore, widgetAfter example.Widget
  rName := acctest.RandStringFromCharSet(10, acctest.CharSetAlphaNum)

  resource.Test(t, resource.TestCase{
    PreCheck:     func() { testAccPreCheck(t) },
    Providers:    testAccProviders,
    CheckDestroy: testAccCheckExampleResourceDestroy,
    Steps: []resource.TestStep{
      {
        Config: testAccExampleResource(rName),
        ConfigStateChecks: []statecheck.StateCheck{
          stateCheckExampleResourceExists("example_widget.foo", &widgetBefore),
        },
      },
      {
        Config: testAccExampleResource_removedPolicy(rName),
        ConfigStateChecks: []statecheck.StateCheck{
          stateCheckExampleResourceExists("example_widget.foo", &widgetAfter),
        },
      },
    },
  })
}

Our recommended approach is to have at least:

A basic test with only required parameters
A more complete or complex test that covers optional parameters

We also recommend implementing some test sweepers to clear out straggling entities on the server that don’t get deleted in the event a test fails.

Run the test

Because acceptance tests run against real infrastructure and can be slow, go test will not run them by default (and instead just runs unit tests). The way that you run the acceptance tests is by setting the TF_ACC environment variable before running go test:

TF_ACC=1 go test ./internal/services/my_service -count 1

Set up acceptance tests in CI

You can run your acceptance tests in CI to gain ongoing confidence in the correctness of the provider.

Sometimes the tests are too slow to be run upon every commit, but you could have them run on a daily schedule, or just manually run via Github’s Actions tab. See the Github Actions workflow on Hashicorp’s site.

It’s highly recommended that you ensure the tests all pass before merging a release PR.

Release a beta

Once your resources are all tested, and you’ve resolved all error diagnostics, you can release a beta terraform provider.

To issue the beta release, set up a production repo and follow the instructions on our publishing guide.

Customizing the provider

Resource-specific configuration

In addition to setting terraform: true, you can provide an object to further customize a resource.

resources:
  accounts:
    terraform:
      resource: true # whether to enable the resource
      data_source: true # whether to enable the data sources
      name: my_custom_name # a custom name for the resource/data source
    methods: ...

See the resource reference for more information.

Changing endpoint mapping and ID properties

Sometimes your API endpoints don’t fully match CRUD semantics. You can customize which endpoints have which Terraform behavior (create, read, update, delete), and how to find the ID parameter.

method specifies which CRUD operation the endpoint should be used for
id_property for create, specifies the request or response property that represents the ID
id_path_param for read, update, and delete, specifies the path param that represents the ID

See the following config where each of the default options is explicitly set:

products:
  terraform: true
  methods:
    list:
      endpoint: get /products
      terraform:
        method: list
    create:
      endpoint: post /products
      terraform:
        id_property: product_id
        method: create
    retrieve:
      endpoint: get /products/{product_id}
      terraform:
        id_path_param: product_id
        method: read
    update:
      endpoint: put /products/{product_id}
      terraform:
        id_path_param: product_id
        method: update
    delete:
      endpoint: delete /products/{product_id}
      terraform:
        id_path_param: product_id
        method: delete

Custom configurability

Every “attribute” (aka property) in a Terraform resource’s schema has a “configurability” setting attached to it.

The “configurability” of an attribute determines how it can change over time:

required: must be provided by the user at all times, cannot be null
optional: must be provided by the user at all times, can be null
computed: cannot be provided by the user, is provided by the API or provider
computed and optional: can be provided by the user or the API or provider

There isn’t quite enough information in the OpenAPI spec to specify each of these precisely, so you can add an annotation to your OpenAPI spec if we don’t infer it correctly.

By default, we infer configurability like so:

required: marked as required in the create endpoint’s request
optional: not marked as required, or
computed: a property only defined in the response of the create or update endpoint, or marked as read-only in the OpenAPI spec

To customize the configurability of a given property, specify the x-stainless-terraform-configurability extension property on your schema. For example:

cardholder_name:
  type: string
  description: The cardholder's name
  x-stainless-terraform-configurability: computed_optional
  # required, optional, computed, computed_optional are all valid values

Next Steps

If you find a bug, have a question, or want to provide product feedback, let us know at support@stainless.com.

Getting started with Terraform