Multiple Terraform providers and dynamic lookup

Multiple Terraform providers are helpful when you are working with multiple accounts (AWS) or working on a solution that spans across multiple accounts. You need an efficient way of defining Multiple Terraform providers to be able to lookup each provider dynamically.

Following example uses terraform modules , refer to this article to understand how terraform modules works with cross account IAM permissions

If you are not familiar with providers, Refer to this terraform documentation to understand providers.

Source Module : where the module (resource) code is declared
Caller Module : where the module code is being used

Source Module code

Define generic provider in file, if needed you can also add multiple providers there and then in the variable file assign the appropriate role ARN.

provider "aws" {
	region = "eu-central-1"
	assume_role {
    role_arn = local.provider_role
 	alias = "account1"

This is the key to define Multiple Terraform providers dynamically. Define a map in variable file that contains a Tag prefix and IAM role as key value pairs – add them to file, make sure these IAM roles exists in each account with the exact tag prefixes.

variable "cross_account_role_arns" {
  type = map
  default = {
    TAG-1        = "arn:aws:iam::XXX:role/TAG-1-IAM-ROLE",
    TAG-2        = "arn:aws:iam::YYY:role/TAG-2-IAM-ROLE",
    TAG-3        = "arn:aws:iam::ZZZ:role/TAG-3-IAM-ROLE",

in the same file, look up for appropriate IAM role based on the given tag prefix, this will return the value of terraform dynamic providers value.

variable "vpc_tag_prefix" {
  description = "Tag Prefix of the caller module account"
  type        = string

locals {
  provider_role = var.cross_account_role_arns["${var.vpc_tag_prefix}"]

In the file where you define your resources, add the provider as shown here

resource "aws_route" "vpc2_routes_public" {
  # all other attributes here..
  provider = aws.account1

Caller module code

In the caller module pass vpc_tag_prefix value as shown in the blow example.
key thing to note here is the value of vpc_tag_prefix variable which is set to “TAG-1”. if you refer to the Source module code, this tag is mapped to an IAM role in the variable map and at run time terrform automatically finds the appropriate aws provider for you.

module "routes-module" {
  source = "./routes"
  #other variable for your vpc routes.
  vpc_tag_prefix = "TAG-1"

Refer to the article on how to setup terraform workspaces which helps better manage your repositories and development environments.


2 thoughts on “Multiple Terraform providers and dynamic lookup

  1. Hi~
    I was very excited about this article.

    Because I thought it could solve the problems with handling multi-account architecture

    So I tried to implement it in exactly the same way you mentioned (copied your example code and pasted it).

    But I encountered the following error. The error was fixed when I removed the provider definition in the source module code.

    I wonder if “Multiple Terraform providers and dynamic lookup” really works.

    Do you have any solution to it?
    Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

    Please see
    for more information about providing credentials.

    Error: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors


Leave a Reply

Your email address will not be published. Required fields are marked *