Solved: Do you separate template browsing from deployment in your internal IaC tooling?
Jenkins
Source: Dev.to | Dec 31, 2025 (UTC)
🚀 Executive Summary
TL;DR: Many organizations struggle with blurred lines between IaC template discovery and deployment, leading to cognitive overload, error-prone provisioning, and slow self-service adoption. This can be solved by separating these processes through dedicated template catalogs, enhanced CLI tools, or GitOps workflows, which improve discoverability, reduce errors, and enforce standardization. By making templates easily accessible and deployments predictable, teams can innovate faster and more securely.
🎯 Key Takeaways
Coupling IaC template browsing and deployment often results in ‘template sprawl,’ cognitive overload, error-prone deployments, and slow self-service adoption due to lack of clear categorization and documentation.
Three primary solutions for separation are: a Dedicated Template Catalog (user-friendly UI), a CLI-Driven approach with enhanced discovery (scripting-friendly), and GitOps with Self-Service via Pull Requests (Git-centric, auditable).
GitOps provides a single source of truth, strong auditability, and built-in review processes through PRs, making it ideal for organizations prioritizing version control and automated reconciliation, despite a higher initial setup complexity.
Navigating and deploying Infrastructure as Code (IaC) templates can be a maze for internal teams. This post explores how to effectively separate template discovery from the deployment process, reducing friction and boosting developer productivity.
Symptoms: When Template Browsing and Deployment Collide
In many organizations, the line between discovering available IaC templates and actually deploying them is blurred, leading to a myriad of operational challenges. This coupling often manifests in several ways, creating friction for development teams and increasing the risk of misconfigurations.
### Cognitive Overload and “Template Sprawl”
Developers are often directed to a large Git repository containing hundreds or thousands of IaC files (Terraform modules, CloudFormation templates, Helm charts, etc.). Without clear categorization, documentation, or an intuitive search mechanism, finding the right template for a specific need becomes a daunting task. This “template sprawl” leads to wasted time, duplicated efforts, and a reluctance to use existing, well-vetted IaC.
### Error-Prone Deployments
When template parameters are poorly documented or require manual extraction from code, users are prone to making mistakes. Misconfigured resource types, incorrect environment variables, or overlooked security settings can lead to failed deployments, security vulnerabilities, or suboptimal infrastructure.
### Slow Provisioning and Low Self-Service Adoption
The complexity of discovering, understanding, and then correctly executing a deployment often forces developers to rely on a central DevOps or Platform team. This creates bottlenecks, delays project timelines, and undermines the goal of empowering teams with self-service infrastructure provisioning.
### Lack of Standardization and Guardrails
Without a clear separation or an opinionated pathway, teams might resort to copying and modifying templates haphazardly. This can lead to deviation from organizational standards, compliance drift, and a lack of consistent security controls across the infrastructure landscape.
A dedicated template catalog or self-service portal provides a user-friendly interface for browsing, understanding, and initiating deployments of approved IaC templates. This completely decouples discovery from the underlying deployment mechanisms, offering a guided experience.
How it Works
Users interact with a web-based interface that presents a curated list of IaC templates. Each template entry typically includes:
Input parameters with descriptions, default values, and validation rules.
Estimated cost (if integrated with cost management tools).
Links to detailed documentation or the source code.
Upon selecting a template, the user provides the necessary parameters via a form. The portal then orchestrates the deployment using an underlying CI/CD pipeline or directly through IaC tools, abstracting away the complexities of commands and configuration files.
Examples & Configuration
Tools like Spotify’s Backstage, HashiCorp’s Terraform Cloud/Enterprise Workspaces, or custom-built internal portals can serve this purpose.
Imagine a template for an “AWS S3 Static Website.” In Terraform Cloud, you’d set up a workspace configured to use this module. The UI would then prompt for specific variables.
# main.tf for S3 static website module (in your template repo) resource "aws_s3_bucket" "website_bucket" { bucket = var.bucket_name acl = "public-read" website { index_document = "index.html" error_document = "error.html" } tags = { Environment = var.environment Project = var.project } } resource "aws_s3_bucket_policy" "website_policy" { bucket = aws_s3_bucket.website_bucket.id policy = jsonencode({ Version = "2012-10-17", Statement = [ { Effect = "Allow", Principal = "*", Action = "s3:GetObject", Resource = "${aws_s3_bucket.website_bucket.arn}/*" } ] }) } variable "bucket_name" { description = "The unique name for the S3 bucket." type = string } variable "environment" { description = "The deployment environment (e.g., dev, staging, prod)." type = string } variable "project" { description = "The project name associated with this resource." type = string }
Enter fullscreen mode
Exit fullscreen mode
In a portal, these variables (bucket\_name, environment, project) would be presented as form fields, with their descriptions guiding the user. The portal would then trigger a Terraform run in the configured workspace using the user-provided values.
Benefits
Enhanced Discoverability: Centralized, searchable catalog makes finding templates easy.
Improved User Experience: Intuitive forms, clear descriptions, and pre-validation reduce errors.
Standardization & Governance: Only approved templates are available, promoting best practices and compliance.
Reduced Cognitive Load: Users don’t need to know IaC syntax or CLI commands.
Drawbacks
Development & Maintenance Overhead: Building and maintaining a custom portal can be resource-intensive.
Potential for Drift: If the portal’s template definitions aren’t kept in sync with the actual IaC code, it can lead to confusion.
Limited Flexibility: May not support highly custom or experimental deployments easily.
Solution 2: CLI-Driven with Enhanced Template Discovery & Validation
This approach emphasizes a robust command-line interface (CLI) that provides specific commands for template discovery, detailed parameter explanation, and pre-deployment validation, separate from the actual deployment command. It’s ideal for teams comfortable with CLI tools and scripting.
How it Works
Instead of a GUI, users interact with a custom CLI tool or a set of well-defined scripts. This CLI provides commands such as list-templates, describe-template, and validate-template before deploy-template. The discovery phase focuses on providing rich metadata directly at the command line, often pulling from structured documentation or code comments.
Examples & Configuration
This often involves a wrapper script around your core IaC tools (e.g., Terraform, CloudFormation CLI, Helm). Let’s consider a custom iac-helper CLI:
Example: Custom iac-helper CLI
Imagine a directory structure for your IaC templates:
Your iac-helper could use jq to parse variables.tf or template.json to extract parameter info. A README.md within each template directory would serve as detailed documentation.
# bash script for iac-helper (simplified) #!/bin/bash IAC_ROOT_DIR="iac" list_templates() { echo "Available IaC Templates:" find "$IAC_ROOT_DIR/modules" -maxdepth 1 -type d -and -not -name "modules" -print0 | xargs -0 -n 1 basename | while read -r template; do echo " - $template" done find "$IAC_ROOT_DIR/templates" -maxdepth 1 -type f \( -name "*.tf" -o -name "*.yaml" -o -name "*.json" \) -print0 | xargs -0 -n 1 basename | sed 's/\.tf\|\.yaml\|\.json//g' | while read -r template; do echo " - $template (Root Module)" done } describe_template() { TEMPLATE_NAME=$1 if [ -d "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME" ]; then echo "Description for module: $TEMPLATE_NAME" cat "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME/README.md" 2>/dev/null || echo "No README.md found." echo "" echo "Variables:" terraform-docs md "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME" | sed -n '/^| Name | Description | Type | Default | Required |/,/^$/p' elif [ -f "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.tf" ]; then echo "Description for root module: $TEMPLATE_NAME.tf" # For Terraform root modules, you might describe input variables terraform-docs md "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.tf" | sed -n '/^| Name | Description | Type | Default | Required |/,/^$/p' elif [ -f "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.yaml" ]; then echo "Description for CloudFormation template: $TEMPLATE_NAME.yaml" # For CloudFormation, parse parameters from YAML yq e '.Parameters | to_entries[] | .key + ": " + .value.Description' "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.yaml" else echo "Template '$TEMPLATE_NAME' not found." fi } deploy_template() { TEMPLATE_NAME=$1 # ... logic to run terraform apply or aws cloudformation create-stack echo "Deploying $TEMPLATE_NAME..." # Example: terraform apply -var="bucket_name=$2" -var="environment=$3" "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME" } case "$1" in list) list_templates ;; describe) if [ -z "$2" ]; then echo "Usage: iac-helper describe " exit 1 fi describe_template "$2" ;; deploy) # This would be much more complex, requiring dynamic var parsing, envs etc. echo "Deploy command not fully implemented in this example." echo "Usage: iac-helper deploy [params...]" ;; *) echo "Usage: iac-helper [list|describe|deploy] ..." ;; esac
Enter fullscreen mode
Exit fullscreen mode
Users would then run:
iac-helper list iac-helper describe s3-website iac-helper deploy s3-website bucket_name=my-unique-site-name environment=dev
Enter fullscreen mode
Exit fullscreen mode
Benefits
Low Overhead: Less development effort compared to a full UI portal.
Scripting Friendly: Easily integrates into CI/CD pipelines or local automation scripts.
Precision & Control: Advanced users can specify exactly what they need.
Version Control Native: Documentation and helper scripts live alongside the IaC.
Drawbacks
Steeper Learning Curve: Requires users to be comfortable with CLI tools and syntax.
Documentation Dependent: Effectiveness heavily relies on comprehensive, up-to-date in-code documentation.
Less Intuitive: Can be less visual and discoverable for non-technical users.
Solution 3: GitOps with Self-Service via Pull Requests
GitOps fundamentally separates the desired state of infrastructure (defined in Git) from the operational process of achieving that state. Template browsing becomes reviewing available IaC modules and applying them by submitting Pull Requests (PRs) to a configuration repository, which then triggers automated deployment by a GitOps operator.
How it Works
In this model, your core IaC templates (e.g., Terraform modules, Helm charts) reside in one or more “template repositories.” Teams needing to provision infrastructure will browse these repositories to identify the appropriate templates. To deploy, they won’t run terraform apply directly. Instead, they create a PR against a separate “configuration repository” (often one per environment, e.g., infra-prod, infra-dev). This PR defines the desired state, typically by referencing the core templates and providing specific values. A GitOps operator (like Argo CD or Flux CD) monitors this configuration repository. When a PR is merged, the operator detects the change and applies the IaC to achieve the declared state.
Examples & Configuration
Example: Deploying an EKS Cluster with Terraform and Argo CD
Let’s assume you have a Terraform module for an EKS cluster in modules/eks-cluster/.
1. Template Repository (iac-templates):
# modules/eks-cluster/main.tf (simplified) resource "aws_eks_cluster" "main" { name = var.cluster_name role_arn = var.cluster_role_arn vpc_config { subnet_ids = var.subnet_ids } # ... other EKS configurations } variable "cluster_name" { description = "Name of the EKS cluster." type = string } variable "cluster_role_arn" { description = "ARN of the IAM role for the EKS cluster." type = string } variable "subnet_ids" { description = "List of subnet IDs for the EKS cluster." type = list(string) }
Enter fullscreen mode
Exit fullscreen mode
2. Configuration Repository (infra-dev):
A developer wants to provision a new EKS cluster for a new project. They would browse the iac-templates repo, understand the eks-cluster module, then create a new directory and main.tf in their infra-dev repository:
The developer creates a PR against infra-dev with these changes.
The PR undergoes code review (colleagues, platform team) and potentially automated checks (e.g., terraform validate, terraform plan output posted as a comment).
Upon merging the PR, Argo CD (configured to watch infra-dev) detects the change.
Argo CD then executes terraform apply with the specified module and variables, provisioning the EKS cluster.
Benefits
Single Source of Truth: Git becomes the definitive desired state for all infrastructure.
Auditability & Rollback: Every change is a commit, providing full history and easy rollback.
Medium (wrapper scripts, structured documentation).
High (GitOps operator, repo structure, CI/CD).
Maintenance Burden
Keeping portal UI/APIs in sync with IaC.
Keeping scripts & documentation updated.
Maintaining GitOps operators, ensuring state reconciliation.
Best For
Large organizations, non-technical users, high standardization needs.
Teams comfortable with CLI, scripting, rapid iteration.
Any organization committed to strong version control, auditability, and automated reconciliation.
Conclusion
Separating template browsing from deployment is a crucial step toward mature IaC adoption. Each solution offers distinct advantages, catering to different organizational structures, technical proficiencies, and governance requirements. Whether you opt for the guided experience of a self-service portal, the flexibility of an enhanced CLI, or the robust, auditable flow of GitOps, the key is to reduce the friction engineers face when trying to provision infrastructure. By making templates easily discoverable and deployments predictable, you empower your teams to innovate faster and more securely.
Залишити відповідь