Docs

# Move module call to config branch and specify Terraform sample into .cycloid.yml

This section introduce Cycloid stacks concept to improve the reusability of your stack.

Pipeline step5

We created a Terraform module during the last step and used it in a main.tf file inside our stacks branch. This step will make it more flexible and follow Cycloid stacks concept.

Keep in mind a stack can be deployed several times for different environments or projects. The standard way to do this is to create a Terraform module in the stacks branch. A Terraform module should be generic and describe an application. This means if we want for an application to create two environments, staging and prod, we might want the same infrastructure but with a smaller instance size on staging. So we will describe the infrastructure in a Terraform module and call this module 2 times (one for each env) using different parameters for the instance type.

As well this concept can also be applied to other technologies like Ansible.

This is where you apply Cycloid stacks concept. To make it simple, there is no technological limitation. It's just a way to split a template (here, the Terraform module) and a parameters of a template (here Terraform module call).

In our use-case, we will transform the call of the Terraform module main.tf in a sample file. Each time someone uses your stack, it will use the Terraform module from the stacks branch and automatically push the module's parameters in a config branch using your sample file. We will go more into detail about it in this step.

Follow those steps to apply all changes described in this step

As described above let's define our new main.tf.sample as a sample in the Cycloid stack (similarly to the pipeline variables sample file), we will define the Terraform sample in .cycloid.yml file.

stack-sample/.cycloid.yml





 
 
 
 

config:
  default:
...
    # Configuration of terraform to setup the infrastructure
    terraform:
      instance:
        path: 'terraform/main.tf.sample'
        destination: '($ project $)/terraform/($ environment $)/main.tf'
1
2
3
4
5
6
7
8

TIPS

Cycloid special vars can be used in .cycloid.yml file, in our case '($ project $)/terraform/($ environment $)/main.tf' will be rendered by 'snowy/terraform/test/main.tf'.

Those lines define the path of your stack sample in the stacks branch and the expected path for the sample after user configuration in the config branch.

After this change, the next time you create a project or a new environment with this stack, the example of Terraform file to put in the Git config branch will be displayed, letting the user specify is own configuration:

wizard

Then it will be automatically pushed into his config repository

Unfortunately, this does not apply to an existing env, and this is why we created the config branch and manually put this file in the commands above.

Now to make your pipeline take into account, you need to apply the concept of merging stacks and config that you will define in the pipeline with the following changes:

stack-sample/pipeline/variables.sample.yml

# Branch used to store the config of the stack
config_git_branch: config
1
2

We define a pipeline variable to specify the branch for our new git resource. And as well create a new git resource for the config branch:

stack-sample/pipeline/pipeline.yml








 


 


resources:
...
# The Terraform config (will be merged with the stack)
- name: git_config
  type: git
  source:
    uri: ((git_repository))
    branch: ((config_git_branch))
    private_key: ((git_ssh_key))
    paths:
      - ((project))/terraform/((env))/*
1
2
3
4
5
6
7
8
9
10
11
12

Notice the paths section here to limit the trigger of the pipeline only if the configuration for this specific project and env change.

Then we need to merge our stack and our config inside the pipeline. We will use the same tips as the previous step based on YAML Alias indicators (opens new window).

stack-sample/pipeline/pipeline.yml



 










 




 



 
 

# YAML anchors
shared:
  - &task-merge-stack-and-config
    task: merge-stack-and-config
    config:
      platform: linux
      image_resource:
        type: docker-image
        source:
          repository: cycloid/cycloid-toolkit
          tag: latest
      run:
        path: /usr/bin/merge-stack-and-config
      inputs:
        - name: git_config
          path: "config"
        - name: git_stack
          path: "stack"
      outputs:
        - name: merged-stack
          path: "merged-stack"
    params:
      CONFIG_PATH: ((project))/terraform/((env))
      STACK_PATH: stack-sample/terraform
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

This task calls a script provided by cycloid in our cycloid-toolkit image (opens new window) to merge our two expected inputs stack and config and put the result of the merge in a merged-stack output. This merge also takes CONFIG_PATH and STACK_PATH as params related to the expected path of Terraform files defined in .cycloid.yml

Last change, add task-merge-stack-and-config task and git_config resource in our Terraform jobs to merge our config and stack before running the Terraform resource.

stack-sample/pipeline/pipeline.yml








 
 




 




 










 
 
 
 







 




 



jobs:
...
  - name: terraform-plan
    plan:
      - do:
        - get: git_stack
          trigger: true
        - get: git_config
          trigger: true
        - get: git_code
          trigger: true
          passed:
            - unittest
        - *task-merge-stack-and-config
        - *task-get-commit
        - put: tfstate
          params:
            plan_only: true
            terraform_source: merged-stack/
            var_files:
              - extracted-vars/terraform.tfvars

  - name: terraform-apply
    plan:
      - do:
        - get: git_stack
          trigger: false
          passed:
            - terraform-plan
        - get: git_config
          trigger: false
          passed:
            - terraform-plan
        - get: tfstate
          trigger: false
          passed:
            - terraform-plan
        - get: git_code
          passed:
            - terraform-plan
        - *task-merge-stack-and-config
        - *task-get-commit
        - put: tfstate
          params:
            plan_run: true
            terraform_source: merged-stack/
            var_files:
              - extracted-vars/terraform.tfvars
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

The code above adds the get call of our git_config resource, then use the YAML anchor to add a merge task before. Then we also changed the terraform_source from terraform (put: tfstate) to use the output of our merge merged-stack instead of the Terraform code from the stack branch.

Add and commit those changes in Git:

git add .
git commit -m "Step 5"
git push origin stacks
1
2
3

Get back to Cycloid's dashboard, and click on the Refresh pipeline button Refresh pipeline.

# Key points to remember

  • A stack can be deployed several times for different environments or projects
  • A Terraform module should be generic and describe an application (same thing with Ansible for example)
  • A stack is a way to split a template (for example a Terraform module) and a parameters of a template (for example a Terraform module call or main.tf)
  • Most of the time, the parameters (for example main.tf) is transformed in a sample file
  • Each time someone uses your stack, it will use the Terraform module from the stacks branch and automatically push the module's parameters in a config branch using your sample file

← Step 6 - Extend our workflow →