I recently returned from a very successful Hashiconf 2019 where lots of new features where announces for the Hashicorp products.  Here are some of the mayor announcements.

Terraform:

• Terraform Cloud (TFC)
  • Rebranding of Terraform Enterprise SaaS to Terraform Cloud.
  • TFC is all about collaboration. When more than 1 person starts working on a Terraform project it requires backend management of the state file and you should start orchestrating Terraform runs using a deployment pipeline.  This is all now provided by Terraform Cloud!
  • Free tier (up to 5 users)
    • User interface
    • Remote state management for storing, view and locking of state files.
    • VCS connection management
    • Collaboration on runs
    • Remote runs and applies
    • Private module registry
  • Paid tiers (more than 5 users)
    • Both the paid tiers are available for free until 01.01.2020!
    • TFC: Teams
      • Create multiple teams
      • Control permissions of users on those teams
    • TFC: Teams & governance
      • This tier is also available for free until 01.01.2020
      • Use Sentinel and Cost Estimation
      • More information and pricing on offerings available here
  • More information here.
• Terraform clustering
  • This is only available with Terraform enterprise (TFE) and current in beta version
  • More information here.
• Terraform Cost Estimation
  • This is available for both TFE and TFC
  • Is executed between the plan and apply phases of a TF run.
  • Can also use Sentinal to control costs with defined policies
  • More information here.

Consul:

• It definitely felt like Consul was the new shiny toy at this years conference and the related sessions were packed.
• HashiCorp Consul Service (HCS) on Azure
  • Native provisioning of a Consul cluster into any region through the Azure marketplace.
  • Although the Pong game live demo did not go as planned I do see the value and potential for this product!
  • Currently only available in private beta
  • More information here
• Consul Enterprise now support VMware NSX Service Mesh Federation
  • Support for the Service Mesh Federation Specification.
  • More information here.

Now back to what we are here for…Terraform Cloud!

After just a couple of weeks of learning & using Terraform I would definitely still consider myself a newbie, so if you do find any inefficiencies, errors or security concerns in my instruction then please let me know in the comments below. I would love to collaborate and update my content where necessary.

Since the start of my Terraform journey I been using Terraform Cloud which provides  remote state management which is a great feature and removes the management of the state file when you start to collaborate with other team members on your TF projects.  Since additional new features have been announced at Hashicorp and made available, I wanted to revisit my initial step by step configuration and include the setup of these new features.

1. The only prerequisite for using TFC is that it requires Terraform 0.11.13 or later, so make sure to verify your local version first!
   1. “terraform -version”
2. TFC setup
   1. Open https://app.terraform.io
   2. Create your free account
   3. Login
   4. Create an Organization
      1. Add organization name
      2. Add email address for notification
   5. Invite team members to organization
      1. In the free tier you cannot manage your teams so please make sure you invite the correct people because there are no member roles for an organization so everyone will have the same permissions as you!
      2. Until 01.01.2020 you do have the option to request the Team & Governance feature sets for free from within TFC, after which this would be a paid subscription. More information here

         1. 

   6. Configure access to Terraform CLI
      1. You can either use the user or a team token to run Terraform. In this case we will just use the user token.
      2. In TFC, select your User icon in top right corner -> User Settings
         1. Select Tokens
         2. Generate Token
         3. Copy the token information and save to desktop if needed.
            1. Protect this date/file because with it anyone can access TFC.
3. Open your terraform cli config file
   1. sudo vi ~/.terraformrc
   2. add the following credential block to the file

   3. credentials "app.terraform.io" {   token = "PASTE_THE_TOKEN_COPIED_IN_EARLIER_STEPS" } 

4. Enable remote state on existing or new Terraform configurations
   1. Create a backed file
      1. vi backend.tf
      2. Add the following backend configuration

      3. terraform {   backend "remote" {     organization = "<ORGANIZATION_NAME_CREATED_EARLIER>"     workspaces {       name = "<WORKSPACE_NAME>"     }   } } 

      4. For the workspaces value, if the configuration manages a unique set or resources then a single workspace should suffice, however if you manage multiple sets of resource you want to use a prefix string value
   2. After the remote backend is configure run “terraform init”
      1. If tfstate exist in working directory you will prompted to copy it to backend.
      2. If no tfstate in working directory and configured a prefix for workspaces you will be asked to create a workspace manually
   3. After terraform init has finished
      1. run terraform plan
      2. run terraform apply
5. Now lets setup the VCS integration, which in my case in GitLab.
   1. First we need to configure GitLab
      1. Login to gitlab.com with a dedicate service user
         1. Account must have admin (master) access
      2.  Select the user ->  settings -> Applications
      3. Enter Name
         1. I recommend something descriptive like TFC (org name)
      4. Enter a temporary Redirect URL, because we do yet have this information from TFC.
         1. https://example.com/will-replace-this-later
      5. Click Save Application
      6. Leave this page open and save the Application ID and secret.
   2. Back in TFC, select settings -> VCS Providers

   3. 

   4. First off as you see we need to configure our OAuth connection.
   5. Click “Add one now”
   6. Select your VCS provider, in my case I’m using Gitlab.com

   7. 

      1. Optionally add a descriptive name if you will have multiple connections to GitLab.
      2. Enter the application ID from GitLab
      3. Enter the secret from GitLab
      4. Click Create VCS provider
      5. On the newly create VSC provider page, copy the Callback URL.
   8. Back on Gitlab application page which we left open
      1. Click Edit
      2. Remove the placeholder URL
      3.  Paste the TFC callback URL from TFC into the Redirect URI field in GitLab.
      4. Click Save Application
   9. Back again on TFC VCS provider page, click “Connect organization <orgname>”
      1. Select Authorize
      2. Make sure in TFC that you get a success notification for linking to GitLab.
   10. You now have GitLab access to TFC and can create workspaces based on your organizations shared repositories!
6. Connect TFC workspace to GitLab repository
   1. In TFC select your workspace.
   2. Click Settings – > Version Control

   3. 

   4. Click “Connect to version control”
   5. Select your VCS provider
      1. In my case GitLab.com
   6. Choose a repository
      1. All repositories you have in GitLab.com should show
      2. Select the corresponding repository
   7. Click “Update VCS Settings”

   8. 

   9. Select your Version Control settings
      1. Choose how you want to Automatic Runs to be triggered
         1. Select “Always trigger runs” or “Only trigger runs when files in specified paths change”
      2. Select VCS branch from which to import new version
      3. Select if you want recursive clone of your repositories submodules
      4. Click update VCS settings

Now we want to change the execution mode for our workspace from local, which runs locally on your machine, to remote which allows your plans and applies to occur on Terraform Cloud’s infrastructure.

1. Open TFC and select your workspace
2. Click Settings -> General
3. 
4. Change the Execution Mode from Local to Remote
5. Select the Apply method
   1. Auto apply will apply changes when a Terraform plan is successful
   2. Manual apply will required an operator to confirm the result of the Terraform plan before applying
6. Select the Terraform version to use in this workspace
7. Select the Terraform working directory
   1. If nothing entered it defaults to the root of the configuration directory
8. Click Save settings

Lastly we want to add version control to our workspace.

1. Open TFC and select your workspace
2. Click Settings -> Version Control
3. 
4. Select “Connect to version control”
5. 
6. We already configured out GitLab.com provider so select that
7. Select your repository from GitLab to associate to your workspace
8. On Confirm Changes select “Update VCS settings”
9. Select your workspace and wait for it to complete the connection to the repository of your VCS provider.

Since I have configured remote execution and using the AWS provider, we need to configure our variables for the workspace to authenticate successfully.  This can now easily be done from the TFC UI.

1. Open TFC and select your workspace.
2. Select the Variables tab  
   1. This tab will only show if you have select Remote execution under Generals settings for the workspace.
3. *Extra info:  During the initial version control configuration of the workspace, the variables specified in *.auto.tfvars file will automatically be imported and shown under Terraform variables. Going forward the TFC Terraform variables will be the source of truth. Also any new variables added to *.auto.tfvars file will not be added automatically to the TFC Terraform variables.
4. There are a couple of ways to go about the authentication, but for this blog I decided to use the Environment Variables, which sets the variables in Terraform shell using export.
5. Under Environment Variables:
   1. Click Add Variable
   2. Add key name AWS_ACCESS_KEY_ID
   3. Add key name AWS_SECRET_ACCESS_KEY
      1. In value, enter the Key and check the Sensitive box!

Everything should now be configured so if you push your commits to Git, it will automatically kick off a Terraform apply with remote execution in TFC… How cool is that!