#spacelift (2022-06)

2022-06-01

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
12:02:39 PM

CI/CD Pipeline : Everything You Need To Know with Examples What makes a good CI/CD pipeline? Take a closer look at the pipeline stages and some examples relating to Terraform.

CI/CD Pipeline : Everything You Need To Know with Examplesattachment image

What makes a good CI/CD pipeline? Take a closer look at the pipeline stages and some examples relating to Terraform.

2022-06-02

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
10:42:38 AM

Drift Detection and How to Fix It With Different IaC Tools What is drift and why does it happen? See how IaC tools, including Spacelift, can help you with drift detection and remediation.

Drift Detection and How to Fix It With Different IaC Toolsattachment image

What is drift and why does it happen? See how IaC tools, including Spacelift, can help you with drift detection and remediation.

2022-06-08

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
10:12:45 AM

Ansible Roles : Basics & How to Combine Them With Playbooks Explore the concept of Ansible roles, their structure, and how we can refactor our playbooks into roles or generate them from scratch.

Ansible Roles : Basics & How to Combine Them With Playbooksattachment image

Explore the concept of Ansible roles, their structure, and how we can refactor our playbooks into roles or generate them from scratch.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

For those using Spacelift, how crazy do you all go with the login/access/approval/plan policies? Looking to glean some ideas to approach implementing this.

2

2022-06-09

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
12:52:36 PM

When to Run Terraform Apply – Quick Usage Examples Terraform apply is a core command in the Terraform workflow. Learn what this command is used for, what it does, the available options, and when to run it.

When to Run Terraform Apply - Quick Usage Examplesattachment image

Terraform apply is a core command in the Terraform workflow. Learn what this command is used for, what it does, the available options, and when to run it.

2022-06-14

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
09:32:37 AM

Adventures in AWS Lambda Land – a Migration Gone Well See the approach we’ve taken that allowed us to simplify our setup and clarify common misconceptions you might have about AWS Lambda development.

Adventures in AWS Lambda Land - a Migration Gone Wellattachment image

See the approach we’ve taken that allowed us to simplify our setup and clarify common misconceptions you might have about AWS Lambda development.

1
1
1

2022-06-15

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
03:32:37 PM

AWS IAM Policies : Best Practices & How to Create an IAM Policy Look into AWS IAM policies with some best practices. Learn how they are structured, how to create them, and how to assign necessary permissions.

AWS IAM Policies : Best Practices & How to Create an IAM Policyattachment image

Look into AWS IAM policies with some best practices. Learn how they are structured, how to create them, and how to assign necessary permissions.

1

2022-06-23

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
04:02:37 PM

9 DevOps Best Practices – What You Should Do and Not-to-Do Explore different DevOps best practices and Anti-Patterns and see how you can incorporate them to accelerate your team performance and value creation.

9 DevOps Best Practices - What You Should Do and Not-to-Doattachment image

Explore different DevOps best practices and Anti-Patterns and see how you can incorporate them to accelerate your team performance and value creation.

2022-06-28

Managing Infrastructure at Scale | Spacelift Blog avatar
Managing Infrastructure at Scale | Spacelift Blog
10:02:39 AM

How to Use Terraform Join and Split Functions with Strings There are many built-in functions available for use with Terraform. Learn do the join and split functions in Terraform do and see examples.

How to Use Terraform Join and Split Functions with Stringsattachment image

There are many built-in functions available for use with Terraform. Learn do the join and split functions in Terraform do and see examples.

azec avatar

After long thread with Andriy in atmos, about how to use atmos and considerations how to get it set up with GitLab CI/CD, and after seeing how greatly it plays with Spacelift, thankfully to SweetOps modules, I want to try Spacelift over the next few weeks.

Before that, I need to write-up a decent SPIKE Jira ticket and hopefully get that approved by my manager. I would like to get inputs from those who already have experience working with Spacelift about what excites them and how did it solve some of the day-to-day problems for Infrastructure work. I would kindly appreciate any info you can share - that would be valuable to corroborate my case on why we should adopt Spacelift.

Thank you!

2
2
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Matt Gowie @Eric Berg

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Alex Jurkiewicz

Matt Gowie avatar
Matt Gowie

@azec few things come to mind:

  1. It using OPA as a policy as code framework gives you endless flexibility with the whole platform. For example, the way your orchestrate approval workflows for your team and infra can be setup exactly how you want it and it’s really not that difficult.
  2. Being able to share a plan and discuss it as a team is huge for working with large infrastructure footprints.
  3. The organization of all of your root modules within their UI folders ends up being a big benefit for understand what is where.
  4. Their before / after hooks enable you to do all the hacky bits that you’d want to do with building your own which is huge. They work great.
azec avatar

Thank you all for following and contributing to this thread.

azec avatar

I am kinda soloing on this, so for yesterday and today, I just stacked ECR private registry and added repository for Spacelift Docker runner. I guess I will be pivoting my focus from here to have at least somewhat useful Docker runner. Just out of curiosity - @Andriy Knysh (Cloud Posse) or @Erik Osterman (Cloud Posse) - does CloudPosse have a public repo of Spacelift Docker runner image that comes with atmos and Terraform > 1.0 ?

azec avatar

I really want to prepare everything I can upfront, before I request SpaceLift trial. This includes two CP modules for day-1 spacelift stack that manages all other atmos stacks as well as module for worker pools.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we use geodesic which comes with terraform and other tools, but we install atmos on it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
ATMOS_VERSION=1.4.22

# Using `registry.hub.docker.com/cloudposse/geodesic:latest-debian` as Spacelift runner image on public worker pool
apt-get update && apt-get install -y --allow-downgrades atmos="${ATMOS_VERSION}-*"

# If runner image is Alpine Linux
# apk add [email protected]~=${ATMOS_VERSION}
Matt Gowie avatar
Matt Gowie

@azec This was something I just put together the other week for a client I have on public runners if it is of interest — https://github.com/masterpointio/spacelift-atmos-runner

If you’re going for private workers, then I would suggest building your own toolbox image from Geodesic like Andriy mentions.

masterpointio/spacelift-atmos-runner

Supporting atmos terraform projects that are using Spacelift with public workers.

1
Alex Jurkiewicz avatar
Alex Jurkiewicz

Sorry, we don’t use Atmos so I can’t really comment on that. But I think the benefit of Spacelift over gitlab ci (or any generic ci platform) is that Spacelift is a) a deployment-aware tool, so dealing with things like failed runs is saner and b) Terraform-aware, so there are escape hatches for running one-off tasks, automatic Terraform version management/init /etc

Eric Berg avatar
Eric Berg

Hey, @azec. Here’s my post-vacation reply. Hope its helpful.

We’re a start-up and pretty much green-field in terms of the technologies we use. We started with TF from the beginning, rolling in infrastructure repos (i.e., IAM, DNS, bastion access, etc.) as well as managing multiple dev and client environments. Knowing we were going in the TACOS direction, we built our processes around gitops, so by the time we got to Spacelift (plus the time it took to get my head around SL and then build out my TF SL implementation) integrating all of those stacks and multi-stacks (that deploy the same codebase to multiple environments) was a matter of updating YAML files.

A few things that thrill me about Spacelift:

  1. Auditing. It can be really helpful to have a list of runs with the plan and apply logs. a. It also makes it much easier to share plans, with links to the SL run
  2. The Github integration goes a long way to providing enough visibility into the runs triggered by PR checks, to make github the authoritative source for approving PRs. The main benefits here are that you don’t have to pass around plan files or even log into Spacelift, unless they see unexpected changes. Along the same lines, there are a number of ways to add the plan output to PRs as well.
  3. Gotta git it up for the Runs view. Once I accepted that time flows from right to left on the graph, I find it very satisfying to see my runs visually.
  4. Consistency. SL makes it easy to keep the TF run environments consistent. We have a Docker container we run on in the SL public worker pool, so every run runs in a consistent managed environment. This benefit of this was reenforced, when one of my devs scrogged his local env to the point that he couldn’t run TF, so he just used local-preview to run it in SL, which unblocked him.
  5. OPA. Lots there and it really gives you a lot of flexibility. We’re getting a lot of mileage out of policies using tag-based triggers WRT day-to-day solutions:

SL allowed me to complete the automation of our bastion-host access. After years of my approving PRs for FW access and then running local scripts to deploy those changes to multiple environments, it’s all in a multi-stack now and fully managed through PRs. When i first set this up, I had neglected to restrict merge access in that repo, so it was a surprise to see that the dev had gotten the PR approved and merged it, and that SL had taken care of the rest, deploying those changes to all client environments….completely without me. We do this now for app deploys, as well as changes to IAM/user mgmt, FW access, and DNS.

We deploy to multiple, essentially identical client accounts. Once i got my TF set up to deploy these “multi-stacks”, it became really easy to add additional stacks, by just copying and updating existing stack or muti-stack config files. I now have 4 multi-stacks that hit 8 environments. The multistacks break out very generally into app-related stuff, non-app infrastructure, non-app access, and account-level stuff. Without a TACOS solution, it would be difficult to manage all of these, both at the config level as well as in terms of running them.

azec avatar

Thank you for the feedback @Eric Berg. Highly appreciated and valued. It provides me with 1st-hand experience that I can benefit from especially as I start narrowing down focus of features of Spacelift I want to self-evaluate once I request the trial. I am wrapping up the documentation related to presenting Spacelift to my team this week and will try to have workers pools and runner by end of week - leveraging Cloudposse’s AWS modules in TF.

Eric Berg avatar
Eric Berg

NP, @azec.

Matt Gowie avatar
Matt Gowie

@azec one thing worth mentioning since you seem focused on a strong initial trial — The folks at SL are super friendly. I imagine they’ll give you a longer trial if you request and share your thoughts and direction why.

1
Eric Berg avatar
Eric Berg

Good point, @Matt Gowie. The SL folks are very helpful. They especially love feature requests!

azec avatar

Thanks @Matt Gowie, I will def. have that in mind.

azec avatar

I have finished writeup of evaluation proposal yesterday. Moving forward with trying to get Spacelift runner ready and hopefully pushed to ECR by EOD tomorrow.

azec avatar

One blocker I have hit is how to safely save GitLab Personal Access Token to be accessible to the Docker runner image when I request Terraform stack plans (through atmos ). The org allows only HTTPS not SSH.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

See how to use netrc files

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That is what we use

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
# The URL "[email protected]:" is used by `git` (e.g. `git clone`)
git config --global url."<https://github.com/>".insteadOf "[email protected]:"

# The URL "<ssh://[email protected]/>" is used by Terraform (e.g. `terraform init --from-module=...`)
# NOTE: we use `--add` to append the second URL to the config file
git config --global url."<https://github.com/>".insteadOf "<ssh://[email protected]/>" --add
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for private workers, we do it in user-data

%{ if github_netrc_enabled }
  export SPACELIFT_WORKER_EXTRA_MOUNTS=/root/.netrc:/conf/.netrc

  export GITHUB_TOKEN=$(aws ssm get-parameters --region=${region} --name ${github_netrc_ssm_path_token} --with-decryption --query "Parameters[0].Value" --output text)
  export GITHUB_USER=$(aws ssm get-parameters --region=${region} --name ${github_netrc_ssm_path_user} --with-decryption --query "Parameters[0].Value" --output text)

  echo "Creating .netrc" | tee -a /var/log/spacelift/info.log
  NETRC="/root/.netrc"
  printf "machine github.com\n" > "$NETRC"
  printf "login %s\n" "$GITHUB_USER" >> "$NETRC"
  printf "password %s\n" "$GITHUB_TOKEN" >> "$NETRC"
  echo "Created .netrc" | tee -a /var/log/spacelift/info.log
%{ endif }
azec avatar

Long thread here, but I finally managed to build Spacelift runner Docker image in our GitLab CI/CD pipeline (which uses Kubernetes runners so I had to use Kaniko to avoid Docker-in-Docker problems with image builds).

azec avatar

This week (starting tomorrow) I am shifting focus to Terraform modules for Spacelift private workers.

1
azec avatar

My dream of trying to get this setup right for my team stops here today In the early morning in a call with sub. director of engineering I have learned that due to the org restructuring, my position is being eliminated at Verizon.

1
Matt Gowie avatar
Matt Gowie

Ah so sorry to hear about that Amer! It seemed like you were well on your way and were doing great work.

Honestly, sounds to me like Verizon’s loss! I’m sure you’ll go and find an amazing role somewhere else where you can implement what you’ve been doing and be well appreciated for it!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That really sucks @azec! I agree with @Matt Gowie - sounds like you can easily bounce back from this and get a job somewhere else. Please post in #jobs!

azec avatar

Can Spacelift get ECR runner images if ECR has KMS key encryption enabled (under the assumption that IAM Role which Spacelift stack assumes has permissions for same KMS key)?

 avatar

I believe so, yes, but I’d imagine this would only work on private workers.

azec avatar

Following up on this. Per https://docs.spacelift.io/integrations/docker#using-private-docker-images it seems that public worker pools only can work with public images, so I would probably have to host runner in public ECR registry.

@, would you know if private worker pools are available in 14-day trial of Spacelift ?

 avatar

Indeed they are, the trial is all features of the enterprise plan

azec avatar

Thank you!

2022-06-29

    keyboard_arrow_up