#gitops (2019-04)

Discuss continuous delivery of infrastructure Archive: https://archive.sweetops.com/gitops/

2019-04-30

btai avatar

So im trying to deploy to my k8s cluster w/ helm via my ci tool and I’d like for those helm deploys to kick off based off merges into special branches. For example, merging a PR into develop would kick off a helm upgrade to the dev cluster and merging a release PR into master would kick off a helm upgrade to the prod cluster. How are those of you following this pattern handling the fact that helm doesn’t redeploy pods using images on a specific tag (develop) even if there is a more recent version of that image?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve used a number of strategies

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for develop you can use helm upgrade --recreate-pods option

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or just always tag yur docker images also with a git ref

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then use the git ref for the image tag

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

--recreate-pods will cause a brief outage

btai avatar

so your repo has a tag per commit?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we tag every docker image with a git ref

btai avatar

ah

btai avatar

i misread that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that way there is a 1:1 relationship between git and docker

btai avatar

so when you merge into master, that will re-build the docker image meaning that you would be technically deploying with a different image (although should be identical)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, but our process is different

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we merge to master, we build

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

especially since we usually squash merge

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use release tags to deploy to environments

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

release tags therefore only retag a docker image. no rebuilding required.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

master is continuously delivered to staging

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

release tags go to production.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

1.2.3-foobar goes to the foobar environment

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. 1.2.3-prod

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

1.2.3-preprod

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

1.2.3 goes no where

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then for PRs we deploy each one into a new namespace on the staging cluster

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. pr1234 of the example repo gets deployed to pr1234-example namespace

:--1:1
btai avatar

so for your prod deploys, they are not being triggered via git

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

they totally are

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

100% git driven

btai avatar

ohh

btai avatar

youre saying

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
btai avatar

by tagging a commit, you are pushing to specific environments?

btai avatar

so when you push that tag, it triggers the deploy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

retag the image

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

btai avatar

thank you, i believe thats the step i was missing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

deploy it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in our case, we use github

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so we use the UI for releases

btai avatar

i do to

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but essentially it’s tags

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

releases also have webhook events

btai avatar

so you create a release, which triggers a deploy in codefresh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we register those with codefresh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yup

btai avatar

nice thats what i’ll do

btai avatar

how do you like codefresh?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and then you can throw in an approval step

btai avatar

i believe you guys are big fans

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so it queues it up for deployment, but someone higher up has to click approve

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, we’re big time fans of #codefresh

btai avatar

approval step is a codefresh thing or github?

btai avatar

codefresh correct

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

approval step is a codefresh step

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
ask_for_permission:
    type: pending-approval
    title: Deploy release?

btai avatar

nice nice

btai avatar

we’re on circle right now, which isnt my favorite

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, circle definitely has the marketshare

btai avatar

but i believe i can perform all of those still

btai avatar

with circle

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and the two have been converging on functionality

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but codefresh has tighter k8s integration and native helm support

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

plus kanban boards so youo can view your releases and what stage they are in

btai avatar

yeah ive built my own helm deploy script to run from circle

btai avatar

thats why i was interested in drone.io but right now its more affordable to pay for circle than run our own 3 node k8s cluster for drone

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, i dn’t like hosting CI/CD

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s foundational infrastructure, that when everything else is done right (IaC) it’s just easier to treat the CI/CD as an axiom; it just exists and we don’t need to worry about how.

btai avatar

same, but drone does run jobs as native k8s jobs which removes the need for managing agents which is nice

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s nice

btai avatar

but yeah, thinking about our CI/CD platform having uptime issues during a client-facing outage would suck

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i feel like we got enough to manage…. kiam, cert-manager, prometheus, grafana, teleport, kibana, fluentd, external-dns, keycloak, etc. . and for those there is no hosted option.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but for CI/CD there is.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

btai avatar

haha yeah

btai avatar

sysdig datadog

btai avatar

but i get it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, true

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sumologic, splunk

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but daaaaaaang the prices $$$

btai avatar

yeah they are not cheap

btai avatar

thanks for the help, answered with the exact solution i was looking for @Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @btai! glad I could help

2019-04-11

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
GitOps with Terraform on Codefresh (Webinar)

Infrastructure as code, pipelines as code, and now we even have code as code! =P In this talk, we show you how we build and deploy applications with Terraform using GitOps with Codefresh. Cloud Posse is a power user of Terraform and have written over 140 Terraform modules. We’ll share how we handl

:--1:1
jamie avatar
jamie

Just read all your slide decks. Very nice demos.

GitOps with Terraform on Codefresh (Webinar)

Infrastructure as code, pipelines as code, and now we even have code as code! =P In this talk, we show you how we build and deploy applications with Terraform using GitOps with Codefresh. Cloud Posse is a power user of Terraform and have written over 140 Terraform modules. We’ll share how we handl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @jamie!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Had a lot of fun making them

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(video will be posted later)

2019-04-10

2019-04-02

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Here are the slides from “Scale 17x: Intro to GitOps”

https://cloudposse.com/slides/scale-17x-intro-to-gitops/

cool-doge2
    keyboard_arrow_up