#release-engineering (2019-11)

All things CI/CD. Specific emphasis on Codefresh and CodeBuild with CodePipeline.

CI/CD Discussions

Archive: https://archive.sweetops.com/release-engineering/

2019-11-29

Brij S

for anyone who has used github actions, if I was to create a git action that had

on:
  push:
    branches:
      - master

would you think this runs when a PR is merged to master?

Brij S

I basically want to run a workflow when master has a change

Vlad Ionescu

You can test it by creating a PR to a dummy branch.

Vlad Ionescu

But yeah, it will run when a PR is merged to master

2019-11-26

Darren Cunningham

I’m very happy with Codefresh, when their macOS supports comes out of beta I’m planning to move our react native app over and shutdown our CircleCI account.

amelia.graycen

I don’t suppose anybody has gotten these Jenkins automated upgrades working with the Docker image? We have them in our current version, but I’m looking in to moving Jenkins off of Windows and want to make sure we keep the feature.

https://wiki.jenkins.io/display/JENKINS/Automated+Upgrade

amelia.graycen

I wouldn’t be shocked if I were misinterpreting this document to some extent.

2019-11-25

imiltchman

CircleCi seems to be having outages on weekly basis. Is this common for other CI/CD platforms?

Steven

Never used to happen. Haven’t used it much in the last year. Suspect they may be going through some growing pains

Chris Fowles

We’ve been pretty happy with BuildKite

2019-11-21

2019-11-20

2019-11-19

I was going to try , https://github.com/cloudposse/terraform-aws-ecs-atlantis, is there anything I should know about using this repo?

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

we want to run atlantis in ECS

Erik Osterman

We assume you BYO Docker container

Erik Osterman

We use geodesic

BYO= ?

build your own ?

Erik Osterman

Bring your own

or I can use the one in docker registry?

Erik Osterman

Perhaps but untested

Erik Osterman

We have a more opinionated approach where by we deploy Atlantis in each AWS account we want to manage

Erik Osterman

And have a one to one correlation between AWS accounts and GitHub repos

Erik Osterman

This ensures webhooks and repos and accounts are all aligned

well we want the same

each account with it’s own atlantis

Erik Osterman

Erik Osterman

Give it a shot

but you have multiples repos per account

Erik Osterman

We have one repo per account

Erik Osterman

Also have you ruled out terraform cloud?

Erik Osterman

It’s a bit more polished

we just the demo with hashicorp

to be honest I do not know if is such a huge advantage now that you have github actions self hosted runners

then we limit the UIs to just github

and PR that is how we work usually

so having a nice UI is not a huge win for Us

to be honest I was pretty dissapointed about the demo

and workspaces can’t assume roles

so it means that each workspace is one AWS account with aws key/secret in the settings

not so clear is those setting can be set trough the API

and one workspace is one repo so if we use 5 repos per account it means we need 5 workspaces all with the keys duplicated

so if you add key rotation to add it becomes pretty redious

and they are not FIPS compliant

so if they get hacked I gess you could get pretty F…..

I guess

Erik Osterman

Join us for #office-hours if you get stuck

I have a strip-down version of the ecs-web-app module that we use for all ECS related tasks

that is very similar to what is on that repo

but this

is kinda weird

you are only allowing https/http and 53 out from atlantis ?

Igor Rodionov

Have anyone try https://www.gocd.org/ ?

Open Source Continuous Delivery and Release Automation Server | GoCD

GoCD is an open source build and release tool from ThoughtWorks. GoCD supports modern infrastructure and helps enterprise businesses get software delivered faster, safer, and more reliably.

Alex Siegman

I used it at a past job a year or two ago running in a windows shop and not containers or anything

Open Source Continuous Delivery and Release Automation Server | GoCD

GoCD is an open source build and release tool from ThoughtWorks. GoCD supports modern infrastructure and helps enterprise businesses get software delivered faster, safer, and more reliably.

Alex Siegman

Any specific questions?

Igor Rodionov

@Alex Siegman not yet. Would be when I will make my hand dirty

Alex Siegman

I think the most difficulty I had is we didn’t use the various stages and pipelines and environments in the way gocd designed. Their organization of those things are a bit weird. I’d have to look again to remember the specifics, but overall it was easy enough to use as anything else like Jenkins, etc.

Igor Rodionov

@Alex Siegman thanks. That is very useful

2019-11-14

Chris Fowles
Introducing Argo Flux - A Weaveworks-Intuit-AWS Collaboration

Today Weaveworks announces a partnership with Intuit to create Argo Flux, a major open source project to drive GitOps application delivery for Kubernetes via an industry-wide community. Argo Flux combines the Argo CD project led by Intuit with the Flux CD project driven by Weaveworks.

aknysh

super, waiting to see what they will have in the end

Chris Fowles

yeh - i’m getting a bit annoyed at the “we’re going to work on a thing” announcements

Chris Fowles

it’s becoming common

2019-11-13

loren

anyone else trying mergify? started using it to auto-approve/merge prs that dependabot creates, with conditions that the required status checks pass… https://doc.mergify.io/

loren
pull_request_rules:
  - name: auto-approve dependabot pull requests
    conditions:
      - author~=dependabot\[bot\]|dependabot-preview\[bot\]
      - status-success=continuous-integration/appveyor/pr
      - status-success=continuous-integration/travis-ci/pr
    actions:
      review:
        type: APPROVE

  - name: auto-merge dependabot pull requests
    conditions:
      - author~=dependabot\[bot\]|dependabot-preview\[bot\]
      - "#approved-reviews-by>=1"
    actions:
      merge:
        method: merge
loren

certainly some overlap in what it does and github actions

Erik Osterman

No, but have somethings bookmarked to achieve the same thing with actions

Erik Osterman

Want to use actions :-)

2019-11-06

MattyB

Anyone have trouble finding the line with abstraction in Jenkins pipeline libs? I forked someone’s codebase at worked and played around with it. While it works, you have to drudge through documentation that’s formatted for github which is terrible compared to typical code documentation. You (or someone) end up trying to write reusable code but end up with a bunch of one-offs (multiple docker run stages). You’re forced to figure out all the variables that are set by hoping the person that wrote the documentation didn’t miss anything, so you end up looking through the code to verify. I think with my current project I’m going to leave most of the logic in our pipeline.yaml and let it tell the story of how our build works instead of the Jenkinsfile. I’m pretty sure I could write a book about it or find someone else that’s already gone through my grief. Just venting a bit too

Erik Osterman

@MattyB today at #office-hours I want to ask more about people’s experience with Jenkins and #kubernetes

Erik Osterman

Join us if free!

Erik Osterman

My motivation is we are starting a Jenkins project next week.

:--1:1
1
MattyB

Thanks for the invite. Just registered.

1
s2504s

That is great @Erik Osterman Jenkins is a swiss knife in the CI/CD world and it must be in SweetOps team’s backpack

Erik Osterman

haha, yea, it was and then it wasn’t it’s a love/hate relationship.

Erik Osterman

but seeing as how many companies still use it, I totally agree!

s2504s

yea, but in most cases Jenkins is very redundant for usual operations - build couple images and push them to registry. For this purpose I choose “free” CI tools linke Drone, GitlabCI and etc. Or if we need self managed solution - GitLab CI

s2504s

But if we need some advanced logic in or pipelines - Jenkins is that we need

Erik Osterman
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

Erik Osterman
  • auto update packages (open a PR every night)
Erik Osterman
  • auto label PR with each package updated (great for mono repos)
Erik Osterman
  • auto clean up branches on merge
Erik Osterman
  • auto assign PR for review
Erik Osterman

(all github actions)

2019-11-05

Sharanya

Need a suggestion : We are hosting a React website on S3, deployed through Jenkins. We have a story to set the site’s environment variables at deployment time (e.g. APIs URL, Vertex Cloud Auth Server , Redirection URL, etc.) but wanted to ping you guys to see if you are doing something like this today

Darren Cunningham

we’re using AWS SSM Parameters – we use a multi-account strategy so each environment: DEV, TEST, PROD is a separate account with their own SSM parameters (keys are the same, values are different). My build process pulls the run time environment variables from SSM param store. for local dev, the team uses a .env file with default values.

:--1:1
Erik Osterman

Yea, so a long these lines you’re basically going to want to do text replacement on the generated react site

Erik Osterman

tools like gomplate or envsubst are what we would use

Erik Osterman

whatever you upload to S3 needs to be static and cannot use envs

Erik Osterman

so you’ll need to use the envs as part of the CD process

vitaly.markov

We use the webpack plugin to setup env variables during build stage on CI (https://www.npmjs.com/package/dotenv-webpack) and as described above you can pull variables from AWS SSM or just put your variables to .env.development, .env.staging, .env.production and keep in git, cause these variables should not contain sensitive data

dotenv-webpack

A simple webpack plugin to support dotenv.

:--1:1
vitaly.markov

otherwise VueJs use the following convention https://cli.vuejs.org/guide/mode-and-env.html

Modes and Environment Variables | Vue CLI

Standard Tooling for Vue.js Development

Erik Osterman

That sounds even better

2019-11-04

Erik Osterman
peter-evans/create-pull-request

A GitHub action to create a pull request for changes to your repository in the actions workspace. - peter-evans/create-pull-request

Erik Osterman

Very cool GitHub action

    keyboard_arrow_up