#release-engineering (2019-11)

jenkins_ci All things CI/CD. Specific emphasis on Codefresh and CodeBuild with CodePipeline.

CI/CD Discussions

Archive: https://archive.sweetops.com/release-engineering/

2019-11-04

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
peter-evans/create-pull-request

A GitHub action to create a pull request for changes to your repository in the actions workspace. - peter-evans/create-pull-request

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Very cool GitHub action

2019-11-05

Sharanya avatar
Sharanya

Need a suggestion : We are hosting a React website on S3, deployed through Jenkins. We have a story to set the site’s environment variables at deployment time (e.g. APIs URL, Vertex Cloud Auth Server , Redirection URL, etc.) but wanted to ping you guys to see if you are doing something like this today

Darren Cunningham avatar
Darren Cunningham

we’re using AWS SSM Parameters – we use a multi-account strategy so each environment: DEV, TEST, PROD is a separate account with their own SSM parameters (keys are the same, values are different). My build process pulls the run time environment variables from SSM param store. for local dev, the team uses a .env file with default values.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, so a long these lines you’re basically going to want to do text replacement on the generated react site

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

tools like gomplate or envsubst are what we would use

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

whatever you upload to S3 needs to be static and cannot use envs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so you’ll need to use the envs as part of the CD process

vitaly.markov avatar
vitaly.markov

We use the webpack plugin to setup env variables during build stage on CI (https://www.npmjs.com/package/dotenv-webpack) and as described above you can pull variables from AWS SSM or just put your variables to .env.development, .env.staging, .env.production and keep in git, cause these variables should not contain sensitive data

dotenv-webpack

A simple webpack plugin to support dotenv.

1
vitaly.markov avatar
vitaly.markov

otherwise VueJs use the following convention https://cli.vuejs.org/guide/mode-and-env.html

Modes and Environment Variables | Vue CLI

Standard Tooling for Vue.js Development

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That sounds even better

2019-11-06

MattyB avatar

Anyone have trouble finding the line with abstraction in Jenkins pipeline libs? I forked someone’s codebase at worked and played around with it. While it works, you have to drudge through documentation that’s formatted for github which is terrible compared to typical code documentation. You (or someone) end up trying to write reusable code but end up with a bunch of one-offs (multiple docker run stages). You’re forced to figure out all the variables that are set by hoping the person that wrote the documentation didn’t miss anything, so you end up looking through the code to verify. I think with my current project I’m going to leave most of the logic in our pipeline.yaml and let it tell the story of how our build works instead of the Jenkinsfile. I’m pretty sure I could write a book about it or find someone else that’s already gone through my grief. Just venting a bit too

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@MattyB today at #office-hours I want to ask more about people’s experience with Jenkins and #kubernetes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Join us if free!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

My motivation is we are starting a Jenkins project next week.

1
1
MattyB avatar

Thanks for the invite. Just registered.

1
s2504s avatar

That is great @Erik Osterman (Cloud Posse) Jenkins is a swiss knife in the CI/CD world and it must be in SweetOps team’s backpack

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha, yea, it was and then it wasn’t it’s a love/hate relationship.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but seeing as how many companies still use it, I totally agree!

s2504s avatar

yea, but in most cases Jenkins is very redundant for usual operations - build couple images and push them to registry. For this purpose I choose “free” CI tools linke Drone, GitlabCI and etc. Or if we need self managed solution - GitLab CI

s2504s avatar

But if we need some advanced logic in or pipelines - Jenkins is that we need

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • auto update packages (open a PR every night)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • auto label PR with each package updated (great for mono repos)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • auto clean up branches on merge
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • auto assign PR for review
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(all github actions)

2019-11-13

loren avatar

anyone else trying mergify? started using it to auto-approve/merge prs that dependabot creates, with conditions that the required status checks pass… https://doc.mergify.io/

loren avatar
pull_request_rules:
  - name: auto-approve dependabot pull requests
    conditions:
      - author~=dependabot\[bot\]|dependabot-preview\[bot\]
      - status-success=continuous-integration/appveyor/pr
      - status-success=continuous-integration/travis-ci/pr
    actions:
      review:
        type: APPROVE

  - name: auto-merge dependabot pull requests
    conditions:
      - author~=dependabot\[bot\]|dependabot-preview\[bot\]
      - "#approved-reviews-by>=1"
    actions:
      merge:
        method: merge
loren avatar

certainly some overlap in what it does and github actions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No, but have somethings bookmarked to achieve the same thing with actions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Want to use actions :-)

2019-11-14

Chris Fowles avatar
Chris Fowles
Introducing Argo Flux - A Weaveworks-Intuit-AWS Collaborationattachment image

Today Weaveworks announces a partnership with Intuit to create Argo Flux, a major open source project to drive GitOps application delivery for Kubernetes via an industry-wide community. Argo Flux combines the Argo CD project led by Intuit with the Flux CD project driven by Weaveworks.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

super, waiting to see what they will have in the end

Chris Fowles avatar
Chris Fowles

yeh - i’m getting a bit annoyed at the “we’re going to work on a thing” announcements

Chris Fowles avatar
Chris Fowles

it’s becoming common

2019-11-19

jose.amengual avatar
jose.amengual

I was going to try , https://github.com/cloudposse/terraform-aws-ecs-atlantis, is there anything I should know about using this repo?

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

jose.amengual avatar
jose.amengual

we want to run atlantis in ECS

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We assume you BYO Docker container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We use geodesic

jose.amengual avatar
jose.amengual

BYO= ?

jose.amengual avatar
jose.amengual

build your own ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Bring your own

jose.amengual avatar
jose.amengual

or I can use the one in docker registry?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Perhaps but untested

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have a more opinionated approach where by we deploy Atlantis in each AWS account we want to manage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

And have a one to one correlation between AWS accounts and GitHub repos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This ensures webhooks and repos and accounts are all aligned

jose.amengual avatar
jose.amengual

well we want the same

jose.amengual avatar
jose.amengual

each account with it’s own atlantis

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Give it a shot

jose.amengual avatar
jose.amengual

but you have multiples repos per account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have one repo per account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also have you ruled out terraform cloud?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s a bit more polished

jose.amengual avatar
jose.amengual

we just the demo with hashicorp

jose.amengual avatar
jose.amengual

to be honest I do not know if is such a huge advantage now that you have github actions self hosted runners

jose.amengual avatar
jose.amengual

then we limit the UIs to just github

jose.amengual avatar
jose.amengual

and PR that is how we work usually

jose.amengual avatar
jose.amengual

so having a nice UI is not a huge win for Us

jose.amengual avatar
jose.amengual

to be honest I was pretty dissapointed about the demo

jose.amengual avatar
jose.amengual

and workspaces can’t assume roles

jose.amengual avatar
jose.amengual

so it means that each workspace is one AWS account with aws key/secret in the settings

jose.amengual avatar
jose.amengual

not so clear is those setting can be set trough the API

jose.amengual avatar
jose.amengual

and one workspace is one repo so if we use 5 repos per account it means we need 5 workspaces all with the keys duplicated

jose.amengual avatar
jose.amengual

so if you add key rotation to add it becomes pretty redious

jose.amengual avatar
jose.amengual

and they are not FIPS compliant

jose.amengual avatar
jose.amengual

so if they get hacked I gess you could get pretty F…..

jose.amengual avatar
jose.amengual

I guess

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Join us for #office-hours if you get stuck

jose.amengual avatar
jose.amengual

I have a strip-down version of the ecs-web-app module that we use for all ECS related tasks

jose.amengual avatar
jose.amengual

that is very similar to what is on that repo

jose.amengual avatar
jose.amengual

but this

jose.amengual avatar
jose.amengual

is kinda weird

jose.amengual avatar
jose.amengual

you are only allowing https/http and 53 out from atlantis ?

Igor Rodionov avatar
Igor Rodionov

Have anyone try https://www.gocd.org/ ?

Open Source Continuous Delivery and Release Automation Server | GoCDattachment image

GoCD is an open source build and release tool from ThoughtWorks. GoCD supports modern infrastructure and helps enterprise businesses get software delivered faster, safer, and more reliably.

Alex Siegman avatar
Alex Siegman

I used it at a past job a year or two ago running in a windows shop and not containers or anything

Open Source Continuous Delivery and Release Automation Server | GoCDattachment image

GoCD is an open source build and release tool from ThoughtWorks. GoCD supports modern infrastructure and helps enterprise businesses get software delivered faster, safer, and more reliably.

Alex Siegman avatar
Alex Siegman

Any specific questions?

Igor Rodionov avatar
Igor Rodionov

@Alex Siegman not yet. Would be when I will make my hand dirty

Alex Siegman avatar
Alex Siegman

I think the most difficulty I had is we didn’t use the various stages and pipelines and environments in the way gocd designed. Their organization of those things are a bit weird. I’d have to look again to remember the specifics, but overall it was easy enough to use as anything else like Jenkins, etc.

Igor Rodionov avatar
Igor Rodionov

@Alex Siegman thanks. That is very useful

2019-11-20

2019-11-21

2019-11-25

Igor avatar

CircleCi seems to be having outages on weekly basis. Is this common for other CI/CD platforms?

Steven avatar

Never used to happen. Haven’t used it much in the last year. Suspect they may be going through some growing pains

Chris Fowles avatar
Chris Fowles

We’ve been pretty happy with BuildKite

2019-11-26

Darren Cunningham avatar
Darren Cunningham

I’m very happy with Codefresh, when their macOS supports comes out of beta I’m planning to move our react native app over and shutdown our CircleCI account.

amelia.graycen avatar
amelia.graycen

I don’t suppose anybody has gotten these Jenkins automated upgrades working with the Docker image? We have them in our current version, but I’m looking in to moving Jenkins off of Windows and want to make sure we keep the feature.

https://wiki.jenkins.io/display/JENKINS/Automated+Upgrade

amelia.graycen avatar
amelia.graycen

I wouldn’t be shocked if I were misinterpreting this document to some extent.

2019-11-29

Brij S avatar

for anyone who has used github actions, if I was to create a git action that had

on:
  push:
    branches:
      - master

would you think this runs when a PR is merged to master?

Brij S avatar

I basically want to run a workflow when master has a change

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

You can test it by creating a PR to a dummy branch.

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

But yeah, it will run when a PR is merged to master

    keyboard_arrow_up