#release-engineering (2020-2)

jenkins_ci All things CI/CD. Specific emphasis on Codefresh and CodeBuild with CodePipeline.

CI/CD Discussions

Archive: https://archive.sweetops.com/release-engineering/

2020-02-12

2020-02-08

2020-02-07

Zachary Loeber avatar
Zachary Loeber
12:48:55 AM

@Zachary Loeber has joined the channel

2020-02-06

PePe avatar

Hi, does cloudposse/atlantis:latest supports 0.12.16 ? AFAIK it downloads any version, right ?

aknysh avatar
aknysh

it can download TF versions specified in config

aknysh avatar
aknysh

in our Docker container, we install atlantis and TF version what we need, and atlantis just uses it

PePe avatar

is that by env variable ?

PePe avatar

ENV DEFAULT_TERRAFORM_VERSION=0.12.16

PePe avatar

or :

PePe avatar

AVAILABLE_TERRAFORM_VERSIONS

PePe avatar

although AVAILABLE_TERRAFORM_VERSIONS contains DEFAULT_TERRAFORM_VERSIONand then it goes on the loop to download the versions

PePe avatar
cloudposse/atlantis

GitOps for Teams (experimental hard fork of atlantis) - cloudposse/atlantis

PePe avatar

my Dockerfile :

FROM segment/chamber:2 AS chamber
FROM cloudposse/atlantis:latest


\# install terraform binaries
ENV DEFAULT_TERRAFORM_VERSION=0.12.16
COPY --from=chamber /chamber /bin/chamber
COPY atlantis-repo-config.yaml /
ENTRYPOINT ["/bin/chamber", "exec", "ecs-atlantis-test", "--", "docker-entrypoint.sh", "server"]
PePe avatar

I just copy the command from the fork that downloads terraform

Erik Osterman avatar
Erik Osterman

I don’t think we shop an atlantis container

Erik Osterman avatar
Erik Osterman

If we do, it is out of date or not maintained

Erik Osterman avatar
Erik Osterman

We distribute Atlantis as an alpine package

Erik Osterman avatar
Erik Osterman

That we install like the other tools

PePe avatar

so you are saying that the image in docker hub is unmaintained and I should not use it ?

PePe avatar

so to use the fork you guys have I will have to build from the repo/fork you guys have

PePe avatar

is that a correct assumption ?

PePe avatar

@Erik Osterman

Erik Osterman avatar
Erik Osterman

Yes, and let me explain

Erik Osterman avatar
Erik Osterman

our fundamental position on this is that running “alantis” from some kind of shared docker image is more or less useless

Erik Osterman avatar
Erik Osterman

it doesn’t solve how custom providers get installed

Erik Osterman avatar
Erik Osterman

it doesn’t solve how any other tools or dependencies will get installed

Erik Osterman avatar
Erik Osterman

if someone depends on helm, helmfile, terragrunt etc…. it won’t do much good

Erik Osterman avatar
Erik Osterman

that’s why we distribute the package instead so that it can be installed in a docker image you control

Erik Osterman avatar
Erik Osterman

In our model, we use cloudposse/geodesic as our base image (which is up to date)

Erik Osterman avatar
Erik Osterman

and then add the exrta tool we depend on.

PePe avatar

I completely agree, make sense

PePe avatar

and you guys do no host those alpine packages I guess

PePe avatar

so I will have to build this : https://github.com/cloudposse/atlantis

cloudposse/atlantis

GitOps for Teams (experimental hard fork of atlantis) - cloudposse/atlantis

PePe avatar

myself

PePe avatar

if I want to use your fork

Erik Osterman avatar
Erik Osterman

oh we do!

Erik Osterman avatar
Erik Osterman
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

Erik Osterman avatar
Erik Osterman

Add something like this to your dockerfile

Erik Osterman avatar
Erik Osterman

\# Install the cloudposse alpine repository
ADD <https://apk.cloudposse.com/[email protected]> /etc/apk/keys/
RUN echo "@cloudposse <https://apk.cloudposse.com/3.11/vendor>" >> /etc/apk/repositories
Erik Osterman avatar
Erik Osterman

then you can do apk add [email protected]

PePe avatar

ahhhhh cool

Erik Osterman avatar
Erik Osterman

where cloudposse-atlantis is the naem of our package from our fork

Erik Osterman avatar
Erik Osterman

Also, check out the other packages we have

Erik Osterman avatar
Erik Osterman

dozens and dozens

Erik Osterman avatar
Erik Osterman

thanks to @Zachary Loeber

PePe avatar

this is awesome, well I’m glad I got it working with the old image at least to do my demo

1
PePe avatar

now I will update

PePe avatar

it was VERY hard to get my head around this

Erik Osterman avatar
Erik Osterman

(i know)

PePe avatar

and….I might have found a bug

Erik Osterman avatar
Erik Osterman

a master class in ECS/Atlatnis/codebuild/modules/etc

Erik Osterman avatar
Erik Osterman

I would like it to be simpler

PePe avatar

I think there is a problem with the example….

PePe avatar
cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar

well what happens is that the ingress module creates a TG and the alb module too

PePe avatar

but the alb module listener rule uses the alb-default TG

PePe avatar
cloudposse/terraform-aws-alb-ingress

Terraform module to provision an HTTP style ingress rule based on hostname and path for an ALB using target groups - cloudposse/terraform-aws-alb-ingress

PePe avatar

I think so….

PePe avatar

I got pretty confused when following the dependencies

PePe avatar

I think I did something wrong

Erik Osterman avatar
Erik Osterman

did you figure it out?

PePe avatar

I have no tried yet again, I’m going to do some cleanup and see but I think this could be the ingress module. I created a PR long ago that was merges to allow to pass a target group arn in case the target groups were created by other means, so i think this could be a case were the alb module is creating the TG and the ingress module too.

PePe avatar

so maybe is a matter of passing the TG arn

PePe avatar

but I need to test my theory

2020-02-05

IckesJ avatar
IckesJ

Any notes on how you guys at Cloud Posse or others that manage many modules as separate repos do your release process? It would be greatly appreciated.

I see the makefile works as the runner for the unit tests which is awesome and I’m assuming happens upon merging to master for module X. But what determines module X is ready to release into an new version? A global find and replace of the old version tag to the new, run all unit & integration tests & if all pass push release?

Erik Osterman avatar
Erik Osterman

@IckesJ we talk about it here: https://github.com/cloudposse/docs/issues/335

Document our Semver Strategy · Issue #335 · cloudposse/docs

what it&#39;s not clear how we currently do versioning why our strategy is unique because we tag every single merge to master our versioning strategy allows us to systematically and consistently in…

Erik Osterman avatar
Erik Osterman

basically, every time you merge to master, you bump the version.

Erik Osterman avatar
Erik Osterman

bugs = patch releases

Erik Osterman avatar
Erik Osterman

every other release is a minor release.

Erik Osterman avatar
Erik Osterman

major releases are milestone driven.

Erik Osterman avatar
Erik Osterman

keep in mind that pre-1.0 has a special meaning in semver

Erik Osterman avatar
Erik Osterman

@Julian Gindi might have some more thoughts on this.

Julian Gindi avatar
Julian Gindi
01:40:19 AM

@Julian Gindi has joined the channel

Santiago Campuzano avatar
Santiago Campuzano

Welcome @Julian Gindi !

Erik Osterman avatar
Erik Osterman

(he just did a big presentation at a local meetup on semver and when/how to bump versions)

Erik Osterman avatar
Erik Osterman

IMO, the main purpose of semver is not to communicate the stability of the functionality. that’s almost impossible to guarantee. even a bug fix can be a breaking change for someone else who had a workaround for that bug.

Erik Osterman avatar
Erik Osterman

I assume that every change could be breaking for someone.

Erik Osterman avatar
Erik Osterman

therefore, IMO the purpose behind semver is to pin software so it only changes when you expect it to.

Erik Osterman avatar
Erik Osterman

thus, I hate it when projects don’t cut a release for every merge to master.

Erik Osterman avatar
Erik Osterman

and that’s why I prefer every merge to master to have release so I can gauge our distance from the latest release.

:--1:2
Erik Osterman avatar
Erik Osterman

git sha’s suck for humans.

Julian Gindi avatar
Julian Gindi

All of this I agree with, I do think you can add safety and a bit of structure to internal API’s and set rules on which services can talk to what, but it’s most powerful when used as a final “resolution” for software and being able to see how things change over time.

Julian Gindi avatar
Julian Gindi

I have a tool to help with this process, but it’s almost identical to what Erik suggested

Erik Osterman avatar
Erik Osterman

do you have a recording of your talk?

Julian Gindi avatar
Julian Gindi

We do

IckesJ avatar
IckesJ

This is awesome info guys! Thanks & would love to see/hear the recording

Julian Gindi avatar
Julian Gindi

Here is a repo that might “automate” the boring mechanical bits of incrementing semver https://github.com/JulianGindi/auto-semver

JulianGindi/auto-semver

A small python tool that aims to let you focus on writing software, while it versions it for you. - JulianGindi/auto-semver

loren avatar
loren

curious about how this might compare to bumpversion? https://github.com/c4urself/bump2version/

c4urself/bump2version

Version-bump your software with a single command. Contribute to c4urself/bump2version development by creating an account on GitHub.

Julian Gindi avatar
Julian Gindi

That looks like a great package, and seems more mature. I think the only issue I have with it is that it seems to require me to pass in the current, while my script automatically determines that based on git tags. Not sure if this script is able to do that, not clear on first glance.

Julian Gindi avatar
Julian Gindi

My approach was also a bit more simplicity, but I’m going to dig into this tool a bit, I rather use and support a tool that has a larger community if it accomplishes my personal needs

Julian Gindi avatar
Julian Gindi

thus, I hate it when projects don't cut a release for every merge to master. 100% @Erik Osterman

Corey Gale avatar
Corey Gale

• Recording of the whole presentation//vimeo.com/388711413> (including @Julian Gindi’s talk on SemVer)

• Julian’s deck and notes on SemVer//gindi.io/semver.html>

• Industry Updates slides//slides.com/coreygale/west-la-devops-5-versioning#/4>

West LA DevOps #5: Versioning attachment image

A presentation created with Slides.

1
1
:--1:2
Erik Osterman avatar
Erik Osterman

Thanks @Corey Gale

Corey Gale avatar
Corey Gale

No problem thanks again for all your support!

Erik Osterman avatar
Erik Osterman

@Julian Gindi did you consider showing a github actions code snippet that can be used to automate the semver stuff with your tool?

Julian Gindi avatar
Julian Gindi

So we have some bits and pieces, but I should 100% add something like that to the repo. My intention was to have it used with CI and it’s certainly how we use it.

Erik Osterman avatar
Erik Osterman
cloudposse/actions

Our Library of GitHub Actions. Contribute to cloudposse/actions development by creating an account on GitHub.

Erik Osterman avatar
Erik Osterman

we have a lot of auto-* type actions

Erik Osterman avatar
Erik Osterman

auto-approve, auto-assign, auto-merge, auto-readme, etc

Julian Gindi avatar
Julian Gindi

Perfect place to slot in an auto-semver

Erik Osterman avatar
Erik Osterman

IckesJ avatar
IckesJ

This actions library is great. I just created a couple days ago to do BulkRepoChanges & pipeline in azdo to be able to do a find and replace in files across all our tf module repos &/or run a cmd like pre-commit run -a….auto creates the branch, pr, etc…i was also wondering how you guys managed hundreds of repos. Digging in reading also brought to light dependabot, i haven’t seen that before…pretty cool as well.

Erik Osterman avatar
Erik Osterman

yea, dependabot is great and pullreminders

michal.matyjek avatar
michal.matyjek

damn, pullreminders no longer allows signups? any alternative?

Erik Osterman avatar
Erik Osterman

ahhh bummer!

Erik Osterman avatar
Erik Osterman
Pull Reminders: Pull request reminders for Slack & GitHub attachment image

Dev teams at 1,000+ companies like Pivotal, Instacart, and WeWork use Pull Reminders to stay on top of code reviews and ship faster.

IckesJ avatar
IckesJ

oh ya that is nice

IckesJ avatar
IckesJ

@Julian Gindi - nice presentation…I like the aviation angles…I have my instrument rating & loved every second of the learning process.

Julian Gindi avatar
Julian Gindi

@IckesJ nice! Absolutely a goal of mine to finish my private and get instrument one day! Will have to talk more…

:--1:1
    keyboard_arrow_up