#release-engineering (2021-11)
All things CI/CD. Specific emphasis on Codefresh and CodeBuild with CodePipeline.
CI/CD Discussions
Archive: https://archive.sweetops.com/release-engineering/
2021-11-22
anyone run into issues with github-actions and using an env value that starts with https://? e.g.
env:
REPO_ENDPOINT: 'https://...'
it keeps stripping the https:// from the env in the log, but i can’t find any docs on that behavior. and i’m pretty sure the yaml is fine?
i even tried an extra set of quotes…
env:
REPO_ENDPOINT: '"https://..."'
which i can see makes it into the env and the run command as "". so it seems to be a specific filter for urls?
mystery deepens… even removing the https:// from the env, and hardcoding it in the run command, it still gets stripped?
env:
REPO_ENDPOINT: ...
run: aws --endpoint-url https://${{ env.REPO_ENDPOINT }}
and it prints as
aws --endpoint-url ...
ok, it’s not the https:// specifically. it happens when the env value contains a period .
what fresh hell is this?
escaping it as \. in the env avoids stripping https:// but doesn’t remove the \ from the command…
leading and trailing periods are fine, so either of these work: .wut or wut. but this does not work: wut.wut
env:
REPO_ENDPOINT: wut.wut
steps:
- name: wut
run: echo https://${{ env.REPO_ENDPOINT }}
Run echo wut.wut
echo wut.wut
shell: /usr/bin/bash -e {0}
env:
REPO_ENDPOINT: wut.wut
wut.wut
omg, can’t even pass it as a literal string expression, ${{ 'wut.wut' }}
omg i am never getting that hour back. the issue with it stripping the https:// from the command is just a wonky issue with how github actions outputs the log. the value is passed to the command just fine. total red herring.
actual issue was due to aws v2 just returning a botocore object when there is an IMDS endpoint on the host, but it can’t figure out the region from that endpoint (which happens on Azure DevOps runners)… https://github.com/aws/aws-cli/issues/5262
added the AWS_DEFAULT_REGION env and it was fine
2021-11-24
hi folks, has anyone managed to get an idea the cost between GHA/ CodeFresh & Spacelift? Asking as i’ve been using GHA which is dead cheap however when i’m looking at the others i see 250$/m and that limited to 5 users.
the billing models are not the same. GHA charges per minute, while Spacelift charges a flat per-worker rate. You would need to do maths based on how many minutes you use per month to compare them
right, i see. Thanks @Alex Jurkiewicz for sharing this info, i’ll dig more into it
Yea, the comparisons won’t work. Spacelift and TFC are purpose built platforms specifically solving one problem for a niche demographic. Their business models necessarily are more expensive because they have a smaller market.
right i see the angle, make sense. thanks!
2021-11-25
2021-11-26
hi folks, in one of the office hours or other videos it was mentioned a CD pattern using GHA instead of codefresh/ spacelift. If i remember correctly it was said that a pattern of below could be achieved:
CI:
• TF config or modules in one git repo
• GHA to build and upload artefacts - ie lambda docker/ layer/ zip file to S3/ ECR (not use tf lambda modul to deal with packaging CD:
• a different / standalone git repo which will run TF plan/ apply to v envs. Q:
What i don’t get how the source repos (in case of poly-repo where various TF modules are scattered in N GH org repos) will:
• be synced to CD git repo? What sort of dir structure layout will be on the CD repo to cover for multiple deployments for different root modules ?
• any metadata consumed by CD and generated by CI - how will that be available ? i.e each lambda zip file or docker image / or TF code will generate a new tag/ release … that needs to be consumed somehow by CD GHA workflows no ? anyway i forgot who made the claim about using GHA as CD, will love to understand more the pattern
any insights of using GHA as CD? thank you
@Erik Osterman (Cloud Posse) sorry to tag you directly … if you don’t mind, can you please shed a bit more context on the story you mentioned in this video (go to 4:29).
• If you have some TF code + other code which is pushed over the “wall” by GHA to “deployment” repo in its own branch, are you saying tools like Spacelift etc can watch a git repo for any dynamic branch name and then trigger a pipe ? I suspect there is more low level info which was not shared w.r.t pre-req on Spacelift side to create a pipe mapped to a known branch name on source (aka eployment repo), have i got it right?
• i guess in your above story, the source git repo TF code relates to a “stack” (root module or solution etc) and not “component” modules of different versions which could be shared between different customers/ env Thanks for taking the time to respond
I am at reinvent this week - will take a look when I get back
now i have a better idea / understanding of various videos watched on your playlist due to the conv we had here , i guess the summary is that you are using the right tool for the job based on its own strengths:
i.e - specific to TF maybe TFC/ Spacelift; for apps (depending on the type/ deployment packaging etc )
@DaniC (he/him), here’s a gist of something I cobbled together for $job[0] while I wait for another team to deliver spacelift. I recommend skipping this and going for spacelift, but this may prove helpful to you if you’re in a similar boat. https://gist.github.com/jim80net/d6a9d291c2f860e14c825a92063aea13
fantastic, looking into it, thanks a lot for sharing it @jimp