#gitops (2023-11)
Discuss continuous delivery of infrastructure
Archive: https://archive.sweetops.com/gitops/
2023-11-16
We’re adopting GitOps with Argo CD and I’m seeking insights on image promotion strategies. Currently, our pipeline builds and pushes images to ECR with a version tag on merging to main. A subsequent job updates deployments with these tags. For release candidates, we pull the image from the dev ECR, push it to the stage ECR, and update the stage deployment—same for production, but with a release tag.
I’m considering simplifying this by having a single image build/push to a CI/CD ECR account, from which each environment would fetch the image upon tag creation. Is this unwise - help me see what I might not be considering?
Your existing setup sounds good to me. What are the issues you have? Do you see any limitations right now?
We don’t exactly have issues, just were thinking of simplifying further esp. since we’re thinking of creating a control plane for argo in the ci/cd account that will then deploy out to the clusters in dev/stage/prod.
But then again, we do worry about that idea and having the single point of failure
Finally hit an issue with this approach. Just curious what people do about hotfixes. Say you need to make a hotfix to prod, right now the devs hate that they have to wait to tag dev, then stage, then prod before the hotfix can hit prod.
You need a “break Glass” workflow that pushes a container image directly to the production registry. There is no other way around it
In the long run however it is better to record how often you have hotfixes and why and try to prevent them in the first place
Ideally every problem that needs a hotfix should be converted to a regression test
That makes sense, I was leaning towards that but wasn’t sure if it was ‘best practice’. And I think the idea of tracking how often it happens and fixing it at the source is fantastic. Thank you!
Hi @Kostis (Codefresh), @Isaac - interesting discussion… I’m new to gitops and argo cd and we are considering it in our company. Do you recommend any good article/workshop that explains the recommendations you are talking about in detail? Thanks!
@Luis Longo depends on exactly what you need. I have personally published several pieces on all these subjects
Check https://codefresh.io/blog/enterprise-ci-cd-best-practices-part-1/ And https://learning.codefresh.io/
Disclaimer: I am the (co)author behind both resources.
Thanks Kostis, I’ll take a look at those