#random (2024-03)
Non-work banter and water cooler conversation
A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.
Archive: https://archive.sweetops.com/random/
2024-03-01
re the whole cp/atmos/sweetops rebrading re logos etc (they look awesome), is there going to be some related swag? with glow in the dark extras? — asking for a friend
we should do that, nothing yet. What’s everyone using these days for that?
(“my friend has”)
Has anyone experienced any longterm affects of not implementing formatting in pipelines? I am curious what potential downfalls could come from a lack of terraform fmt
in pipelines
Having this fmt being done in the pipeline save a lot of review remarks re fmtting (anf this the braincells during a pr can be used for actual reviews).
Recently somebody in my team made a remark re some space/tab thing that is not picked up by tf fmt
and that was quickly resolve b “if fmt does not & cannot fix it, it’s not an issue”
Also look into pre-commit for terraform fmt, white space, and end of line issues. A huge time saver when it comes to reviews
I’m currently working with an organization that did not turn on formatting from the start, so utilizing pre-commit and pipelines now formats each commit, and the team is resistant to implement formatting now because they prioritize git history and blame over formatted code. I am trying to rationalize an argument to begin implementing it now, but it isn’t looking promising
was initially also a bit hessitant re auto-formatting and the likes (but in favor of linting etc though), but noticing it saves a lot of non-discussion and provides more content focused reviews. Initially it’s a single sweep, and as we combine it with merged commits combined with small commits, the “blame”/history is a non-issue. (having al these “fix small typo” commut msgs is alo adding more noise as value)
but still quite opposite to mandatory pre-commit hooks or things that run pipeline-side only
I agree, can you elaborate on how the history is a non issue from your experience? @Hans D
we do more or less trunk based development re our infra. Short lived branches, and many ieteration. That way we already get a nice sense of the evolution of the concept - and allowes for faster reaction to the actual production interaction. Having long lived branches (> 3 days) with prs that take days for a proper review and understanding, it tends to generate lot of noisy commits - making the case for “keep the history” already something to be like - too much noise/detail, skip looking at those. Or getting things like, you should do a squash, having essentially the same effect. In combination with github and having the review remarks in there still attached to the commits (good for referencing the blamce etc). And as time continues, that details git history becomes like having 5 sec resolution on your monitoring becoming less relevant/usuable as time continues. having worked with “working on a branch for weeks” combined with all the details and merging paths in git, and the “multiple prs / day” with sweet linear history, I do prefer the latter one. Nice test in that case: if somebody has been on leave for a couple of day, how do they catch up with the changes? I prefer a summary, diff on now…last known situation and look at the interesting bits over having to go through each pr/commit.
Combine that with the non-default formatting/linting with most prs consisting of shoving code around, cause I like to do it in my style …
dont get me wrong, blame is still good to have. Having small prs where just a single person is working on, provides better understanding as lets-have-pre-integration branches where the whole team touches stuff (as the latter does require you keep the whole history).
(some context, our team of 3 + renovatebot generated over 1000 prs in the last 5 months - all to trunk and almost all prs have an additional fmt/code-generation commit - but those get squashed on the merge. Some small change of 3 lines can be amplified into the actual commit being 1000s of lines in what we call dont-panic prs)
@Jeremy White (Cloud Posse) you’ve been witnessing this at our side, perhaps you can share your observations in how we progressed on this bit
in my opinion, formatting terraform code will not bring any benefit at all. Some times, it will introduce false changes that are not necessary, like aligning after equal sign.
alfa = "v1"
beta = "v2"
vs
alfa = "v1"
beta = "v2"
long_variable_name = "v3"
The change on the first two lines are unnecessary and usually noise in the PRs. I totally understand formatting and linters in the real programming languages like php, python, java, rust, and so on, but on terraform not really necessary. So, I would keep the part of the team that doesn’t want to introduce another problem that they don’t have it already.
Would be better even for you to focus more on the correctness of the code itself, not on how it looks.
And here is another crazy idea that we have in my company and I am totally against it, but the team likes it. Generating readme files for terraform projects that nobody reads.
Would be better even for you to focus more on the correctness of the code itself, not on how it looks.
Human nature: they will first pick on these kind of things, before the actual code. Primary reason why golang was implemented to do the “we determine the formatting, you take care of the code”.
| Generating readme files for terraform projects that nobody reads We do tons of readme generations. Not always read right now, but our futurse self and future team mate will thank that us we did…
well, I disagree on the part with readme generation. While it totally makes sense for terraform module and documentation that somebody actually writes with valuable information, for the terraform projects where the tf-docs only generates a list of resources and modules, I haven’t seen anyone using those files in years
i’ll post my picture, as I’ve been using some of those readme files (also from the cloudposse repos)
again, I am not talking about the terraform modules readme files, those make perfect sense to have and use. What I am talking about is the terraform code that we write as projects to create our infrastructure.
Thinking about this again, I think code generation follows lock step with how to DRY things. After you start copy pasting, you always hit a point where things are ‘human effort’. You have to ask yourself, when does human effort reach a critical point. I advocate generating configuration when you actually find the golden pattern, and that is something you need to soul search for while you work on other DRY efforts.
As for whitespace changes in code formatting, don’t forget that whitespace changes are a matter of configuring your view in PRs. Here’s a screenshot of a public PR on our components repo. Note the setting “Hide Whitespace” in the lower right:
Moreover, you can use .gitattributes to auto fold/accept readme changes if you feel like they are irrelevant to a change: https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github
@Michael JFYI you can ignore formatting commits for git blame: https://www.stefanjudis.com/today-i-learned/how-to-exclude-commits-from-git-blame/ This is also supported by Github: https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view
(jm2c/unpopular opinion: if they want a clean git history, they should’ve done it right from the beginning)
2024-03-04
2024-03-05
I’m excited to share Quantm (https://quantm.io), a platform I founded that aims to automate and simplify your release workflows. I’m curious to hear your thoughts and experiences. Do any of you currently use tools for deployments? What are the biggest pain points you encounter? Quantm focuses on addressing common issues like:
• Complex release configurations
• Inefficient collaboration during deployments
• Repetitive and time-consuming release tasks We believe Quantm can help streamline your process and free you up to focus on building amazing things. I’d love to hear your feedback and answer any questions you might have. Feel free to share in this channel or reach out directly. Thanks, Yousuf (CEO, Quantm)x
Automate with proven release engineering best practices. Accelerate releases, gather rapid customer insights, and maintain rock-solid reliability.
2024-03-07
2024-03-08
over 9000
(almost)
2024-03-10
Due to the insane demand for our null label module, we have launched a clothing brand. Because these days that’s what you do when you become famous.
Null Label - Unleash Your Unique Style with Curated Streetwear Discover a diverse world of streetwear at Null Label. We curate the latest and most unique selections from various labels, allowing you to express your individuality with confidence. Explore our collection today!
Null Label - Unleash Your Unique Style with Curated Streetwear Discover a diverse world of streetwear at Null Label. We curate the latest and most unique selections from various labels, allowing you to express your individuality with confidence. Explore our collection today!
2024-03-12
2024-03-14
Anyone tried Terraform stacks? I’m curious about your experience and how you think it stacks (pun intended) against tools like Terragrunt
Is it exclusive to Terraform cloud?
Yeah, it’s currently exclusive to Terraform Cloud. I’m interested in the feature, but I don’t use TFC.
what do you do now instead? I used to use Terragrunt, but now I use code generation and a Taskfile to manage inter-module dependencies and envs
Task is a task runner / build tool that aims to be simpler and easier to use
Terramate also launched this https://terramate.io/docs/cli/stacks/
Learn how stacks help you efficiently build and manage infrastructure as code projects at any scale with technologies such as Terraform.
I still use terragrunt mostly. Also spacelift
I think it’s important to first clarify what a stack is. From Cloud Posse’s perspective, a stack is defined quite differently from others. We define a stack as a collection of loosely coupled components. Components are things like root modules, helm charts, etc. Each of those components has its own lifecycle. From a (cloud posse) best practices perspective, we like to decompose a stack of its parts, separated by lifecycle. The VPC has a different lifecycle, from the databases, clusters, and applications that sit on top of the cluster. Similarly two different stacks might be more or less identical, but deployed in different regions. They should share nothing, including provider configuration. E.g. one component should almost never provision resources in more than one region, to preserve DR capabilities.
For example, in Spacelift, a Stack is simply an invocation of a terraform root module. Is that really anything different from a Root module? When we use spacelift, we deploy what we call “components” as what space lift calls “Stacks”
Terramate defines it yet differently. From their example, a stack seems similarly defined as in Spacelift, and allowing it to depend on other root modules, but again, just a root module configured with dependencies.
# stack.tm.hcl
stack {
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Configure the metadata of a stack
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
name = "Terraform Example Stack"
description = "An awesome stack for demo purposes"
id = "780c4a63-79c2-4725-81f0-06d7c0435426"
tags = [
"terraform",
"prd",
"service-abc"
]
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Optionally the orchestration behavior can be configured
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Ensures that the current stack is executed before the following stacks
before = [
"../stack-a",
"../stack-b",
]
# Ensures that the current stack is executed after the following stacks
after = [
"../stack-c",
]
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Optionally the trigger behavior can be configured
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# If any of the configured files changed, this stack will be marked as changed in the change detection.
watch = [
"/policies/mypolicy.json"
]
# Forces the execution of a list of stacks whenever the current stack is executed
# even if those don't contain any changes
wants = [
"../stack-d",
]
# Ensures that the current stack always gets executed when a list of configured
# stacks are executed even if the current stack doesn't contain any changes
wanted_by = [
"../stack-e",
]
}
Are you generating a root module based on other modules using Taskfile?
We use a cli tool we built to generate all the terraform in different local directories (modules) with an accompanying auto-generated Taskfile that applies them in order + some convenience management commands for devs
We generate this code for every environment explicitly, so we can check the git repo at any time and see the corresponding TF code in the appropriate directory
So, on the one hand, I agree with committing the TF code for its immutability/auditability (it’s why we vendor & commit components). The practical downside is that no engineer likes doing code reviews of machine-generated code, so large amounts of code go unchecked from a practical point of view.
This is ultimately why we built atmos because we wanted to rely minimally on meta programming for Terraform. This is a departure from a lot of other tools that automate terraform. Cloud Posse has generally not relied on code generation of Terraform, and when we’ve been tempted, realized that our architecture was wrong.
@george.m.sedky I think I might be confusing what you’re doing with how other tools work in the Terraform ecosystem. From our earlier conversations (not in this thread), I recall now that you are developing more of visual CAD software for Terraform, which is different from the problem I address, using Terraform code generation to overcome the perceived limitations of Terraform.
This is the one I’m referring to in this thread https://github.com/stakpak/devx
A tool for generating, validating & sharing all your configurations, powered by CUE. Works with Kubernetes, Terraform, Compose, GitHub actions and much more…
We’re moving away from this approach to work on the visual CAD / Copilot interface I showed you
Ohhhhhh hah, cool. Fun coincidence since you used cue and were asking about stacks. Check this out https://github.com/DavidGamba/dgtools/tree/master/bt#stacks-a-different-take
We discussed it on #office-hours last week
Yes we built a code-gen + stacks approach (as you explained different people define stacks differently) and have been using it with multiple clients over the past year
Devs were split about using CUE, half of the people hate it and the other half love it
We built a lot of tooling around CUE, like a package manager and an LSP to make it easier
But CUE made onboarding take way longer than what the average developer expected
I could imagine
Sounds like a good blog post: lessons we learned trying to adopt cue with Terraform. The good (awesome?), the bad, the ugly.
2024-03-15
2024-03-16
Reduces build times and repository maintenance costs through high-quality developer tooling.
pretty wild stuff in here. For instance, they have their own query lang for your monorepo: https://moonrepo.dev/docs/concepts/query-lang
moon supports an integrated query language, known as MQL, that can be used to filter and select
Time to make a PR for supporting atmos
they already have terraform https://moonrepo.dev/docs/proto/tools#terraform
Looks fascinating
2024-03-18
hard to sell in a good price, the potential buyers know if opponents invest into OpenTofu, Hashicorp’s stock is down with no limit
have seen this in many products… it is a dillema