#random (2024-02)

This is my Makefile:

install:
	@go mod vendor
	@go install github.com/golang/mock/mockgen@latest

and this is my renovate.json file

{
  "$schema": "<https://docs.renovatebot.com/renovate-schema.json>",
  "extends": [
    "config:best-practices"
  ],
  "customManagers": [
    {
      "customType": "regex",
      "fileMatch": [
        "^Makefile$"
      ],
      "matchStrings": [
        "@go install (?<depName>[^@]+)@(?<currentValue>[^\s]+)"
      ],
      "datasourceTemplate": "go"
    }
  ]
}

I am getting error like this

"regex": [
      {
        "deps": [
          {
            "depName": "github.com/golang/mock/mockgen",
            "currentValue": "latest",
            "datasource": "go",
            "replaceString": "@go install github.com/golang/mock/mockgen@latest",
            "updates": [],
            "packageName": "github.com/golang/mock/mockgen",
            "versioning": "semver",
            "warnings": [],
            "skipReason": "invalid-value"
          }
        ],
        "matchStrings": [
          "@go install (?<depName>[^@]+)@(?<currentValue>[^\s]+)"
        ],
        "datasourceTemplate": "go",
        "packageFile": "Makefile"
      }
    ]
  }
}

I’m speculating a bit here but might be worth looking into. It’s parsing current value correctly (“latest”) but since your versioning is set to “semver” I don’t think it can understand “latest” in terms of semver

Might have to change [github.com/golang/mock/mockgen@latest](http://github.com/golang/mock/mockgen@latest) to [github.com/golang/mock/[email protected]](http://github.com/golang/mock/[email protected])

FWIW, a quick search shows that that project is archived and no longer maintained, so renovate will never update that dependency, anyways

May want to consider using the maintained fork https://github.com/uber-go/mock if you want security updates in the future

tfsec (now under trivy) is used quite often too

https://github.com/antonbabenko/pre-commit-terraform is a good place to start, IMHO. (I don’t know if there are any other commonly used terraform pre-commit hooks somewhere else ) I personally like tfupdate to update the provider versions automatically, I think it’s a bit underrated. But it’s quite practical for maintenance

I’m using trunk.io now. It has most of these tools and has very little wrangling required. It’s my go to right now esp on open source or private projects.

I might have to give this a try! Looks awesome!

@Hans D

thanks @Erik Osterman (Cloud Posse). Prefer to to be able to run everything also outside a ci/cd pipeline, so i’ve setup 1 workflow which picks up the things it needs to do. phase 1: do any kind of generation / reformatting, so the pr can be updated (single gha step) phase 2: do all the validation/test checks (gha matrix)

Using go-task for the feeding/execution (but you can use what ever you want), which we also use for locally running various things. terraform specific:

• tflint

• terraform fmt (both format and linting)

• checkov / trivy are in the works (some poc is done, but other prios need to be done first)

• terraform plan (via spacelift)

• composing terraform modules (vendir for vendoring and composing + some bash and jq)

• terraform docs

ORM is a good toy for small projects where database performance is not critical, and nobody looks at the database at all (load, queries, slow queries, etc.). Once the project is big enough, and the database gets high load, ORM becomes a disaster (there are many many examples of that in real companies).

After the disaster starts, nobody knows how to check the queries and how to optimize them - indexes, sub-queries optimization (creating indexes on the relevant columns, rewriting the subquery to use a different approach, or reducing the complexity of the subquery), normalization - all of that is unknown and even worst, hidden from the view b/c the ORM generates something that nobody knows about.

So ORM is good for some website to get data for the UI, but it’s an absolute disaster for high-speed transactional databases with many relations and foreign keys.

The best solution is to use stored procedures (not to hardcode SQL queries in the app). DBAs can see them and optimize them anytime w/o affecting the apps (keeping the same interface). The devs just create a DAL that calls the stored procedures. Every layer does just what it’s supposed to do. Easy to optimize all layers (even in the future with diff loads or number of concurrent users).

That’s what I’ve been realizing, but so many just keep gaslighting on about ORM value. These folks will learn with enough exposure to scale. Until then I’ll agree to disagree. At least now I’m confident that there’s no “magic”… And even if there were, we already hate magic numbers, so why would any magic be acceptable. :)

I don’t see why you could not do indexes, query optimization, normalization, look at slow queries, etc. while using an ORM? In fact, that’s what I’ve always seen being done, But probably I’m just gaslit. And stored procedures? Well, better not open that can of worms.

If you ever created more than a few resources on AWS with multiple people/teams involved you would have seen that resource id’s and tags start getting inconsistent very quickly.

The people from cloudposse can correct me but as far as I understand it, null-label is all about consistent naming and consistent tags for resources.

null-label helps us prevent mentioned inconsistencies by us passing it contextual values like name, environment, tenant, etc. and null-label outputting an id and tags for us that we can use for our resources in a consistent and reproducible manner.

If you look at e.g. cloudposse’s s3 bucket module, they call null-label here and use it, among other places, here for the bucket name if no explicit bucket name is provided.

https://github.blog/2019-10-14-introducing-autolink-references/

You mean this?

Autolinking ticket numbers are great. The annoying thing here is that typing ‘README.md’, etc., results in a link to some external resource.

Basically, the logic: IF the string matches a local asset, then maaaaybe link to that asset, but NEVER link to some 3rd party website on a .md gTLD.

Hah, true

README.md 301 >> goats who see… ?

who knows

documentation is hard and always lacking. Let me give you an example of how we do it with Atmos https://atmos.tools

before, we had the CLI and almost no docs, and we notice that we were spending more and more time trying to explain diff features to diff people

then we spent 2 months on the docs trying to get it to the state describing all the CLI features

then we decided, if we add/improve/update anything in Atmos, we update/improve the docs at the same time. This is “annoying” (who wants to write docs?), but pays big time

now we update the CLI and the docs at the same time in the same PR, so everything is up to date and in sync - takes more time for the PR, but pays in the long run

now almost all PRs have “Update docs” item, e.g. https://github.com/cloudposse/atmos/pull/525

That’s what i’m thinking of introducing part of the acceptance criteria for any ticket that a developer works. Documentation should be part of the requirements. Yeah, similar to the what in your example

does it mean that you guys do documentation after you’ve checked off the other bullet points ?

or as you finish each bullet point as an example

for Atmos yes. When features are implemented and tested, we write some docs for it

i mean, that all new features in a PR (regardless it’s just one or a few) should be covered by the changes to the docs

that makes sense to me, I was being asked why wouldn’t we document as we are working on the feature instead of waiting until the end and then writing the documentation

it’s better b/c when you are working on something, you are in the context and can better describe what you are doing

@aj_baller23 I wrote some thoughts down based on your question and published them here. I thought perhaps the POV would be helpful in some way. https://wahlnetwork.com/2024/03/12/how-to-improve-documentation-practices-in-engineering-teams/

@Chris Wahl thanks for the feedback great article

You cannot since AWS is in a different geographical sovereignty. You need to go back to your business and discuss with your regulatory team options.

Contact AWS for localised region, AWS outposts perhaps, AWS partner in your region.

https://www.infoq.com/news/2023/09/aws-dedicated-local-zones/

Best for atmos

So it depends what your pipeline is.

Are you using Spacelift, GItHub Actions or something else?

GitHub actions

I’ll move to atmos, thanks!

Sounds cool, I’ll give it a download and a read

they should make it into competition with leaderboard etc