#general (2022-01)

Hi from a new helmfile user.

Hi all

welcome!

are you using mssql on rds? if so check the section “Restoring a database” on https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html

exec msdb.dbo.rds_restore_database 
	@restore_db_name='database_name', 
	@s3_arn_to_restore_from='arn:aws:s3:::bucket_name/file_name.extension',
	@with_norecovery=0|1,
	[@kms_master_key_arn='arn:aws:kms:region:account-id:key/key-id'],
	[@type='DIFFERENTIAL|FULL'];

I’m not sure where it is hosted. But I need to do it on a mac

here’s the official way from microsoft

https://docs.microsoft.com/en-us/sql/t-sql/statements/restore-statements-transact-sql?view=sql-server-ver15

 Restores SQL database backups taken using the BACKUP command.

note that i haven’t tried either of these techniques for ms sql

dbatools.io has powershell modules to restore backups. I have not tested them on a mac, however.

Newer to asdf. Really DO NOT like that the asdf install doesn’t just setup plugins + install in a project, as this means you still need a bootstrap step to install plugins.

I’ve used ansible before to do some of this but it’s not as focused on shims which I absolutely love (stop debugging path issues and let it just work!)

I’m rethinking my codespace/devtooling for 2022 and thinking of any new approach.

I currently use Chezmoi (which is awesome!) but really it’s more than dotfiles, I want a common set of tools like bat, fzf, and such to all be bootstrapped on Debian, Ubuntu, WSL2 ubuntu, mac all with 1-2 liner when possible.

ANSIBLE

So… I primarily work in devcontainers right now.

Assuming I have python3 installed everywhere I could start to wrap up my toolkits in Ansible playbooks and just run the ansible comand to use my remote repo.

Ansible is kinda slow here but I can tolerate it. Not perfect and more work to debug overall, but probably the most mature, albeit overkill Benefit is reuse of the knowledge and tooling for other things whereas chezmoi and others are very much not reusable across other domains.

Chezmoi

Works pretty well. Go templates, cross platform. Install scripts oriented, so not declarative so combining with asdf works well, but I need to maintain the plugin install + asdf install setup .

Most likely to succeed.

Asdf

Combining with chezmoi seems powerful. I’m really not wowed by the first experience. I freaking wish it was asdf install and the .tool-versions (nitpick should be .tools-version) had the plugin path like

something like this.

pulumi, repopath, version

Having to run asdf plugin add pulmi && asdf install pulumi latest && asdf local pulumi latest to get my first tool file synced up is annoying and doesn’t provide a great experience.

Docker

I’ve decided as much as I want a wrapper for docker cli tools, I think for dev tooling they complicate the basics tools in the environment. Since I already work in a container mostly nested containers add complexity. I prefer to run something that installs the tool for the platform a bit more intelligently, like asdf should (mac or linux, it should just find the right now). Ansible can but it’s more work.

it also complicates things if the tools are desired by vscode and other dev tools. That level of abstraction ends up being more work I feel.

…. would love to hear what others do for those jumping between container driven workflows like Gitpod/Codespaces and such.

We recently moved to a monorepo (whole other topic) and as part of that we created a setup script so all you have to do to get your machine ready to go is run ./setup.

We’re using asdf in that setup script, and I’m a big fan. Our monorepo includes a .tool-versions file, and the script uses it to set things up. Here’s what I did in that script:

CURRENT_ASDF_PLUGINS=$(asdf plugin list || true)

# shellcheck disable=SC2034
while read -r PLUGIN VERSION || [[ -n "$PLUGIN" ]]; do
    if ! (echo "$CURRENT_ASDF_PLUGINS" | grep "$PLUGIN" &>/dev/null); then
        asdf plugin-add "$PLUGIN" || true
    fi

    # Install tool
    asdf install "$PLUGIN" "$VERSION"
    # enable running tool outside of monorepo directory
    asdf global "$PLUGIN" "$VERSION"
done < "${DIR}/.tool-versions"

@Igor Rodionov is a big fan of https://github.com/EnvCLI/EnvCLI

it’s a hybrid between using containers and native

Nice!

@bradym I use a similar approach with Mage (Go based Make files, super cool!) right now. Asdf wasn’t as smooth of a setup as I had hoped in general, but it might be one of the best options.

it’s funny because all of this was initiated by my movement to a monorepo structure too! :slightly_smiling_face:

Right now I run mage init and it runs all dev tooling setup. Asdf might be the best (was considering replacing nvm), but was just checking. @Erik Osterman (Cloud Posse) i’ll check that out. I actually haven’t seen that so I’ll take a look. Thank you!

I use mage right now because it is the closest thing to the build harness style project Cloudposse put together.

I can have one file doing my import from remote repo and all my tasks are autodisovered. I created this repo based on that https://github.com/sheldonhull/magetools

Working pretty well though because it does a bit of magic for commands you can’t easily test your commands like in standard Go.

It gets out of the way though so overall the experience has been stellar

Now I just use something like

go
//go:build mage
package main 

import (

	// mage:import
	_ "github.com/sheldonhull/magetools/gotools" // gotools provides Go tasks such as linting and testing

	// mage:import
	_ "github.com/sheldonhull/magetools/licensing" // licensing provides a license checker and vendor tooling for the project
)

And when you run go run mage.go init or mage all the remote tasks get imported and used. I loved that because of how cool build-harness was but I wanted to write reusable common tasks in Go I could use everywhere.

now i just drop those files in, run go mod tidy and it’s ready. If a linter/tool is missing it installs it when it finds it missing.

Non-go tools are a different matter though, and was looking for something inbetween scripts and ansible, but I guess asdf is going to be the closest to this.

I’d like to take another look at this now and see if it would add some value to tie it into atmos (if possible)

It’s not easy to test because it does autodiscovery of functions based on signature. But overall, i’m really happy with it since it gets out of the way pretty much like make/task style files do. I don’t personally want to write a lot of tests on my tasks since they often interact with files and such.

Really the only big complaint is that flags aren’t supported, so it focuses on very simple inputs like

mage publish myproject dev and I personally prefer mage publish -project myproject -env dev as it’s less error prone. For that compromise though I’m fine as the remote import of tasks works so well I can reuse all my work in many places.

This is one thing I really like with the power of go.

I wrap the start of the function with something like this

	start := time.Now()
	defer func(start time.Time) {
		if err == nil {
			pterm.Success.Printf(":white_check_mark: BuildAll[%s]\n", relTime(start))
		}
	}(start)

I then also have a defer function that builds a report of all the variables as I go through a more complex workflow. Then no matter if it fails or not that defer executes and I get a nice table output for debugging or summarization of all the configured values I used. Saves me time debugging stuff.

Cheers!

Was just checking out the S3 bucket w/ DynamoDB lock yesterday. Nice one! Though i put backend.tf in a different folder than the bucket config to not have to do any Voodoo by migrating state around (new project)

yes the comment is true

if you cannot update the readme md locally using make readme, the bot will do it when you submit the pr

it will only fail if you’re working off a fork in an company org because a forked company org may not give explicit perms to the cloudposse bot

a personal fork will allow the bot to run as expected

Cool, thanks

Yes, we are using in some projects for complex pipelines.

Like it? Seems really interesting!

Yes, it’s really easy to use.

At first, I saw this and I was like: Gawd.. another CI/CD system.

but this is actually cool - it’s a way to abstract away the standard CI processes so they can run locally as well as the CI dujour