#atmos-dev (2025-01)

Discuss atmos core development (golang). If you want to help out, reach out to <@UB2EH889X>

2025-01-01

2025-01-02

Vinny avatar

@Andriy Knysh (Cloud Posse) this is ready for review https://github.com/cloudposse/atmos/pull/853

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny the PR looks very good. Just one misconfiguration in atmos.yaml, please see the comment (and ping me when you fix it)

Vinny avatar
1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it was fixed in another PR, we can close this ticket

1

2025-01-03

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this LGTM https://github.com/cloudposse/atmos/pull/905

#905 Support Default Values for Arguments in Custom Commands

what

The task is was “If a default is provided for a custom command, and required is true, then it shouldn’t error.”

before

before

a custom cli-command is defined

custom_cli_command

after

after

why

Provide a deafult value in atmos.yaml for a custom CLI command, so it doesn’t error out (before it did).

references

Link to the ticket - https://linear.app/cloudposse/issue/DEV-2327/default-value-for-demo-argument-should-not-cause-an-error

Summary by CodeRabbit

New Features
• Enhanced custom command argument handling with required and default value support.
• Added new “greet” command with a configurable name argument. • Documentation
• Updated documentation for custom commands, including command name change from “hello” to “greet”.
• Clarified positional argument requirements and default value handling. • Bug Fixes
• Improved argument validation logic for custom commands.
• Added more robust error messaging for invalid command configurations. • Tests
• Introduced new test cases for the “greet” command to validate argument handling and default values.

1
1
Vinny avatar

@Andriy Knysh (Cloud Posse) looks like good now https://github.com/cloudposse/atmos/pull/853

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny this config in atmos.yaml

# Syntax highlighting configuration
  syntax:
    enabled: true
    theme: "dracula"      # or use custom colors from theme
    line_numbers: true
    wrap: false
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

looks like it’s a separate sub-section under settings

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but it’s not in schema.go

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s eigher needs to be removed from atmos.yaml and the docs, or used in schema.go and in the code

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this should be settings.terminal.syntax

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(Which it currently is)

Vinny avatar

I think we are good to remove this syntax

Vinny avatar

colorize should handle yaml

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny please ping me here when you finish all the changes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Why remove? What does removing it solve

Vinny avatar

we don’t need it there at all as we have custom colors

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But themes is nice too :-)

Vinny avatar

yeah indeed its your guys decision at the end

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But I don’t understand why we have both “syntax” and “syntax_highlighting”

Vinny avatar

will be only this syntax_highlighting we can rename it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s combine

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But yes, I prefer we keep support for themes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But document where the list of themes is found

Vinny avatar

so to summarize right now we have

Glamour with markdown with custom colors

Themes for yaml colorized

• (chroma themes for Glamour also) This is not implemented now

Vinny avatar

I can do this chroma in the same glamour PR or we can create another feature to implement it, up to you @Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s do that in a new PR

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But yes, let’s do that. Can you write a linear task and self assign

Vinny avatar

sounds good

Vinny avatar

@Andriy Knysh (Cloud Posse) good to check now please

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) I think this is also ready https://github.com/cloudposse/atmos/pull/906

#906 Support for a circuit breaker to avoid recursive calls to atmos commands (infinite loop)

what

This PR contains the code that counts the number of recursive calls, and in case it exceeds the threshold, terminates the processes (parent + all children).

Add-on to atmos.yaml

atmos_yaml_addon

The circuit breaker in use

atmos_loop_termination

why

Avoid infinite loops

references

• Closes #765DEV-2688

Summary by CodeRabbit

New Features
• Added a circuit breaker mechanism to prevent infinite shell command loops.
• Implemented maximum shell depth limit of 10 levels.
• Introduced a new command loop for testing infinite loops in the CLI.
• Enhanced environment variable tracking for nested shell executions. • Tests
• Added a new test case to verify circuit breaker functionality.
• Ensured CLI handles infinite loop scenarios gracefully.

1
Vinny avatar

This is good to start review @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/907

1

2025-01-04

Vinny avatar

This is good to start review @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/909

1

2025-01-06

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) note this failed on tests during release due to github API rate limits.

https://github.com/cloudposse/atmos/actions/runs/12637455883

I think this might be fixed when we support bearer tokens as part of https://github.com/cloudposse/atmos/pull/912

cc @Aaron Bidderman

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

this failed b/c (for some reason) the tests are failing b/c the etst code is not valid

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i’ll fix the tesst in this PR https://github.com/cloudposse/atmos/pull/916 from @Vinny

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

something about the output format changed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We should be sure that was deliberate

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in pkg/merge/merge_test.go

Vinny avatar

yes we are using workdir: “../”

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Settings: schema.AtmosSettings{
			ListMergeStrategy: ListMergeStrategyReplace,
		},
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

is not valid anymore ^

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

should be

Settings: &schema.AtmosSettings{
			ListMergeStrategy: ListMergeStrategyReplace,
		},
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny i pushed a fix in pkg/merge/merge_test.go to your PR

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

please check all other tests

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(on my local, running make testacc never finishes, we need to check it)

1
Vinny avatar

@Andriy Knysh (Cloud Posse) hotfix for Prod https://github.com/cloudposse/atmos/pull/916

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Vinny looks like it’s related to some other change

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) accidentally merged your previous PR

Vinny avatar

yep it’s the pointers

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i did not release anything, let’s fix all of that in this PR

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

these tests are still failing

internal/exec/stack_processor_utils.go

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny i suggest we reverse the commit from https://github.com/cloudposse/atmos/pull/907 so the main branch is clean

#907 Colorize Atmos Describe Commands when TTY attached

what

Colorize output when we attach TTY

Before:
Screenshot 2025-01-03 at 16 15 15
Screenshot 2025-01-03 at 16 15 43

After
Example 1
Screenshot 2025-01-03 at 16 17 55

Example 2
Screenshot 2025-01-03 at 16 19 04

references

https://github.com/alecthomas/chroma

Summary by CodeRabbit

New Features
• Added terminal configuration options for syntax highlighting.
• Introduced customizable settings for YAML and JSON output display.
• Enabled line numbers, color schemes, and pagination for terminal output. • Documentation
• Created comprehensive documentation for terminal configuration settings.
• Explained syntax highlighting options and usage in Atmos CLI.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

this PR broke a few things

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and internal/exec/stack_processor_utils.go still failing

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(the PR was prob not synched with the main branch for some time, and I forgot to check that)

Vinny avatar

which commit you want to revert or entire PR?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we can revert the entire PR or maybe you can fix it in https://github.com/cloudposse/atmos/pull/916

internal/exec/stack_processor_utils.go is failing

#916 Fixed settings in atmos yaml

what

Fix duplicated terminal settings in atmos yaml

why

Duplicated Prop broken CI

references

Summary by CodeRabbit

Configuration Update
• Enhanced terminal output settings in the configuration file.
• Added syntax highlighting with Dracula style.
• Enabled line numbers.
• Disabled line wrapping.
• Configured YAML lexer and terminal formatter.

Vinny avatar

hold on a bit

Vinny avatar

let me understand the errors

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i would revert the PR b/c it was prob old (not synched with main) and it broke at least 2 things (I fixed one)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
=== RUN   TestStackProcessor
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x1053c7604]
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i already fixed a similar error (and the fix was already in main)

Vinny avatar

lets wait CI on this and see

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i also test PRs locally (you should do it too)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

make testacc

1
Vinny avatar

good @Andriy Knysh (Cloud Posse)

Vinny avatar

will create another PR with updated main

Vinny avatar

thanks !

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i merged the PR

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, please open a new one instead of https://github.com/cloudposse/atmos/pull/907

#907 Colorize Atmos Describe Commands when TTY attached

what

Colorize output when we attach TTY

Before:
Screenshot 2025-01-03 at 16 15 15
Screenshot 2025-01-03 at 16 15 43

After
Example 1
Screenshot 2025-01-03 at 16 17 55

Example 2
Screenshot 2025-01-03 at 16 19 04

references

https://github.com/alecthomas/chroma

Summary by CodeRabbit

New Features
• Added terminal configuration options for syntax highlighting.
• Introduced customizable settings for YAML and JSON output display.
• Enabled line numbers, color schemes, and pagination for terminal output. • Documentation
• Created comprehensive documentation for terminal configuration settings.
• Explained syntax highlighting options and usage in Atmos CLI.

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Vinny avatar

ok

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

please make sure you add all the sections correctly in atmos.yaml in your new PR

1
Vinny avatar

yes this conflict from other change

Vinny avatar

I think the big mistake here was outdated PR

2025-01-07

Aaron Bidderman avatar
Aaron Bidderman
07:21:52 PM

@Andriy Knysh (Cloud Posse) hi, having a discussion with Erik on go-getter v1 vs v2. Would appreciate if you share your view, see below.

@Erik Osterman (Cloud Posse) hi, here is a status update on the research for this (I have updated the version PR with a new commit - see https://github.com/cloudposse/atmos/pull/912).

  1. Go-getter 1 and Go-getter 2 are two completly different products. They are developed in parallel, and v2 isn’t actually an upgrade of v1. It is more a migration to new product.
  2. Go-getter 2 has an option of downloading folders via http protocol where as our old go getter 1 had only git clone option. If we decide now to download with getter 2 over http (not git) I built a working a working example for this using github token.
  3. The downside is that in configuration files you now have to explititly specify the protocol (like this now - “https://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for https, and “git://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for old git clone functionality as we had before. Please advise if and how you want to proceed with this. In the meantime, I’m returning back to my main tickets (docs generation).
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Aaron Bidderman i’ll review this

@Erik Osterman (Cloud Posse) hi, here is a status update on the research for this (I have updated the version PR with a new commit - see https://github.com/cloudposse/atmos/pull/912).

  1. Go-getter 1 and Go-getter 2 are two completly different products. They are developed in parallel, and v2 isn’t actually an upgrade of v1. It is more a migration to new product.
  2. Go-getter 2 has an option of downloading folders via http protocol where as our old go getter 1 had only git clone option. If we decide now to download with getter 2 over http (not git) I built a working a working example for this using github token.
  3. The downside is that in configuration files you now have to explititly specify the protocol (like this now - “https://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for https, and “git://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for old git clone functionality as we had before. Please advise if and how you want to proceed with this. In the meantime, I’m returning back to my main tickets (docs generation).
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

did you estimate how much effort it would take to upgrade to v2? and how many breaking changes we will have?

Aaron Bidderman avatar
Aaron Bidderman

@Andriy Knysh (Cloud Posse) thanks, no, haven’t made any estimations yet, that’s more an MVP of how that actually works. I only tested in on vendor pull and managed to build a working example. The bright side seems is that now we can download stuff over http not only git clone. Wanted more to check with you and Erik if this is a go/no go.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it will depend on LOE

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are you absolutely certain there’s no way to do it in v1

Aaron Bidderman avatar
Aaron Bidderman

@Erik Osterman (Cloud Posse) thanks, my current understanding is that with git clone approach of v1 we can not use a gitub token as it isn’t an http request (it is a git clone).

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

maybe we can combine v1 and v2 for now, detect the scheme and call one or the other (w/o making too many changes to the current code)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya, I was thinking something along those lines as well.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


my current understanding is that with git clone approach of v1
My observation is that it downloads a tarball archive and decompresses it, and that it’s not actually doing a git clone

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and that that download is over https

Aaron Bidderman avatar
Aaron Bidderman

@Erik Osterman (Cloud Posse) i might be wrong but I think it does do git clone. Download with v2 over https works lot faster. So do you propose to do a check of incoming scheme and if it ha https - call v2, and if does not - v1, for backward compatibility?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ok, I think you’re right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
package getter

import (
	"fmt"
	"net/url"
	"strings"
)

// GitHubDetector implements Detector to detect GitHub URLs and turn
// them into URLs that the Git Getter can understand.
type GitHubDetector struct{}

func (d *GitHubDetector) Detect(src, _ string) (string, bool, error) {
	if len(src) == 0 {
		return "", false, nil
	}

	if strings.HasPrefix(src, "github.com/") {
		return d.detectHTTP(src)
	}

	return "", false, nil
}

func (d *GitHubDetector) detectHTTP(src string) (string, bool, error) {
	parts := strings.Split(src, "/")
	if len(parts) < 3 {
		return "", false, fmt.Errorf(
			"GitHub URLs should be github.com/username/repo")
	}

	urlStr := fmt.Sprintf("https://%s", strings.Join(parts[:3], "/"))
	url, err := url.Parse(urlStr)
	if err != nil {
		return "", true, fmt.Errorf("error parsing GitHub URL: %s", err)
	}

	if !strings.HasSuffix(url.Path, ".git") {
		url.Path += ".git"
	}

	if len(parts) > 3 {
		url.Path += "//" + strings.Join(parts[3:], "/")
	}

	return "git::" + url.String(), true, nil
}

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It converts https urls to git::

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What if instead we just write a custom detector fornow.

package main

import (
	"fmt"
	"net/url"
	"strings"

	"github.com/hashicorp/go-getter"
)

// Custom GitHub detector
func CustomGitHubDetector(u *url.URL) (string, map[string]string, error) {
	if u.Host != "github.com" {
		return "", nil, nil // Not a GitHub URL, skip this detector
	}

	parts := strings.SplitN(u.Path, "/", 4)
	if len(parts) < 3 {
		return "", nil, fmt.Errorf("invalid GitHub URL %q", u)
	}

	// Extract GitHub token from the environment
	githubToken := os.Getenv("GITHUB_TOKEN")
	if githubToken != "" {
		// Add the token to the URL
		u.User = url.User(githubToken)
	}

	// Convert to a git:: URL
	if len(parts) == 4 {
		return "git::" + u.String(), map[string]string{}, nil
	}

	return "git::" + fmt.Sprintf("<https://%s/%s/%s.git>", u.Host, parts[1], parts[2]), map[string]string{}, nil
}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
// Register custom detectors
	getter.Detectors = []getter.Detector{
		// Add the custom GitHub detector first
		getter.DetectorFunc(CustomGitHubDetector),
		// Add other default detectors as fallback
		getter.DetectGitHub,  // Original GitHub detector (optional)
		getter.DetectGzip,    // Detect .gz files
		getter.DetectGit,     // Detect git:: URLs
		getter.DetectHTTP,    // Detect HTTP(S) URLs
	}

	// Use the getter.Client with the modified detectors
	client := &getter.Client{
		Src:  "<https://github.com/user/repo.git>",
		Dst:  "./repo",
		Mode: getter.ClientModeAny,
	}

	if err := client.Get(); err != nil {
		fmt.Printf("Error: %s\n", err)
	}
Aaron Bidderman avatar
Aaron Bidderman

Actually i tried doing git::https://<token>@github.com/... and it still does git clone not using github token at all, or does not if repo is private. If the token doesn’t have the exact right Git credentials/scopes, git clone won’t accept it. So the v1 would not use token.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


or does not if repo is private.
Do you mean then it doesn’t work with a private repo even when you have a hardcoded token (just for testing)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Comment on #449 Is it possible to copy private github repositories with parameters : token

I’ve just found that forcing the protocol by prefixing the url works for example: git://[email protected]/org/repos)

Aaron Bidderman avatar
Aaron Bidderman

@Erik Osterman (Cloud Posse) Hi, Just checked again in the morning and yes, it git://<token>@github.com/… does allow me to download from my private repo. So, the solution for now is to stay on v1 and implement custom detector? Im going to rollback my latest commit with upgrade then and do this instead.

this1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s do that for now, to reduce the scope

1
Vinny avatar

@Andriy Knysh (Cloud Posse) PR TTY back ready for review https://github.com/cloudposse/atmos/pull/919

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this LGTM https://github.com/cloudposse/atmos/pull/845 cc @Shubham Tholiya

#845 Update the help descriptions for commandsattachment image

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

what

We have updated the help strings of the commands

why

Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image

references

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

Summary by CodeRabbit

Release Notes

New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities. • Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd, awsCmd, describeCmd, validateCmd, vendorCmd, and many others to better reflect their purposes. • Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this is ready for review. Tests are passing. We went through a few rounds of review on the language. You are our in-house grammarian, so if you see any improvements, have at it.

#845 Update the help descriptions for commandsattachment image

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

what

We have updated the help strings of the commands

why

Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image

references

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

Summary by CodeRabbit

Release Notes

New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities. • Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd, awsCmd, describeCmd, validateCmd, vendorCmd, and many others to better reflect their purposes. • Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i like the new descriptions. made a few minor changes, will merge

1

2025-01-08

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Heads up, k3s-demo is failing because bitnami chart repo is having problems: https://github.com/bitnami/charts/issues/31257

#31257 helm repo update fails intermittently

What steps will reproduce the bug?

Add the Bitnami Helm repo and try to update it.

What is the expected behavior?

helm repo update succeeds consistently.

What do you see instead?

Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "<https://charts.bitnami.com/bitnami>" chart repository:
	stream error: stream ID 1; INTERNAL_ERROR; received from peer

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i restarted it and it succeeded second time

#31257 helm repo update fails intermittently

What steps will reproduce the bug?

Add the Bitnami Helm repo and try to update it.

What is the expected behavior?

helm repo update succeeds consistently.

What do you see instead?

Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "<https://charts.bitnami.com/bitnami>" chart repository:
	stream error: stream ID 1; INTERNAL_ERROR; received from peer

Shubham Tholiya avatar
Shubham Tholiya

https://github.com/cloudposse/atmos/pull/845 Merged latest main branch But hopefully we merge it today if everything looks good

#845 Update the help descriptions for commandsattachment image

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

what

We have updated the help strings of the commands

why

Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image

references

issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands

Summary by CodeRabbit

Release Notes

New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities. • Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd, awsCmd, describeCmd, validateCmd, vendorCmd, and many others to better reflect their purposes. • Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.

Vinny avatar

this is good to review please @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/913

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) LGTM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(we went through a few rounds of reviews already)

Michael avatar
Michael

Major GitHub outages for PRs, Git operations, actions, and webhooks for awareness

1

2025-01-09

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this is almost ready. Just a small comment from me regarding timestamps.

https://github.com/cloudposse/atmos/pull/919

#919 Colorize Atmos Describe Commands when TTY attachedattachment image

what

Colorize output when we attach TTY

Before:
Screenshot 2025-01-07 at 19 05 26

After:
Screenshot 2025-01-07 at 15 37 10

references

Summary by CodeRabbit

New Features
• Added syntax highlighting configuration for terminal outputs.
• Introduced customizable settings for terminal display, including line numbers and color styles.
• Enhanced configuration options for YAML and JSON output formatting. • Documentation
• Added comprehensive documentation for terminal settings and syntax highlighting configuration. • Chores
• Updated configuration schema to support new terminal display options.
• Implemented utility functions for managing syntax highlighting.
• Standardized naming conventions in configuration settings.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is ready for review https://github.com/cloudposse/atmos/pull/891

Definitely should be tested on Windows. Tests are passing now.

One minor command remains. @Dan Miller (Cloud Posse) is going to update some of the example/demos-* to use relative paths.

#891 feat: Support Relative Path Imports

what

• Add support for relative paths in imports, when the path starts with ./ or ../ • Support all path types:

import:

  • ./relative_path
  • ../relative_path

or

import:

  • path: ./relative_path
  • path: ../relative_path

why

• Allow less path duplication and support simpler code

references

DEV-1856 • Depends on #918 • Depends on #908 (for test_case directory support)

Summary by CodeRabbit

New Features
• Introduced new YAML configuration files with schema declarations for better validation and modularity.
• Added variable declarations for stage, is_prod, and namespace to enhance environment settings.
• Enhanced import sections to include mixins and defaults for improved configuration management.
• Added a new Terraform module for fetching weather information with customizable display options. • Bug Fixes
• Improved handling of relative paths in import sections for better file path resolution. • Tests
• Expanded test cases to include new YAML files, updating expected outcomes accordingly.

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

@Erik Osterman (Cloud Posse) I’m still having trouble when I remove atmos schema

#891 feat: Support Relative Path Imports

what

• Add support for relative paths in imports, when the path starts with ./ or ../ • Support all path types:

import:

  • ./relative_path
  • ../relative_path

or

import:

  • path: ./relative_path
  • path: ../relative_path

why

• Allow less path duplication and support simpler code

references

DEV-1856 • Depends on #918 • Depends on #908 (for test_case directory support)

Summary by CodeRabbit

New Features
• Introduced new YAML configuration files with schema declarations for better validation and modularity.
• Added variable declarations for stage, is_prod, and namespace to enhance environment settings.
• Enhanced import sections to include mixins and defaults for improved configuration management.
• Added a new Terraform module for fetching weather information with customizable display options. • Bug Fixes
• Improved handling of relative paths in import sections for better file path resolution. • Tests
• Expanded test cases to include new YAML files, updating expected outcomes accordingly.

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

even though this PR doesnt add anything new to the schema

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

Is this supposed to work?

$ atmos version

 █████  ████████ ███    ███  ██████  ███████
██   ██    ██    ████  ████ ██    ██ ██
███████    ██    ██ ████ ██ ██    ██ ███████
██   ██    ██    ██  ██  ██ ██    ██      ██
██   ██    ██    ██      ██  ██████  ███████


👽 Atmos 1.142.0 on darwin/arm64


examples/quick-start-simple$ tree
.
├── .gitignore
├── atmos.yaml
├── components
│   └── terraform
└── stacks
    ├── catalog
    └── deploy

6 directories, 2 files


examples/quick-start-simple$ atmos validate stacks

Schema file 'stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json' not found. Configure via:
1. 'schemas.atmos.manifest' in atmos.yaml
2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var
3. --schemas-atmos-manifest flag

Accepts: absolute path, paths relative to base_path, or URL
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

jumping on a call

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sounds like a regression

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nothing is configured to point to stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

can you grep for atmos-manifest

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

Yes when nothing is specified

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

I thought it was supposed to use some remote source

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yep

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

I’m just using the latest version of Atmos. Not a local build

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

But it’s the same either way

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

[here’s the issue//github.com/cloudposse/atmos/blob/main/internal/exec/validate_stacks.go#L96-L112)

	if u.FileExists(atmosConfig.Schemas.Atmos.Manifest) {
		atmosManifestJsonSchemaFilePath = atmosConfig.Schemas.Atmos.Manifest
	} else if u.FileExists(atmosManifestJsonSchemaFileAbsPath) {
		atmosManifestJsonSchemaFilePath = atmosManifestJsonSchemaFileAbsPath
	} else if u.IsURL(atmosConfig.Schemas.Atmos.Manifest) {
		atmosManifestJsonSchemaFilePath, err = downloadSchemaFromURL(atmosConfig.Schemas.Atmos.Manifest)
		if err != nil {
			return err
		}
	} else {
		return fmt.Errorf("Schema file '%s' not found. Configure via:\n"+
			"1. 'schemas.atmos.manifest' in atmos.yaml\n"+
			"2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var\n"+
			"3. --schemas-atmos-manifest flag\n\n"+
			"Accepts: absolute path, paths relative to base_path, or URL",
			atmosConfig.Schemas.Atmos.Manifest)
	}
	if u.FileExists(atmosConfig.Schemas.Atmos.Manifest) {
		atmosManifestJsonSchemaFilePath = atmosConfig.Schemas.Atmos.Manifest
	} else if u.FileExists(atmosManifestJsonSchemaFileAbsPath) {
		atmosManifestJsonSchemaFilePath = atmosManifestJsonSchemaFileAbsPath
	} else if u.IsURL(atmosConfig.Schemas.Atmos.Manifest) {
		atmosManifestJsonSchemaFilePath, err = downloadSchemaFromURL(atmosConfig.Schemas.Atmos.Manifest)
		if err != nil {
			return err
		}
	} else {
		return fmt.Errorf("Schema file '%s' not found. Configure via:\n"+
			"1. 'schemas.atmos.manifest' in atmos.yaml\n"+
			"2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var\n"+
			"3. --schemas-atmos-manifest flag\n\n"+
			"Accepts: absolute path, paths relative to base_path, or URL",
			atmosConfig.Schemas.Atmos.Manifest)
	}
Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

TLDR if no option is specified for schema, it fails

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

the default should alternatively be to use our own remote URL schema

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

(or go.embed. it should behave the same as atmos.yaml)

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

cc @Andriy Knysh (Cloud Posse)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we will have go:embed today

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

can you add the same functionality for schema as atmos config too?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

please provide more details

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Not to config.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s a separate task.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
#808 Supported the atmos.d, allowing automatic inclusion of configuration files from the atmos.d directory

what

• Supported the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
Implemented deep merging of configurations in lexicographical order, recursively processing files within specified directories to ensure consistent and predictable configuration outcomes.
Added support for the import key inside atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from. • expose env variable ATMOS_CLI_CONFIG_PATH and ATMOS_BASE_PATH before run terraform and helm cmd • demos :
• examples/demo-env for expose env variable
• demo-atmos-cli-imports for custom inclusion of configuration files from the atmos.d directory
• demo-atmos.d for automatic inclusion of configuration files from the atmos.d directory

why

Simplifies configuration management by adopting the atmos.d directory convention, making it easier to include additional configurations without explicit declarations.

details

Load Config

flowchart TD A[“Load Configuration File”] –> B{“Import Section Exists?”}

B -- Yes --> C["Process Imports in Order"]
C --> D{"Import Type?"}
D --> E["Remote URL"]
D --> F["Specific Path"]
D --> G["Wildcard Globs"]

E --> H["Fetch Config from Remote URL"]
F --> I["Read Config from Filesystem"]
G --> I["Read Config from Filesystem"]

H --> J["Call Load Configuration File (Recursively)"]
I --> J["Call Load Configuration File (Recursively)"]

J --> L["Deep Merge with Current Config in Memory"]
L --> K{"More Imports to Process?"}
K -- Yes --> C
K -- No --> M["Configuration Processing Complete"]

%% Loopback for recursion
J -.-> A

%% Styling for clarity
style A fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style B fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style C fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style D fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style E fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style F fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style H fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style I fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style J fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style L fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style M fill<i class="em em-#1D3557,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF

classDef recursion stroke-dasharray: 5 5;

Loading

3 Stages


config: layout: fixed — flowchart TD A[“Start Configuration Process”] –> Z1[“Load Atmos Schema Defaults”] Z1 –> Z2[“Load // go:embed cloudposse/atmos/atmos.yaml”] Z2 –> Z3[“Deep Merge Schema Defaults and Embedded Config”] Z3 –> B{“Is –config Provided?”}

%% If --config is provided
B -- Yes --> C["Stage 1: Load Explicit Configurations via --config"]
C --> C1{"Load each --config path in order"}
C1 --> C2["Update ATMOS_CLI_CONFIG_PATH with loaded config absolute paths (separated by delimiter)"]
C2 --> D["Process Imports and Deep Merge"]
D --> E["Final Merged Config"]
E --> F["Output Final Configuration"]

%% If --config is not provided
B -- No --> G["Stage 1: Load System Configurations"]
G --> G1{"Check System Paths"}
G1 -- Found --> G2["Load First Found Config: %PROGRAMDATA%/atmos.yaml (Windows), /usr/local/etc/atmos.yaml, or /etc/atmos.yaml"]
G2 --> G3["Update ATMOS_CLI_CONFIG_PATH"]
G3 --> H["Process Imports and Deep Merge"]
G1 -- Not Found --> H["Process Imports and Deep Merge"]

H --> I["Stage 2: Discover Additional Configurations"]
I --> I1{"Check ATMOS_CLI_CONFIG_PATH"}
I1 -- Found --> I2["Load First Found Config: atmos.yaml or atmos.d/**/* from ATMOS_CLI_CONFIG_PATH"]
I2 --> I4["Update ATMOS_CLI_CONFIG_PATH with loaded absolute paths"]
I4 --> J["Process Imports and Deep Merge"]
I1 -- Not Found --> I5{"Check Git Repository Root"}
I5 -- Found --> I6["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/*, or .github/atmos.yaml from Git Repo Root"]
I6 --> I4
I5 -- Not Found --> I12{"Check Current Working Directory"}

%% New branch for Current Working Directory (note it's not identical to repo root)
I12 -- Found --> I13["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/* from CWD"]
I13 --> I4
I12 -- Not Found --> I18["No configuration found in Stage 2"]
I18 --> K["Stage 3: Apply User Preferences"]

J --> K["Stage 3: Apply User Preferences"]
K --> K1{"Check $XDG_CONFIG_HOME/atmos.yaml"}
K1 -- Found --> K2["Load First Found Config: $XDG_CONFIG_HOME/atmos.yaml"]
K2 --> K3["Update ATMOS_CLI_CONFIG_PATH with $XDG_CONFIG_HOME/atmos.yaml"]
K3 --> L["Process Imports and Deep Merge"]
K1 -- Not Found --> K4{"Check User's Home Directory"}
K4 -- Found --> K5["Load First Found Config: %LOCALAPPDATA%/atmos/atmos.yaml (Windows), ~~~/.config/atmos/atmos.yaml, or ~~~atmos/atmos.yaml (Linux/macOS)"]
K5 --> K3
K4 -- Not Found --> K7["No configuration found in Stage 3"]
K7 --> M["Final Merged Config"]

L --> M["Final Merged Config"]
M --> F["Output Final Configuration"]

%% Styling for clarity
style A fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style Z1 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style Z2 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style Z3 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style B fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style C fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style C1 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style C2 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style D fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style E fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style F fill<i class="em em-#1D3557,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style G1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G3 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style H fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I4 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style J fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I5 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I6 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I12 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I13 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I18 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style K fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K3 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style K4 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K5 f…
Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

oh I see. As far as I can tell, there is no default fallback if schema isnt provided - remote URL or otherwise

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@haitham911eg is working on that. See diagrams in PR description. Under 3 stages, you will see the first step is to add go:embed for atmos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
#777 Set Default Schema to Remote Schemaattachment image

what

if schemas.atmos.manifest empty atmos validate not work work.
if schemas.atmos.manifest empty is set Default Schema to Remote Schema .
default remote url https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json

why

We should set the default schema to the remote atmos schema so that atmos validate work

references

DEV-2727
https://linear.app/cloudposse/issue/DEV-2727/set-default-schema-to-remote-schema
DEV-2720
https://linear.app/cloudposse/issue/DEV-2720/error-if-remote-schema-for-validation-not-set

2024-11-17 14_35_09-Window
2024-11-17 14_42_51-Window

Summary by CodeRabbit

New Features
• Introduced a default URL for the Atmos JSON Schema manifest, ensuring a consistent configuration experience. • Improvements
• Simplified logic for determining the Atmos manifest file path, enhancing error handling and control flow.

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

ah okay this is a bug. I’ll fix it

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

this was a red-herring on my part. It’s because I had an old atmos config locally that had an explicit path for schema

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

with that, now this PR is ready for a final review: https://github.com/cloudposse/atmos/pull/891

#891 feat: Support Relative Path Imports

what

• Add support for relative paths in imports, when the path starts with ./ or ../ • Support all path types:

import:

  • ./relative_path
  • ../relative_path

or

import:

  • path: ./relative_path
  • path: ../relative_path

why

• Allow less path duplication and support simpler code

references

DEV-1856 • Depends on #918

Summary by CodeRabbit

New Features
• Added a new function to resolve relative paths in stack imports.
• Introduced a new test case for processing YAML configurations with relative paths.
• Added a new YAML configuration file for stage-specific settings. • Tests
• Added comprehensive test for stack processor with relative path scenarios. • Chores
• Updated .gitignore to ignore cache-related text files.
• Modified import paths in several YAML configuration files for improved clarity and consistency.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Dan, I’m fixing it (JSON schema) in my PR, there are more issues there

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

ah okay thanks. But this PR is ready for a review either way – tests are passing now

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

LGTM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Dan Miller (Cloud Posse) something I missed in your PR. I don’t like the name of these test-cases: complete because it’s neither complete, nor based on a convention. If it were based on a scenario or demo, I think it would make more sense.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So I think for now, we create a .yaml config for the test cases of each scenario or demo

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

that is everything that used to be just the 1 example

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

rather than breaking it up, I just called it “complete”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i know, but your next PR will introduce a new file, no?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then the file called complete is not complete.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

And complete has no meaning.

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

yeah each new PR creates a new example

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

what’s a better name?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

like I suggested, naming it after the scenario makes sense to me.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so we have a lot of tests that are for complete, but then the tests should actually be changing to that directory

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in this case, the tests are changing to a different directory

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

what is the scenario though, if the scenario is all existing tests

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

it used to just be called examples/test

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

yes but what would you call a scenario that we have been using with all existing tests?

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

legacy sounds bad

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m okay with calling the scernario complete. My point is that test-cases/complete.yaml tests things that are not in compelte

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

agree, but naming is hard

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so if a test is testing relative-paths, I’m saying call the file test-cases/relative-paths.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if a test is testing scenarios/complete, I’m saying call the file test-cases/complete.yaml

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

agree on that too, that’s what I believe I did

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no, because test-cases/complete is testing things that is not in scenarios/complete1

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

ohh

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

i see what you mean

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

okay

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

just need to break up the test-cases/complete into test-cases/complete and test-cases/demos

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

or would you rather having 1 per demo?

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

also probably should create one called core

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)
/tests/test-cases
  core.yaml
  complete.yaml
  demos.yaml
  relative-paths.yaml
  metadata.yaml
Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)
#924 chore: Separate Test Cases

what

• Break up the “complete” test case into each respective type

why

complete is neither complete, nor based on a convention. Directly associate each test case file name with the scenario or functionality it is testing

references

Slack

2025-01-10

Carlos Reyna avatar
Carlos Reyna

Good morning

wave3
2
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

good morning Carlos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Carlos is going to help with some atmos tasks!

Vinny avatar

I see terraform is failing on latest version CI @Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse)

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have a link to the failed build?

Vinny avatar

still failing for me in local

Vinny avatar

``` package exec

import ( “fmt” “os” osexec “os/exec” “path/filepath” “strings”

"github.com/pkg/errors"
"github.com/spf13/cobra"

tuiUtils "github.com/cloudposse/atmos/internal/tui/utils"
cfg "github.com/cloudposse/atmos/pkg/config"
"github.com/cloudposse/atmos/pkg/schema"
u "github.com/cloudposse/atmos/pkg/utils" )

const ( autoApproveFlag = “-auto-approve” outFlag = “-out” varFileFlag = “-var-file” skipTerraformLockFileFlag = “–skip-lock-file” everythingFlag = “–everything” forceFlag = “–force” )

// ExecuteTerraformCmd parses the provided arguments and flags and executes terraform commands func ExecuteTerraformCmd(cmd *cobra.Command, args []string, additionalArgsAndFlags []string) error { info, err := ProcessCommandLineArgs(“terraform”, cmd, args, additionalArgsAndFlags) if err != nil { return err } return ExecuteTerraform(info) }

func shouldProcessStacks(info *schema.ConfigAndStacksInfo) (bool, bool) { shouldProcessStacks := true shouldCheckStack := true

if info.SubCommand == "clean" &&
	(u.SliceContainsString(info.AdditionalArgsAndFlags, everythingFlag) ||
		u.SliceContainsString(info.AdditionalArgsAndFlags, forceFlag)) {
	if info.ComponentFromArg == "" {
		shouldProcessStacks = false
	}

	shouldCheckStack = info.Stack != ""

}

return shouldProcessStacks, shouldCheckStack }

func generateBackendConfig(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, workingDir string) error { // Auto-generate backend file if atmosConfig.Components.Terraform.AutoGenerateBackendFile { backendFileName := filepath.Join(workingDir, “backend.tf.json”)

	u.LogDebug(*atmosConfig, "\nWriting the backend config to file:")
	u.LogDebug(*atmosConfig, backendFileName)

	if !info.DryRun {
		componentBackendConfig, err := generateComponentBackendConfig(info.ComponentBackendType, info.ComponentBackendSection, info.TerraformWorkspace)
		if err != nil {
			return err
		}

		err = u.WriteToFileAsJSON(backendFileName, componentBackendConfig, 0644)
		if err != nil {
			return err
		}
	}
}

return nil }

func generateProviderOverrides(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, workingDir string) error { // Generate providers_override.tf.json file if the providers section is configured if len(info.ComponentProvidersSection) > 0 { providerOverrideFileName := filepath.Join(workingDir, “providers_override.tf.json”)

	u.LogDebug(*atmosConfig, "\nWriting the provider overrides to file:")
	u.LogDebug(*atmosConfig, providerOverrideFileName)

	if !info.DryRun {
		var providerOverrides = generateComponentProviderOverrides(info.ComponentProvidersSection)
		err := u.WriteToFileAsJSON(providerOverrideFileName, providerOverrides, 0644)
		return err
	}
}
return nil }

// ExecuteTerraform executes terraform commands func ExecuteTerraform(info schema.ConfigAndStacksInfo) error { atmosConfig, err := cfg.InitCliConfig(info, true) if err != nil { return err }

if info.NeedHelp {
	return nil
}

// If the user just types `atmos terraform`, print Atmos logo and show terraform help
if info.SubCommand == "" {
	fmt.Println()
	err = tuiUtils.PrintStyledText("ATMOS")
	if err != nil {
		return err
	}

	err = processHelp(atmosConfig, "terraform", "")
	if err != nil {
		return err
	}

	fmt.Println()
	return nil
}

if info.SubCommand == "version" {
	return ExecuteShellCommand(atmosConfig,
		"terraform",
		[]string{info.SubCommand},
		"",
		nil,
		false,
		info.RedirectStdErr)
}

// Skip stack processing when cleaning with --everything or --force flags to allow cleaning without requiring stack
// configuration
shouldProcessStacks, shouldCheckStack := shouldProcessStacks(&info)

if shouldProcessStacks {
	info, err = ProcessStacks(atmosConfig, info, shouldCheckStack, true)
	if err != nil {
		return err
	}

	if len(info.Stack) < 1 && shouldCheckStack {
		return errors.New("stack must be specified when not using --everything or --force flags")
	}
}

if !info.ComponentIsEnabled {
	u.LogInfo(atmosConfig, fmt.Sprintf("component '%s' is not enabled and skipped", info.ComponentFromArg))
	return nil
}

err = checkTerraformConfig(atmosConfig)
if err != nil {
	return err
}

// Check if the component (or base component) exists as Terraform component
componentPath := filepath.Join(atmosConfig.TerraformDirAbsolutePath, info.ComponentFolderPrefix, info.FinalComponent)
componentPathExists, err := u.IsDirectory(componentPath)
if err != nil || !componentPathExists {
	return fmt.Errorf("'%s' points to the Terraform component '%s', but it does not exist in '%s'",
		info.ComponentFromArg,
		info.FinalComponent,
		filepath.Join(atmosConfig.Components.Terraform.BasePath, info.ComponentFolderPrefix),
	)
}

// Check if the component is allowed to be provisioned (`metadata.type` attribute is not set to `abstract`)
if (info.SubCommand == "plan" || info.SubCommand == "apply" || info.SubCommand == "deploy" || info.SubCommand == "workspace") && info.ComponentIsAbstract {
	return fmt.Errorf("abstract component '%s' cannot be provisioned since it's explicitly prohibited from being deployed "+
		"by 'metadata.type: abstract' attribute", filepath.Join(info.ComponentFolderPrefix, info.Component))
}

// Check if the component is locked (`metadata.locked` is set to true)
if info.ComponentIsLocked {
	// Allow read-only commands, block modification commands
	switch info.SubCommand {
	case "apply", "deploy", "destroy", "import", "state", "taint", "untaint":
		return fmt.Errorf("component '%s' is locked and cannot be modified (metadata.locked = true)",
			filepath.Join(info.ComponentFolderPrefix, info.Component))
	}
}

if info.SubCommand == "clean" {
	err := handleCleanSubCommand(info, componentPath, atmosConfig)
	if err != nil {
		u.LogTrace(atmosConfig, fmt.Errorf("error cleaning the terraform component: %v", err).Error())
		return err
	}
	return nil
}

varFile := constructTerraformComponentVarfileName(info)
planFile := constructTerraformComponentPlanfileName(info)

// Print component variables and write to file
// Don't process variables when executing `terraform workspace` commands
if info.SubCommand != "workspace" {
	u.LogDebug(atmosConfig, fmt.Sprintf("\nVariables for the component '%s' in the stack '%s':", info.ComponentFromArg, info.Stack))

	if atmosConfig.Logs.Level == u.LogLevelTrace || atmosConfig.Logs.Level == u.LogLevelDebug {
		err = u.PrintAsYAMLToFileDescriptor(atmosConfig, info.ComponentVarsSection)
		if err != nil {
			return err
		}
	}

	// Write variables to a file (only if we are not using the previously generated terraform plan)
	if !info.UseTerraformPlan {
		var varFilePath, varFileNameFromArg string

		// Handle `terraform varfile` and `terraform write varfile` legacy commands
		if info.SubCommand == "varfile" || (info.SubCommand == "write" && info.SubCommand2 == "varfile") {
			if len(info.AdditionalArgsAndFlags) == 2 {
				fileFlag := info.AdditionalArgsAndFlags[0]
				if fileFlag == "-f" || fileFlag == "--file" {
					varFileNameFromArg = info.AdditionalArgsAndFlags[1]
				}
			}
		}

		if len(varFileNameFromArg) > 0 {
			varFilePath = varFileNameFromArg
		} else {
			varFilePath = constructTerraformComponentVarfilePath(atmosConfig, info)
		}

		u.LogDebug(atmosConfig, "Writing the variables to file:")
		u.LogDebug(atmosConfig, varFilePath)

		if !info.DryRun {
			err = u.WriteToFileAsJSON(varFilePath, info.ComponentVarsSection, 0644)
			if err != nil {
				return err
			}
		}
	}
}

// Handle `terraform varfile` and `terraform write varfile` legacy comm…
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it must be related to https://github.com/cloudposse/atmos/pull/928

#928 update eol settings

what

Ensure the EOL for files in this repo are always lf regardless of platform the user is developing on (windows, mac, linux)

why

Ensure consistency and avoid diffs being shown solely for line endings.

references

#896

Summary by CodeRabbit

Chores
• Updated .editorconfig to enforce LF line endings
• Updated .gitattributes to standardize line endings across all files
• Maintained existing documentation and generated content configurations

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Matt Calhoun I renormalized all the files, and the setting broke images. I think we cannot use * and should use specific file extensions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
#929 Renormalize files

what

• Renormalize all files (git add --renormalize .) to respect new defaults

why

• We changed the file formatting for better cross-platform support

references

#928

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Matt Calhoun - no good deed goes unpunished

The eol PR broke releases

https://github.com/cloudposse/atmos/actions/runs/12714711809/job/35446199346

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) ready for final review: https://github.com/cloudposse/atmos/pull/919

#919 Colorize Atmos Describe Commands when TTY attachedattachment image

what

Colorize output when we attach TTY

Before:
Screenshot 2025-01-07 at 19 05 26

After:
Screenshot 2025-01-07 at 15 37 10

references

Summary by CodeRabbit

New Features
• Added syntax highlighting configuration for terminal outputs.
• Introduced customizable settings for terminal display, including line numbers and color styles.
• Enhanced configuration options for YAML and JSON output formatting. • Documentation
• Added comprehensive documentation for terminal settings and syntax highlighting configuration. • Chores
• Updated configuration schema to support new terminal display options.
• Implemented utility functions for managing syntax highlighting.
• Standardized naming conventions in configuration settings.
• Removed deprecated timestamps property from terminal settings.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this is also ready for final review: https://github.com/cloudposse/atmos/pull/912

#912 Support `GITHUB_TOKEN` for HTTP Requests to github.comattachment image

what

After the internal discussion and descoping of this PR there 2 parts left:

  1. the support of the github token for the version command. The code checks if the token is set (e.g. via an env var) and if it is, the requests are done using the token. In case no token is provided, the requests to github are made without it.
  2. for github links to folders and in case the ATMOS_GITHUB_TOKEN or GITHUB_TOKEN variable is set, the files are downloaded with go-getter (v1) that uses the token. The token is put into the url under the hood, so for the end user/custmer the interaction/syntax remains the same. I.e. the following commands specified in the vendor.yaml pull out the files from github using the token (not subject to anonymous users rate limits)

NB. github.com/analitikasi/Coonector.git - is a private repo

- component: “weather” source: “github.com/analitikasi/Coonector.git//quick-start-simple/components/terraform/{{ .Component }}?ref={{.Version}}” version: “main” targets: - “components/terraform/{{ .Component }}/{{.Version}}” tags: - demo

Token_private_repo

why

Bypass github ratelimits for non-auth requests (use a token instead)

references

DEV-2778 (1 part out of 4 initially requested + 1 extra feature for repo cloning using the token) • Supersedes #871

Summary by CodeRabbit

Summary by CodeRabbit

New Features
• Enhanced GitHub client authentication by supporting optional token-based authentication.
• Added custom GitHub URL detection and transformation for improved package downloading.
• Introduced a new configuration option for GitHub token injection in the Atmos CLI.
• Added a new environment variable ATMOS_GITHUB_TOKEN for Bearer token usage in GitHub API requests. • Improvements
• Introduced more flexible GitHub client creation process.
• Added custom detection mechanisms for package downloads.
• Improved URL handling for GitHub-based resources.
• Established a default configuration setting for GitHub token injection. • Documentation
• Updated documentation to include the new inject_github_token configuration option in atmos.yaml.

2025-01-12

Michael avatar
Michael

Did the Atmos 1.146.0 Cloudsmith package release complete? Looks like it isn’t available

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’re investigating why a binary was not created with this release. Stay tuned.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Prob will get fixed tomorrow

Michael avatar
Michael

Awesome, just wanted to make sure the team knew

1

2025-01-14

Vinny avatar

@Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/923 this is good for review

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Vinny please address Erik’s comment

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

please ping me when done

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) this is ready for review https://github.com/cloudposse/atmos/pull/896

#896 Add editorconfig validation

issue: https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig

what

• Added functionality to atmos validate to validate against the .editorconfig file if it is defined. • Integrated the editorconfig-checker library to perform the validation. • Modified the validation process to support OPA/JSON schema validation alongside the new .editorconfig validation.

why

• The change ensures that the .editorconfig file is validated, which helps maintain consistency in code formatting across teams. • By incorporating editorconfig-checker, the validation process becomes more comprehensive, ensuring that both OPA/JSON schemas and editor configuration are correct. • This update was made to improve development workflow by catching formatting issues early during the validation process.

references

https://github.com/editorconfig-checker/editorconfig-checkerhttps://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig

Summary by CodeRabbit

New Features
• Added EditorConfig validation command to check and enforce coding style rules.
• Introduced configuration options for EditorConfig validation in Atmos configuration.
• Implemented comprehensive EditorConfig checking across project files. • Documentation
• Added detailed documentation for the EditorConfig validation command.
• Created new documentation explaining the EditorConfig validation process and usage. • Configuration
• Added .editorconfig files to enforce consistent coding styles.
• Updated Atmos configuration to support EditorConfig validation settings.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Shubham Tholiya please address the comment

#896 Add editorconfig validation

issue: https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig

what

• Added functionality to atmos validate to validate against the .editorconfig file if it is defined. • Integrated the editorconfig-checker library to perform the validation. • Modified the validation process to support OPA/JSON schema validation alongside the new .editorconfig validation.

why

• The change ensures that the .editorconfig file is validated, which helps maintain consistency in code formatting across teams. • By incorporating editorconfig-checker, the validation process becomes more comprehensive, ensuring that both OPA/JSON schemas and editor configuration are correct. • This update was made to improve development workflow by catching formatting issues early during the validation process.

references

https://github.com/editorconfig-checker/editorconfig-checkerhttps://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig

Summary by CodeRabbit

New Features
• Added EditorConfig validation command to check and enforce coding style rules.
• Introduced configuration options for EditorConfig validation in Atmos configuration.
• Implemented comprehensive EditorConfig checking across project files. • Documentation
• Added detailed documentation for the EditorConfig validation command.
• Created new documentation explaining the EditorConfig validation process and usage. • Configuration
• Added .editorconfig files to enforce consistent coding styles.
• Updated Atmos configuration to support EditorConfig validation settings.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) simple one liner from @Dan Hansen https://github.com/cloudposse/atmos/pull/936

#936 [Vendor] Print error message when an error occurs during installation of the last vendored component

what

Adds an error message to the output of atmos vendor pull for the last component in a vendor manifest

references

closes #935

Summary by CodeRabbit

Bug Fixes
• Improved package installation error reporting by displaying more detailed error messages during the installation process.

    keyboard_arrow_up