#atmos-dev (2025-01)
Discuss atmos core development (golang). If you want to help out, reach out to <@UB2EH889X>
2025-01-01
2025-01-02
@Andriy Knysh (Cloud Posse) this is ready for review https://github.com/cloudposse/atmos/pull/853
@Vinny the PR looks very good.
Just one misconfiguration in atmos.yaml
, please see the comment (and ping me when you fix it)
I am not able to reproduce this anymore https://linear.app/cloudposse/issue/DEV-2845/atmos-panics-instead-of-showing-error-for-incorrect-command
2025-01-03
@Andriy Knysh (Cloud Posse) this LGTM https://github.com/cloudposse/atmos/pull/905
what
The task is was “If a default is provided for a custom command, and required is true, then it shouldn’t error.”
before
a custom cli-command is defined
after
why
Provide a deafult value in atmos.yaml for a custom CLI command, so it doesn’t error out (before it did).
references
Link to the ticket - https://linear.app/cloudposse/issue/DEV-2327/default-value-for-demo-argument-should-not-cause-an-error
Summary by CodeRabbit
• New Features
• Enhanced custom command argument handling with required and default value support.
• Added new “greet” command with a configurable name argument.
• Documentation
• Updated documentation for custom commands, including command name change from “hello” to “greet”.
• Clarified positional argument requirements and default value handling.
• Bug Fixes
• Improved argument validation logic for custom commands.
• Added more robust error messaging for invalid command configurations.
• Tests
• Introduced new test cases for the “greet” command to validate argument handling and default values.
@Andriy Knysh (Cloud Posse) looks like good now https://github.com/cloudposse/atmos/pull/853
@Vinny this config in atmos.yaml
# Syntax highlighting configuration
syntax:
enabled: true
theme: "dracula" # or use custom colors from theme
line_numbers: true
wrap: false
looks like it’s a separate sub-section under settings
but it’s not in schema.go
it’s eigher needs to be removed from atmos.yaml
and the docs, or used in schema.go
and in the code
this should be settings.terminal.syntax
(Which it currently is)
I think we are good to remove this syntax
colorize should handle yaml
@Vinny please ping me here when you finish all the changes
Why remove? What does removing it solve
because syntax themes is implemented here https://github.com/cloudposse/atmos/pull/907/files#diff-c0d6f95f5584bc195d902401b649de3778ba9f88b3758b5f22a9bec32afa7cd8R319
we don’t need it there at all as we have custom colors
But themes is nice too :-)
yeah indeed its your guys decision at the end
But I don’t understand why we have both “syntax” and “syntax_highlighting”
will be only this syntax_highlighting we can rename it
Let’s combine
But yes, I prefer we keep support for themes
But document where the list of themes is found
so to summarize right now we have
• Glamour with markdown with custom colors
• (chroma themes for Glamour also) This is not implemented now
I can do this chroma in the same glamour PR or we can create another feature to implement it, up to you @Erik Osterman (Cloud Posse)
Let’s do that in a new PR
But yes, let’s do that. Can you write a linear task and self assign
sounds good
@Andriy Knysh (Cloud Posse) I think this is also ready https://github.com/cloudposse/atmos/pull/906
what
This PR contains the code that counts the number of recursive calls, and in case it exceeds the threshold, terminates the processes (parent + all children).
Add-on to atmos.yaml
The circuit breaker in use
why
Avoid infinite loops
references
Summary by CodeRabbit
• New Features
• Added a circuit breaker mechanism to prevent infinite shell command loops.
• Implemented maximum shell depth limit of 10 levels.
• Introduced a new command loop
for testing infinite loops in the CLI.
• Enhanced environment variable tracking for nested shell executions.
• Tests
• Added a new test case to verify circuit breaker functionality.
• Ensured CLI handles infinite loop scenarios gracefully.
This is good to start review @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/907
2025-01-04
This is good to start review @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/909
2025-01-06
@Andriy Knysh (Cloud Posse) note this failed on tests during release due to github API rate limits.
https://github.com/cloudposse/atmos/actions/runs/12637455883
I think this might be fixed when we support bearer tokens as part of https://github.com/cloudposse/atmos/pull/912
cc @Aaron Bidderman
this failed b/c (for some reason) the tests are failing b/c the etst code is not valid
i’ll fix the tesst in this PR https://github.com/cloudposse/atmos/pull/916 from @Vinny
something about the output format changed
We should be sure that was deliberate
in pkg/merge/merge_test.go
yes we are using workdir: “../”
Settings: schema.AtmosSettings{
ListMergeStrategy: ListMergeStrategyReplace,
},
is not valid anymore ^
should be
Settings: &schema.AtmosSettings{
ListMergeStrategy: ListMergeStrategyReplace,
},
please check all other tests
(on my local, running make testacc
never finishes, we need to check it)
@Andriy Knysh (Cloud Posse) hotfix for Prod https://github.com/cloudposse/atmos/pull/916
@Vinny looks like it’s related to some other change
@Andriy Knysh (Cloud Posse) accidentally merged your previous PR
yep it’s the pointers
i did not release anything, let’s fix all of that in this PR
Joining it with this thread: https://sweetops.slack.com/archives/C07RMALBX7V/p1736194804030919?thread_ts=1736188448.206099&cid=C07RMALBX7V
these tests are still failing
internal/exec/stack_processor_utils.go
@Vinny i suggest we reverse the commit from https://github.com/cloudposse/atmos/pull/907 so the main
branch is clean
what
Colorize output when we attach TTY
Before:
Screenshot 2025-01-03 at 16 15 15
Screenshot 2025-01-03 at 16 15 43
After
Example 1
Screenshot 2025-01-03 at 16 17 55
Example 2
Screenshot 2025-01-03 at 16 19 04
references
https://github.com/alecthomas/chroma
Summary by CodeRabbit
• New Features
• Added terminal configuration options for syntax highlighting.
• Introduced customizable settings for YAML and JSON output display.
• Enabled line numbers, color schemes, and pagination for terminal output.
• Documentation
• Created comprehensive documentation for terminal configuration settings.
• Explained syntax highlighting options and usage in Atmos CLI.
this PR broke a few things
and internal/exec/stack_processor_utils.go
still failing
(the PR was prob not synched with the main
branch for some time, and I forgot to check that)
which commit you want to revert or entire PR?
we can revert the entire PR or maybe you can fix it in https://github.com/cloudposse/atmos/pull/916
internal/exec/stack_processor_utils.go is failing
what
Fix duplicated terminal settings in atmos yaml
why
Duplicated Prop broken CI
references
Summary by CodeRabbit
• Configuration Update
• Enhanced terminal output settings in the configuration file.
• Added syntax highlighting with Dracula style.
• Enabled line numbers.
• Disabled line wrapping.
• Configured YAML lexer and terminal formatter.
hold on a bit
let me understand the errors
i would revert the PR b/c it was prob old (not synched with main
) and it broke at least 2 things (I fixed one)
=== RUN TestStackProcessor
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x1053c7604]
i already fixed a similar error (and the fix was already in main
)
lets wait CI on this and see
i also test PRs locally (you should do it too)
https://github.com/cloudposse/atmos/pull/917 passes local tests
good @Andriy Knysh (Cloud Posse)
will create another PR with updated main
thanks !
yes, please open a new one instead of https://github.com/cloudposse/atmos/pull/907
what
Colorize output when we attach TTY
Before:
Screenshot 2025-01-03 at 16 15 15
Screenshot 2025-01-03 at 16 15 43
After
Example 1
Screenshot 2025-01-03 at 16 17 55
Example 2
Screenshot 2025-01-03 at 16 19 04
references
https://github.com/alecthomas/chroma
Summary by CodeRabbit
• New Features
• Added terminal configuration options for syntax highlighting.
• Introduced customizable settings for YAML and JSON output display.
• Enabled line numbers, color schemes, and pagination for terminal output.
• Documentation
• Created comprehensive documentation for terminal configuration settings.
• Explained syntax highlighting options and usage in Atmos CLI.
i will also delete this branch https://github.com/cloudposse/atmos/tree/hotfix-atmos-yaml
ok
please make sure you add all the sections correctly in atmos.yaml
in your new PR
yes this conflict from other change
I think the big mistake here was outdated PR
2025-01-07
@Andriy Knysh (Cloud Posse) hi, having a discussion with Erik on go-getter v1 vs v2. Would appreciate if you share your view, see below.
@Erik Osterman (Cloud Posse) hi, here is a status update on the research for this (I have updated the version PR with a new commit - see https://github.com/cloudposse/atmos/pull/912).
- Go-getter 1 and Go-getter 2 are two completly different products. They are developed in parallel, and v2 isn’t actually an upgrade of v1. It is more a migration to new product.
- Go-getter 2 has an option of downloading folders via http protocol where as our old go getter 1 had only git clone option. If we decide now to download with getter 2 over http (not git) I built a working a working example for this using github token.
- The downside is that in configuration files you now have to explititly specify the protocol (like this now - “https://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for https, and “git://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for old git clone functionality as we had before. Please advise if and how you want to proceed with this. In the meantime, I’m returning back to my main tickets (docs generation).
@Aaron Bidderman i’ll review this
@Erik Osterman (Cloud Posse) hi, here is a status update on the research for this (I have updated the version PR with a new commit - see https://github.com/cloudposse/atmos/pull/912).
- Go-getter 1 and Go-getter 2 are two completly different products. They are developed in parallel, and v2 isn’t actually an upgrade of v1. It is more a migration to new product.
- Go-getter 2 has an option of downloading folders via http protocol where as our old go getter 1 had only git clone option. If we decide now to download with getter 2 over http (not git) I built a working a working example for this using github token.
- The downside is that in configuration files you now have to explititly specify the protocol (like this now - “https://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for https, and “git://github.com/cloudposse/atmos.git//examples/demo-library/{{ .Component }}?ref={{.Version}}” for old git clone functionality as we had before. Please advise if and how you want to proceed with this. In the meantime, I’m returning back to my main tickets (docs generation).
did you estimate how much effort it would take to upgrade to v2? and how many breaking changes we will have?
@Andriy Knysh (Cloud Posse) thanks, no, haven’t made any estimations yet, that’s more an MVP of how that actually works. I only tested in on vendor pull and managed to build a working example. The bright side seems is that now we can download stuff over http not only git clone. Wanted more to check with you and Erik if this is a go/no go.
I think it will depend on LOE
Are you absolutely certain there’s no way to do it in v1
@Erik Osterman (Cloud Posse) thanks, my current understanding is that with git clone approach of v1 we can not use a gitub token as it isn’t an http request (it is a git clone).
maybe we can combine v1 and v2 for now, detect the scheme and call one or the other (w/o making too many changes to the current code)
Ya, I was thinking something along those lines as well.
my current understanding is that with git clone approach of v1
My observation is that it downloads a tarball archive and decompresses it, and that it’s not actually doing a git clone
and that that download is over https
@Erik Osterman (Cloud Posse) i might be wrong but I think it does do git clone. Download with v2 over https works lot faster. So do you propose to do a check of incoming scheme and if it ha https - call v2, and if does not - v1, for backward compatibility?
Ok, I think you’re right
package getter
import (
"fmt"
"net/url"
"strings"
)
// GitHubDetector implements Detector to detect GitHub URLs and turn
// them into URLs that the Git Getter can understand.
type GitHubDetector struct{}
func (d *GitHubDetector) Detect(src, _ string) (string, bool, error) {
if len(src) == 0 {
return "", false, nil
}
if strings.HasPrefix(src, "github.com/") {
return d.detectHTTP(src)
}
return "", false, nil
}
func (d *GitHubDetector) detectHTTP(src string) (string, bool, error) {
parts := strings.Split(src, "/")
if len(parts) < 3 {
return "", false, fmt.Errorf(
"GitHub URLs should be github.com/username/repo")
}
urlStr := fmt.Sprintf("https://%s", strings.Join(parts[:3], "/"))
url, err := url.Parse(urlStr)
if err != nil {
return "", true, fmt.Errorf("error parsing GitHub URL: %s", err)
}
if !strings.HasSuffix(url.Path, ".git") {
url.Path += ".git"
}
if len(parts) > 3 {
url.Path += "//" + strings.Join(parts[3:], "/")
}
return "git::" + url.String(), true, nil
}
It converts https urls to git::
What if instead we just write a custom detector fornow.
package main
import (
"fmt"
"net/url"
"strings"
"github.com/hashicorp/go-getter"
)
// Custom GitHub detector
func CustomGitHubDetector(u *url.URL) (string, map[string]string, error) {
if u.Host != "github.com" {
return "", nil, nil // Not a GitHub URL, skip this detector
}
parts := strings.SplitN(u.Path, "/", 4)
if len(parts) < 3 {
return "", nil, fmt.Errorf("invalid GitHub URL %q", u)
}
// Extract GitHub token from the environment
githubToken := os.Getenv("GITHUB_TOKEN")
if githubToken != "" {
// Add the token to the URL
u.User = url.User(githubToken)
}
// Convert to a git:: URL
if len(parts) == 4 {
return "git::" + u.String(), map[string]string{}, nil
}
return "git::" + fmt.Sprintf("<https://%s/%s/%s.git>", u.Host, parts[1], parts[2]), map[string]string{}, nil
}
// Register custom detectors
getter.Detectors = []getter.Detector{
// Add the custom GitHub detector first
getter.DetectorFunc(CustomGitHubDetector),
// Add other default detectors as fallback
getter.DetectGitHub, // Original GitHub detector (optional)
getter.DetectGzip, // Detect .gz files
getter.DetectGit, // Detect git:: URLs
getter.DetectHTTP, // Detect HTTP(S) URLs
}
// Use the getter.Client with the modified detectors
client := &getter.Client{
Src: "<https://github.com/user/repo.git>",
Dst: "./repo",
Mode: getter.ClientModeAny,
}
if err := client.Get(); err != nil {
fmt.Printf("Error: %s\n", err)
}
Actually i tried doing git::https://<token>@github.com/...
and it still does git clone not using github token at all, or does not if repo is private. If the token doesn’t have the exact right Git credentials/scopes, git clone won’t accept it. So the v1 would not use token.
or does not if repo is private.
Do you mean then it doesn’t work with a private repo even when you have a hardcoded token (just for testing)?
I’ve just found that forcing the protocol by prefixing the url works for example: git://[email protected]/org/repos)
@Erik Osterman (Cloud Posse) Hi, Just checked again in the morning and yes, it git://<token>@github.com/… does allow me to download from my private repo. So, the solution for now is to stay on v1 and implement custom detector? Im going to rollback my latest commit with upgrade then and do this instead.
@Andriy Knysh (Cloud Posse) PR TTY back ready for review https://github.com/cloudposse/atmos/pull/919
@Andriy Knysh (Cloud Posse) this LGTM https://github.com/cloudposse/atmos/pull/845 cc @Shubham Tholiya
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
what
We have updated the help strings of the commands
why
Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image
references
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
Summary by CodeRabbit
Release Notes
• New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities.
• Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd
, awsCmd
, describeCmd
, validateCmd
, vendorCmd
, and many others to better reflect their purposes.
• Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.
@Andriy Knysh (Cloud Posse) this is ready for review. Tests are passing. We went through a few rounds of review on the language. You are our in-house grammarian, so if you see any improvements, have at it.
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
what
We have updated the help strings of the commands
why
Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image
references
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
Summary by CodeRabbit
Release Notes
• New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities.
• Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd
, awsCmd
, describeCmd
, validateCmd
, vendorCmd
, and many others to better reflect their purposes.
• Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.
i like the new descriptions. made a few minor changes, will merge
2025-01-08
Heads up, k3s-demo is failing because bitnami chart repo is having problems: https://github.com/bitnami/charts/issues/31257
What steps will reproduce the bug?
Add the Bitnami Helm repo and try to update it.
What is the expected behavior?
helm repo update
succeeds consistently.
What do you see instead?
Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "<https://charts.bitnami.com/bitnami>" chart repository:
stream error: stream ID 1; INTERNAL_ERROR; received from peer
i restarted it and it succeeded second time
What steps will reproduce the bug?
Add the Bitnami Helm repo and try to update it.
What is the expected behavior?
helm repo update
succeeds consistently.
What do you see instead?
Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "<https://charts.bitnami.com/bitnami>" chart repository:
stream error: stream ID 1; INTERNAL_ERROR; received from peer
https://github.com/cloudposse/atmos/pull/845 Merged latest main branch But hopefully we merge it today if everything looks good
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
what
We have updated the help strings of the commands
why
Previously the help descriptions metioned a cyclic description.
Example:
image
So we had to improve the descriptions.
Example:
image
references
issue: https://linear.app/cloudposse/issue/DEV-2823/atmos-commands-should-describe-what-they-do
parent issue: https://linear.app/cloudposse/issue/DEV-2740/atmos-help-should-show-meaningful-descriptions-of-commands
Summary by CodeRabbit
Release Notes
• New Features
• Enhanced command descriptions across various commands for improved clarity and specificity regarding their functionalities.
• Documentation Updates
• Updated short and long descriptions for commands including atlantisCmd
, awsCmd
, describeCmd
, validateCmd
, vendorCmd
, and many others to better reflect their purposes.
• Chores
• Introduced a new variable for customized help messages to allow tailored command descriptions based on command names.
this is good to review please @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/913
@Andriy Knysh (Cloud Posse) LGTM
(we went through a few rounds of reviews already)
Major GitHub outages for PRs, Git operations, actions, and webhooks for awareness
2025-01-09
@Andriy Knysh (Cloud Posse) this is almost ready. Just a small comment from me regarding timestamps.
what
Colorize output when we attach TTY
Before:
Screenshot 2025-01-07 at 19 05 26
After:
Screenshot 2025-01-07 at 15 37 10
references
Summary by CodeRabbit
• New Features
• Added syntax highlighting configuration for terminal outputs.
• Introduced customizable settings for terminal display, including line numbers and color styles.
• Enhanced configuration options for YAML and JSON output formatting.
• Documentation
• Added comprehensive documentation for terminal settings and syntax highlighting configuration.
• Chores
• Updated configuration schema to support new terminal display options.
• Implemented utility functions for managing syntax highlighting.
• Standardized naming conventions in configuration settings.
This is ready for review https://github.com/cloudposse/atmos/pull/891
Definitely should be tested on Windows. Tests are passing now.
One minor command remains. @Dan Miller (Cloud Posse) is going to update some of the example/demos-*
to use relative paths.
what
• Add support for relative paths in imports, when the path starts with ./
or ../
• Support all path types:
import:
- ./relative_path
- ../relative_path
or
import:
- path: ./relative_path
- path: ../relative_path
why
• Allow less path duplication and support simpler code
references
• DEV-1856 • Depends on #918 • Depends on #908 (for test_case directory support)
Summary by CodeRabbit
• New Features
• Introduced new YAML configuration files with schema declarations for better validation and modularity.
• Added variable declarations for stage
, is_prod
, and namespace
to enhance environment settings.
• Enhanced import sections to include mixins and defaults for improved configuration management.
• Added a new Terraform module for fetching weather information with customizable display options.
• Bug Fixes
• Improved handling of relative paths in import sections for better file path resolution.
• Tests
• Expanded test cases to include new YAML files, updating expected outcomes accordingly.
@Erik Osterman (Cloud Posse) I’m still having trouble when I remove atmos schema
what
• Add support for relative paths in imports, when the path starts with ./
or ../
• Support all path types:
import:
- ./relative_path
- ../relative_path
or
import:
- path: ./relative_path
- path: ../relative_path
why
• Allow less path duplication and support simpler code
references
• DEV-1856 • Depends on #918 • Depends on #908 (for test_case directory support)
Summary by CodeRabbit
• New Features
• Introduced new YAML configuration files with schema declarations for better validation and modularity.
• Added variable declarations for stage
, is_prod
, and namespace
to enhance environment settings.
• Enhanced import sections to include mixins and defaults for improved configuration management.
• Added a new Terraform module for fetching weather information with customizable display options.
• Bug Fixes
• Improved handling of relative paths in import sections for better file path resolution.
• Tests
• Expanded test cases to include new YAML files, updating expected outcomes accordingly.
even though this PR doesnt add anything new to the schema
Is this supposed to work?
$ atmos version
█████ ████████ ███ ███ ██████ ███████
██ ██ ██ ████ ████ ██ ██ ██
███████ ██ ██ ████ ██ ██ ██ ███████
██ ██ ██ ██ ██ ██ ██ ██ ██
██ ██ ██ ██ ██ ██████ ███████
👽 Atmos 1.142.0 on darwin/arm64
examples/quick-start-simple$ tree
.
├── .gitignore
├── atmos.yaml
├── components
│ └── terraform
└── stacks
├── catalog
└── deploy
6 directories, 2 files
examples/quick-start-simple$ atmos validate stacks
Schema file 'stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json' not found. Configure via:
1. 'schemas.atmos.manifest' in atmos.yaml
2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var
3. --schemas-atmos-manifest flag
Accepts: absolute path, paths relative to base_path, or URL
Sounds like a regression
nothing is configured to point to stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json
?
can you grep for atmos-manifest
Yes when nothing is specified
I thought it was supposed to use some remote source
yep
I’m just using the latest version of Atmos. Not a local build
But it’s the same either way
[here’s the issue//github.com/cloudposse/atmos/blob/main/internal/exec/validate_stacks.go#L96-L112)
if u.FileExists(atmosConfig.Schemas.Atmos.Manifest) {
atmosManifestJsonSchemaFilePath = atmosConfig.Schemas.Atmos.Manifest
} else if u.FileExists(atmosManifestJsonSchemaFileAbsPath) {
atmosManifestJsonSchemaFilePath = atmosManifestJsonSchemaFileAbsPath
} else if u.IsURL(atmosConfig.Schemas.Atmos.Manifest) {
atmosManifestJsonSchemaFilePath, err = downloadSchemaFromURL(atmosConfig.Schemas.Atmos.Manifest)
if err != nil {
return err
}
} else {
return fmt.Errorf("Schema file '%s' not found. Configure via:\n"+
"1. 'schemas.atmos.manifest' in atmos.yaml\n"+
"2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var\n"+
"3. --schemas-atmos-manifest flag\n\n"+
"Accepts: absolute path, paths relative to base_path, or URL",
atmosConfig.Schemas.Atmos.Manifest)
}
if u.FileExists(atmosConfig.Schemas.Atmos.Manifest) {
atmosManifestJsonSchemaFilePath = atmosConfig.Schemas.Atmos.Manifest
} else if u.FileExists(atmosManifestJsonSchemaFileAbsPath) {
atmosManifestJsonSchemaFilePath = atmosManifestJsonSchemaFileAbsPath
} else if u.IsURL(atmosConfig.Schemas.Atmos.Manifest) {
atmosManifestJsonSchemaFilePath, err = downloadSchemaFromURL(atmosConfig.Schemas.Atmos.Manifest)
if err != nil {
return err
}
} else {
return fmt.Errorf("Schema file '%s' not found. Configure via:\n"+
"1. 'schemas.atmos.manifest' in atmos.yaml\n"+
"2. ATMOS_SCHEMAS_ATMOS_MANIFEST env var\n"+
"3. --schemas-atmos-manifest flag\n\n"+
"Accepts: absolute path, paths relative to base_path, or URL",
atmosConfig.Schemas.Atmos.Manifest)
}
TLDR if no option is specified for schema, it fails
the default should alternatively be to use our own remote URL schema
(or go.embed. it should behave the same as atmos.yaml)
cc @Andriy Knysh (Cloud Posse)
we will have go:embed
today
can you add the same functionality for schema as atmos config too?
please provide more details
Not to config.
That’s a separate task.
what
• Supported the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
Implemented deep merging of configurations in lexicographical order, recursively processing files within specified directories to ensure consistent and predictable configuration outcomes.
Added support for the import key inside atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from.
• expose env variable ATMOS_CLI_CONFIG_PATH and ATMOS_BASE_PATH before run terraform and helm cmd
• demos :
• examples/demo-env for expose env variable
• demo-atmos-cli-imports for custom inclusion of configuration files from the atmos.d directory
• demo-atmos.d for automatic inclusion of configuration files from the atmos.d directory
why
Simplifies configuration management by adopting the atmos.d directory convention, making it easier to include additional configurations without explicit declarations.
details
Load Config
flowchart TD A[“Load Configuration File”] –> B{“Import Section Exists?”}
B -- Yes --> C["Process Imports in Order"]
C --> D{"Import Type?"}
D --> E["Remote URL"]
D --> F["Specific Path"]
D --> G["Wildcard Globs"]
E --> H["Fetch Config from Remote URL"]
F --> I["Read Config from Filesystem"]
G --> I["Read Config from Filesystem"]
H --> J["Call Load Configuration File (Recursively)"]
I --> J["Call Load Configuration File (Recursively)"]
J --> L["Deep Merge with Current Config in Memory"]
L --> K{"More Imports to Process?"}
K -- Yes --> C
K -- No --> M["Configuration Processing Complete"]
%% Loopback for recursion
J -.-> A
%% Styling for clarity
style A fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style B fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style C fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style D fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style E fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style F fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style H fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style I fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style J fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style L fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style M fill<i class="em em-#1D3557,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
classDef recursion stroke-dasharray: 5 5;
Loading
3 Stages
config: layout: fixed — flowchart TD A[“Start Configuration Process”] –> Z1[“Load Atmos Schema Defaults”] Z1 –> Z2[“Load // go:embed cloudposse/atmos/atmos.yaml”] Z2 –> Z3[“Deep Merge Schema Defaults and Embedded Config”] Z3 –> B{“Is –config Provided?”}
%% If --config is provided
B -- Yes --> C["Stage 1: Load Explicit Configurations via --config"]
C --> C1{"Load each --config path in order"}
C1 --> C2["Update ATMOS_CLI_CONFIG_PATH with loaded config absolute paths (separated by delimiter)"]
C2 --> D["Process Imports and Deep Merge"]
D --> E["Final Merged Config"]
E --> F["Output Final Configuration"]
%% If --config is not provided
B -- No --> G["Stage 1: Load System Configurations"]
G --> G1{"Check System Paths"}
G1 -- Found --> G2["Load First Found Config: %PROGRAMDATA%/atmos.yaml (Windows), /usr/local/etc/atmos.yaml, or /etc/atmos.yaml"]
G2 --> G3["Update ATMOS_CLI_CONFIG_PATH"]
G3 --> H["Process Imports and Deep Merge"]
G1 -- Not Found --> H["Process Imports and Deep Merge"]
H --> I["Stage 2: Discover Additional Configurations"]
I --> I1{"Check ATMOS_CLI_CONFIG_PATH"}
I1 -- Found --> I2["Load First Found Config: atmos.yaml or atmos.d/**/* from ATMOS_CLI_CONFIG_PATH"]
I2 --> I4["Update ATMOS_CLI_CONFIG_PATH with loaded absolute paths"]
I4 --> J["Process Imports and Deep Merge"]
I1 -- Not Found --> I5{"Check Git Repository Root"}
I5 -- Found --> I6["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/*, or .github/atmos.yaml from Git Repo Root"]
I6 --> I4
I5 -- Not Found --> I12{"Check Current Working Directory"}
%% New branch for Current Working Directory (note it's not identical to repo root)
I12 -- Found --> I13["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/* from CWD"]
I13 --> I4
I12 -- Not Found --> I18["No configuration found in Stage 2"]
I18 --> K["Stage 3: Apply User Preferences"]
J --> K["Stage 3: Apply User Preferences"]
K --> K1{"Check $XDG_CONFIG_HOME/atmos.yaml"}
K1 -- Found --> K2["Load First Found Config: $XDG_CONFIG_HOME/atmos.yaml"]
K2 --> K3["Update ATMOS_CLI_CONFIG_PATH with $XDG_CONFIG_HOME/atmos.yaml"]
K3 --> L["Process Imports and Deep Merge"]
K1 -- Not Found --> K4{"Check User's Home Directory"}
K4 -- Found --> K5["Load First Found Config: %LOCALAPPDATA%/atmos/atmos.yaml (Windows), ~~~/.config/atmos/atmos.yaml, or ~~~atmos/atmos.yaml (Linux/macOS)"]
K5 --> K3
K4 -- Not Found --> K7["No configuration found in Stage 3"]
K7 --> M["Final Merged Config"]
L --> M["Final Merged Config"]
M --> F["Output Final Configuration"]
%% Styling for clarity
style A fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style Z1 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style Z2 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style Z3 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style B fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style C fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style C1 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style C2 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style D fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style E fill<i class="em em-#457B9D,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style F fill<i class="em em-#1D3557,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style G1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style G3 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style H fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I4 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style J fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I5 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I6 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I12 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I13 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style I18 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style K fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K1 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K2 fill<i class="em em-#E63946,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K3 fill<i class="em em-#A8DADC,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#000000
style K4 fill<i class="em em-#F4A261,stroke"></i>#1D3557,stroke-width<i class="em em-2px,color"></i>#FFFFFF
style K5 f…
oh I see. As far as I can tell, there is no default fallback if schema isnt provided - remote URL or otherwise
@haitham911eg is working on that. See diagrams in PR description. Under 3 stages, you will see the first step is to add go:embed for atmos
Remote schema was in https://github.com/cloudposse/atmos/pull/777
what
if schemas.atmos.manifest empty atmos validate not work work.
if schemas.atmos.manifest empty is set Default Schema to Remote Schema .
default remote url https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json
why
We should set the default schema to the remote atmos schema so that atmos validate work
references
DEV-2727
https://linear.app/cloudposse/issue/DEV-2727/set-default-schema-to-remote-schema
DEV-2720
https://linear.app/cloudposse/issue/DEV-2720/error-if-remote-schema-for-validation-not-set
2024-11-17 14_35_09-Window
2024-11-17 14_42_51-Window
Summary by CodeRabbit
• New Features
• Introduced a default URL for the Atmos JSON Schema manifest, ensuring a consistent configuration experience.
• Improvements
• Simplified logic for determining the Atmos manifest file path, enhancing error handling and control flow.
ah okay this is a bug. I’ll fix it
this was a red-herring on my part. It’s because I had an old atmos config locally that had an explicit path for schema
with that, now this PR is ready for a final review: https://github.com/cloudposse/atmos/pull/891
what
• Add support for relative paths in imports, when the path starts with ./
or ../
• Support all path types:
import:
- ./relative_path
- ../relative_path
or
import:
- path: ./relative_path
- path: ../relative_path
why
• Allow less path duplication and support simpler code
references
Summary by CodeRabbit
• New Features
• Added a new function to resolve relative paths in stack imports.
• Introduced a new test case for processing YAML configurations with relative paths.
• Added a new YAML configuration file for stage-specific settings.
• Tests
• Added comprehensive test for stack processor with relative path scenarios.
• Chores
• Updated .gitignore
to ignore cache-related text files.
• Modified import paths in several YAML configuration files for improved clarity and consistency.
Dan, I’m fixing it (JSON schema) in my PR, there are more issues there
ah okay thanks. But this PR is ready for a review either way – tests are passing now
LGTM
@Dan Miller (Cloud Posse) something I missed in your PR. I don’t like the name of these test-cases: complete
because it’s neither complete, nor based on a convention. If it were based on a scenario or demo, I think it would make more sense.
So I think for now, we create a .yaml config for the test cases of each scenario or demo
that is everything that used to be just the 1 example
rather than breaking it up, I just called it “complete”
i know, but your next PR will introduce a new file, no?
Then the file called complete
is not complete.
And complete has no meaning.
yeah each new PR creates a new example
what’s a better name?
like I suggested, naming it after the scenario makes sense to me.
so we have a lot of tests that are for complete
, but then the tests should actually be changing to that directory
in this case, the tests are changing to a different directory
what is the scenario though, if the scenario is all existing tests
it used to just be called examples/test
yes but what would you call a scenario that we have been using with all existing tests?
legacy sounds bad
I’m okay with calling the scernario complete. My point is that test-cases/complete.yaml
tests things that are not in compelte
agree, but naming is hard
so if a test is testing relative-paths, I’m saying call the file test-cases/relative-paths.yaml
if a test is testing scenarios/complete, I’m saying call the file test-cases/complete.yaml
agree on that too, that’s what I believe I did
no, because test-cases/complete
is testing things that is not in scenarios/complete
1
ohh
i see what you mean
okay
just need to break up the test-cases/complete into test-cases/complete and test-cases/demos
or would you rather having 1 per demo?
also probably should create one called core
/tests/test-cases
core.yaml
complete.yaml
demos.yaml
relative-paths.yaml
metadata.yaml
what
• Break up the “complete” test case into each respective type
why
• complete
is neither complete, nor based on a convention. Directly associate each test case file name with the scenario or functionality it is testing
references
• Slack
2025-01-10
Good morning
good morning Carlos
Carlos is going to help with some atmos tasks!
I see terraform is failing on latest version CI @Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse)
have a link to the failed build?
still failing for me in local
comming from https://github.com/cloudposse/atmos/blob/main/internal/exec/terraform.go locked component
``` package exec
import ( “fmt” “os” osexec “os/exec” “path/filepath” “strings”
"github.com/pkg/errors"
"github.com/spf13/cobra"
tuiUtils "github.com/cloudposse/atmos/internal/tui/utils"
cfg "github.com/cloudposse/atmos/pkg/config"
"github.com/cloudposse/atmos/pkg/schema"
u "github.com/cloudposse/atmos/pkg/utils" )
const ( autoApproveFlag = “-auto-approve” outFlag = “-out” varFileFlag = “-var-file” skipTerraformLockFileFlag = “–skip-lock-file” everythingFlag = “–everything” forceFlag = “–force” )
// ExecuteTerraformCmd parses the provided arguments and flags and executes terraform commands func ExecuteTerraformCmd(cmd *cobra.Command, args []string, additionalArgsAndFlags []string) error { info, err := ProcessCommandLineArgs(“terraform”, cmd, args, additionalArgsAndFlags) if err != nil { return err } return ExecuteTerraform(info) }
func shouldProcessStacks(info *schema.ConfigAndStacksInfo) (bool, bool) { shouldProcessStacks := true shouldCheckStack := true
if info.SubCommand == "clean" &&
(u.SliceContainsString(info.AdditionalArgsAndFlags, everythingFlag) ||
u.SliceContainsString(info.AdditionalArgsAndFlags, forceFlag)) {
if info.ComponentFromArg == "" {
shouldProcessStacks = false
}
shouldCheckStack = info.Stack != ""
}
return shouldProcessStacks, shouldCheckStack }
func generateBackendConfig(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, workingDir string) error { // Auto-generate backend file if atmosConfig.Components.Terraform.AutoGenerateBackendFile { backendFileName := filepath.Join(workingDir, “backend.tf.json”)
u.LogDebug(*atmosConfig, "\nWriting the backend config to file:")
u.LogDebug(*atmosConfig, backendFileName)
if !info.DryRun {
componentBackendConfig, err := generateComponentBackendConfig(info.ComponentBackendType, info.ComponentBackendSection, info.TerraformWorkspace)
if err != nil {
return err
}
err = u.WriteToFileAsJSON(backendFileName, componentBackendConfig, 0644)
if err != nil {
return err
}
}
}
return nil }
func generateProviderOverrides(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, workingDir string) error {
// Generate providers_override.tf.json
file if the providers
section is configured
if len(info.ComponentProvidersSection) > 0 {
providerOverrideFileName := filepath.Join(workingDir, “providers_override.tf.json”)
u.LogDebug(*atmosConfig, "\nWriting the provider overrides to file:")
u.LogDebug(*atmosConfig, providerOverrideFileName)
if !info.DryRun {
var providerOverrides = generateComponentProviderOverrides(info.ComponentProvidersSection)
err := u.WriteToFileAsJSON(providerOverrideFileName, providerOverrides, 0644)
return err
}
}
return nil }
// ExecuteTerraform executes terraform commands func ExecuteTerraform(info schema.ConfigAndStacksInfo) error { atmosConfig, err := cfg.InitCliConfig(info, true) if err != nil { return err }
if info.NeedHelp {
return nil
}
// If the user just types `atmos terraform`, print Atmos logo and show terraform help
if info.SubCommand == "" {
fmt.Println()
err = tuiUtils.PrintStyledText("ATMOS")
if err != nil {
return err
}
err = processHelp(atmosConfig, "terraform", "")
if err != nil {
return err
}
fmt.Println()
return nil
}
if info.SubCommand == "version" {
return ExecuteShellCommand(atmosConfig,
"terraform",
[]string{info.SubCommand},
"",
nil,
false,
info.RedirectStdErr)
}
// Skip stack processing when cleaning with --everything or --force flags to allow cleaning without requiring stack
// configuration
shouldProcessStacks, shouldCheckStack := shouldProcessStacks(&info)
if shouldProcessStacks {
info, err = ProcessStacks(atmosConfig, info, shouldCheckStack, true)
if err != nil {
return err
}
if len(info.Stack) < 1 && shouldCheckStack {
return errors.New("stack must be specified when not using --everything or --force flags")
}
}
if !info.ComponentIsEnabled {
u.LogInfo(atmosConfig, fmt.Sprintf("component '%s' is not enabled and skipped", info.ComponentFromArg))
return nil
}
err = checkTerraformConfig(atmosConfig)
if err != nil {
return err
}
// Check if the component (or base component) exists as Terraform component
componentPath := filepath.Join(atmosConfig.TerraformDirAbsolutePath, info.ComponentFolderPrefix, info.FinalComponent)
componentPathExists, err := u.IsDirectory(componentPath)
if err != nil || !componentPathExists {
return fmt.Errorf("'%s' points to the Terraform component '%s', but it does not exist in '%s'",
info.ComponentFromArg,
info.FinalComponent,
filepath.Join(atmosConfig.Components.Terraform.BasePath, info.ComponentFolderPrefix),
)
}
// Check if the component is allowed to be provisioned (`metadata.type` attribute is not set to `abstract`)
if (info.SubCommand == "plan" || info.SubCommand == "apply" || info.SubCommand == "deploy" || info.SubCommand == "workspace") && info.ComponentIsAbstract {
return fmt.Errorf("abstract component '%s' cannot be provisioned since it's explicitly prohibited from being deployed "+
"by 'metadata.type: abstract' attribute", filepath.Join(info.ComponentFolderPrefix, info.Component))
}
// Check if the component is locked (`metadata.locked` is set to true)
if info.ComponentIsLocked {
// Allow read-only commands, block modification commands
switch info.SubCommand {
case "apply", "deploy", "destroy", "import", "state", "taint", "untaint":
return fmt.Errorf("component '%s' is locked and cannot be modified (metadata.locked = true)",
filepath.Join(info.ComponentFolderPrefix, info.Component))
}
}
if info.SubCommand == "clean" {
err := handleCleanSubCommand(info, componentPath, atmosConfig)
if err != nil {
u.LogTrace(atmosConfig, fmt.Errorf("error cleaning the terraform component: %v", err).Error())
return err
}
return nil
}
varFile := constructTerraformComponentVarfileName(info)
planFile := constructTerraformComponentPlanfileName(info)
// Print component variables and write to file
// Don't process variables when executing `terraform workspace` commands
if info.SubCommand != "workspace" {
u.LogDebug(atmosConfig, fmt.Sprintf("\nVariables for the component '%s' in the stack '%s':", info.ComponentFromArg, info.Stack))
if atmosConfig.Logs.Level == u.LogLevelTrace || atmosConfig.Logs.Level == u.LogLevelDebug {
err = u.PrintAsYAMLToFileDescriptor(atmosConfig, info.ComponentVarsSection)
if err != nil {
return err
}
}
// Write variables to a file (only if we are not using the previously generated terraform plan)
if !info.UseTerraformPlan {
var varFilePath, varFileNameFromArg string
// Handle `terraform varfile` and `terraform write varfile` legacy commands
if info.SubCommand == "varfile" || (info.SubCommand == "write" && info.SubCommand2 == "varfile") {
if len(info.AdditionalArgsAndFlags) == 2 {
fileFlag := info.AdditionalArgsAndFlags[0]
if fileFlag == "-f" || fileFlag == "--file" {
varFileNameFromArg = info.AdditionalArgsAndFlags[1]
}
}
}
if len(varFileNameFromArg) > 0 {
varFilePath = varFileNameFromArg
} else {
varFilePath = constructTerraformComponentVarfilePath(atmosConfig, info)
}
u.LogDebug(atmosConfig, "Writing the variables to file:")
u.LogDebug(atmosConfig, varFilePath)
if !info.DryRun {
err = u.WriteToFileAsJSON(varFilePath, info.ComponentVarsSection, 0644)
if err != nil {
return err
}
}
}
}
// Handle `terraform varfile` and `terraform write varfile` legacy comm…
@Matt Calhoun - no good deed goes unpunished
The eol PR broke releases
https://github.com/cloudposse/atmos/actions/runs/12714711809/job/35446199346
I think it must be related to https://github.com/cloudposse/atmos/pull/928
what
Ensure the EOL for files in this repo are always lf
regardless of platform the user is developing on (windows, mac, linux)
why
Ensure consistency and avoid diffs being shown solely for line endings.
references
Summary by CodeRabbit
• Chores
• Updated .editorconfig
to enforce LF line endings
• Updated .gitattributes
to standardize line endings across all files
• Maintained existing documentation and generated content configurations
@Matt Calhoun I renormalized all the files, and the setting broke images. I think we cannot use *
and should use specific file extensions
this might fix it https://github.com/cloudposse/atmos/pull/929
what
• Renormalize all files (git add --renormalize .
) to respect new defaults
why
• We changed the file formatting for better cross-platform support
references
• #928
@Matt Calhoun - no good deed goes unpunished
The eol PR broke releases
https://github.com/cloudposse/atmos/actions/runs/12714711809/job/35446199346
@Andriy Knysh (Cloud Posse) ready for final review: https://github.com/cloudposse/atmos/pull/919
what
Colorize output when we attach TTY
Before:
Screenshot 2025-01-07 at 19 05 26
After:
Screenshot 2025-01-07 at 15 37 10
references
Summary by CodeRabbit
• New Features
• Added syntax highlighting configuration for terminal outputs.
• Introduced customizable settings for terminal display, including line numbers and color styles.
• Enhanced configuration options for YAML and JSON output formatting.
• Documentation
• Added comprehensive documentation for terminal settings and syntax highlighting configuration.
• Chores
• Updated configuration schema to support new terminal display options.
• Implemented utility functions for managing syntax highlighting.
• Standardized naming conventions in configuration settings.
• Removed deprecated timestamps property from terminal settings.
@Andriy Knysh (Cloud Posse) this is also ready for final review: https://github.com/cloudposse/atmos/pull/912
what
After the internal discussion and descoping of this PR there 2 parts left:
- the support of the github token for the version command. The code checks if the token is set (e.g. via an env var) and if it is, the requests are done using the token. In case no token is provided, the requests to github are made without it.
- for github links to folders and in case the ATMOS_GITHUB_TOKEN or GITHUB_TOKEN variable is set, the files are downloaded with go-getter (v1) that uses the token. The token is put into the url under the hood, so for the end user/custmer the interaction/syntax remains the same. I.e. the following commands specified in the vendor.yaml pull out the files from github using the token (not subject to anonymous users rate limits)
NB. github.com/analitikasi/Coonector.git - is a private repo
- component: “weather” source: “github.com/analitikasi/Coonector.git//quick-start-simple/components/terraform/{{ .Component }}?ref={{.Version}}” version: “main” targets: - “components/terraform/{{ .Component }}/{{.Version}}” tags: - demo
why
Bypass github ratelimits for non-auth requests (use a token instead)
references
• DEV-2778 (1 part out of 4 initially requested + 1 extra feature for repo cloning using the token) • Supersedes #871
Summary by CodeRabbit
Summary by CodeRabbit
• New Features
• Enhanced GitHub client authentication by supporting optional token-based authentication.
• Added custom GitHub URL detection and transformation for improved package downloading.
• Introduced a new configuration option for GitHub token injection in the Atmos CLI.
• Added a new environment variable ATMOS_GITHUB_TOKEN
for Bearer token usage in GitHub API requests.
• Improvements
• Introduced more flexible GitHub client creation process.
• Added custom detection mechanisms for package downloads.
• Improved URL handling for GitHub-based resources.
• Established a default configuration setting for GitHub token injection.
• Documentation
• Updated documentation to include the new inject_github_token
configuration option in atmos.yaml
.
2025-01-12
Did the Atmos 1.146.0
Cloudsmith package release complete? Looks like it isn’t available
We’re investigating why a binary was not created with this release. Stay tuned.
Prob will get fixed tomorrow
2025-01-14
@Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/923 this is good for review
@Vinny please address Erik’s comment
please ping me when done
@Andriy Knysh (Cloud Posse) this is ready for review https://github.com/cloudposse/atmos/pull/896
issue: https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig
what
• Added functionality to atmos validate to validate against the .editorconfig file if it is defined. • Integrated the editorconfig-checker library to perform the validation. • Modified the validation process to support OPA/JSON schema validation alongside the new .editorconfig validation.
why
• The change ensures that the .editorconfig file is validated, which helps maintain consistency in code formatting across teams. • By incorporating editorconfig-checker, the validation process becomes more comprehensive, ensuring that both OPA/JSON schemas and editor configuration are correct. • This update was made to improve development workflow by catching formatting issues early during the validation process.
references
• https://github.com/editorconfig-checker/editorconfig-checker • https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig
Summary by CodeRabbit
• New Features
• Added EditorConfig validation command to check and enforce coding style rules.
• Introduced configuration options for EditorConfig validation in Atmos configuration.
• Implemented comprehensive EditorConfig checking across project files.
• Documentation
• Added detailed documentation for the EditorConfig validation command.
• Created new documentation explaining the EditorConfig validation process and usage.
• Configuration
• Added .editorconfig
files to enforce consistent coding styles.
• Updated Atmos configuration to support EditorConfig validation settings.
@Shubham Tholiya please address the comment
issue: https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig
what
• Added functionality to atmos validate to validate against the .editorconfig file if it is defined. • Integrated the editorconfig-checker library to perform the validation. • Modified the validation process to support OPA/JSON schema validation alongside the new .editorconfig validation.
why
• The change ensures that the .editorconfig file is validated, which helps maintain consistency in code formatting across teams. • By incorporating editorconfig-checker, the validation process becomes more comprehensive, ensuring that both OPA/JSON schemas and editor configuration are correct. • This update was made to improve development workflow by catching formatting issues early during the validation process.
references
• https://github.com/editorconfig-checker/editorconfig-checker • https://linear.app/cloudposse/issue/DEV-2836/implement-atmos-validate-editorconfig
Summary by CodeRabbit
• New Features
• Added EditorConfig validation command to check and enforce coding style rules.
• Introduced configuration options for EditorConfig validation in Atmos configuration.
• Implemented comprehensive EditorConfig checking across project files.
• Documentation
• Added detailed documentation for the EditorConfig validation command.
• Created new documentation explaining the EditorConfig validation process and usage.
• Configuration
• Added .editorconfig
files to enforce consistent coding styles.
• Updated Atmos configuration to support EditorConfig validation settings.
@Andriy Knysh (Cloud Posse) simple one liner from @Dan Hansen https://github.com/cloudposse/atmos/pull/936
what
Adds an error message to the output of atmos vendor pull
for the last component in a vendor manifest
references
closes #935
Summary by CodeRabbit
• Bug Fixes
• Improved package installation error reporting by displaying more detailed error messages during the installation process.