#office-hours (2021-01)
Meeting password: sweetops
Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours
Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours
Meeting password: sweetops
2021-01-21
2021-01-20
@here office hours is starting in 30 minutes! Remember to post your questions here.
Will this affect Terraform ?
<https://aws.amazon.com/about-aws/whats-new/2021/01/aws-sdk-for-go-version-2-now-generally-available/>
Does anyone follow a git flow (i.e. develop + master branches) pattern for larger Terraform repositories? How has that worked out for you?
I tried using develop and master and found it hard to manage. So, I just do PR’s off of master. I create tags for each module as well similar to this source = [email protected]<name-of-repo?ref=v1.0.0 so that when the module changes it’s not pointing to a local reference of my modules. There could be better ways to do it though. We are running terraform locally though and have a small team. I’d be interested in what others are doing as well.
I tried lot of approaches, let me list them first, then discuss them
• one repo - a branch per ENV
• multiple repos - a repo per env
• one repo - a folder per ENV all of these has pros and cons, but I move to something called stacks
So for any account I have stacks, one for network, others for Data, compute, app, CICD, so any account use one repo, and in that repo I have stacks each stack points to folder. then using a makefile I can have the sequence I want, and using remote state data source I can any output I want.
• one account reflect an ENV in one repo, and within multiple stacks with a makefile If I need to do any change I would create aPR on target account repo, run TF plan, approve it and merge, then run terraform apply, branch name cloud be issue-X or feat-X that match our backlog
some PR require changes on multiple stacks, and that can be easily done with this approach
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
David Lundgren has joined Public “Office Hours”
Phil Hershkowitz has joined Public “Office Hours”
any tips for improving global s3 upload speed? (think india, hong kong, etc) what other optimizations could I possibly make after turning on s3 transfer acceleration and using multipart uploads?
mb Branski has joined Public “Office Hours”
Sam C has joined Public “Office Hours”
Colton Wrisner has joined Public “Office Hours”
James Thalacker has joined Public “Office Hours”
Brian Tai has joined Public “Office Hours”
Jim Park has joined Public “Office Hours”
sri has joined Public “Office Hours”
Cosmin Drimba has joined Public “Office Hours”
Andy Miguel (Cloud Posse) has joined Public “Office Hours”
Patrick Jahns has joined Public “Office Hours”
Tim Gourley has joined Public “Office Hours”
Sebastian Stadil has joined Public “Office Hours”
Mansoor Ebrahim has joined Public “Office Hours”
Cosmin Drimba has joined Public “Office Hours”
James Haughey has joined Public “Office Hours”
Bill Clark has joined Public “Office Hours”
15139103984 has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Srikar Ananthula has joined Public “Office Hours”
Neil Gealy has joined Public “Office Hours”
Leia Renée has joined Public “Office Hours”
Andy Roth has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
venkatamutyala has joined Public “Office Hours”
Todd Thomas has joined Public “Office Hours”
Guelor Emanuel has joined Public “Office Hours”
Loren Gordon has joined Public “Office Hours”
Mohammed Yahya has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Oskar Maria Grande has joined Public “Office Hours”
mike dizon has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
Petros Kolyvas has joined Public “Office Hours”
Robert Jackson has joined Public “Office Hours”
Question re: office hours. What types of internal users have you seen typically use these prebuilt stacks / catalog?
Catalogs is a general concept
Here’s our catalog of datadog monitors:
Here’s our catalog for AWS Config (with of the rules for CIS 1.2)
…so now we are extending our catalog approach to infrastructure using “Stacks”
and since we’ve abstracted the concept of a stack as YAML, now we can support that with virtually any TACOS provider.
Like scalr
Blaisep has joined Public “Office Hours”
Jose Franco has joined Public “Office Hours”
Sheldon Hull has joined Public “Office Hours”
Are you guys going to update examples in atmos? I’d love to get it going!
Whatever cloud provider, a registry for easy review and calling of modules is key. I checked and Env0 didn’t have it at this time. Scalyr i think did. Spacelift i haven’t checkout out, but will look too.
I really want to towards that curated high quality modules for teams to use this way.
Dennis Lipovsky has joined Public “Office Hours”
Miguelangel Freitas has joined Public “Office Hours”
Cloudposse modules are epic I’m always looking to leverage them. I just painfully used a new datadog monitor project that wasn’t Cloudposse as it was a bit more extensive and it felt like pulling teeth.
Much
Ah what did the DD monitor module not have that you were looking for? I was just in there, so I’m curious.
1 - preset messages already built (maybe you had that too) 2 - 10-20 prebuilt monitors for different services already ready. I wanted to convert but didn’t have time. https://registry.terraform.io/modules/claranet/monitors/datadog/latest
I would like to use cloudposse, but need to have time to convert the checks into yaml to use that.
@sheldonh did you see the catalog?
There’s a lot there, but maybe not for your services
e.g. we have 20 monitors just for EKS
We’ll gladly accept PRs for more monitors that we can add to our catalog.
If you compare our module to the one by claranet, it’s a pretty big difference. The claranet one requires a submodule for every one.
While using the YAML config pattern, we add monitors very easily:
k8s-deployment-replica-pod-down:
name: "(k8s) Deployment Replica Pod is down"
type: query alert
query: |
avg(last_15m):avg:kubernetes_state.deployment.replicas_desired{*} by {cluster_name,deployment} - avg:kubernetes_state.deployment.replicas_ready{*} by {cluster_name,deployment} >= 2
message: |
({{cluster_name.name}}) More than one Deployments Replica's pods are down on {{deployment.name}}
escalation_message: ""
tags: [ "ManagedBy:Terraform" ]
notify_no_data: false
notify_audit: true
require_full_window: true
enable_logs_sample: false
force_delete: true
include_tags: true
locked: false
renotify_interval: 0
timeout_h: 0
evaluation_delay: 60
new_host_delay: 300
no_data_timeframe: 5
threshold_windows: { }
thresholds:
critical: 2
For what it’s worth I’ve read that this type of concept:
You want to simplify, that’s great! Simplifying complex systems though often is basically shifting around complexity. In this case… do you want Terragrunt to manage the complexity and it’s own flow, or do you shift it to the user, or a yaml config for example.
I think it’s a case of where do you feel the complexity should be best moved.
Food for though.
Jeremy Branham has joined Public “Office Hours”
Udit Dave has joined Public “Office Hours”
Jose Franco has joined Public “Office Hours”
Hao Wang has joined Public “Office Hours”
WOOT WOOT. I didn’t see atmos. Is this built on top of variant2?
It’s a master class in atmos.
I’d like to say it was “Easy”, but it’s been quite challenging. We’ve had to work a lot with Mumoshu to get to where we are on it today.
We’re also developing a companion cli in pure-go
still deciding on name of that one. that cli is for working around limitations in terraform providers in order for us to provde SOC2 compliance for customers.
Andrew Thompson has joined Public “Office Hours”
Seriously you guys are on FIRE
tfswitch — https://tfswitch.warrensbox.com/
asdf is about the ultimate expression of lazy typing i’ve ever seen. Absolutely love it!
Go is HUGE on backward compatibility. They have a promise even on “deprecated” features to never break functionality.
I’d say even if they move to SDK v2, it’s not going to impact us overall.
Pretty sure SDK v2 has been out for years before this announcement too, so it’s not “new”. It’s been in usage from 2018 i believe. It simplifies a lot of code too.
New Zoom Recording from our Office Hours session on 2021-01-20 is now available.
2021-01-14
2021-01-13
@here office hours is starting in 30 minutes! Remember to post your questions here.
Andy Miguel has joined Public “Office Hours”
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Ian Groff has joined Public “Office Hours”
Mohammed Yahya has joined Public “Office Hours”
James Haughey has joined Public “Office Hours”
Hemanth Kumar has joined Public “Office Hours”
Bircan Bilici has joined Public “Office Hours”
Tim Gourley has joined Public “Office Hours”
Andy Miguel has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
mb Branski has joined Public “Office Hours”
15139103984 has joined Public “Office Hours”
Patrick Jahns has joined Public “Office Hours”
Udit Dave has joined Public “Office Hours”
Mazin Ahmed has joined Public “Office Hours”
Michael Martin has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Kenji Nakamura has joined Public “Office Hours”
Isa Aguilar has joined Public “Office Hours”
Bill Clark has joined Public “Office Hours”
Dave Lundgren has joined Public “Office Hours”
Working with the SSO resources - but would be great to also define at least Groups via code
PePe Amengual has joined Public “Office Hours”
Kenji Nakamura has joined Public “Office Hours”
@ You mentioned you were getting started with Terraform - this is a GREAT book for “getting up and running”. https://www.amazon.com/Terraform-Running-Writing-Infrastructure-Code/dp/1491977086
There is a second edition of this book - https://www.amazon.com/gp/aw/d/1492046906/ref=dp_ob_neva_mobile
Yes. I am reading this one!
Oskar Maria Grande has joined Public “Office Hours”
@ I think this is one of the outstanding SSO resources that’s still in draft: https://github.com/hashicorp/terraform-provider-aws/pull/15322
I’ve been following that PR and it seems like that might be what you were missing.
https://www.terraform.io/docs/configuration/types.html#experimental-optional-object-type-attributes
variable "with_optional_attribute" {
type = object({
a = string # a required attribute
b = optional(string) # an optional attribute
})
}
awesome to use
here is a use case in my ecs-cluster module https://github.com/mhmdio/terraform-aws-ecs-cluster/blob/master/variables.tf
Shouky Dan has joined Public “Office Hours”
Marc Tamsky has joined Public “Office Hours”
mb Branski has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
rajiv ranjan has joined Public “Office Hours”
use this CFN templates until TF AWS SSO is ready
Looks like AWS SSO assignment resources are dropping today in v3.24.0 — https://github.com/hashicorp/terraform-provider-aws/issues/15108#issuecomment-760421304
AWSTemplateFormatVersion: 2010-09-09
Description: Configure AWS SSO
Parameters:
AwsSsoInsanceArn:
Type: String
Default: 'arn:aws:sso:::instance/ssoins-XXXXXXXXXXXXX'
Description: 'AWS SSO Instance ARN.'
Mappings:
Groups:
Admins:
Id: 'XXXXXXX-xxxxx-xxxx-xxxx-xxxx-XXXXX'
Developers:
Id: 'XXXXXXX-xxxxx-xxxx-xxxx-xxxx-XXXXX'
Developers-CodeCommit:
Id: 'XXXXXXX-xxxxx-xxxx-xxxx-xxxx-XXXXX'
Accounts:
master:
Id: '111111111111'
shared:
Id: '222222222222'
dev:
Id: '333333333333'
prod:
Id: '444444444444'
Resources:
adminsPermissionSet:
Type: AWS::SSO::PermissionSet
Properties:
Description: Admins Group Administrator Access Permission Set
InstanceArn: !Ref AwsSsoInsanceArn
ManagedPolicies:
- arn:aws:iam::aws:policy/AdministratorAccess
Name: Admins
Tags:
- Key: 'CFN'
Value: 'Yes'
- Key: 'Project'
Value: 'Awesome'
developersPermissionSet:
Type: AWS::SSO::PermissionSet
Properties:
Description: Developers Group Read Only Access Permission Set
InstanceArn: !Ref AwsSsoInsanceArn
ManagedPolicies:
- arn:aws:iam::aws:policy/ReadOnlyAccess
Name: Developers
Tags:
- Key: 'CFN'
Value: 'Yes'
- Key: 'Project'
Value: 'Awesome'
developersCodeCommitPermissionSet:
Type: AWS::SSO::PermissionSet
Properties:
Description: Developers Group CodeCommit Access Permission Set
InstanceArn: !Ref AwsSsoInsanceArn
ManagedPolicies:
- arn:aws:iam::aws:policy/AWSCodeCommitPowerUser
Name: Developers-CodeCommit
Tags:
- Key: 'CFN'
Value: 'Yes'
- Key: 'Project'
Value: 'Awesome'
# masterAssignmentAdmins:
# Type: AWS::SSO::Assignment
# Properties:
# InstanceArn: !Ref AwsSsoInsanceArn
# PermissionSetArn: !GetAtt adminsPermissionSet.PermissionSetArn
# TargetId: !FindInMap [ Accounts, master, Id ]
# TargetType: 'AWS_ACCOUNT'
# PrincipalType: 'GROUP'
# PrincipalId: !FindInMap [ Groups, Admins, Id ]
sharedAssignmentAdmins:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt adminsPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, shared, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Admins, Id ]
prodAssignmentAdmins:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt adminsPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, prod, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Admins, Id ]
devAssignmentAdmins:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt adminsPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, dev, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Admins, Id ]
devAssignmentDevelopers:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt developersPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, dev, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Developers, Id ]
prodAssignmentDevelopers:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt developersPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, prod, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Developers, Id ]
sharedAssignmentDevelopers:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt developersPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, shared, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Developers, Id ]
sharedAssignmentDevelopersCodeCommit:
Type: AWS::SSO::Assignment
Properties:
InstanceArn: !Ref AwsSsoInsanceArn
PermissionSetArn: !GetAtt developersCodeCommitPermissionSet.PermissionSetArn
TargetId: !FindInMap [ Accounts, shared, Id ]
TargetType: 'AWS_ACCOUNT'
PrincipalType: 'GROUP'
PrincipalId: !FindInMap [ Groups, Developers-CodeCommit, Id ]
If you have time and interest talk about your Codefresh GitOps method. And did you look and consider the Gitlab terraform automation? Pros and Cons
Not using it, recommending it or promoting it any more for the same reasons we don’t recommend github actions
use a purpose built platform. don’t roll your own using a general purpose CI/CD solution.
So neg on the Gitlab and GitHub. But you still like and use Codefresh right? I ask as we have enterprise Bitbucket which has CI/CD capabilities. Im not sure I like it for more than a repo though.
Yes we still use a lot of Codefresh.
But based on some of our engagements last year, I just don’t recommend building your own terraform CD solution. The problem I think is that teams who want to do it are solving the wrong problem and haven’t yet practiced gitops enough to know the challenges. If after having used TACOS, the team still believes it can do a better job, then they can try it.
We also use a lot of GitHub actions. Love them. But we don’t use them for terraform .
(Other than for automated testing)
Rohit Koimattur has joined Public “Office Hours”
Phil Hersh has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
hari b has joined Public “Office Hours”
New Zoom Recording from our Office Hours session on 2021-01-13 is now available.
2021-01-11
2021-01-09
I know y’all explained the CloudPosse way of defining envs and propagating changes many times, but for the life of me I can’t find that recording. Anybody know where I can find it? That part of Office Hours where @Erik Osterman (Cloud Posse) explained the base Terraform infra, and then the DBs, and then the apps thing. With a flowchart with arrows and nice diagram of things building on top of the previous stages
AHA!
The 4 layers of infrastructure! They’re explained starting here: https://youtu.be/fVRy3qpTxME?t=2249
Yes! that was it.
I’ve also added the lucid chart here: https://cloudposse.com/big-picture/
but I think i’ll change it to a image so it’s easier to share
2021-01-07
2021-01-06
Question for today’s discussion: When deploying via helm, do you use hacked versions of the full values.yaml files or a file with just diffs? Which is better for managing upgrades of things like the datadog agent, going forward?
@here office hours is starting in 30 minutes! Remember to post your questions here.
Thank you for announcing these! I totally forgot what day it was
@Erik Osterman (Cloud Posse) Q : In https://github.com/cloudposse/reference-architectures#3-delegate-dns Can some one explains An available domain we can use for DNS-base service discovery (E.g. [ourcompany.co](http://ourcompany.co)). This domain must not be in use elsewhere as the master account will need to be the authoritative name server (SOA).
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Tarlan Isaev has joined Public “Office Hours”
charles pogi has joined Public “Office Hours”
Weston Platter has joined Public “Office Hours”
Vlad Ionescu has joined Public “Office Hours”
Jesse Cafarelli has joined Public “Office Hours”
uri unger has joined Public “Office Hours”
Ken Y.y has joined Public “Office Hours”
Raymond Mazurik has joined Public “Office Hours”
Andy Miguel has joined Public “Office Hours”
Shouky Dan has joined Public “Office Hours”
Adam Crown has joined Public “Office Hours”
Julian Severino has joined Public “Office Hours”
Mohammed Yahya has joined Public “Office Hours”
Brian Tai has joined Public “Office Hours”
Shouky Dan has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
Randy Bridges has joined Public “Office Hours”
Florain Drescher has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
Bill Clark has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Neil Gealy has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Brandon vh has joined Public “Office Hours”
Tim Gourley has joined Public “Office Hours”
15139103984 has joined Public “Office Hours”
Hyejin Song has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Is this per field?
or is this total?
Troy Taillefer has joined Public “Office Hours”
@Erik Osterman (Cloud Posse) how to preserve most significant digit being at the end of the id?
James Haughey has joined Public “Office Hours”
yes, thanks!
Martin Mazurik has joined Public “Office Hours”
Rashid Boyko has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
Kareem Shahin has joined Public “Office Hours”
Related thread: https://sweetops.slack.com/archives/CCT1E7JJY/p1609797027221600
Vicken Simonian has joined Public “Office Hours”
Petros K has joined Public “Office Hours”
Anere Faithful has joined Public “Office Hours”
Question for me: We are adopting the terraform-aws-jenkins infra and are pretty impressed with it. I have noticed some issues with the use of EFS though which I wanted to ask about. I am not sure if we’re somehow doing it wrong or if this is a genuine issue. 1- throughput in terms of single file operations: dsl jobs (as well as master startup time) are quite longer compared to what we experienced with using local store (which we have done in the past). benchmarking on the master node indeed shows a round trip for a single operation is 5-10ms. 2- normal “nfs pain”- network “hickups” results with locked threads in jenkins master that never unlocks.
Rohit Koimattur has joined Public “Office Hours”
Has anyone looked at and/or considered Terraspaces as a framework for terraform? I have setup a quick and dirty environment in a Cloud9 instance and so far am loving it.
Michael Jenkins has joined Public “Office Hours”
hello, what do you guys use for something like constant configuration changes? i was leaning towards ansible but just wanted to see if there is anything out there that works ok too
I am still on bit.y/oauth2_proxy … it seems like maybe the project itself has moved on and is active still… (I’m on a very old version on the “legacy” services I front with oauth2 proxy still) – https://github.com/oauth2-proxy/oauth2-proxy
f5 BIG-IP 3DNS?
Abisoye Olaomi has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
I have not heard of anyone using a dedicated DNS account before
Would you talk a little about your thoughts limits around multi-account setup. I think a limit of 10 or perhaps up 20 is manageable, but beyond that I think you should think of creating a new org for more member accounts
This is a good question. Will pin it for next week.
Excellent. I have been wrestling with this for a while. Obviously there are exceptions. But I like the multi-account member account approach to help isolate and demarc things, but I also worry about sprawl and have seen for some plans of a 100 or more accounts under an org. Myself I see it being more approachable somewhere in the 10 - 25 range.
It certainly helps because spreading the zones out is horrible. You might end up needing to use the “prod” DNS from a non-prod account etc…
ruby gem
Any good news about Waypoint?)
I don’t have any news yet on this. We’re waiting probably until > Q2 before taking a serious look at it.
Very excited about what it is aspiring to do.
I am too.. but… it’s always tricky with projects which try to do “allthethings”. If you start on a clean slate it can be absolutely wonderful.
Going all-in on something, regardless of what it is though… can be amazing.
Oh you’re already talking about)
No such thing as “Best Practice.” There are only tendrils of innovation that become increasingly adopted.
I wasn’t in office hours. Is this in reference to a particular tool, or the DevOps world in general?
Overall I totally agree, but I think there have been a couple of things that should be considered a best practice and used by all/almost all
- Use Git
- Use a modern Git-based VCS like GitHub/GitLab/BitBucket/etc
- Do CI, with an automated testing pipeline
- Containerize
Other more controversial ones that are, in my opinion, best practices
- Kubernetes in all but the most basic of use cases
- Throw out your style guide and automate it with hooks/CI/etc
- …I’ll think about some more. I really like going over stuff like this
IaC, probably in the top list…
This was a general statement. The point I was trying to make is that “Best Practice” today feels less like “use drbd, pacemaker, corosync and this my.cnf” and more like the following picture:
But I agree, there are definitely some practices that are mature and broadly adopted, like your list above.
What I like about Terraspaces is that you can still go an do it with Terraform, but gives some better structure and easier safeties for a small group.
I think one of the most important questions to ask is where on the adoption curve does my organization want to be?
Thanks all! I appreciate the insights.
New Zoom Recording from our Office Hours session on 2021-01-06 is now available.
2021-01-04
A question I have about https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms … I’m using it to scale up/down the number of Fargate Tasks for an ECS Service. My issue is that it the scale down action continues to scale the task count below the min desired count. What I’m trying to achieve is for the scale down process to not scale below the min desired count.
@PePe have you run into this?
not exactly
We enabled a capacity provider which caused and issue similar
maybe you have a capacity provider at the cluster level @?
Checking ….
I don’t have a capacity provider configured. Do I need that?
no no you do not
Capacity providers are a completely different animal
Asking this here so I can ask this during this week’s office hours.