#office-hours (2023-04)
“Office Hours” are every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cloudposse.com/office-hours
Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers!
https://cpco.io/slack-office-hours
Meeting password: sweetops
2023-04-01
Thing related to the discussion we had last week: https://github.com/toricls/aws-fargate-with-step-functions
2023-04-02
Is there any video library link where we can see all office hours
If yes pls share link
Thanks in advance
2023-04-04
2023-04-05
I’m setting up an Elastic stack (Elasticsearch, Kibana) which will mainly be ingesting time-based data (via Filebeat/Metricbeat in EKS clusters); this new stack will replace an older, legacy Elastic stack (version 7.9). This is a self-hosted stack deployed as a series of AWS ASGs.
The legacy stack is only configured to use the hot tier. I’d like to set up an ILM policy for the new stack to leverage hot/warm/cold tiers.
Few questions:
- Do you have recommendations on an example ILM policy that we could start with?
- What instance types or other resources would you recommend for each data tier?
@here office hours is starting in 30 minutes! Remember to post your questions here.
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Vlad Ionescu has joined Public “Office Hours”
Taylor Turner has joined Public “Office Hours”
Linda Pham (Cloud Posse) has joined Public “Office Hours”
Marc Slayton has joined Public “Office Hours”
Amer Zec has joined Public “Office Hours”
Andy Wortman has joined Public “Office Hours”
Jeremy / Nuru (Cloud Posse) has joined Public “Office Hours”
Paul Marcelin has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Jeremy Bouse has joined Public “Office Hours”
tyler has joined Public “Office Hours”
Isaac M has joined Public “Office Hours”
Fireflies.ai Notetaker has joined Public “Office Hours”
Alex Atkinson has joined Public “Office Hours”
dave lundgren has joined Public “Office Hours”
dag viggo lokoeen has joined Public “Office Hours”
Roy Sprague has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
Jim Park has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Jonathan Eunice has joined Public “Office Hours”
David Hawthorne has joined Public “Office Hours”
Risikat Yewande Raji has joined Public “Office Hours”
Alejandro Calbazana has joined Public “Office Hours”
Kris Musard has joined Public “Office Hours”
Alejandro Calbazana has joined Public “Office Hours”
Jeremy White has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Oskar Maria Grande has joined Public “Office Hours”
Michael Vasilenko has joined Public “Office Hours”
Radha Ramadoss has joined Public “Office Hours”
Anup Dubey has joined Public “Office Hours”
Michael Vasilenko has joined Public “Office Hours”
Radha Ramadoss has joined Public “Office Hours”
Links from today’s office hours: https://spacelift.io/blog/introducing-spacelift-self-hosted https://aws.amazon.com/about-aws/whats-new/2023/04/aws-service-catalog-terraform-open-source/ https://github.com/aws-samples/service-catalog-engine-for-terraform-os https://github.blog/2023-03-28-introducing-self-service-sboms/ https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/network/dependencies https://aws.amazon.com/blogs/aws/simplify-service-to-service-connectivity-security-and-monitoring-with-amazon-vpc-lattice-now-generally-available/ https://github.com/hashicorp/terraform-provider-aws/issues/30380 https://github.com/AlexNabokikh/tfsort? https://github.com/sylwit/terraform-cleaner https://github.com/SpotOnInc/renovate-config https://github.com/alfasoftware/astra github/cloudposse/packages Promptops.com https://github.com/jerryjliu/llama_index https://github.com/marketplace/actions/gitops-automatic-versioning
2023-04-06
2023-04-08
This is horribly written
did you guys discussed this before?
2023-04-10
2023-04-11
2023-04-12
@here office hours is starting in 30 minutes! Remember to post your questions here.
I won’t make it today, sorry
In case you missed it, EKS just got 1.26 support https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-26/
Thanks!
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Taylor Turner has joined Public “Office Hours”
Michael Pursifull has joined Public “Office Hours”
Nenna Salinas has joined Public “Office Hours”
When updating EKS how do you folks do it? I tried doing it via terraform and got the error below, but when i updated it manually in the AWS Console/UI it worked fine and my terraform plan was clean without changes.
When using the cloudposse modules to manage EKS how do you folks do your EKS upgrades?
module.cluster.aws_iam_role_policy_attachment.ebs_csi: Refreshing state... [id=AmazonEKS_EBS_CSI_DriverRole-2023041217083972830000000a]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
<= read (data resources)
Terraform planned the following actions, but then encountered a problem:
# module.cluster.module.kubernetes.data.aws_eks_cluster_auth.eks[0] will be read during apply
# (depends on a resource or a module with changes pending)
<= data "aws_eks_cluster_auth" "eks" {
+ id = (known after apply)
+ name = "cluster"
+ token = (sensitive value)
}
# module.cluster.module.kubernetes.data.tls_certificate.cluster[0] will be read during apply
# (depends on a resource or a module with changes pending)
<= data "tls_certificate" "cluster" {
+ certificates = (known after apply)
+ id = (known after apply)
+ url = "<https://oidc.eks.us-west-2.amazonaws.com/id/190881CA4EA9C23E6847BDCA0F149F8F>"
}
# module.cluster.module.kubernetes.aws_eks_cluster.default[0] will be updated in-place
~ resource "aws_eks_cluster" "default" {
id = "cluster"
name = "cluster"
tags = {
"Attributes" = "cluster"
"Name" = "cluster"
}
~ version = "1.24" -> "1.25"
# (10 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
# module.cluster.module.kubernetes.aws_iam_openid_connect_provider.default[0] will be updated in-place
~ resource "aws_iam_openid_connect_provider" "default" {
id = "arn:aws:iam::739737747774:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/190881CA4EA9C23E6847BDCA0F149F8F"
tags = {
"Attributes" = "cluster"
"Name" = "cluster"
}
~ thumbprint_list = [
- "9e99a48a9960b14926bb7f3b02e22da2b0ab7280",
] -> (known after apply)
# (4 unchanged attributes hidden)
}
Plan: 0 to add, 2 to change, 0 to destroy.
╷
│ Error: configmaps "aws-auth" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
│
│ with module.cluster.module.kubernetes.kubernetes_config_map.aws_auth_ignore_changes[0],
│ on .terraform/modules/cluster.kubernetes/auth.tf line 118, in resource "kubernetes_config_map" "aws_auth_ignore_changes":
│ 118: resource "kubernetes_config_map" "aws_auth_ignore_changes" {
│
Alex Atkinson has joined Public “Office Hours”
Robert Jordan has joined Public “Office Hours”
venkata mutyala has joined Public “Office Hours”
David Hawthorne has joined Public “Office Hours”
Ian Bartholomew has joined Public “Office Hours”
Joe Caulfield has joined Public “Office Hours”
Jim Park has joined Public “Office Hours”
Isaac M has joined Public “Office Hours”
Ozzy has joined Public “Office Hours”
tyler has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Andrew Vitko has joined Public “Office Hours”
AJ Junior has joined Public “Office Hours”
John Mitchell has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
Johnmary Odenigbo has joined Public “Office Hours”
Antarr Byrd has joined Public “Office Hours”
Allen Lyons has joined Public “Office Hours”
Jeremy White has joined Public “Office Hours”
Adedapo Ajuwon has joined Public “Office Hours”
Kris Musard has joined Public “Office Hours”
mtamsky has joined Public “Office Hours”
Marc Slayton has joined Public “Office Hours”
Brandon Nason has joined Public “Office Hours”
Ozzy has joined Public “Office Hours”
Ralf Pieper has joined Public “Office Hours”
Ozzy has joined Public “Office Hours”
Links from today’s office hours:
https://atmos.tools/cli/configuration/#logs https://clark.center/browse?collection=nccp&currPage=1 https://bashdb.sourceforge.net/remake/ https://www.hashicorp.com/blog/vault-secrets-operator-a-new-method-for-kubernetes-integration https://www.hashicorp.com/blog/introducing-hermes-an-open-source-document-management-system https://github.com/future-architect/tftarget https://aws.amazon.com/about-aws/whats-new/2023/04/aws-app-runner-compute-configurations/ https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-26/ https://news.ycombinator.com/item?id=32600821 http://mynixos.com/ https://www.chezmoi.io/ https://github.com/aws-samples/eks-cluster-upgrade https://github.com/terraform-aws-modules https://github.com/vmware-archive/kubewatch https://github.com/hwchase17/langchain
yes @Erik Osterman (Cloud Posse) with 12 GB and 4vCpus you can definitely run atlantis in app-runner
1
I think this + a CloudFormation template would be a rad way to bootstrap GitOps for terraform
…requiring minimal scaffolding to get off the ground
cloudformation????? no no no I prefer to run a bash script with aws cli commands that that thing
That’s not very declarative
If it’s a oneliner AWS cli command I will forgive you ;)
lol
ChatGPT
AWSTemplateFormatVersion: 2010-09-09
Resources:
MyECRRepo:
Type: AWS::ECR::Repository
Properties:
RepositoryName: my-container-repo
MyAppRunnerService:
Type: AWS::AppRunner::Service
Properties:
ServiceName: my-app-runner-service
InstanceConfiguration:
InstanceRoleArn: !GetAtt MyIAMRole.Arn
InstanceType: FARGATE
SourceConfiguration:
RepositoryType: ECR
RepositoryUrl: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/my-container-repo"
ImageIdentifier: 1.0
HealthCheckConfiguration:
HealthyThreshold: 1
UnhealthyThreshold: 1
Interval: 60
Path: /
Protocol: HTTP
AutoScalingConfiguration:
MinConcurrency: 1
MaxConcurrency: 2
AuthenticationConfiguration:
ConnectionArn: !Ref MySecretsManagerConnection
AccessRoleArn: !GetAtt MyIAMRole.Arn
MyAppRunnerDeployment:
Type: AWS::AppRunner::Deployment
Properties:
ServiceArn: !Ref MyAppRunnerService
DeploymentName: my-app-runner-deployment
SourceConfiguration:
CodeRepository:
RepositoryUrl: <https://github.com/my/repo.git>
SourceCodeVersion: main
InstanceConfiguration:
InstanceCount: 1
EnvironmentVariables:
◦ Name: ENV_VAR_1 ```
Value: my-value-1 ```
◦ Name: ENV_VAR_2 ```
Value: my-value-2 ``` ``` MyIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement: ```
◦ Effect: Allow ```
Principal:
Service: apprunner.amazonaws.com
Action: sts:AssumeRole
Policies: ```
◦ PolicyName: MyECRPolicy ```
PolicyDocument:
Version: 2012-10-17
Statement: ```
◦ Effect: Allow ```
Action: ```
▪︎ ecr:GetAuthorizationToken
▪︎ ecr:BatchCheckLayerAvailability
▪︎ ecr:GetDownloadUrlForLayer
▪︎ ecr:BatchGetImage
▪︎ ecr:DescribeImages ```
Resource: '*' ``` ``` MySecretsManagerConnection:
Type: AWS::SecretsManager::SecretTargetAttachment
Properties:
SecretId: my-secrets-manager-secret
TargetId: !Ref MyAppRunnerService
TargetType: AWS::AppRunner::Service ```
Grr on my phone, so that didn’t paste well
But to me, that’s a trivial amount of YAML
interesting
2023-04-13
2023-04-17
Compliance has been around for a while, but still I see folks thinking that they can “agreement” away the personal rights of their data subjects. As far as I know regulations are pretty straight forward. You collect Protected Data, you’re subject to the related laws, regardless of whether your product is “beta”. This article from Instabug suggest otherwise with statements like: “the developer disclaims any liability for data loss, damages”, and “Testing is the only purpose behind using the application and the developer disclaims any liability for data loss, damages,…”.
Is there a resource, such as a guide, that someone knows about that specifically kneecaps this line of thought? As far as I’m concerned, this shouldn’t need such elaboration, but people……… Unless I’m 100% wrong and you can easily skirt data protection regulations… But if that’s the case, why is compliance and infosec a billion dollar industry? So I’m pretty sure this isn’t the case.
2023-04-19
@here office hours is starting in 30 minutes! Remember to post your questions here.
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Oliver Schoenborn has joined Public “Office Hours”
Rizky Ramadhan has joined Public “Office Hours”
Nenna Salinas has joined Public “Office Hours”
Robert Jordan has joined Public “Office Hours”
Andy Wortman has joined Public “Office Hours”
Andrew Vitko has joined Public “Office Hours”
Jacob Hudson has joined Public “Office Hours”
Isaac M has joined Public “Office Hours”
Jared Richards has joined Public “Office Hours”
Opeyemi folorunsho has joined Public “Office Hours”
Michael Holt has joined Public “Office Hours”
Abdul Aziz Tetteh has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Isa Aguilar has joined Public “Office Hours”
Michael Pursifull has joined Public “Office Hours”
Diego Maia has joined Public “Office Hours”
Eduardo Wohlers has joined Public “Office Hours”
Chad Thomas has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Jim C has joined Public “Office Hours”
Kris Musard has joined Public “Office Hours”
Abdul Aziz Tetteh has joined Public “Office Hours”
Rizky Ramadhan has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
Joko Sarjono has joined Public “Office Hours”
Jim Park has joined Public “Office Hours”
Paul Marcelin has joined Public “Office Hours”
Adedapo Ajuwon has joined Public “Office Hours”
Austin Blythe has joined Public “Office Hours”
Alex Atkinson has joined Public “Office Hours”
What have you found is THE simplest way to deploy a container for a prototype application? ECS+fargate?
Amaan Khan has joined Public “Office Hours”
Links from today’s office hours:
https://registry.terraform.io/modules/cloudposse/amplify-app/aws/latest https://github.com/kayac/ecspresso https://github.com/cloudposse/github-action-release-label-validator https://github.com/cloudposse/github-action-release-branch-manager https://github.com/cloudposse/github-action-major-release-tagger https://www.chatpdf.com/ https://techcrunch.com/2023/04/19/atlassian-brings-an-ai-assistant-to-jira-and-confluence/amp/? https://betterprogramming.pub/iac-secrets-management-from-github-secrets-to-aws-secrets-manager-63bb4fdd5992 https://github.com/jatalocks/terracove https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-efs-10-gibs-throughput/ https://github.com/dagger/dagger https://sweetops.slack.com/archives/CHDR1EWNA/p1681756391429779 https://github.com/cloudposse/terraform-aws-components
2023-04-26
@here office hours is starting in 30 minutes! Remember to post your questions here.
I don’t really have a question and I’m unable to join because of a conflict, but I’d love to bring some attention to this if people are spending some time doing community stuff. I’ve been working with igor a bit on one of the github actions and I have a PR that fixes a breaking bug I found in the latest release: https://sweetops.slack.com/archives/CQA2BH8AG/p1682523233931669
@Linda Pham (Cloud Posse)
Influx data lunches : https://www.influxdata.com/blog/introducing-influxdb-3-0/
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Vlad Ionescu has joined Public “Office Hours”
Kevin Moore has joined Public “Office Hours”
Nenna Salinas has joined Public “Office Hours”
Alejandro Calbazana has joined Public “Office Hours”
Robert Jordan has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Jonathan Eunice has joined Public “Office Hours”
Isaac M has joined Public “Office Hours”
Alex Atkinson has joined Public “Office Hours”
Diego Maia has joined Public “Office Hours”
Alejandro Calbazana has joined Public “Office Hours”
Michael Pursifull has joined Public “Office Hours”
John Mitchell has joined Public “Office Hours”
Kris Musard has joined Public “Office Hours”
Jared Richards has joined Public “Office Hours”
Jonathan Poczatek has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Max Lobur has joined Public “Office Hours”
emem u has joined Public “Office Hours”
Charles Smith has joined Public “Office Hours”
Adedapo Ajuwon has joined Public “Office Hours”
Evan Pitstick has joined Public “Office Hours”
Charles Smith has joined Public “Office Hours”
Abdul Aziz Tetteh has joined Public “Office Hours”
Alejandro Calbazana has joined Public “Office Hours”
Jose Figueredo has joined Public “Office Hours”
0433 - VPC CNI Network Policy Support 0423 - Cluster Access Management 0403 - IRSAv2 0423 - Cluster updates 0458 - Karpenter Automatic Compute
VPC CNI Network Policy Support, will no longer need Calico
Cluster updates → updated will become a lot easier
Keynote from new GM of EKS project
@Jeremy G (Cloud Posse) @Andriy Knysh (Cloud Posse)
Michael Pursifull has joined Public “Office Hours”
Links from today’s office hours:
https://infracopilot.io/ https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-guardduty-monitors-runtime-activity-containers-eks/ https://github.com/robusta-dev/krr https://github.blog/changelog/2023-04-20-github-actions-create-and-share-your-own-deployment-protection-rules-for-safe-and-controlled-deployments/ https://github.com/tf-libsonnet/core https://www.thoughtworks.com/radar?utm_source=marketo&utm_medium=email&utm_campaign=techradar-vol28_2023-04&mkt_tok=MTk5LVFERS0yOTEAAAGLW_VMeFYc69h2Mrph2Fz21dF73QVy2VL6vkxJHEL95eBNlyOojdNoqXFgWaSuKRXyRNalW7SiQgvgM6JmbR6thye_lD1TYRVUBZAsXHDTDnYGlqc https://www.reddit.com/r/Terraform/comments/12yvukb/mac_literally_just_started_blocking_terraform/ https://github.com/stefanprodan/timoni https://atlasgo.io/blog/2023/04/21/terraform-v050 https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-ec2-ubuntu-pro-subscription-model/ https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-guardduty-aws-lambda/ https://www.youtube.com/watch?v=LGD52z0LxAA https://github.com/klothoplatform/klotho https://ermetic.com/solution/just-in-time/ https://github.com/bmarsh9/gapps https://github.com/strongdm/comply https://github.com/cloudposse/terraform-aws-amplify-app https://www.datacenterdynamics.com/en/news/water-leak-at-paris-global-switch-data-center-causes-fire-leads-to-outages-at-google/#<i class="em em-~~~"https://www.datacenterdynamics.com/en/news/water-leak-at-paris-global-switch-data-center-causes-fire-leads-to-outages-at-google/#:~~~ext=[…]a%20fire