#office-hours (2020-11)

Meeting password: sweetops

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Meeting password: sweetops

2020-11-29

2020-11-28

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hi all - what are some approaches for application config management in kubernetes? A few topics I’m interested in • dynamic configuration (say for example configuration a feature flag in an app) • deploying applications that share configuration (or secrets) Just looking for some projects to look into to get a feel for what people are doing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hello, I’m a bit unsure what channel it would fit to ask this question.

I’ll try and keep this short (new listener to the office-hours, inspired to ask this in Slack from the “finding a mentor” conversation). My day job is with a team focused on AWS, doing traditional operations and some light automation work.

I haven’t been given a “roughly-defined” roadmap on how to succeed in this type of role or career. I am curious on how other people have gotten to where they were in this (or their own) space.

2020-11-27

voidSurfr avatar
voidSurfr

is there a Google calendar I can subscribe to for Wednesday Office Hours by chance?

JJ avatar

Yeah, as @jose.amengual suggested, when you subscribe to the Zoom invitation, it’s got a link to add the events to a few different calendars, including Google’s.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, if you register here: cloudposse.com/office-hours you’ll get a calendar invite

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andy Miguel (Cloud Posse) we should create a public calendar too for people to subscribe to

jose.amengual avatar
jose.amengual

there is something somewhere because I get notifications

voidSurfr avatar
voidSurfr

@jose.amengual excellent, thank you

2020-11-26

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

Ah, missed office hours, sorry My schedule’s super chaotic with re:Invent prep and related things. I hope I’ll make it to the next one, but it’s not looking good

2020-11-25

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:00:26 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

David Napier avatar
David Napier
07:01:17 PM

Using a for_each loop with the terraform-aws-route53-alias module, how would I specify a parent_zone_id from a resource. This (parent_zone_id = aws_route53_zone[each.key].zone_id) returns Invalid reference.

btai avatar

is there a best practice to follow w/ helmfiles in terms of inheritance / keeping things dry?

currently my setup is a remote monochart chart with zero assumptions made (i.e only default labels like app: {{ include "common.name" . }}). However I have a bunch of services that essentially follow the same pattern (all use traefik ingress annotations, redis pod). Using the traefik ingress annotation example, Instead of having all those annotations copied to every service’s helmfile, should I have a remote “parent” helmfile I can reference (is this even a thing?) or should I have a separate monochart with more assumptions (all the default traefik ingress annotations I want like HSTS enabled)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm This looks like helm and not helmfile notation

btai avatar

@Erik Osterman (Cloud Posse) what do you mean by notation?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

{{ include "common.name" . }} looks like what you would write in a helm chart and not in a helmfile

btai avatar

yeah, i was just giving an example of how my monochart helm chart doesn’t make any assumptions currently except things like those very high level labels

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

You can use bases in helmfiles to layer options.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

go here and scroll UP to Advanced Configuration: Layering above  bases in code block

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Cloud Posse

We’re a DevOps accelerator. That means we help companies own their infrastructure in record time by building it with you and then showing you the ropes. If t…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Add Binary Releases · Issue #57 · hashicorp/terraform-config-inspect

what Add semver releases Attach precompiled binaries to the releases why Not everyone has a go setup GitHub actions make it trivial to build and distribute binary releases (happy to provide example)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
New – Attribute-Based Access Control with AWS Single Sign-On | Amazon Web Services attachment image

Starting today, you can pass user attributes in the AWS session when your workforce sign-in into the cloud using AWS Single Sign-On. This gives you the centralized account access management of AWS Single Sign-On and ABAC, with the flexibility to use AWS SSO, Active Directory, or an external identity provider as your identity source. To […]

David Napier avatar
David Napier

No worries! Thanks!

voidSurfr avatar
voidSurfr

I put my question in the zoom chat; is it better here?

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

please post here

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-kubernetes-tfc-cloud-agent

Provision a Terraform Cloud Agent on an existing Kubernetes cluster. - cloudposse/terraform-kubernetes-tfc-cloud-agent

voidSurfr avatar
voidSurfr

I’m just dipping my toe in with the https://github.com/cloudposse/terraform-aws-tfstate-backend - can’t get it to create the bucket?! Is there a repo that has this configured in as a working example?

vicken avatar
vicken

@Matt Gowie What about an aws managed vpn service? https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html

What is AWS Client VPN? - AWS Client VPN

Enable access to your VPC and on-premises network from anywhere, on any device.

Matt Gowie avatar
Matt Gowie

I wanted to avoid a VPN if possible. Plus AWS VPN is terribly expensive from what I remember.

What is AWS Client VPN? - AWS Client VPN

Enable access to your VPC and on-premises network from anywhere, on any device.

1
jose.amengual avatar
jose.amengual

and TF does not support all the confid for aws vpn

Vugar avatar
Vugar

@Matt Gowie I haven’t tried it myself yet, but maybe this one? https://github.com/abutaha/aws-es-proxy/

abutaha/aws-es-proxy

aws-es-proxy is a small web server application sitting between your HTTP client (browser, curl, etc…) and Amazon Elasticsearch service. - abutaha/aws-es-proxy

Vugar avatar
Vugar

I don’t know if this is intended to work with signed requests to Kibana… or ES only

Joe Niland avatar
Joe Niland

Hey @Matt Gowie I just had to solve this for a client. We were using AWS Client VPN but it wasn’t ideal.

The app @ linked to publishes a Docker image so I used it with cloudposse/terraform-aws-ecs-alb-service-task.

Then used the alb (for TLS and easy target registration with ECS) and alb-ingress modules to set up authentication to Google OIDC.

The modules are customised for the client, but let me know if I can send you some snippets.

cloudposse/terraform-aws-ecs-alb-service-task

Terraform module which implements an ECS service which exposes a web service via ALB. - cloudposse/terraform-aws-ecs-alb-service-task

Vugar avatar
Vugar

Thank you @Joe Niland! Sounds great! I So I take it ‘aws-es-proxy’ works both for signing requests to Kibana and ES… , am I correct?

Joe Niland avatar
Joe Niland

Yes it seems to @

1
Matt Gowie avatar
Matt Gowie

Thanks for the suggestion @Joe Niland. I’ve added it to the list.

Matt Gowie avatar
Matt Gowie

I’ve been looking into multiple solutions for this… I think I’m moving towards something like https://tailscale.com/. It’s a SaaS product, but damn is it impressive!

pjaudiomv avatar
pjaudiomv
Duo Network Gateway attachment image

Duo Network Gateway allows users to access your on-premises websites and applications without worrying about managing VPN credentials or installing software on their devices.

Duo Authentication Proxy Reference attachment image

Duo’s trusted access solution enables organizations to secure access to all work applications, for all users, from anywhere, with any device they choose.

1
pjaudiomv avatar
pjaudiomv

I use a reverse proxy fargate cluster with okta in front on lb for aws es

1
jose.amengual avatar
jose.amengual

mmmm I can’t access it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

wierd!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i see what you mean

Zoom avatar
Zoom
12:06:04 AM

New Zoom Recording from our Office Hours session on 2020-11-25 is now available.

Zoom avatar
Zoom
04:50:16 AM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
04:50:46 AM

emem umoh has joined Public “Office Hours”

Zoom avatar
Zoom
04:50:54 AM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
04:50:55 AM

Mikael Fridh has joined Public “Office Hours”

Zoom avatar
Zoom
04:51:12 AM

David Lundgren has joined Public “Office Hours”

Zoom avatar
Zoom
04:51:13 AM

David Napier has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:06 AM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:07 AM

Jeremy Branham has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:08 AM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:09 AM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:09 AM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:09 AM

James Haughey has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:10 AM

matt simonsen has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:10 AM

will Salt has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:11 AM

Patrick Joyce has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:11 AM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:21 AM

Abisoye Olaomi has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:23 AM

Jason Walsh has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:23 AM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:23 AM

Brian Tai has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:24 AM

Todd Thomas has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:24 AM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:24 AM

Mahesh Yadav has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:25 AM

15139103984 has joined Public “Office Hours”

Zoom avatar
Zoom
04:54:30 AM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
04:55:01 AM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
04:55:31 AM

Hilal Jaffan has joined Public “Office Hours”

Zoom avatar
Zoom
04:55:50 AM

vicken has joined Public “Office Hours”

Zoom avatar
Zoom
04:56:21 AM

Matthew Zeemann has joined Public “Office Hours”

Zoom avatar
Zoom
04:57:06 AM

Baba Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
04:57:08 AM

Michael Jenkins has joined Public “Office Hours”

Zoom avatar
Zoom
04:57:08 AM

Dale-Kurt Murray has joined Public “Office Hours”

Zoom avatar
Zoom
04:57:09 AM

Nick James has joined Public “Office Hours”

jose.amengual avatar
jose.amengual

WTH?

jose.amengual avatar
jose.amengual

ohhhhhhhh Amazon is up now LOL

jose.amengual avatar
jose.amengual

no it is not….

Zoom avatar
Zoom
05:05:24 AM

Abisoye Olaomi has joined Public “Office Hours”

Zoom avatar
Zoom
05:05:28 AM

Robert Jackson has joined Public “Office Hours”

Zoom avatar
Zoom
05:05:30 AM

Emile Fugulin has joined Public “Office Hours”

Zoom avatar
Zoom
05:05:31 AM

Neil Gealy has joined Public “Office Hours”

Zoom avatar
Zoom
05:13:00 AM

Ion Munteanu has joined Public “Office Hours”

Zoom avatar
Zoom
05:19:03 AM

Bade has joined Public “Office Hours”

Zoom avatar
Zoom
05:19:04 AM

Abisoye Olaomi has joined Public “Office Hours”

2020-11-21

2020-11-20

2020-11-19

2020-11-18

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:00:40 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Tomek avatar
Tomek

would a question like this be acceptable for office hours? https://sweetops.slack.com/archives/CB6GHNLG0/p1605655766298600

:wave: Is there a way to define the session expiration time for the role an ECS task assumes in terraform? The AWS docs state that the default is 6 hours. max_session_duration for aws_iam_role only sets the allowed max session but it looks like when changing that to 12 hours, the ECS task’s role still uses the default 6 hour session duration

Tomek avatar
Tomek

maybe too specific

Blaise Pabon avatar
Blaise Pabon

I’ll be in today! I’m curious if anyone has an opinionated pattern for running a personal cluster (independent of employer) that is portable enough to generally run on a laptop and occasionally push to a cloud. https://sweetops.slack.com/archives/CHDR1EWNA/p1604519888152500

This could be a question for next week…. I have a desire to have a little dev environment on my laptop… most of the time I am pulling open source projects and self hosting them with some small modifications… Sometimes it’s a monolith project, sometimes a Docker image, sometimes a k8s microservice…. So I thought I would conjure up a local haproxy/dnsmasq and have all my http traffic go through there, where it would get redirected to a local port, or to the ingress of my k3d cluster.

Does anyone already do this? Is there a smarter way?

Blaise Pabon avatar
Blaise Pabon

@, I think the #aws channel is good for the role TTL question. IMHO, office hours is good for questions that may have multiple solutions or that might elicit the wisdom borne from painful experience.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

anyone have recommendations on how to find a good mentor for the whole SRE and devops world ? or maybe even tech in general ?

Chris Picht avatar
Chris Picht

Q: How are people capturing Cognito configuration via code? Anyone using Terraform to accomplish IaC for Cognito?

1
roth.andy avatar
roth.andy

Pseudo-poll: Do you turn on “Dismiss PR approvals when new commits are pushed”? Why/why not?

Zoom avatar
Zoom
07:29:19 PM

Andy Miguel has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:20 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:21 PM

Victor Fondevilla has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:22 PM

Oliver Schoenborn has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:26 PM

Scott Rogers has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:56 PM

Jagan Rajagopal has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:22 PM

Blaise pabon has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:30 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:02 PM

Mikael Fridh has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:10 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:15 PM

Sam C has joined Public “Office Hours”

OliverS avatar
OliverS

I’m trying to create a kubernetes cluster in EKS, this time with the terraform-eks-aws module in terraform registry, and after a while the node group creation fails because

NodeCreationFailure: Instances failed to join the kubernetes cluster

I’m looking into it but was wondering if any had any suggestions, since this is a rather vanilla setup that I’m creating

Zoom avatar
Zoom
07:31:22 PM

Nicolás de la Torre has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:30 PM

majan paul has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:33 PM

Tomek Rabczak has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:37 PM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:46 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:58 PM

Guelor Emanuel has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:03 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:32 PM

Stefan Andonov has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:43 PM

15139103984 has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:26 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:42 PM

Jagan Rajagopal has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:49 PM

Arjun Venkatesh has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:56 PM

Eric Berg has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Zoom avatar
Zoom
07:46:03 PM

Tarlan Isaev has joined Public “Office Hours”

Zoom avatar
Zoom
07:47:02 PM

Juan Soto has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-components

Catalog of reusable Terraform components and blueprints for provisioning reference architectures - cloudposse/terraform-aws-components

Zoom avatar
Zoom
07:51:51 PM

Kevin Chan has joined Public “Office Hours”

Zoom avatar
Zoom
07:56:52 PM

Omer Sen has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
derailed/k9s

Kubernetes CLI To Manage Your Clusters In Style! - derailed/k9s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Kubernetes Horror Stories attachment image

Kubernetes is a complex container management system. So it’s no surprise that it’s often the lead character in application or infrastructure horror stories.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Products attachment image

Cloud Posse Merch

Zoom avatar
Zoom
08:10:21 PM

Abisoye Olaomi has joined Public “Office Hours”

Zoom avatar
Zoom
08:17:57 PM

Derek Davis has joined Public “Office Hours”

organicnz avatar
organicnz

Yo! Could you guys please explain a little bit in details what a main purpose and power of Waypoint? Has anyone had a recent chance to touch its guts? Thanks for office hours

Andy Miguel (Cloud Posse) avatar
Andy Miguel (Cloud Posse)

hi Tarlan! we uploaded this segment a few weeks ago on Waypoint, hope it helps https://www.youtube.com/watch?v=DoE6OteOUSo&feature=youtu.be

1
organicnz avatar
organicnz

Awesome, thanks mate

1
Blaise Pabon avatar
Blaise Pabon

A few items about time: chrony is the modern NTP client: https://chrony.tuxfamily.org/comparison.html Software developers have a lot to learn about time: https://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time If you know what you’re doing with time, you can do some pretty cool stuff, like:

• Generate crypto (https://blog.cloudflare.com/secure-time/ )

• Google spanner database (or, how to have data every where in general and nowhere in particular)

More falsehoods programmers believe about time; "wisdom of the crowd" edition attachment image

A couple of days ago I decided to [write down some of the things I’ve learned about testing][testing_post] over the course of the last [several years.][codeascraft] In the course of enumerating the…

Introducing time.cloudflare.com attachment image

Cloudflare has always been a leader in deploying secure versions of insecure Internet protocols and making them available for free for anyone to use. In 2014, we launched one of the world’s first free, secure HTTPS service (Universal SSL) to go along with our existing free HTTP plan.

1
Zoom avatar
Zoom
09:40:34 PM

New Zoom Recording from our Office Hours session on 2020-11-18 is now available.

2020-11-16

2020-11-15

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

For the people who find out difficult to attend the office hours meetings and too lazy to watch an entire hour of video, it would be nice to have meeting minutes or short summaries to some of the discussion topics.

It would make for good sweet ops blog entries or a newsletter.

2
1
Andy Miguel (Cloud Posse) avatar
Andy Miguel (Cloud Posse)

@RB (Ronak) (Cloud Posse) agreed! we started doing this. you can find notes in the youtube video descriptions with timestamps to jump to those, check out last week’s video for example: https://www.youtube.com/watch?v=XR9pIWAMNlE&list=PLhRztDM6Uvne8MUuwXrv2truMl6gVZ0D8&index=1&t=160s

we sometimes also create short segments for interesting topics you can find in our playlists “Cloud Posse Explains” and “Terraform News”

notes are also uploaded below the videos on our blog: https://cloudposse.com/blog/

2

2020-11-12

2020-11-11

EvanG avatar
EvanG

Hi there, I am working implementing CIS Benchmark requirements in AWS. My goal is to go through the CIS_Amazon_Web_Services_Foundations_Benchmark_v1.3.0.pdf and manually connect the resources. Then I plan to automate the creation of the resources through terraform. I feel like this is a pretty standard thing, but I’m new to cloud and terraform. Has anyone done this before?

antonbabenko avatar
antonbabenko

I did this for the customer, but didn’t find anything complete, so used to follow items one by one from the list and adjust each service. I am interested to know if you find something better (preferably not very $$$).

1
EvanG avatar
EvanG

I plan on making it from scratch. I have to figure it out by next tuesday

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) thank you for the reference. I’m using that doc to guide me through the process. I’m going to add a cloudtrail resource, s3 resources, and KMS key resource to encrypt the logs. Hopefully, things go smoothly. Still learning a lot though.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-security-hub

Terraform module to provision AWS Security Hub. Contribute to cloudposse/terraform-aws-security-hub development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-cloudtrail-cloudwatch-alarms

Terraform module for creating alarms for tracking important changes and occurrences from cloudtrail. - cloudposse/terraform-aws-cloudtrail-cloudwatch-alarms

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) I don’t really understand the point of the security-hub or cloudtrail-cloudwatch-alarms. I think I only need to create a cloudtrail resource, s3 resource, and kms resource. I’m still learning AWS and Terraform. My boss basically handed me an ambiguous project and now I need to figure it out.

EvanG avatar
EvanG

cloudtrail.tf

resource "aws_cloudtrail" "nfcisbenchmark" {
  name                          = "nf-cis-benchmark"
  s3_bucket_name                = aws_s3_bucket.nfcisbenchmark.id
  enable_logging                = var.enable_logging
  enable_log_file_validation    = var.enable_log_file_validation
  is_multi_region_trail         = var.is_multi_region_trail
  include_global_service_events = var.include_global_service_events
  is_organization_trail         = var.is_organization_trail

  # CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
  # for a multi-regions trail, ensuring that management events configured for all type of
  # Read/Writes ensures recording of management operations that are performed on
  # all resources in an AWS account
  event_selector {
    # Specify if you want your event selector to include management events for your trail.
    include_management_events = true
    # Specify if you want your trail to log read-only events, write-only events, or all. By default, 
    # the value is All. Needed for logging management events.
  }
}

resource aws_s3_bucket "nfcisbenchmark" {
  bucket = "nf-cis-benchmark"
  acl  = "private"
}

variables.tf

# CIS Benchmark 3.2 Ensure CloudTrail log file validation is enabled (Automated)
variable "enable_log_file_validation" {
  type        = bool
  default     = true
  description = "Specifies whether log file integrity validation is enabled. Creates signed digest for validated contents of logs"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# ensuring that a multi-regions trail exists will ensure that unexpected activity
# occurring in otherwise unused regions is detected
variable "is_multi_region_trail" {
  type        = bool
  default     = true
  description = "Specifies whether the trail is created in the current region or in all regions"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# ensuring that a multi-regions trail exists will ensure that Global Service Logging
# is enabled for a trail by default to capture recording of events generated on AWS
# global services
variable "include_global_service_events" {
  type        = bool
  default     = true
  description = "Specifies whether the trail is publishing events from global services such as IAM to the log files"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# Ensure Logging is set to ON
variable "enable_logging" {
  type        = bool
  default     = true
  description = "Enable logging for the trail"
}

variable "is_organization_trail" {
  type        = bool
  default     = false
  description = "The trail is an AWS Organizations trail"
}
EvanG avatar
EvanG

Actually here is everything for CIS Benchmark 3 Logging

resource "aws_cloudtrail" "nfcisbenchmark" {
  name                          = "nf-cis-benchmark"
  s3_bucket_name                = aws_s3_bucket.nfcisbenchmark.id
  enable_logging                = var.enable_logging
  enable_log_file_validation    = var.enable_log_file_validation
  is_multi_region_trail         = var.is_multi_region_trail
  include_global_service_events = var.include_global_service_events
  is_organization_trail         = var.is_organization_trail
  kms_key_id                    = aws_kms_key.nfcisbenchmark.arn

  # CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
  # for a multi-regions trail, ensuring that management events configured for all type of
  # Read/Writes ensures recording of management operations that are performed on
  # all resources in an AWS account
  event_selector {
    # Specify if you want your event selector to include management events for your trail.
    include_management_events = true
    # Specify if you want your trail to log read-only events, write-only events, or all. By default, 
    # the value is All. Needed for logging management events.
  }
}

resource aws_s3_bucket "nfcisbenchmark" {
  bucket = "nf-cis-benchmark"
  acl  = "private"
  #  ensure the CloudTrail S3 bucket has access logging is enabled.
  logging {
    target_bucket = aws_s3_bucket.log_bucket.id
    target_prefix = "log/"
  } 
}

resource aws_s3_bucket "log_bucket" {
  bucket = "nfcisbenchmark-log-bucket"
  acl    = "log-delivery-write"
}

# 3.7 Ensure CloudTrail logs are encrypted at rest using KMS CMKs (Automated)
# Defaults to ENCRYPT_DECRYPT
resource aws_kms_key "nfcisbenchmark" {
  description             = "nf-cis-benchmark"
}

and here are the vars

variable workspace_iam_roles {
  type = map
  default = {
    nf-sandbox         = "arn:aws:iam::721086286010:role/ops-mgmt-admin"
    nf-integration-int = "arn:aws:iam::472879144981:role/ops-mgmt-admin"
    nf-staging-int     = "arn:aws:iam::560269805515:role/ops-mgmt-admin"
    nf-staging-cust    = "arn:aws:iam::337684097865:role/ops-mgmt-admin"
    nf-devops-tools    = "arn:aws:iam::447513199460:role/ops-mgmt-admin"
    nf-prod-int        = "arn:aws:iam::745435643501:role/ops-mgmt-admin"
  }
}

# CIS Benchmark 3.2 Ensure CloudTrail log file validation is enabled (Automated)
variable "enable_log_file_validation" {
  type        = bool
  default     = true
  description = "Specifies whether log file integrity validation is enabled. Creates signed digest for validated contents of logs"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# ensuring that a multi-regions trail exists will ensure that unexpected activity
# occurring in otherwise unused regions is detected
variable "is_multi_region_trail" {
  type        = bool
  default     = true
  description = "Specifies whether the trail is created in the current region or in all regions"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# ensuring that a multi-regions trail exists will ensure that Global Service Logging
# is enabled for a trail by default to capture recording of events generated on AWS
# global services
variable "include_global_service_events" {
  type        = bool
  default     = true
  description = "Specifies whether the trail is publishing events from global services such as IAM to the log files"
}

# CIS Benchmark 3.1 Ensure CloudTrail is enabled in all regions
# Ensure Logging is set to ON
variable "enable_logging" {
  type        = bool
  default     = true
  description = "Enable logging for the trail"
}

variable "is_organization_trail" {
  type        = bool
  default     = false
  description = "The trail is an AWS Organizations trail"
}

I stole some of this from your repos

EvanG avatar
EvanG

I don’t really understand a lot of the dynamic blocks and what not. I’m just getting started in terraform and I would really like some help with this project if anyone has time.

EvanG avatar
EvanG

I figured it all out btw.

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) wow actually I just looked at SecurityHub and I can see that it’s an auditing tool. I made the terraform for cloudtrail, kms, and s3, for CIS. Would you recommend using Security Hub to audit the CIS benchmark foundations or create custom config rules in AWS config?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@matt has maybe more insight on this

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) @matt I’d love to get some more insight on this. I’ve been having issues with the aws_config_delivery_channel most of the code out there is quite old. One repo was using format for parsing an arn. I think I’m going to need to make it all from scratch. My plan is to set up an AWS Config resource with a delivery channel, s3 bucket, and sns messaging queue. I’ll use the same s3 bucket that I’m using for cloudtrail, but use two different prefixes i.e. cloudtrail and config. Then I will add an aws_securityhub_standards_subscription. I’m not sure how cost effective this solution is though. Also, I’m learning terraform and AWS at the same time. Probably going to need to make it all manually and then figure out the terraform. Is this cost effective? I’m trying to stay lean, but I gotta meet CIS Foundations.

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) still running this down. I’ve knocked out 1, 2, and 3. I am running into a bump on 4.1 Ensure no security groups allow ingress from 0.0.0.0/0 to port 22. Did you guys find a way to solve this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-config

This module configures AWS Config, a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. - cloudposse/terraform-aws-config

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Which leverages the full catalog of prepared config rules by AWS: https://github.com/cloudposse/terraform-aws-config/tree/master/catalog

cloudposse/terraform-aws-config

This module configures AWS Config, a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. - cloudposse/terraform-aws-config

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@matt can maybe talk to the specific question you have though

matt avatar

Yes, the AWS Config rule resstricted-ssh can detect that this is compliant and let you know of any SGs that are violating it…

matt avatar
restricted-ssh - AWS Config

Use the restricted-ssh AWS Config managed rule to check whether security groups that are in use disallow unrestricted incoming SSH traffic.

matt avatar

We are in the process of putting together a series of modules and configurations that allow our customers to achieve CIS compliance. So hopefully we’ll have a good “end-to-end” story on this to share shortly.

EvanG avatar
EvanG
04:18:33 PM

@Erik Osterman (Cloud Posse) does it make sense to use the newer version of CIS?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Not really, because SecurityHub doesn’t yet support it.

EvanG avatar
EvanG

Okay, thank you. I am struggling to find an optimal way to knock out the security groups no ingress 0.0.0.0/0 to port 22. Seems like the solution is to go through and individually remove the non-compliant sgs, add a cloudwatch alert, trigger an alert when an sg with port 22 open is created, then use auto-remediation to delete it. I feel like this is going to be painful.

EvanG avatar
EvanG

Actually better idea. Get a list of running instances, run a port scan for each ip, if 22 is open and not a bastion then remove the ingress rule

EvanG avatar
EvanG

Also, has anyone figured out why vpc_flow_log resources always need to be recreated? It looks like the AWS iam arn isn’t getting attached.

EvanG avatar
EvanG

@Erik Osterman (Cloud Posse) are you enabling AWS accessanalyzer for via

aws_accessanalyzer_analyzer 

resource? https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/accessanalyzer_analyzer I’m very close to being done with the entire automation project for CIS foundations.

EvanG avatar
EvanG

Oh also I’m handing the no ingress on port 22 or port 3389 by creating config rules that check each condition. Then if the compliance status on either of those rules change I’ve set up a cloudwatch rule that will send an alert to a central email account. It’s better than the nmap script that I was previously using.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Haven’t looked into that one yet. Looks nice - I think we should build support for that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:00:20 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

aravind.kandalam498 avatar
aravind.kandalam498

Using the https://github.com/cloudposse/terraform-aws-ec2-autoscale-group. I am following the example. I am able to plan it but getting the following error.

module.autoscale_group.aws_autoscaling_group.default[0]: Creating... Error: One of `id` or `name` must be set for `launch_template

`

cloudposse/terraform-aws-ec2-autoscale-group

Terraform module to provision Auto Scaling Group and Launch Template on AWS - cloudposse/terraform-aws-ec2-autoscale-group

aravind.kandalam498 avatar
aravind.kandalam498

See you in 30 mins

Ikana avatar
Ikana

What would be the best way to modify the terraform on this package: https://github.com/terraform-aws-modules/terraform-aws-ec2-instance so that we could optionally have it create a ENI and bind it, as an alternative to just providing the ENI

terraform-aws-modules/terraform-aws-ec2-instance

Terraform module which creates EC2 instance(s) on AWS - terraform-aws-modules/terraform-aws-ec2-instance

chris avatar
chris

From my post in #general: I have searched through the archives and google is not helping either… so hopefully this isn’t described somewhere I have just missed. I am working on setting up a new infra utilizing cloudposse OSS but have bumped into a few issued. I got all the accounts setup and that is working great, also got EKS and other application backing services up (eg RDS) all using terraform… but I am having troubles finding out how to make the next steps through IaC.

In particular I am trying to get Codefresh setup for GitOps but it is not clear how to do this as IaC. I got my Codefresh account setup (using Github to auth) but to add the K8s EKS integration for example I used the helm chart (cloudposse-incubator/codefresh-service-account ). But I did this by hand which seem wrong. Then to setup the project and pipelines I can only find how to do this by hand.

I am working through the cloudposse/codefresh repo but this seems to be make based, not terraform. So it is not clear how this would be applied using gitops. So I come into the same problem as with the helm chart above.

Does anyone have information about how to do these programmatically? Is just doesn’t seem correct to have these be done by hand.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-eks-workers

Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers - cloudposse/terraform-aws-eks-workers

vicken avatar
vicken

Hi, have folks ever tried hardening the EKS worker node images and what would be the best way to go about it? Using one of the CIS hardened AMI images as a base then adding and configuring kube components to it initially comes to mind (unless one already exists?).

Steve Neuschotz avatar
Steve Neuschotz

I have just gone through this for PCI compliance. You do not want to use the CIS images. Although they are hardened they are not complete with the docker and Kubernetes components. Installing those is not trivial and will not result in a process you can easily repeat. Instead use the recommended Amazon Linux 2 Optimized AMi ( ami-0c62450bce8f4f57f). It works perfectly with EKS. If you need to show hardening use kube-bench tool found here (https://aws.amazon.com/blogs/containers/introducing-cis-amazon-eks-benchmark and https://www.eksworkshop.com/intermediate/300_cis_eks_benchmark/).

Introducing The CIS Amazon EKS Benchmark | Amazon Web Services attachment image

Today, we’re announcing a new Center for Internet Security (CIS) benchmark for Amazon Elastic Kubernetes Service (EKS). This new benchmark is optimized to help you accurately assess the security configuration of Amazon EKS clusters, including security assessments for nodes to help meet security and compliance requirements. Security is a critical consideration when configuring and maintaining […]

EKSworkshop.com attachment image

Amazon EKS Workshop

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Gravity - Run Applications Anywhere attachment image

Gravity is an application delivery system allowing engineers to deliver & run cloud-native applications in regulated, restricted, remote environments

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Hardened Images for Amazon EKS

Ramblings about Cloud, Containers, and Other Stuff

Zoom avatar
Zoom
07:28:55 PM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:04 PM

charles pogi has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:09 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:10 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:14 PM

Alexis Concepcion has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:27 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:32 PM

vicken has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:34 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:48 PM

sivo has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:56 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:13 PM

Andy Miguel has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:18 PM

Dale-Kurt Murray has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:18 PM

15139103984 has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:31 PM

Adam Kaplun has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:59 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:07 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:19 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:32 PM

Aravind k has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:34 PM

emem umoh has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:34 PM

Sam C has joined Public “Office Hours”

Zoom avatar
Zoom
07:32:13 PM

Chris Dutton has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:34 PM

David Lozano has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:46 PM

Marcos Soutullo has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-mq-broker

Terraform module for provisioning an AmazonMQ broker - cloudposse/terraform-aws-mq-broker

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS Announces Amazon MQ Will Support RabbitMQ attachment image

AWS announced Amazon MQ will now support RabbitMQ, a popular open-source message broker. With the support for RabbitMQ, customers can migrate their existing RabbitMQ message brokers to AWS without rewriting code.

1
Zoom avatar
Zoom
07:34:05 PM

David Lundgren has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:18 PM

Rainer Schuth has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:28 PM

Rodrigo Quezada has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
S3 Intelligent-Tiering Adds Archive Access Tiers | Amazon Web Services attachment image

We launched S3 Intelligent-Tiering two years ago, which added the capability to take advantage of S3 without needing to have a deep understanding of your data access patterns. Today we are launching two new optimizations for S3 Intelligent-Tiering that will automatically archive objects that are rarely accessed. These new optimizations will reduce the amount of […]

Zoom avatar
Zoom
07:36:09 PM

Guelor Emanuel has joined Public “Office Hours”

Zoom avatar
Zoom
07:38:04 PM

Babajide Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
07:38:11 PM

Aarat has joined Public “Office Hours”

Zoom avatar
Zoom
07:45:09 PM

Marc Tamsky has joined Public “Office Hours”

Zoom avatar
Zoom
07:47:03 PM

Mikael Fridh has joined Public “Office Hours”

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

Shoutout to Bottlerocket: https://aws.amazon.com/bottlerocket/

Zoom avatar
Zoom
07:56:49 PM

mallory mabe has joined Public “Office Hours”

matt avatar
Can we implement EKS-AMI hardening? · Issue #245 · awslabs/amazon-eks-ami

As per Our Infosec team, Every server should be using Hardened AMI according to there policies. While we do the same for EKS AMI worker nodes are terminated before starting. Any suggestions?

matt avatar
Consider applying kube-bench/CIS k8s benchmark to node · Issue #99 · awslabs/amazon-eks-ami

I've attached a run of kube-bench that applies the k8s CIS benchmark against the nodes. The remediations are included in the output of the tool. Should this be the defaults for the created AMI …

Steve Neuschotz avatar
Steve Neuschotz

having just done this - kube-bench and the Optimized Amazon Linux 2 AMI are the best way to go.

Steve Neuschotz avatar
Steve Neuschotz

Kube-bench has a –benchmark parameter which can be set to EKS-1.0. This enables a paired down but accurate view of the Nodes.

Steve Neuschotz avatar
Steve Neuschotz

Kube-bench also does not care if the nodes are EKS control plane managed or self-managed.

Zoom avatar
Zoom
08:01:42 PM

emem has joined Public “Office Hours”

roth.andy avatar
roth.andy
Why We Care About FIPS (And You Should Too) UPDATE! | D2iQ attachment image

UpdateWe are very pleased to announce that we have received Federal Information Processing Standards (FIPS) Validation (CMVP Cert #3702) for the…

Zoom avatar
Zoom
08:06:08 PM

emem umoh has joined Public “Office Hours”

Zoom avatar
Zoom
08:06:52 PM

Isa Aguilar has joined Public “Office Hours”

Jim Park avatar
Jim Park

@Erik Osterman (Cloud Posse), thoughts on comparisons between Terraform SaaS services?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’re working to have all the SaaS services on an office hours showdown

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@marcinw @Sebastian Stadil @Jake Lundberg (HashiCorp) so the vendors can showcase their differentiators.

2
David Lozano avatar
David Lozano

@Erik Osterman (Cloud Posse) you mentioned before bridgecrew for enforcing sec policies in terraform. What advantage you get with this tool over TFC Sentinel sec policies?

Bridgecrew - Secure public cloud infrastructure attachment image

Built to be simple- Protecting infrastructure in the public cloud is a software engineering challenge. We solve it like one.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Multiple ways to skin the cat, but one thing bridgecrew does is automatically open PRs to Fix problems (optional)

Bridgecrew - Secure public cloud infrastructure attachment image

Built to be simple- Protecting infrastructure in the public cloud is a software engineering challenge. We solve it like one.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@

barak avatar
barak

@Erik Osterman (Cloud Posse) thanks for the tagging! @David Lozano sorry for the late reply. bridgecrew is the company behind checkov, and it is part of the platform. Some features that are only in the platform:

  1. Remediation - the platform can open “fix” pull requests where possible to a terraform/cloudformation misconfig
  2. Compliance reporting
  3. Runtime analysis of misconfig of you Cloud (AWS, GCP, AZURE) and K8 projects
  4. AWS IAM rightsizing - analysing AWS IAM policies and recommending editions by the principle of least privilege
  5. Dependency graph analysis ( if a resource, depends on a variable in another file - the platform will know that)
  6. Organization dashbaord
  7. Notifications (jira, slack, splunk etc..) and many more to come

I’m here if you want to drill into any of those in DM

1
Zoom avatar
Zoom
08:28:09 PM

Abisoye Olaomi has joined Public “Office Hours”

Jake Lundberg (HashiCorp) avatar
Jake Lundberg (HashiCorp)
09:11:44 PM

@Jake Lundberg (HashiCorp) has joined the channel

Zoom avatar
Zoom
09:43:25 PM

New Zoom Recording from our Office Hours session on 2020-11-11 is now available.

2020-11-10

2020-11-09

Tim Gourley avatar
Tim Gourley

Question – how to deal with resources/processes that change resource state intentionally outside of terraform?

Hi All, This is my first post/question. I’m just getting to know cloudposse/slack, and finding the few office-hours sessions I’ve been able to attend to be very insightful. Thanks to all that are contributing!

We are working with terraform modules, and terragrunt live repo currently deploying to AWS accounts.

My question is about how resources that change (outside of terraform) are managed – For example - we have a mongo cluster setup consisting of 9 ec2 instances in 3x3x3 configuration. (the usual mongos, config, and data nodes) Creating this cluster in terraform is straight forward. But as part of our security processes we have a lambda which listens to AMI update events and updates the cluster with new nodes in a very complicated dance of configuration updates. In the end all the nodes (EC2 instances) are replaced and the cluster has had no downtime (perfect!) but, now the terraform state is completely out of sync with what is deployed. How is this problem generally handled?

Another use-case could be similar AMI updates for an ECS cluster. In that case EC2 instances are not tracked via terraform but the autoscaling group/ launch configurations are.

Thanks in advance. Best Regards, Tim

tim.j.birkett avatar
tim.j.birkett

Hi Tim, Tim here… You could use ignore_changes: https://www.terraform.io/docs/configuration/resources.html#ignore_changes

To ignore changes in AMI IDs but that might lead to another change causing old AMIs to be deployed if you use a static AMI ID in your Terraform code…

You could retrieve the AMI ID using the aws_ami data source: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami based on a name regex or tags.

Maybe combining the 2 ignore_changes and aws_ami would be a reasonable experience.

You could also have the lambda that updates everything store the AMI IDs in something like parameter store and look up the IDs in Terraform… that might add a bit of complexity when you can probably get by with the above option ( combining the 2 ignore_changes and aws_ami ).

Resources - Configuration Language - Terraform by HashiCorp

Resources are the most important element in a Terraform configuration. Each resource corresponds to an infrastructure object, such as a virtual network or compute instance.

1

2020-11-06

Eric Berg avatar
Eric Berg

Another topic for next time: SLA monitoring for upstream dependencies. If we need to make a claim against a SaaS provider, what kind of monitoring do you use to generate evidence?

2

2020-11-04

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:00:43 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Zoom avatar
Zoom
07:28:38 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:28:47 PM

Tarlan Isaev has joined Public “Office Hours”

Zoom avatar
Zoom
07:28:50 PM

Andrew Elkins has joined Public “Office Hours”

Zoom avatar
Zoom
07:28:53 PM

Haribabu Balagani has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:07 PM

usha dav has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:32 PM

Chris Dutton has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:39 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
07:29:50 PM

Nick James has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:00 PM

David Lozano has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:00 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:14 PM

Andy Miguel has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:30 PM

Patrick Joyce has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:56 PM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
07:30:58 PM

Sheldon Hull has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:31 PM

Michael Londeen has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:40 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
07:31:46 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
07:32:05 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
07:32:07 PM

Mikael Fridh has joined Public “Office Hours”

Zoom avatar
Zoom
07:32:24 PM

15139103984 has joined Public “Office Hours”

Zoom avatar
Zoom
07:32:29 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
07:33:47 PM

Michael Martin has joined Public “Office Hours”

Jim Park avatar
Jim Park
cloudposse/terraform-tfe-cloud-infrastructure-automation

Terraform Enterprise/Cloud Infrastructure Automation - cloudposse/terraform-tfe-cloud-infrastructure-automation

Zoom avatar
Zoom
07:34:07 PM

Alexis Concepcion has joined Public “Office Hours”

Zoom avatar
Zoom
07:34:50 PM

Alexis Concepcion has joined Public “Office Hours”

Zoom avatar
Zoom
07:35:17 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
07:35:42 PM

venkatamutyala has joined Public “Office Hours”

Zoom avatar
Zoom
07:39:31 PM

pepe amengual has joined Public “Office Hours”

Zoom avatar
Zoom
07:40:14 PM

Marcos Soutullo has joined Public “Office Hours”

Zoom avatar
Zoom
07:41:03 PM

Sebastian Borrajo has joined Public “Office Hours”

Zoom avatar
Zoom
07:41:29 PM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:42:20 PM

Wesley Chiang has joined Public “Office Hours”

Zoom avatar
Zoom
07:44:55 PM

Fernando Castillo has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS to create its own public container registry in response to Docker pull rate limit attachment image

Plus: Faster on-demand supercomputing. P4d VMs with Nvidia A100 and GPUDirect RDMA

Zoom avatar
Zoom
07:45:30 PM

Derek Davis has joined Public “Office Hours”

Jim Park avatar
Jim Park
Advice for customers dealing with Docker Hub rate limits, and a Coming Soon announcement | Amazon Web Services attachment image

Many container customers building applications use common software packages (e.g. operating systems, databases, and application components) that are publicly distributed as container images on Docker Hub. Docker, Inc. has announced that the Hub service will begin limiting the rate at which images are pulled under their anonymous and free plans. These limits will progressively take […]

Zoom avatar
Zoom
07:46:13 PM

vicken has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS hearts multi-cloud? It's gonna happen. attachment image

And if you’re confused about why, you’re still thinking of multi-cloud the way vendors tried to sell it to you in 2016.

roth.andy avatar
roth.andy
Multi-Cloud is the Worst Practice - Last Week in AWS attachment image

Multi-cloud (that is, running the same workload across multiple cloud providers in a completely agnostic way) is absolutely something you need to be focusing on—at least, according to two constituencies: Declining vendors that realize that if you don’t go multi-cloud, they’ll have nothing left to sell you. AWS isn’t going to build a multi-cloud dashboard, […]

Zoom avatar
Zoom
07:48:32 PM

Isa Aguilar has joined Public “Office Hours”

Jim Park avatar
Jim Park

You could hijack DNS…

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Using Admission Controllers

This page provides an overview of Admission Controllers. What are they? An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. The controllers consist of the list below, are compiled into the kube-apiserver binary, and may only be configured by the cluster administrator. In that list, there are two special controllers: MutatingAdmissionWebhook and ValidatingAdmissionWebhook.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
New – Application Load Balancer Support for End-to-End HTTP/2 and gRPC | Amazon Web Services attachment image

Thanks to its efficiency and support for numerous programming languages, gRPC is a popular choice for microservice integrations and client-server communications. gRPC is a high performance remote procedure call (RPC) framework using HTTP/2 for transport and Protocol Buffers to describe the interface. To make it easier to use gRPC with your applications, Application Load Balancer (ALB) […]

Zoom avatar
Zoom
07:57:02 PM

Matthew Zeemann has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Better Managing Cost in AWS with Budgets Actions attachment image

Recently, AWS announced budgets actions allowing customers to define actions to take when a budget exceeds its threshold (actual or forecasted amounts). With budget actions, customers will have more control over their AWS Budgets in order to reduce unintentional overspending in their AWS accounts.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This could be a question for next week…. I have a desire to have a little dev environment on my laptop… most of the time I am pulling open source projects and self hosting them with some small modifications… Sometimes it’s a monolith project, sometimes a Docker image, sometimes a k8s microservice…. So I thought I would conjure up a local haproxy/dnsmasq and have all my http traffic go through there, where it would get redirected to a local port, or to the ingress of my k3d cluster.

Does anyone already do this? Is there a smarter way?

Zoom avatar
Zoom
07:58:17 PM

Matt Gowie has joined Public “Office Hours”

matt avatar

Speaking of costs, I saw that over the course of the 66-hour Prime Day, Amazon made 16.4 trillion calls to the DynamoDB API, peaking at 80.1 million requests per second. I wonder what that would have cost a non-amazon customer?

matt avatar
Amazon Prime Day 2020 – Powered by AWS | Amazon Web Services attachment image

Tipped off by a colleague in Denmark, I bought the LEGO Star Wars Stormtrooper Helmet, which turned out to be a Prime Day best-seller! As I like to do every year, I would like to share a few of the many ways that AWS helped to make Prime Day a reality for our customers. Back […]

Zoom avatar
Zoom
07:59:44 PM

Marc T has joined Public “Office Hours”

roth.andy avatar
roth.andy
inlets/inlets-operator

Add public LoadBalancers to your local Kubernetes clusters. - inlets/inlets-operator

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

• Start by checking out this long blog post which is the best introduction, summary, and reference cheat sheet on Kubernetes we have yet to find. Reviewing this will shave weeks off the learning curve.

emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html

kubernetesbyexample.com/

learnk8s.io/blog/kubectl-productivity

www.katacoda.com/courses/kubernetes

matt avatar

I can add that I didn’t grok a lot of key k8s concepts until I went through https://github.com/kelseyhightower/kubernetes-the-hard-way

kelseyhightower/kubernetes-the-hard-way

Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts. - kelseyhightower/kubernetes-the-hard-way

1
kareem.shahin avatar
kareem.shahin

Super k8s noob here. Going thru this book and its been great so far: https://www.manning.com/books/kubernetes-in-action

Jim Park avatar
Jim Park

I feel like Rich Hickey is a relevant tangent to this conversation: https://www.youtube.com/watch?v=kGlVcSMgtV4

tamsky avatar
tamsky

I feel like the framework here is also a good read for managing k8s “at-scale”

tamsky avatar
tamsky
Managing Thousands of Edge Kubernetes Clusters with GitOps | Volterra

At Volterra, the SRE team’s job is to operate a global SaaS-based edge platform. We have to solve various challenges in managing a large number of application clusters in various states (i.e. online, offline, admin-down, etc.) and we do this by leveraging the Kubernetes ecosystem and tooling with a declarative pull-based model using GitOps.

Jim Park avatar
Jim Park
Team Topologies

Organizing business and technology teams for fast flow: book + training + consulting from Matthew Skelton and Manuel Pais

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If we have time at the end….

I’m trying to implement a better backup solution for SQL Server in AWS EC2 instances. EFS has been mentioned . I haven’t done this before and wondering if anyone has had luck with using EFS for backups from an ec2 instance as a “network storage” solution backed by AWS.

Most other solutions I’ve seen use EBS/AMI snapshots which i doubt will give me RPO of 15

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-efs-backup

Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline - cloudposse/terraform-aws-efs-backup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-backup

Terraform module to provision AWS Backup, a fully managed backup service that makes it easy to centralize and automate the back up of data across AWS services such as EBS volumes, RDS databases, Dy…

sheldonh avatar
sheldonh

I want to use AWS backup, but it does EBS snapshots only, not native backups.

If I have a 2TB drive with databases and assuming i enable VSS aware mode… I’m pretty sure I won’t get 15 min RPO from EBS snapshots with AWS Backup.

If I’m wrong and they are fast then let me know! Last time I checked it took a while and also freezes sets of db’s during the snapshots

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)



I want to use AWS backup, but it does EBS snapshots only, not native backups.
Our AWS Backup module also does EFS.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what do you mean by “Native backups”? … especially when a database is involved.

sheldonh avatar
sheldonh

SQL Server (MSSQL) has it’s own backup mechanism. They are compressed and per individual database. If I use EBS i need it to be where i can maintain 15 min max data loss which I doubt EBS snapshots will give me.

EFS is promising for where to push my backups but might seems simplier to just backup and sync to s3

tamsky avatar
tamsky

Thanks Erik for hosting another great session.

2
1
Zoom avatar
Zoom
09:44:48 PM

New Zoom Recording from our Office Hours session on 2020-11-04 is now available.

    keyboard_arrow_up