#office-hours (2020-12)
“Office Hours” are every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cloudposse.com/office-hours
Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers!
https://cpco.io/slack-office-hours
Meeting password: sweetops
2020-12-02
Oh bummer. I couldn’t make for previous office hours and today when I can it’s not on xD Hi anyways
hey Marcin! it will be on today at 11:30 AM PT; https://cloudposse.com/office-hours/ to register
Oh man double bummer. It was so quite here plus on this website yesterday 2.12.2020 was missing, maybe its a bug. To add it to my google calendar zoom ask for full RW permissions, so no can do. Better luck next time
@here office hours is starting in 30 minutes! Remember to post your questions here.
recently I needed a module that creates RDS instances to be executed only after RDS parameter groups were created. So I used depends_on, but honestly I’m surprised this works:
resource aws_db_parameter_group "rds_pg" {
for_each = local.rds_param_groups
...
}
module rds_instance {
for_each = local.rds_instances
source = "./rds-instance-module"
...
param_group_name = aws_db_parameter_group.rds_pg[each.value.parameterGroupRef].name
depends_on = [aws_db_parameter_group.rds_pg]
}
Is there a chance that I’m just in the “undefined behavior” zone, is there anything in the docs that says this should work, I would have thought that the depends would need specific resources (ie rds_pg[something]
) not a “map” of resources. If the above is correct, it’s awesome!
That is exactly what depends_on
is for
yeah woohoo!
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Vlad Ionescu has joined Public “Office Hours”
Andy Roth has joined Public “Office Hours”
faithful anere has joined Public “Office Hours”
Jay Zalowitz has joined Public “Office Hours”
Michael Martin has joined Public “Office Hours”
Evan Gertis has joined Public “Office Hours”
Andy Miguel has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Colton Wrisner has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Sam C has joined Public “Office Hours”
Jeremy (Cloud Posse) has joined Public “Office Hours”
Tim Gourley has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
Natalia Woroniec has joined Public “Office Hours”
Hey what’s the password again I cannot find it in the invite
sweetops
just saw it in the announcements (honestly I don’t remember needing one previous weeks but I’ve missed a few)
15139103984 has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
Oliver Schoenborn has joined Public “Office Hours”
Ramesh Yelishala has joined Public “Office Hours”
majan paul has joined Public “Office Hours”
Arca Admin has joined Public “Office Hours”
Andrey Nazarov has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
vicken has joined Public “Office Hours”
Todd Thomas has joined Public “Office Hours”
venkatamutyala has joined Public “Office Hours”
Brandon vh has joined Public “Office Hours”
David Caccavella has joined Public “Office Hours”
JJ Ferman has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
- AWS Lambda Now Billed per Millisecond (ms)
- AWS Lambda Now Supports Docker
- AWS Supports EC2 with MacOS (Mac Minis with 32GB ram)
- AWS EKS Distro - a hardened OS for kubernetes on Amazon and On-prem
- AWS EKS Managed Node Groups Support Spot Instances (via @Vlad Lonescu) AWS Proton – Automated Management for Container and Serverless Deployment
Derek Davis has joined Public “Office Hours”
Omer Sen has joined Public “Office Hours”
Jeremy Branham has joined Public “Office Hours”
Chris Dutton has joined Public “Office Hours”
Amazon Web Services has 264 repositories available. Follow their code on GitHub.
This is the public roadmap for AWS Proton, available in preview - aws/aws-proton-public-roadmap
Adeoye Remi has joined Public “Office Hours”
tamsky has joined Public “Office Hours”
Extend support on AWS Proton to enable defining and provisioning infrastructure using Hashicorp Terraform
Jim Park has joined Public “Office Hours”
The CloudFormation Resource Provider Package For AWS CloudFormation - aws-cloudformation/aws-cloudformation-resource-providers-cloudformation
Terraform Pull Request Automation. Contribute to runatlantis/atlantis development by creating an account on GitHub.
So it seems we should give Atlantis another try then)
Matthew Zeemann has joined Public “Office Hours”
Derek Davis has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Stakater/reloader or pusher/wave or weaveworks/watch
Oh thanks for sharing! wasn’t aware of the other 2
First two is cluster level and last one is sidecar (application level)
So secret and configmap changes deployment restarts
Unleash is the open source feature toggle service. - Unleash/unleash
Or use terraform for so it can detect changes and restarts deployments dependant on secret/configmap whose entries change
Kubernetes terraform provider
Deji has joined Public “Office Hours”
New Zoom Recording from our Office Hours session on 2020-12-02 is now available.
@Erik Osterman (Cloud Posse) during a discussion about DevOps mentoring and stuff you articulated one thing about if you think that things are done in a wrong way and you see that it must be much better approaches here and there but nobody cares in the organisation you should think about switching a job (I hope I remembered it correctly). But what if this happens in a relationship with a client. Say, you provide solutions based on requirements and agreements and everything works smoothly until the client gets the ownership and starts doing things by himself - your solutions always get removed, substituted with crazy hacks, the least optimal, complicated monsters, etc. They might still pay for the support and stuff, but you feel that something goes completely wrong, but there is no feedback why it’s done in this way, no explanation at all. It seems they are just used to leveraging hacks and taking obsolete approaches. I’ve heard these kind of stories quite some time, not always for sure, but it happens. Have you ever faced something like this? Any tips?
By the way, this might be a good start for sharing some tips and tricks about how to deal with clients/customers if you are willing to share of course:) This seems to be an interesting topic and not that trivial, requiring skills (soft skills which are sometimes overlooked) and experience.
Following this conversation !
Say, you provide solutions based on requirements and agreements and everything works smoothly until the client gets the ownership and starts doing things by himself - your solutions always get removed, substituted with crazy hacks, the least optimal, complicated monsters, etc.
The gist of this is a failure on both sides. As a “DevOps Accelerator” our mission is to help our customers own their infrastructure. Ownership is more than paying the bill. It’s taking mental ownership of what’s delivered. For that to happen, there’s got to be a path to ownership that includes incremental and continuous handoff. Waiting until the end is too late. It needs to start almost from the beginning.
I’ll talk about this more next week on #office-hours.
I see you point and agree. Thank you. Looking forward to hearing more on this.
Will you be on the call today by any chance?
I’m sorry, can’t participate today:(
No problem! I’m going to go deep on this in an upcoming office hours. Probably first week or January or so.
2020-12-03
Don’t Panic: Kubernetes and Docker | Kubernetes https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/
Authors: Jorge Castro, Duffie Cooley, Kat Cosgrove, Justin Garrison, Noah Kantrowitz, Bob Killen, Rey Lejano, Dan “POP” Papandrea, Jeffrey Sica, Davanum “Dims” Srinivas Kubernetes is deprecating Docker as a container runtime after v1.20. You do not need to panic. It’s not as dramatic as it sounds. tl;dr Docker as an underlying runtime is being deprecated in favor of runtimes that use the Container Runtime Interface(CRI) created for Kubernetes. Docker-produced images will continue to work in your cluster with all runtimes, as they always have.
2020-12-04
2020-12-08
Does anyone do terraform tests against their root modules? I’m assuming no, but if anyone is I’d like to hear your experience.
Just a heads up, Wednesday 11/8 is a special office hours with @Lee Skillen from Cloudsmith. We’re talking modern day package management and how to make it easier. Cloud Posse uses Cloudsmith, and so we’re excited to have them on the show tomorrow.
@Lee Skillen has joined the channel
2020-12-09
@here office hours is starting in 30 minutes! Remember to post your questions here.
If we have time, wondering if anyone has had success (or assessed the tradeoffs) with an implementation for multi-tenancy where the key goal is to keep data for “organizations” within the platform segmented/segregated. Some options I can think of off top:
• Single app with multiple databases - more complex to manage in the app, not fully segmented if someone were to gain access to the app instance as they would potentially have access to all the data. Easier to manage deployments
• Multiple instances of the app with their own databases - could be segmented by network/vpc or accounts. Slightly more complicated to manage deployments but I feel like once everything is in IaC and the pipeline is automated, it should be reasonable. The app is a rails 4 backend (i dont think multiple databases were native to rails until rails 6) on a single ec2 instance with MySQL Aurora. We plan on re-arch for high availability on something like ECS or EKS.
I could also redirect this to #aws or something for discussion if it’s not a good use of time for office hours!
Hi Kareem, did you get any interesting replies to this? Multi-tenancy is fun! There’s pros and cons to each approach.
@Tim Birkett sorry just noticed you responded! Thanks, no not yet but i’ve went through the exercise of laying out the pros and cons. We’ve decided, for now, to continue with having a single instance of the platform. Should still be an interesting topic to discuss as i’d love to here from other’s experiences
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Mike Drummond has joined Public “Office Hours”
Andrew Speed has joined Public “Office Hours”
Adam Watson has joined Public “Office Hours”
Greg Rynkowski has joined Public “Office Hours”
Andrew Speed has joined Public “Office Hours”
mallory mabe has joined Public “Office Hours”
venkatamutyala has joined Public “Office Hours”
James Haughey has joined Public “Office Hours”
Andy Miguel has joined Public “Office Hours”
Biswajit Das has joined Public “Office Hours”
Robert Horrox has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
Ramesh Yelishala has joined Public “Office Hours”
Matt Ferris has joined Public “Office Hours”
Dan McKinney has joined Public “Office Hours”
David Test has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
Chris Dutton has joined Public “Office Hours”
Kareem Shahin has joined Public “Office Hours”
Tarlan Isaev has joined Public “Office Hours”
Matthew Zeemann has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Tarlan Isaev has joined Public “Office Hours”
Dale-Kurt Murray has joined Public “Office Hours”
Oskar Maria Grande has joined Public “Office Hours”
Jeremy (Cloud Posse) has joined Public “Office Hours”
John Jarvis has joined Public “Office Hours”
Scott Rogers has joined Public “Office Hours”
Adam Crown has joined Public “Office Hours”
Marc Tamsky has joined Public “Office Hours”
Tarlan Isaev has joined Public “Office Hours”
Oliver Schoenborn has joined Public “Office Hours”
Ramesh Yelishala has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
James Haughey has joined Public “Office Hours”
Isa Aguilar has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
rei has joined Public “Office Hours”
Using GitLab to publish packages to Cloudsmith: https://www.youtube.com/watch?v=9WpvfegCgBY Create a Cloudsmith repository in under 60 seconds: https://www.youtube.com/watch?v=yzBEWrgbvQQ Read the docs https://help.cloudsmith.io/docs/welcome-to-cloudsmith-docs
Explore our guides and examples to integrate Cloudsmith
As a special promotion for those listening, you can register to receive your exclusive Cloudsmith + Cloud Posse socks arriving January 2021: https://form.typeform.com/to/XM6QreBu Sign-up for a free trial of Cloudsmith before 20th December to receive an extended trial of one month! Sign up here: https://cloudsmith.com/signup/
Turn data collection into an experience with Typeform. Create beautiful online forms, surveys, quizzes, and so much more. Try it for FREE.
Get your private, secure and ultra-fast repository in 60 seconds - and start handling packages and dependencies the right way.
Shouky Dan has joined Public “Office Hours”
also, we have a cloudsmith channel if you have any questions for @Lee Skillen & team
PePe Amengual has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
2020-12-10
2020-12-11
2020-12-14
2020-12-15
Remember to prepare your questions for our special office hours session tomorrow with Env0, Scalr, Spacelift, and Terraform Cloud.
@Erik Osterman (Cloud Posse) Hi Can you send the link I usually watch office hours after the fact on youtube but this session is particularly interesting because I am a happy atlantis user but looking to see what these offer ?
2020-12-16
@here office hours is starting in 30 minutes! Remember to post your questions here.
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
Emile Fugulin has joined Public “Office Hours”
Zeid Derhally has joined Public “Office Hours”
Hao Wang has joined Public “Office Hours”
Jesse Cafarelli has joined Public “Office Hours”
https://stackshare.io –> nifty site that lays out various organization technical stacks. They offer some kind of service for internal teams in larger orgs to share such stuff as well
All the best Open Source & Software as a Service (SaaS) tools in one place, ranked by developers and companies using them. Compare and browse tech stacks from thousands of companies and software developers from around the world.
Sebastian Stadil has joined Public “Office Hours”
Leia Renée has joined Public “Office Hours”
Zachary Loeber has joined Public “Office Hours”
Shaun Ward has joined Public “Office Hours”
Jake Lundberg has joined Public “Office Hours”
ohad maislish has joined Public “Office Hours”
Adriaan Beiertz has joined Public “Office Hours”
Tim Davis has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Jay Zalowitz has joined Public “Office Hours”
Jameson Welch has joined Public “Office Hours”
Eric Berg has joined Public “Office Hours”
Luis Navarro has joined Public “Office Hours”
Michael Martin has joined Public “Office Hours”
Roni Frantchi has joined Public “Office Hours”
Andy Roth has joined Public “Office Hours”
Casey Webster has joined Public “Office Hours”
Cameron Zeeb has joined Public “Office Hours”
JB Beck has joined Public “Office Hours”
Troy Taillefer has joined Public “Office Hours”
JB Beck has joined Public “Office Hours”
Ian Groff has joined Public “Office Hours”
David Lundgren has joined Public “Office Hours”
Kuba Martin has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
Michiel De Wilde has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Jeremy Branham has joined Public “Office Hours”
Brian Tai has joined Public “Office Hours”
Florain Drescher has joined Public “Office Hours”
Matt Calhoun has joined Public “Office Hours”
Udit Dave has joined Public “Office Hours”
Chris Dutton has joined Public “Office Hours”
Joaquin Menchaca has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
Matt Zeemann has joined Public “Office Hours”
Gareth Frost has joined Public “Office Hours”
Sam C has joined Public “Office Hours”
Marcin Wyszynski has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
James Thalacker has joined Public “Office Hours”
Jonathan Le has joined Public “Office Hours”
Igor Bronovskyi has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
Adam Crown has joined Public “Office Hours”
gkhan khan has joined Public “Office Hours”
John Mitchell has joined Public “Office Hours”
David Hayes has joined Public “Office Hours”
Kareem Shahin has joined Public “Office Hours”
Mazin Ahmed has joined Public “Office Hours”
Jeremy (Cloud Posse) has joined Public “Office Hours”
Hi Eric, Sebastian, Jake, Ohad and Everybody,
I’m working as SRE in an online education platform. I ’m about to migrate current on-prem infrastructure into AWS Cloud. I work with Terraform since 0.12 along with Terragrunt. I like the way how Terragrunt organizes TF in a more structured way.
I’m about to present a automatic deployment pipeline solution to the company based on TF, TG along with Atlantis pull request automation. I’ve already implemented EKS managed K8s as IAC and deployments will be on that. What do you think? I’m I on the right path? What do you suggest?
After TF 0.13 is published, I started to question if TG is still necessary. Thank you very much. What are other alternatives to use TF as a CI/CD tool.
Leia
If you use Terragrunt, be sure to check out what Ohad is going to present. we support Terraform and Terragrunt with env0.
Hey @Leia Renée I hear you. TG still does offer a few benefits over TF although there’s most definitely a good portion of the gap that was narrowed with TF13.
It really depends on which features of TG you’re using.
But all are great options - so I’d say you’re most definitely on the right path
Worth mentioning env0 for your use case, because it is flexible enough so you can juggle both TerraGrunt and Terraform templates, but also it has a lot of what Atlantis offers as well - so pretty much all the options you brought up covered there.
We generally suggest describing your ideal workflow and mapping the strengths/weaknesses of the tools against those workflows. Quite a bit of the reason folks have traditionally used TerraGrunt may now be addressed by Terraform Cloud (or other tools) natively.
Not to say that Atlantis is dead, but the both Luke and Mishra work for HashiCorp now. Much of what they developed has been rolled into Terraform Cloud.
Again, focus on your ideal workflows and map that to the least amount of tools to reduce complexity.
Thank you guys for great advises. Off course everything is on the table right now. For all of these great companies here as well as cloudposse we are potential customers. Before convincing my company I should convince my self. That’s why I’m here and trying to benefit from this great event. Also there are some other questions flying on my mind such as :
• Is Terraform best solution to K8s apps or we should choose other Gitops based solutions such as Flux or Helm Operator,
• Is AWS CDK considered as IAC tool since one of my college suggests using it instead of TF
Is Terraform best solution to K8s apps
Hard to say without hearing more about the specifics of your use-case, but I’d say, more often than not - yes it is.
Is AWS CDK considered as IAC
Yes. It’s relatively new, vendor specific, and offers the benefit of using familar general purpose programming languages to describe your infra - in some ways, trading off the predictable declarative form of markup-like IaC.
Thnks @Roni Frantchi (env0) for the great opinions, and @Jake Lundberg (HashiCorp) for now we are seriously considering Vault to be a central repo for secrets. Vault seems to be unrivaled in terms of some features.
For TF Cloud I experienced it 1 year ago and it was very new and very simple not offering too much. After your great presentation I should maybe re-examine it to test new features.
TFC is constantly evolving. We released 34 new features this year so far, and we listen to our customers.
Andrew Thompson has joined Public “Office Hours”
Shouky Dan has joined Public “Office Hours”
Isa Aguilar has joined Public “Office Hours”
David Knell has joined Public “Office Hours”
Loren Gordon has joined Public “Office Hours”
Shawn Rushefsky has joined Public “Office Hours”
Fernando Castillo has joined Public “Office Hours”
Phred Lane has joined Public “Office Hours”
David Lozano has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Does env0 support bitbucket cloud well ?
We support any Git provider. We just need the URL and Git Token. But, as of now, GitHub is the only provider with “deep” integration.
@Jake Lundberg (HashiCorp) is there any public roadmap for Terraform Cloud?
We have customer-friendly roadmaps, but I’m not sure if we discuss roadmaps with folks that don’t have NDAs. I can check.
So no, we don’t have publicly published roadmaps. We definitely cover with anyone under NDA however.
Jesse Cafarelli has joined Public “Office Hours”
Michiel De Wilde has joined Public “Office Hours”
@Jake Lundberg (HashiCorp) Moved to terraform cloud last year and ran into this issue as a big user of Terraform CLI Workspaces: https://github.com/hashicorp/terraform/issues/22802 As you can read from the issues thread, many decided to go with a workaround which I also ended up doing. It’s been roughly a year since I’ve moved and I’m starting to wonder if I should wait for Hashicorp to provide a fix/update on how to proceed or migrate off TF CLI workspaces. Also, do you have a particular best practice that we should follow if we were to move off Terraform CLI workspaces?
Also, is there any work in TFC on shared env vars across TFC Workspaces?
Terraform Version Terraform v0.12.8 Terraform Configuration Files resource "aws_instance" "example" { count = "${terraform.workspace == "default" ? 5 : 1}" #…
What’s the advantage of staying with the CLI workspace versus moving to the cloud based workspace (aside from having to refactor, which I can understand may be painful depending on code).
We do have shared variables on the roadmap, but as I highlighted, we take issues of scale and security very seriously. Global variables have some hidden gotchas when it comes to authorization and access models. We want to make sure we release a highly secure solution when released.
Terraform Version Terraform v0.12.8 Terraform Configuration Files resource "aws_instance" "example" { count = "${terraform.workspace == "default" ? 5 : 1}" #…
@Jake Lundberg (HashiCorp) I initially chose to use CLI workspaces because it helped keep my Terraform code dry. I can have a single set of terraform files in a repo that I can have up to 5 or 6 different TF CLI workspaces which followed a specific pattern ({environment}_{region}
) Using the native terraform.workspace
variable, I could make specific decision based on the environment within my code i.e. if environment == prod don't spin up a bastion node by default
on shared vars being on the roadmap
To be clear, I’m open on moving away from TF CLI workspaces, but I don’t want to necessarily give up the code dry-ness I was able to achieve utilizing TF CLI workspaces.
You can have the same pattern with TFC workspaces. Use the same code base, just vary the workspace variables. In fact, this is how I normally demo TFC where various environments consume the same code base but vary by variables. And even launch into different cloud environments based on credentials/roles if need be.
Much of this is going to depend on if you want to move to a TFC model or have a more generic capability. If you’re planning on using TFC, I’d suggest migrating to TFC workspaces.
Todd Thomas has joined Public “Office Hours”
Useful links for Terraform Cloud
• Get Started with Terraform Cloud
Oliver Schoenborn has joined Public “Office Hours”
Question: How difficult is it to create a new resource type for AWS provider? There is a new type needed due to recent AWS “key group” resource in cloudfront, it is not represented in TF so prevents use of the key-based trusted signers. There is a ticket open with lots of votes (over 50), but no indication that work is happening, wondering if it’s something I could volunteer to do.
Spacelift — Can you cover your pricing details? I don’t see them on your site.
We’re still working on the right approach but we’ll bill by 2 things: the number of seats and the number of concurrent private (self-hosted) agents.
Re: seats we want the whole company to be able to use Spacelift to free up DevOps resources so after first 5 seats that we’ll price at ~~~$50 and there will be a nominal fee of ~~~ after that.
Private workers are a premium feature and we’re looking at ~$400/month a piece.
Terraform config for each workpsace is pretty annoying
thats pretty much the only thing left in terragrunt
We have a thing in Spacelift called contexts that allow you to share config between stacks.
Christos Pashcalidis has joined Public “Office Hours”
Terraform Cloud — Tough questions, but they’re out of love
- Will we see the SSO Tax come down (i.e. can we get SSO support at the team level instead of biz level)?
- Will we see TF Agents outside of Biz Level? This is a big limiting factor for automating internal network resources (Data tier layer), which is behind a paywall.
If we have enough requests for a specific feature set in different levels we’ll certainly entertain them. The best thing to do is speak with your sales teams or post in Discuss as the product teams will pay attention here.
Gotcha — Thanks Jake, much appreciated.
Not TFC, but Scalr has no SSO tax and TF Agents coming soon and affordable (like the rest of Scalr)
What is it looking like for getting these tools to be certified with use by the various GovClouds?
Hey guys, I’ve tested out the https://registry.terraform.io/modules/cloudposse/rds/aws/latest module; it has issues. I’ve tested/documented everything - there are some easy fixes in here. The only problem is there are a lot of issues; when would you have time to go over it?
@Erik Osterman (Cloud Posse) https://docs.google.com/document/d/1fbHUUonGETj_NJRf5lnAu0fhhJQE0J6bBNFLx28AHw4/edit?usp=sharing
let me know if you need any context for this.
@voidSurfr perhaps you could add each issue on the GitHub repo? There are already a few raised. https://github.com/cloudposse/terraform-aws-rds/issues
Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.
Thanks for the heads up. Due to the holidays things will be slower. As joe recommends, let’s make sure we’re tracking an issue for each one.
Let’s use #pr-reviews to move PRs forward. (we literally have hundreds of outstanding PRs). @Maxim Mironenko (Cloud Posse) on our team can help you get things tested and merged.
@voidSurfr thanks for the professionalism of your bug report!
I took a quick look and I think the problem stems from:
@Andriy Knysh (Cloud Posse)
can you take a look at the google doc
@voidSurfr I requested access to https://docs.google.com/document/d/1fbHUUonGETj_NJRf5lnAu0fhhJQE0J6bBNFLx28AHw4/edit?usp=sharing
or you can post those issues here
the issues here https://github.com/cloudposse/terraform-aws-rds/issues are very old, and many of them are probably fixed already (since the module was updated many times)
Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.
you prob have some recent issues, please approve access to the doc
hey guys, as @Joe Niland suggested, these of course should be moved into issues; just trying to catch up myself. please enjoy the holidays, no rush on my part I’ll get them in there soonish.
Useful Scalr links to get started Homepage Documentation Free account (GitHub/Google SSO - no credit card required) Pricing
Which of the 4 can
a) be run On-Premise? b) be run On-Premise in a disconnected environment?
answers:
TFE: yes to both env0: yes to a, no to b Scalr: yes to both Spacelift: no to both
what was the answer to this one?
haha tks
Nadtakan Jones has joined Public “Office Hours”
If i understand scalr works with terraform cli ? If that is the case since terragrunt wraps terraform cli can scalr work with terragrunt ?
@Troy Taillefer to clarify my answer, you really just need Scalr as the remote backend in your code
Thanks
Example:
terraform {
backend "remote" {
hostname = "my-account.scalr.io"
environment = "some-env"
workspaces {
name = "some-name"
}
}
}
@Erik Osterman (Cloud Posse) afterward can you collect and share technical and/or sales POCs for each? I know each of them had it in their presentations but I didn’t get them written down
Slack contacts: @Jake Lundberg (HashiCorp) (TFC/E), @ohad (env0), @Sebastian Stadil (Scalr), @marcinw (Spacelift)
Ohad got the CEO lobotomy
LOL
RBAC was pretty tough, lots of ways to do it wrong, only a few to do it right
Same for the inheritance model Ryan is talking about now
For us one of the biggest engineering challenges was building the whole backend (sampling, simulations) for the policy IDE.
Thanks to the presenters and the organizer for this great session. Need to drop have a great day.
Also, getting the self-hosted agent flow and boundary right.
Env0 Resources:
• Website: https://www.env0.com/
• Documentation: https://docs.env0.com/
• Terratag OSS on Github: https://github.com/env0/terratag
• Twitter: https://twitter.com/envZero
• IaC Automation Demo Video (3 min): https://youtu.be/AKBJMO-G4sQ
Spacelift Resources:
• Website//spacelift.io/>
• Docs//docs.spacelift.io>
• Quickstart Repo: https://github.com/spacelift-io/terraform-starter
I agree with Erik, policies are hard
especially for smaller startups
OPA samples to help you get started: https://github.com/Scalr/sample-tf-opa-policies
Contribute to Scalr/sample-tf-opa-policies development by creating an account on GitHub.
Thanks everyone !
Great session y’all. Love to see the creativity and love for the infrastructure community.
feel free to check out our blog at Rancher labs from today: https://rancher.com/blog/2020/speed-development-automated-kubernetes-deployments
Automating the deploy and destroy of your environments and speed up development using the env0 infrastructure automation platform with Rancher’s Kubernetes management platform.
So much for airgapped networks: https://arxiv.org/abs/2012.06884
In this paper, we show that attackers can exfiltrate data from air-gapped computers via Wi-Fi signals. Malware in a compromised air-gapped computer can generate signals in the Wi-Fi frequency…
that team has been busy the past few years on airgap exfiltration research - here’s their paper and demo video roundup: https://cyber.bgu.ac.il/advanced-cyber/airgap , see also https://www.wired.com/story/air-gap-researcher-mordechai-guri/
Researcher Mordechai Guri has spent the last four years exploring practically every method of stealthily siphoning data off of a disconnected computer.
2020-12-18
2020-12-21
Hi all - really enjoyed the recent cast on TACOS and I’m really interested in not having to manage my own Terraform or create the governance that I want around our infra on my own. Basically (and I understand that this is a really broad question, that I expect to differ between Terraform Cloud, Env0, Scalr, Spacelift) I would to ask how you transition your self-hosted Terraform solution to one of these SaaS providers without downtime and, maybe more importantly, how your previous small-team customers have driven buy-in from their wider org that this stuff is really important (please don’t sell me on it, I know it’s critical)
2020-12-23
Any recommendations on a tool for release notes? Right now our developers/product team update a confluence page whenever they remember. This feels a bit error prone and ends up being an after thought since the code is already deployed. So to make it part of the process I am thinking about… having everyone just add a CHANGELOG.MD to their repo and when a new git tag gets dropped a webhook gets fired and a centralized service just aggregates all the CHANGELOG.MD’s. I have a feeling I shouldn’t be building anything to do this and that someone else may have solved this problem and/or solved it a different way. Any recommendations/suggestions?
May be this https://github.com/semantic-release/semantic-release ?
Fully automated version management and package publishing - semantic-release/semantic-release
sold.
https://github.com/git-chglog/git-chglog is also pretty good.
CHANGELOG generator implemented in Go (Golang). Contribute to git-chglog/git-chglog development by creating an account on GitHub.
You’ll find that most things require conventional commit messages: https://www.conventionalcommits.org/en/v1.0.0/ to function well.
A specification for adding human and machine readable meaning to commit messages
Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.
Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.
Geodesic is a DevOps Linux Distro. We use it as a cloud automation shell. It's the fastest way to get up and running with a rock solid Open Source toolchain. ★ this repo! https://slack.cloud…
Thank you!
@here office hours is starting in 30 minutes! Remember to post your questions here.
Andy Miguel has joined Public “Office Hours”
Oskar Maria Grande has joined Public “Office Hours”
Raghavendra V has joined Public “Office Hours”
Phil Hershkowitz has joined Public “Office Hours”
Aarat Nathwani has joined Public “Office Hours”
Patrick Joyce has joined Public “Office Hours”
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
venkatamutyala has joined Public “Office Hours”
Tom Dugan has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
majan paul has joined Public “Office Hours”
PePe Amengual has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
Brian Tai has joined Public “Office Hours”
Troy Taillefer has joined Public “Office Hours”
Michael Jenkins has joined Public “Office Hours”
Juan Soto has joined Public “Office Hours”
David Lundgren has joined Public “Office Hours”
Kareem Shahin has joined Public “Office Hours”
Michael York has joined Public “Office Hours”
Chris Ferrer has joined Public “Office Hours”
Ola Ade has joined Public “Office Hours”
Adriaan Beiertz has joined Public “Office Hours”
Ola Ade has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
Omer Sen has joined Public “Office Hours”
Neil Gealy has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
2020-12-24
just tried getting docker installed in CloudShell without success. Anyone here do it successfully before?
30 sudo amazon-linux-extras install docker
31 sudo service docker start
32 docker
33 docker run -it nodejs
34* service
35 systemctl
36 systemctl status docker
37 sudo systemctl docker
38 sudo systemctl status docker
39 sudo systemctl docker
40 sudo systemctl status docker
41 sudo service docker start
42 docker --version
43 history
[cloudshell-user@ip-10-1-17-184 ~]$ docker run -it nodejs
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
[cloudshell-user@ip-10-1-17-184 ~]$ service
bash: service: command not found
[cloudshell-user@ip-10-1-17-184 ~]$ systemctl
Failed to get D-Bus connection: Operation not permitted
[cloudshell-user@ip-10-1-17-184 ~]$ systemctl status docker
Failed to get D-Bus connection: Operation not permitted
[cloudshell-user@ip-10-1-17-184 ~]$ sudo systemctl docker
Unknown operation 'docker'.
[cloudshell-user@ip-10-1-17-184 ~]$ sudo systemctl status docker
Failed to get D-Bus connection: Operation not permitted
[cloudshell-user@ip-10-1-17-184 ~]$ sudo systemctl docker
Unknown operation 'docker'.
[cloudshell-user@ip-10-1-17-184 ~]$ sudo systemctl status docker
Failed to get D-Bus connection: Operation not permitted
[cloudshell-user@ip-10-1-17-184 ~]$ sudo service docker start
sudo: service: command not found
[cloudshell-user@ip-10-1-17-184 ~]$ docker --version
Docker version 19.03.13-ce, build 4484c46
I think the docs currently explicitly say it isn’t supported.
https://docs.aws.amazon.com/cloudshell/latest/userguide/vm-specs.html
Currently, the AWS CloudShell compute environment doesn’t support Docker containers.
Provides details about the virtual machine and pre-installed tools that support your AWS CloudShell environment.
Thanks! For some reason in the officehours I swore cloudshell w/ docker support was mentioned verbally. Perhaps I misunderstood.
I also thought I heard Erik say they got docker working in cloudshell.
Yes, I did say it was supported. mea culpa.
I just checked again, I got in and validated that it works:
sudo -- sh -c 'amazon-linux-extras enable docker && yum clean metadata && yum install docker'
[cloudshell-user@ip-10-0-145-19 ~]$ docker --version
Docker version 19.03.13-ce, build 4484c46
Sorry, I had not independently validated it. big bummer.
I wonder if there’s an alternative way that the conatiners can be run without using docker?
2020-12-25
2020-12-27
2020-12-28
2020-12-29
2020-12-30
New Zoom Recording from our Office Hours session on 2020-12-23 is now available.
What will be the recommended way when using EKS cluster for lets say for CD/CD or Control plane management and yo wanted to keep the ingress in a private subnet, will that work? ( we keep our CI/CD systems behind vpn and since I was playing with ArgoCD I was using the port-forwarding option)
i’ve got a probably dumb question about using docker containers… is there a simple/automatic way to refer to local files from the host, within the container environment? i was just playing with the terraform container, which says to do this:
docker run -i -t hashicorp/terraform:light plan [main.tf](http://main.tf)
but of course that fails because 1) it’s invalid syntax for terraform and 2) the container workdir does not have my main.tf. i do know about -v of course, and can mount $PWD to /, but what i’m more interested in is the idea of using a docker image to replace a binary installed to my system. if i have to mount $PWD to the workdir every time, that seems a little more annoying?
@here office hours is starting in 30 minutes! Remember to post your questions here.
Any recommendations on a pen test vendor? Or perhaps a better question, what should i be looking for before I pick a vendor?
Is this for PCI compliance? Do you already have a QSA?
I’ve worked with https://securisea.com/ in the past and they were great. I know the founder and whole heartedly recommend them for smaller sized infras.
Erik Osterman (Cloud Posse) has joined Public “Office Hours”
venkatamutyala has joined Public “Office Hours”
charles pogi has joined Public “Office Hours”
Jesse Cafarelli has joined Public “Office Hours”
Jesse Cafarelli has joined Public “Office Hours”
Vicken Simonian has joined Public “Office Hours”
Neil Gealy has joined Public “Office Hours”
Mohammed Yahya has joined Public “Office Hours”
Shouky Dan has joined Public “Office Hours”
Andrew Thompson has joined Public “Office Hours”
Mikael Fridh has joined Public “Office Hours”
Chris Brooks has joined Public “Office Hours”
Hao Wang has joined Public “Office Hours”
Joey Freeland has joined Public “Office Hours”
Andy Miguel has joined Public “Office Hours”
Russell Sherman has joined Public “Office Hours”
Troy Taillefer has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Vlad Ionescu has joined Public “Office Hours”
Adam Crown has joined Public “Office Hours”
Matt Gowie has joined Public “Office Hours”
Tarlan Isaev has joined Public “Office Hours”
Hari Prasad has joined Public “Office Hours”
Phil Hershkowitz has joined Public “Office Hours”
Abisoye Olaomi has joined Public “Office Hours”
Todd Thomas has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Martin Mazurik has joined Public “Office Hours”
Andrew Elkins has joined Public “Office Hours”
Blue Pisces Consulting Inc is hiring for a DevOps Engineer in Los Angeles. Find more details about the job and how to apply at Built In Los Angeles.
We Need Your Help! Are you a regular contributor to this module? Have you opened multiple Pull Requests against Cloud Posse repos? Would you like to help us maintain this module? Have you checked o…
Rashid Boyko has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Kareem Shahin has joined Public “Office Hours”
Rashid Boyko has joined Public “Office Hours”
Ray Mazurik has joined Public “Office Hours”
emem umoh has joined Public “Office Hours”
Tochukwu Nwoko has joined Public “Office Hours”
anyone try “werf” yet? https://github.com/werf/werf
GitOps delivery tool. Contribute to werf/werf development by creating an account on GitHub.
The “Cloud Posse” Distribution of Kubernetes Applications - cloudposse/charts
This post is a continuation into the world of locally executing CI/CD for developers, with my prior post being about Skaffold. In this post I’ll look at another one of these tools called Tilt. Background The world of software development and how apps are run in production environments has come a long way over the years. Starting with bare metal physical servers, we evolved to virtual machines, onward to LXC, Docker daemons, and now our current state of container orchestration via things like Kubernetes. The other side of the world… that which defines how software developers locally develop, test, iterate, package, build and deploy those apps to their final execution environments likewise has varied wildly. Much of this is due to obvious things like choice of language and frameworks, but another factor in it is the final execution environment by which the application will live. As target runtime environments has evolved from bare-metal to containers, much of the complexity of configuration and “installing an application” has now been pushed down to the developer’s plate, as the developer is now responsible for defining the context by which the application will execute in using container images. With this comes more responsibility for the developer of not only defining and documenting an apps dependencies, but now also implementing all of it via Dockerfiles; building those Dockerfiles into images, then pushing them to an artifact repository. The containerization standards over the past few years has certainly offloaded more “DevOps” like work on the developers plate but with that extra work comes a big benefit: Like never before, developers can now test their apps locally in much more realistic execution environments as they will run in production (i.e. local Minkube, Docker, k3s etc). However in order to be able to test the artifacts locally, they still need to be built and deployed (locally or remotely) to a container execution engine. Typically this can just be a centralized CI/CD service which handles all of these extra steps in reaction to a developer just pushing a commit; but what if a developer wants to do all of this in a more real-time fashion and avoid pushing/deploying artifacts to remote environments on every change over numerous iterations? i.e. just iterate locally. Well, over the past few years several tools have evolved which bring powerful CI/CD capabilities right to the developer’s laptop, enabling them to harness the power of container automation using standard CI/CD tooling to build, package, test and deploy both remotely OR locally… even in real time as local files are being changed. Let’s take a brief look at another one of these tools: Tilt. Please keep in mind that my coverage here is based primarily on my personal experience using it which was very specific to certain use-cases. This article is not an exhaustive overview of all the capabilities.
Tilt
Tilt is another locally executing CI/CD tool for developers, similar to Skaffold, the key differences being the lack of formal “stages” as well as Tilt’s extremely flexible configuration format which is a derivative of Python called Starlark. As opposed to Skaffold where your pipeline configuration file is defined in YAML with very limited support for any variables much less any logic, Tilt’s choice of Starlark for its Tiltfile format, gives it a massive edge (IMHO) when compared with Skaffold’s less-flexible YAML syntax. If you need the ability to fully customize your local Tiltfile…. well the sky is the limit as your Tiltfile is basically a Python program. With Tilt’s exposure of its “local()” or “custom_build()” functions you can pretty much execute any 3rd party tool you wish as part of a Tiltfile definition. Note that “local()” invocations only run on “tilt up | down” but are still quite useful. The other key thing to note is even though Tilt doesn’t have any formal first class “stages” defined like “testing” etc, but you could still wrap those calls somewhere within the other functions that Tilt provides. Tilt’s key workflow paradigm to understand is that when a file changes, something is built (i.e. docker image), k8s YAML manifests are generated and finally the k8s YAML manifests are applied to the target k8s cluster. To get started the developer installs Tilt locally, creates a “Tiltfile”, then on to the CLI to ”tilt up” a project. The “tilt up” command starts a Tilt daemon locally that is watching the project folder for changes and then executes the commands defined in your project’s “Tiltfile”. When the Tile daemon starts, it also launches a nice little SPA (see further below in the article). When you are done, you can call “tilt down” which will also run your “local()” functions. I’d like to mention that if your Tiltfile needs to do some initialization things only on daemon start, you need to do a hack like the below, due to the lack of well defined Tilt lifecycle hooks that are made available to the Tiltfile developer. Here is what a Tiltfile looks like below: Example of a custom Tiltfile which only reacts to Git commits, makes multiple local() calls, builds a custom Dockerfile, invokes Helm template and applies the resulting YAML to the cluster via k8s_yaml()
In the Tiltfile above we only react to Git commits (rather than any random file change), and only do certain operations to initialize some things on the initial “tilt up”, that are not done on every Git commit. The “custom_build()” action occurs on every reactive change the Tilt daemon detects as well as the “k8s_yaml()” calls. Note that we also always call “kubectl delete” via “local()” to ensure old objects are being cleaned up on “tilt down”. Note that “local()” invocations only run on “tilt up | down” but are still quite useful. Tilt has a first class preference for dealing with raw k8s YAML manifests but Helm install/upgrade support does not appear to be directly supported. What do I mean by that? Well Tilt provides a “helm()” function which you can leverage in your Tiltfile, but it only invokes Helm’s “template” command to generate YAML and then applies it directly to the k8s cluster (via “k8s_yaml()”) rather than letting Helm’s “install, upgrade” commands do it for you (and properly track things). This was something I didn’t care for as Tilt can result in orphaned objects due to it’s architecture with regards to how deployments are tracked and cleaned up (i.e. via “tilt down”). It assumes for example, that the chart you are using to generate the YAML will always create objects w/ the same names… but what if it doesn’t? For example what if the image tag you generate has a commit ID in it, and this commit ID is also consumed by the chart as part of the object names? This can lead to orphans. For example if your first git commit generates k8s object names with “myapp-XYZ”, then you commit again and yield “myapp-ABC”…. what happens to “myapp-XYZ” names objects on the k8s cluster? This could however be worked around w/ good Kubernetes object labels and some additional calls to “kubectl delete” via “local()” when “tilt down” occurs (or embedded in your “custom_build()” or overloaded in a “k8s_yaml(local())” call. My biggest concern with this was that in a large team environment, each individual laptop is the only thing “aware” of the collections of objects that each local Tilt instance generated/applied to the cluster via …
Helm Plugin - Install Helm Charts directly from Git repositories - aslafy-z/helm-git
anyone played with codefresh’s argocd integration?
Collection of Makefiles and test scripts to facilitate testing Terraform modules, Kubernetes resources, Helm charts, and more - cloudposse/test-harness
re: the serverless vs. kubernetes discussion, i listened to https://softwareengineeringdaily.com/2020/12/29/kubernetes-vs-serverless-with-matt-ward-repeat/ yesterday and it’s a bit old, but it’s still very relevant and a pretty good discussion
Originally published May 29, 2020 Kubernetes has become a highly usable platform for deploying and managing distributed systems. The user experience for Kubernetes is great, but is still not as simple as a full-on serverless implementation–at least, that has been a long-held assumption. Why would you manage your own infrastructure, even if it is Kubernetes?
@Erik Osterman (Cloud Posse) It never came up an opportunity to ask. Your dig at either gatekeeper or keycloak or the combination of both - Could you elaborate why you opt to nix them? (I’m looking at replacing or complementing Cognito with Keycloak at the moment)
I will stay away from keycloak
There is few bugs that are very important that have not been fixed
It does not have user onboarding
If you compare it with Octa it lacks a lot off stuff
Okta is just off limits to us. We need an intermediary. You know how it is… big corp.
It’s also possible the major problem is tooling too. So will try to work on that to alleviate the Cognito head aches before any keycloak road is paved.
New Zoom Recording from our Office Hours session on 2020-12-30 is now available.