#announcements (2018-10)
Cloud Posse Open Source Community #geodesic #terraform #release-engineering #random #releases #docs
This channel is for workspace-wide communication and announcements. All members are in this channel.
Archive: https://archive.sweetops.com
2018-10-01
There is 1 event this week
October 3rd, 2018 from 11:00 AM to 11:50 AM GMT-0700 at https://zoom.us/j/299169718
Agenda is here: https://cloudposse.quip.com/VHF9A1Qrp3eR
When00am PST
Please feel free to update
hey everyone give a warm welcome to @MattN! Good to have you here
hey everyone give a warm welcome to @dserodio! Good to have you here
Hi! I just discovered the CloudPosse website and Geodesic, and I was wondering why you chose Chamber for secrets management instead of Hashicorp Vault
@dserodio welcome!
Sure - so Hashicorp vault is a great solution, but takes considerably more effort to manage over AWS SSM+Parameter Store.
Nothing about our strategy precludes the use of hashicorp vault, but using parameter store + ssm is a turnkey solution with no new overhead to manage.
this is the prescribed architecture for managing vault, which doesn’t take into account the SDLC including upgrades, backups, DR, etc.
hey everyone give a warm welcome to @jarv! Good to have you here
Hey Jarv! What brings you around?
Just a fan of you guy’s work! Figured I’d stop by and say hi
awesome! always great to hear when we’re helping out
Been messing around with the TF modules lately, but impressed with everything I’ve seen. Appreciate all your guy’s work on everything.
have your seen our latest modules for EKS?
@Andriy Knysh (Cloud Posse) has been hard at work on those
Not yet, will likely soon though. Taking a look at the user/chamber stuff currently actually
Cool - yea, we developed those mostly for external CI/CD systems (in our case Codefresh)
Was planning on doing something similar to what you used to do before moving to 99designs/aws-vault, at least for the moment. But figured it was worth taking a quick look at how you have that set up now.
oh, so we still use aws-vault
and chamber
they are complimentary
aws-vault
is the best way to manage local AWS credentials for “assuming roles” (and not to be confused with hashicorp’s vault - in anyway)
Ah yeah, sorry was more referring to this: https://docs.cloudposse.com/announcements/aws-assumed-roles-repo-deprecated/
so we like to use aws-vault
for all local development contexts
aha, yes! as soon as we took a look at aws-vault
we knew that our own implementation was not worth maintaining anymore
haha
So I guess my question is (and totally realize I can look this up as well), how is user/permission management done with the new setup?
Previously I was thinking designated IAM account with users getting something like:
"Action": "sts:*",
"Resource": "arn:aws:iam::*:role/${aws:username}",
"Condition": {
"StringEquals": {
"aws:PrincipalType": "User"
}
}
then to provision a user in other account’s (that trust’s the iam one) you just need to create a role named after their username. (I think this is somewhat similar to what you did before, could be wrong there though)
hey @jarv
take a look at our reference architectures https://docs.cloudposse.com/reference-architectures/
in short, we create a few AWS accounts:
root
(which is billing account) to store all IAM users and roles
yep, this was just another setup I was considering vs chamber, etc.. (iam account is seperate but doesn’t need to be, similar to the reference-architecture that way)
this is complementary to chamber. we provision a separate chamber IAM user in all accounts (prod, staging, dev, testing) and use it to access SSM secrets
ok cool, yeah so sounds like what I need to look into
I went through the cold start on a test setup a bit ago, didn’t quite understand how this part worked exactly though.
(I imagine I just need to dig into it a bit though)
ohh, we keep forgetting it as well
can show you if you are stuck
heh, yeah for sure. Might need to take you up on that
I’ll go ahead and make an attempt here though first, appreciate the offer
ok
@Andriy Knysh (Cloud Posse) I think he’s talking specifically about Chamber
yea ok
using our terraform modules
Yeah actually personally would like to adopt the whole platform, but more longer term goal I think.
I’m kinda hoping we can use what we need for now and adopt where it makes sense to dedup work.. which also makes getting stuff approved from my end a lot easier as well.
and tbh I like to do with anything that get’s adopted really, need to know how it works first imo..
Chamber is pretty solid, but be careful/aware that AWS has API rate limits on the param store, so many companies after a certain size end up returning to Consul. But for startups/small businesses, I agree Chamber is too easy to ignore. Unless you have some really fancy dynamic auth/cert needs.
Thanks good to know, shouldn’t be too many users for now.
I could see that being a problem where there are hundreds of containers getting spawned in an account per minute for example
but also mitigated by having multiple accounts and never co-mingling prod with dev or staging
2018-10-02
It all depends on use case of course. I first ran into the issue at a Serverless shop. In that scenario it wasn’t just secrets but nearly 100% of the behavior config was stored in param store. So I think we hit Params more than typical.
went looking for a published rate limit on AWS’s site later, but never found one, so no idea where that threshold sits. Best I could find where other people also seeing the boto3 failures and asking the same questions.
Luckily we DID do everything in unique accounts, and we first noticed it while load testing in Stage … so we were able to pivot before being SOL in prod.
for serverless i found that secrets manager worked nicely for me and my use cases
still an aws service but i never ran into any rate limit issues with 50+ lambdas
is anybody else using that?
hey everyone give a warm welcome to @tomweston! Good to have you here
hey everyone give a warm welcome to @mallen! Good to have you here
Hey @mallen! what’cha working on?
Hi Erik, I was intrigued by your Unlimited Staging Environments vid on codefresh.
Oh cool! We’ve actually taken that a bit further now and extended it with automatic destruction of environments when PRs are closed
and using Chamber for secrets and helmfile for deployment of helm releases
nice, makes a real lot of sense. I would like to take a similar route.
are you using k8s + helm?
yeah we do, i spent a lot of time writing jenkins grooviness and to calm my rage I started to look for fresh ideas.
so true…
haha yea, Jenkins is a common route, but frankly after working with Codefresh for the past year couldn’t imagine going back.
I’m not really a jenkins hater if it works i’m ok with it. codefresh and ideas like the unlimited dynamic staging environments are just making soo much better use of the power available (without investing lots of time). I’m also thinking of self service environment with some sort of live code sync would be a great pain easier, something like devspace, but Im in the early research stages.
hey everyone give a warm welcome to @dan! Good to have you here
Welcome @dan ! let me know if you need any assistance with our stuff
hey everyone give a warm welcome to @Fizz! Good to have you here
hi @Fizz
2018-10-03
Town hall meeting today at 11am PST, 6pm GMT. For those interested, it’s a chance to meet others “face-to-face” via zoom and hear what everyone is working on.
Agenda: https://cloudposse.quip.com/VHF9A1Qrp3eR Quip: https://zoom.us/j/299169718
When00am PST
hey everyone give a warm welcome to @nathan! Good to have you here
October 3rd, 2018 from 11:00 AM to 11:50 AM GMT-0700 at https://zoom.us/j/299169718
hey everyone give a warm welcome to @ankur! Good to have you here
Hey @ankur ! Let me know if you need help with anything…
2018-10-04
hey everyone give a warm welcome to @yudi.phanama! Good to have you here
2018-10-05
2018-10-06
hey everyone give a warm welcome to @blake! Good to have you here
2018-10-07
So many new faces here! Welcome all!
Welcome all! If you came for questions regarding terraform, be sure to check out the #terraform channel
2018-10-08
There are no events this week
September 19th, 2018 from 11:00 AM to 11:50 AM GMT-0700 Recurring every 2 weeks on Wednesday at https://zoom.us/j/299169718
I’m going to be reincarnating this event - we had some good discussions on how to make it better last week
Proposed format is something like: 1) Show & Tell - present a live demo of something cool (recorded) 2) Gossip - share some cool new link or project, why you liked it; let others share what they’ve done in the past (recorded)
3) Off the Record - talk shop (not recorded)
Office Hours - Weekly Q&A, AMA for people who need to ask cloudposse for assistance on our tools and projects
Please share your thoughts on what you’d like to see!
hey everyone give a warm welcome to @vinay.nair! Good to have you here
hey everyone give a warm welcome to @geertn! Good to have you here
2018-10-09
hey everyone give a warm welcome to @Nathan Preen! Good to have you here
hey @Nathan Preen and welcome
hey everyone give a warm welcome to @Gaurav! Good to have you here
Thank you
I need a help
[root@server kubernetes]# kubectl run hello-minikube –image=worpress The connection to the server localhost:8080 was refused - did you specify the right host or port?
In kubectl
@Gaurav are you on OSX with Docker for Mac?
let’s move to #kubernetes
hey everyone give a warm welcome to @sebas! Good to have you here
2018-10-10
I am using Centos7
GUI
hey everyone give a warm welcome to @ben! Good to have you here
Woohoo @ben ! Welcome. Good seeing you tonight at the Github meetup.
2018-10-11
@maarten created a new channel #azure. Join if this sounds interesting!
hey everyone give a warm welcome to @bober2000! Good to have you here
Hi all. I’m quite new to Terraform so my question could be newbie but I need help. I’m using https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment to create beanstalk nodejs environment. I need to change NodeJS version - as I see I could do it using this namespace https://docs.aws.amazon.com//elasticbeanstalk/latest/dg/command-options-specific.html#command-options-nodejs but I don’t know where could I inject settings for this
Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment
Configure platform-specific options for your Elastic Beanstalk environment.
Like this:
setting {
namespace = "aws:elasticbeanstalk:container:nodejs"
name = "NodeVersion"
value = "8.6.4"
}
@bober2000 solution_stack_name
“Elastic Beanstalk stack, e.g. Docker, Go, Node, Java, IIS. http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html string no
`
“
Elastic Beanstalk provides pre-configured platforms for programming languages (Java, PHP, Python, Ruby, Go), web containers (Tomcat, Passenger, Puma), and Docker containers, with multiple configurations of each.
check out their example https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/master/examples/complete/main.tf . They specify solution_stack_name = "64bit Amazon Linux 2018.03 v2.12.2 running Docker 18.03.1-ce"
Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment
@Andy thanks - solution stack is ok, Node env is installed - but it’s using Node 6.x by default - in AWS Console - I could change version to 8.x/10.x etc - I need to do this in tearraform
Supported versions of Node on Beanstalk: https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platforms-supported.html#platforms-supported.nodejs
Elastic Beanstalk provides pre-configured platforms for programming languages (Java, PHP, Python, Ruby, Go), web containers (Tomcat, Passenger, Puma), and Docker containers, with multiple configurations of each.
Ah I see they combine multiple version for NodeJs
Default platform: 6.14.4 - I need it to be set to 8.11.4 at
terraform apply
So the setting you pasted above you’d have to add
The Node.js platform includes a few Node.js versions in a single configuration. The following table lists them. The default version applies when the NodeVersion option in the aws:elasticbeanstalk:container:nodejs namespace isn't set. For details, see Node.js Platform Options in the AWS Elastic Beanstalk Developer Guide.
That is the question - where should I put those settings section ?
Let’s move the terraform discussion to #terraform
Currently I’m using example
Oh, sorry
No prob! Welcome to the community :-)
hey everyone give a warm welcome to @GFox)(AWSDevSecOps! Good to have you here
2018-10-12
hey everyone give a warm welcome to @samh! Good to have you here
2018-10-14
hey everyone give a warm welcome to @tobiaswi! Good to have you here
2018-10-15
There are no events this week
hey everyone give a warm welcome to @nicgrayson! Good to have you here
welcome @nicgrayson and @tobiaswi!
hey everyone give a warm welcome to @Miguel Mendez! Good to have you here
hey everyone give a warm welcome to @Nizam! Good to have you here
Hi There!
I’m setting up prometheus to cloudwatch
hey @Nizam welcome to the community
I’m getting these logs:
prometheus-to-cloudwatch: published 0 metrics to CloudWatch
there is no error
what can be the issue?
did you specify the correct scrapping URL?
PROMETHEUS_SCRAPE_URL: “http://hardy-ocelot-kube-state-metrics.default.svc.cluster.local:8080/metrics”
its right, name of service is same
in another form I only mentioned PROMETHEUS_SCRAPE_URL: “http://hardy-ocelot-kube-state-metrics:8080/metrics”
did you install kube-state-metrics
?
Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch - cloudposse/prometheus-to-cloudwatch
yes
hardy-ocelot-kube-state-metrics
above is the name of my service
I’ve given instance IAM role to all cloudwatch permissions, its in aws
any idea?
can you get the pods and look into the logs? maybe they were not started correctly. Look into kube-state-metrics
pods and prometheus-to-cloudwatch
pods
kubectl get pods --all-namespaces
kubectl logs <xxxxxxx> -n <namespace>
let me check
User “systemdefault:default” cannot list replicationcontrollers at the cluster scope
this is user account permission issue
there are many more like this on other resources
assigning rbac to default
2018-10-16
hey everyone give a warm welcome to @shaiss! Good to have you here
hey everyone give a warm welcome to @Suresh! Good to have you here
2018-10-17
thought it could be useful for you to have PR templates on your repos… made an example here https://github.com/cloudposse/terraform-aws-dynamic-subnets/pull/35
What Add PR Template Why Makes it standard for new PRs to include required information. Providing this PR as an example if you want to include on other repos.
Thanks @Gabe! You’re right. We should roll this out to all repos.
I made a small comment. What do you think about using .github/
folder?
Yeah that’s how I was going to do it originally lol
It could be a little nicer since it keeps it hidden
Happy to make the change if that’s your preference
done
hey everyone give a warm welcome to @Trey! Good to have you here
Thanks @Gabe
I think I’ll have @vadzim on our team add this to all of our repos.
Welcome @Trey!
hi @Erik Osterman (Cloud Posse) Saw this yesterday https://github.com/skyscrapers/terraform-kubernetes. They are using null resources to call kops and helm.
Contribute to skyscrapers/terraform-kubernetes development by creating an account on GitHub.
@Igor Rodionov @Andriy Knysh (Cloud Posse)
Contribute to skyscrapers/terraform-kubernetes development by creating an account on GitHub.
oh cool, and using teleport for ssh
looks like a lot of work went into this….
2018-10-18
speaking of teleport for ssh. i have a customer thats asking for ssh type access into a fargate container… as much as ive tried to persuade him against it. hes pretty steadfast in that requirement. looked at ecsctl
but wondering if anyone around here has experience with a tool
unfortunately, not… this cannot be achieved on fargate using docker exec
since that capability is not exposed
it would require deploying an SSH daemon alongside the other processes inside of the container, which I’m sure you don’t want to do
if you switch to ECS “classic” then you can do whatever you want since you BYOM to the cluster.
but, hypotheticaly, if you did, here’s a base image for ya: https://github.com/cloudposse/bastion
Secure Bastion implemented as Docker Container running Alpine Linux with Google Authenticator & DUO MFA support - cloudposse/bastion
@Ryan Ryke would maybe a bastion suffice?
this wouldl allow you to deploy an ECS task to the cluster that you could expose to the internet
if you ssh to the bastion, you could then curl
and access the internal services running on private subnets
of course, SSH would still require that those services have that running
yeah bastion is in there, from there i was curious how we could do it
run the SSM agent in each container?
…which now has some kind of remote term capability in AWS web console
run tmate in every container.
oh i wonder if we could do that
that would be hot
let me know what you come up with
hey everyone give a warm welcome to @joshmyers! Good to have you here
hey @joshmyers
hey everyone give a warm welcome to @ivodvb! Good to have you here
Howdy!
I’ve been following CloudPosse work for a while but just came across geodesic, which looks awesome. How battle tested is it? Am OOO atm but looking forward to getting hands on with it ASAP
we’ve been using it for many projects and clients. We are constantly improving it, but it mostly does what we want. @Erik Osterman (Cloud Posse) has more to say about it and future plans
hey everyone give a warm welcome to @ndobbs! Good to have you here
@joshmyers it’s currently in use a handful (7+) organizations to power their production, staging, dev, etc. environments
it’s actively maintained, supported and receiving a continuous stream of enhancements and updates.
the power here is that by collaborating on a similar base infrastructure, we all benefit. i’ve been absolutely amazed by all the wonderful contributions (and bug fixes!) we continually receive from the community. would be happy to share more details…..
hey everyone give a warm welcome to @Ion! Good to have you here
welcome @Ion!
hey everyone give a warm welcome to @lvh! Good to have you here
awesome! great to see you here @lvh
hey everyone give a warm welcome to @jsanchez! Good to have you here
Hey Jesse!
hi. saw the slack channel on a terraform module. figured to join the group.
awesome! well definitely join the #terraform channel. which module, btw?
Terraform Module to Define Vanity Host/Domain (e.g. [brand.com](http://brand.com)
) as an ALIAS record - cloudposse/terraform-aws-route53-alias
Cool, what are you working on?
old project at work. cleaning up some old code and trying to use modules already out there.
hey everyone give a warm welcome to @Vi! Good to have you here
Hi everyone
hey @Vi! what brings you around?
I am a university graduate, I have started with terraform past week.
I was looking for automating the creation of cloudfront, route53 records and creation of certificate and I found this github which has helped me alot.
so you found our terraform-aws-cloudfront-*
modules?
yes
hey everyone give a warm welcome to @sigafoose! Good to have you here
tanks a latte
@sigafoose what are you up to?
<– just joined a devops team
im in info overload mode
yea, there’s a lot to pick up
what’s the tech stack?
we are still working on that
right now we’ve decided on aws
terraform/ansible is about all I am aware is in stone
your teams git is a really good resource
thanks! @sigafoose hope you get some good use out of our modules.
2018-10-19
Hi @Erik Osterman (Cloud Posse). I had a blats watching the video you did with codefresh on unlimited staging environments. bookmarked it and wanted to rewatch it now but looks like its no longer available on youtube. any idea where i can find it?
it used to be here: https://www.youtube.com/watch?v=HJ7HkiGTQ48
How to run complete, disposable apps on Kubernetes for Staging and Development What if you could rapidly spin up new environments in a matter of minutes entirely from scratch, triggered simply by the push of a button or automatically for every Pull Request or Branch. Would that be cool? That’s
@tobiaswi ^
This is really nice, we did same thing using spinnaker though
@rohit.verma I’d like a demo
let’s sync up next week
Sure, my laptop screen broke yesterday night. Will connect back when repair complete. On a temp laptop now
hey, when do you want to have the demo?
i will explain what we needed and how we did
@tobiaswi thanks for bringing that to my attention !
codefresh will be reuploading the video.
That is awesome. Thank you so much. We are currently building out our ci/cd on codefresh and your video was a huge inspiration for me
Thanks so much! We have a lot of updates since then - including automatic destruction when PR closed
I am going to reach out to Codefresh to find out what happened.
There is a new video, but it’s only 10 minutes
hey everyone give a warm welcome to @Jay! Good to have you here
Hey fellers!
Hola @Jay!
hey, I had a question for you guys, is there anyway to do bind mounts with your “terraform-aws-ecs-container-definition” module?
I’ve been going through the readme but I don’t see mounts mentioned anywhere
oh hrm….
so we’re using it with Fargate mostly, so maybe we didn’t add the fields for that
@Jay if you want to open a PR, we’ll promptly review it
sure thing! it would be awesome if I could bind mount volumes using the same logic as the port mapping works
hey everyone give a warm welcome to @joe! Good to have you here
2018-10-20
hey everyone give a warm welcome to @Nate! Good to have you here
hey everyone give a warm welcome to @sixarm! Good to have you here
hey everyone give a warm welcome to @George! Good to have you here
2018-10-21
hey everyone give a warm welcome to @SigmundFried! Good to have you here
Welcome @SigmundFried! What brings you around?
hey everyone give a warm welcome to @EinavF! Good to have you here
@EinavF welcome!
thank
hey Erik ! I just recently discovered Cloud Posse SweetOps and I have to say it is very inline with my current role
awesome! good to hear it
What are you working on?
A fairly substantial cloud transformation that includes HPC on AWS
moving towards containerization
Biotech
Cool - so using Terraform presumably?
more and more
Are you using ECS or EKS (or kops)?
currently ECS but kops is pretty alluring
kinda what brought me in
yea, kops >> EKS >> ECS
kops does a great job of simplifying the long-term maintenance of kubernetes on AWS.
simplifying the long-term maintenance of kubernetes on AWS you read my mind
of course it’s programming people that is always the initial challenge. That’s where I am knee-deep right now
specially the sciency types
haha, yea, getting buy-in is the most important part.
are you using Tensorflow?
no
not yet anyhow
GPUs?
yes
LOTS
cool
protein engineering takes tons
yea, that’s one area it really shines… making it easy to autoscale GPU node pools
all the way from 0 to N
ya so we have (historically) hand rolled our own auto-scaling solutions but I am convinced there is a far better way
or at least a better way to roll em
yea, it’s basically reinventing the wheel. kubernetes is basically a cloud framework, like rails is a web framework
exactly
preaching to the choir !!
hehe
haha
well, if you ever need more material - hit me up
I will for sure…. Thanks !!
preaching is my pasttime
coo
hey everyone give a warm welcome to @catdevman! Good to have you here
@catdevman thanks for the PRs! I just want to have @Andriy Knysh (Cloud Posse) review them tomorrow as well. Also, as you’ve probably noticed, GitHub is barely functional right now
2018-10-22
@Erik Osterman (Cloud Posse) My pleasure. I have been using FOSS for years now and I am just glad to have found a place that I can give back in.
I was looking at what you do at CloudPosse and I really like the idea of being part of an organization like that even if it is just adding/editing some Terraform code on a repo
You found your spot! In addition to accepting nearly all PRs, just to let you know we also accept module donations as well - so long as they conform to some of our standards (like using the terraform-null-label
or terraform-terraform-label
modules)
Okay, for now the modules I have written are pretty specific to the company I work at. I am working on making them more configurable and generic picked up a few lessons from Gruntwork talk https://www.youtube.com/watch?v=LVgP63BkhKQ. I am even looking into adding some tests with terratest https://github.com/gruntwork-io/terratest. (I hope that Gruntwork isn’t a curse word to you all at CloudPosse)
Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. - gruntwork-io/terratest
As a remote DevOps Engineer that is a single guy at the company I am at, managing all our servers from EC2 to RDS any help I can get is much appreciated!
Yea exactly! There are so many engineers as yourself in one-man silos. It’s so important to be able talk shop and bang ideas off peers in order to grow in the position.
For sure. It is funny my background was systems/networking engineer (Linux - Cisco and HP mostly, some Motorola wireless controller stuff as well) then I started working in Software Development and now I found the happy middle ground. I would love to find out more about the process that CloudPosse goes through… I have found it very difficult to get buy-in on true DevOps. Non-Siloed teams where they can deploy their own code whenever they would like and get features out faster… and as someone that worked in Software Development I know I would like to have that power if all it took was learning a few extra tools but management seems to be the stumbling block for me. I have found that I really dislike the title DevOps Engineer… like makes it seem like one group/person is suppose to take care of a lot more then what is possible.
sure - would be happy to share some thoughts on that. you can always schedule time with me here: https://calendly.com/cloudposse/
hey @catdevman and welcome
GitHub’s database has suffered network partition. https://www.theinquirer.net/inquirer/news/3064898/github-down-major-outage They clearly use a distributed system, in which according to CAP theorem you can have only two out of three:
Consistency: Every read receives the most recent write or an error Availability: Every request receives a response that is not an error Partition tolerance: The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes
They chose CP (no availability)
https://en.wikipedia.org/wiki/CAP_theorem https://towardsdatascience.com/cap-theorem-and-distributed-database-management-systems-5c2be977950e https://jepsen.io/analyses https://github.com/aphyr/partitions-post https://www.consul.io/docs/internals/jepsen.html https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions
seems like this time consistency was an issue too
There are no events this week
hey everyone give a warm welcome to @serhat! Good to have you here
I guess they viewed data loss as worse than downtime
hey everyone give a warm welcome to @sprutner! Good to have you here
Hey @sprutner!
what are you working on?
I was trolling through the issues on Atlantis and saw some thoughtful commentary by you and some enhancements you’re working on
Oh cool! Yea, we’re having a lot of fun with Atlantis. We have an experimental fork going on. Hoping that we can get some of these features upstreamed eventually.
Yeah Cloudposse rung a bell, not sure where I heard of you before. But why not say hello.
Hrmm… maybe some of our terraform modules?
Where are you based?
I’m in SF
hey everyone give a warm welcome to @steve! Good to have you here
@Erik Osterman (Cloud Posse) created a new channel #atlantis. Join if this sounds interesting!
hey everyone give a warm welcome to @Benn! Good to have you here
hey everyone give a warm welcome to @Wes! Good to have you here
hi @Wes, welcome
Hey @Wes!
I’ve been pulling a lot of your terraform modules so I had to drop by and say hello
Day 2 of Terraform…I think I understand it?
haha
awesome - glad you’re enjoying them!
haha
yea - there’s a lot of to learn
yeah I need to move my AWS account from one to another so I figured this may be a good time to learn how to automate building out AWS accounts
yea, it’s a bit of upfront investment
yeah and will be helpful when we need a QA env set up quickly and what not
hey everyone give a warm welcome to @seses! Good to have you here
2018-10-23
HashiCorp announces improved configuration language, remote operations, and new, free collaboration features for HashiCorp Terraform….
Atlantis Team Joins HashiCorp
Atlantis is an open source tool designed to help teams collaborate on Terraform. It provides a workflow for reviewing and executing Terraform directly from GitHub pull requests.
Anubhav Mishra and Luke Kysow are the engineers behind Atlantis. Mishra started the project at Hootsuite and has been a developer advocate at HashiCorp for the past year. Luke joined Mishra to help open source the project and has been leading the project for the past nine months.
Over the past few months, we have had many discussions between Mishra and Luke and the folks on the Terraform team here at HashiCorp. Through these conversations, we have come to understand that we have a shared vision for providing solutions for Terraform collaboration for teams large and small.
Today we are pleased to announce that both Mishra and Luke are on board as HashiCorp employees and we'll be working together to solve Terraform collaboration for everyone. In the near term, nothing will change for Atlantis and its users. Luke will continue to maintain Atlantis, review pull requests, triage issues, and write code.
We are still working out the details of how Atlantis will fit into the Terraform portfolio, but whatever direction we take, we're committed to keeping Atlantis functionality free and open source.
Atlantis Maintainer joins HashiCorp
@Erik Osterman (Cloud Posse) created a new channel #terraform-0_12. Join if this sounds interesting!
hey everyone give a warm welcome to @Jeremy! Good to have you here
Hi
am looking for help on tomcat puppet module
I am trying to create tomcat image using packer sourcing puppet tomcat module. It is throwing an error -unknown resource type: “concat”
I understand the tomcat module is missing dependency of concat & stdlib which I included then in .fixtures file as fixtures: repositories: stdlib: “https://github.com/puppetlabs/puppetlabs-stdlib.git” concat: “https://github.com/puppetlabs/puppetlabs-concat.git” archive: “https://github.com/voxpupuli/puppet-archive.git” augeas_core: “https://github.com/puppetlabs/puppetlabs-augeas_core.git”
Puppet Labs Standard Library module. Contribute to puppetlabs/puppetlabs-stdlib development by creating an account on GitHub.
File concatenation system for Puppet. Contribute to puppetlabs/puppetlabs-concat development by creating an account on GitHub.
Compressed archive file download and extraction with native types/providers for Windows and Unix - voxpupuli/puppet-archive
Manage files using Augeas. Contribute to puppetlabs/puppetlabs-augeas_core development by creating an account on GitHub.
am I missing something
welcome @praveen!
Hello Erik
I can’t speak for others, but cloudposse doesn’t support any classic configuration management systems (cfengine, salt, chef, ansible, puppet, etc)
we’re using strictly terraform+containers or kubernetes+containers
maybe checkout the hangops
slack team
sure, thank you for letting me know erik
will checkout hangops
though many here use packer, so you can probably get some tips on that
packer is running fine Erik. Thank you very much for the swift responce
hey everyone give a warm welcome to @fnova! Good to have you here
2018-10-24
@Erik Osterman (Cloud Posse) created a new channel #terragrunt. Join if this sounds interesting!
Hi, I am looking for information on how to source passwords from azure keyvault using remote-exec(terraform). Basically I will have to copy a property file to the server which I will source it from git. I do not want the properties file to have sensitive information like secrets/ passwords. so I would want to append the file to password/secret’s from keyvault in azure platform
Who will be the 200th SweetOps member.. 7 more to go
2 more to go :-)
i’m not counting the bots @Erik Osterman (Cloud Posse)
Haha
hey everyone give a warm welcome to @Bong Aquino! Good to have you here
2018-10-25
hey everyone give a warm welcome to @s.p.i! Good to have you here
hey @s.p.i welcome
hi
what are you working on?
As a follow-up to the HashiCorp Consul plus Kubernetes blog series, this blog highlights our recent work to provide first-c…
Today at HashiConf 2018 in San Francisco, we are announcing major updates across our entire suite of open source and enterprise products. Our mission…
Lol
hehe
Is there a way to get an ECS task to assume a foreign role (as in from a different account) as the task role? I think the answer is no because the way you make that work is you add a trust policy that allows ecs-tasks.amazonaws.com to assume the role, but i don’t own the role
@lvh you can give the role the permission to assume the role of a different account
Trust policies on services only work within the same account afaik.
What are you trying to achieve ?
I want to run an ECS container in my AWS account with permissions to view infra; I want to view said infra from my account
ideally I would do that with the application itself being entirely role-oblivious
I don’t think I follow, what do you mean by infra ?
some infrastructure; could be anything
imagine I’m deploying something with terraform in a foreign account
(actually what I’m doing is auditing the foreign account, but who cares – there’s a foreign role that has some permissions that software I control needs; I would like to run said software on ECS in my account)
Yes, that should be possible
but ECS already has a concept of giving some permissions to a task: it’s just that it expects that role to be in the same account
Datadog for example does this
when you say should be possible you mean possible by explicitly assuming a role inside the application? or by assigning it to ECS directly
And it’s a pattern they recommend with Atlantis to assume role into foreign accounts
They application needs to support it
However!!
One Sec
You can do use this: https://github.com/cloudposse/packages/issues/85
what Add simple assume-role utility why For CI/CD contexts with codefresh, where aws-vault is not suitable references https://github.com/remind101/assume-role/
This will facilitate that programmatically I such a way you get a shell that has access to that env.
But if your are writing something from the ground up, you should just incorporate it into your app
ah neat, I’ve been using aws-vault but obvs “keychain” makes less sense in a container
Right, this is like AWS vault but for services rather than humans
well all I really need is something that takes the output of aws sts assume-role and sets some env vars which sounds pretty doable
it’s also easy to script with just aws sts
jynx
that tool suggests I configure ~/.aws/config
– one would hope it automagically works with the metadata service or env vars too because it just uses the aws sdk
yeah I’ll probably just script this manually
well; thanks
It just uses go sdk for aws
Which works with metadata api
AWS config is probably for local dev (which aws-vault is better suited for). Shouldn’t be required. I was looking to do exactly what you are doing which is what led me down this path
AWS cli for STS falls short of what I would want from it.
It’s like curl for AWS api. Great that it returns json
, but if I have to “program” to use it, I might as well use the library for my language
That’s why I like the cli approach that provides me an operational environment. I don’t know why aws cli for sts still doesn’t support that exec capability.
I say this after having used it before I found the simple cli tools
:-)
Easily assume roles that require MFA on AWS. Contribute to cloudposse/aws-assumed-role development by creating an account on GitHub.
2018-10-26
hey everyone give a warm welcome to @Kenny Inggs! Good to have you here
Hi Kenny, what are you working on ?
hey everyone give a warm welcome to @Jake Lundberg (HashiCorp)! Good to have you here
whoot! welcome @Jake Lundberg (HashiCorp); thanks for stopping by
#terraform and #terraform-0_12 are our hashicorp specific channels at this time.
hey everyone give a warm welcome to @gk! Good to have you here
Welcome @gk! Anything we can help out with?
2018-10-27
hey everyone give a warm welcome to @Dombo! Good to have you here
Welcome @Dombo ! Working on the weekend?
hey everyone give a warm welcome to @hemanth_jayaraman! Good to have you here
Welcome @hemanth_jayaraman !
2018-10-28
hey everyone give a warm welcome to @Pablo Costa! Good to have you here
hey everyone give a warm welcome to @OCHOA! Good to have you here
2018-10-29
@Erik Osterman (Cloud Posse) just doing some contract work outside of the normal employment. Evaluating atlantis currently actually! Been a terraform user for a year or two now.
Hi @maarten & @Erik Osterman (Cloud Posse) - sorry, I joined last week, but only now got time to read through things. I’m co-founder of two new fin-tech startups in Cape Town South Africa, and we’ve been putting a lot of effort into “doing things right”. Heaps of Terraform, Docker and Serverless stuff (soon to move more towards k8s) in various guises. I kept on bumping into the SweetOps stuff everywhere I go, so decided to look at it in earnest. I feel a little like the local woodworker who became better than anybody else in the village, and then visits the master woodworkers in Japan, only to realise how little he really knew about woodworking. I’m now seriously considering scrapping much of what we did and embracing the SweetOps way. Maybe with Atlantis thrown in. Planning on working through the cold start in an isolated environment within the next week or two.
Haha, well, you’re welcome here! we all learn/grow from each other
did you hear AWS is launching a new region in south africa?
Last year we launched new AWS Regions in France and China (Ningxia), and announced that we are working on regions in Bahrain, Hong Kong SAR, Sweden, and a second GovCloud Region in the United States. South Africa in Early 2020 Today, I am happy to announce that we will be opening an AWS Region in […]
btw, this was just such an awesome analogy. I think we’re going to have to borrow it.
I did hear about aws in Cape Town yes. We’ve been eagerly anticipating it. The only let-down is that it will only be in 2020. So Microsoft will be here way before them.
And you are of course welcome to borrow it.
hey everyone give a warm welcome to @Jon Monts! Good to have you here
hi @Kenny Inggs nice to meet you
welcome @Jon Monts
Thanks, just discovered the site cloud posse site. Nice site thus far, I am I am ingesting it all.
nice, let us know what you are working on, can help you find more info to ingest
hey everyone give a warm welcome to @granville! Good to have you here
There are no events this week
welcome @granville
Has anyone seen Fargate errors a la: STOPPED (CannotStartContainerError: API error (500): failed)
I have no idea what’s up with that; I gave the task role AdministratorAccess and put it in a completely permissive SG just in case, no dice
logs are empty
happens repeatedly
I haven’t seen that particular error. Fwiw, we’re running #atlantis in ECS/Fargate using AdministratorAccess
and using this tool: https://github.com/jpignata/fargate
CLI for AWS Fargate. Contribute to jpignata/fargate development by creating an account on GitHub.
Usually, when I don’t see any logs it’s because the task doesn’t even attempt to start which is b/c I use the wrong image tag or don’t have permissions to pull the image
@maarten might have some other ideas
Outbound sg rules should be permissive
Otherwise check your routing
And if you think ECR might be the problem, try nginx:latest as image to rule that out.
hey everyone give a warm welcome to @ALI! Good to have you here
Another thing to make sure is that the cloudwatch log group exists and that the task definition has those properties setup correctly including the correct region. Can’t think of anything else.
@Erik Osterman (Cloud Posse) created a new channel #test. Join if this sounds interesting!
2018-10-30
hey everyone give a warm welcome to @nukepuppy! Good to have you here
hey everyone give a warm welcome to @mmarseglia! Good to have you here
Hey there! Whatcha up to?
hi @mmarseglia
working with some of these cloudposse modules to build a s3/cloudfront cdn
Aha! Bummer about the production fire
I know there’s an open issue on one of the CDN modules related to regional s3 endpoints
Let me know if you run into that
@mmarseglia if your S3 bucket is a website, here is a working example using S3/CDN that we use for our docs (https://docs.cloudposse.com/): https://github.com/cloudposse/terraform-root-modules/blob/master/aws/docs/main.tf
Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules
In-depth analysis of the incident that impacted GitHub services on October 21 and 22.
2018-10-31
hey everyone give a warm welcome to @Maycon Santos! Good to have you here
hey @Maycon Santos
hey everyone give a warm welcome to @tchia04! Good to have you here
welcome @tchia04
Hey @Andriy Knysh (Cloud Posse)
welcome!!