#terraform-aws-modules (2019-11)
Terraform Modules
Discussions related to https://github.com/terraform-aws-modules
Archive: https://archive.sweetops.com/terraform-aws-modules/
2019-11-01
Hey Guys - Quick Question — if any one has every come across this “Is there any way we can make our S3 bucket Private and then Cloudfront provides - URL , Which can be viewed from this Private S3 Bucket” ?
2019-11-02
Oh with cloud front.. we have something for that too
Terraform module to easily provision CloudFront CDN backed by an S3 origin - cloudposse/terraform-aws-cloudfront-s3-cdn
I think that does what you want
(Not a github.com/terraform-aws-module though which is what this channel is for)
2019-11-03
Hey guys, I have been using cloudposse for getting our emr cluster up, though I am able to get my cluster up in our vpc , but our autoscaling policy is unable get triggered with the error Failed to provision the AutoScaling policy: Unable to assume IAM role: arn:aws:iam::216727*****:role/emr-stage-dataorc-emr-ec2-autoscaling
, though in the module <https://github.com/cloudposse/terraform-aws-emr-cluster>
i can see most of the assume policy requirement given, any direction I can look up or something I am missing
i gues i found the issue , below policy requires one more trustee application-autoscaling.amazonaws.com
data "aws_iam_policy_document" "assume_role_ec2" {
count = var.enabled ? 1 : 0
statement {
effect = "Allow"
principals {
type = "Service"
identifiers = ["elasticmapreduce.amazonaws.com","application-autoscaling.amazonaws.com"]
}
actions = ["sts:AssumeRole"]
}
}
good for a PR ?
thanks @navdeep, we’ll review it
2019-11-04
Hi! I use this module https://github.com/cloudposse/terraform-aws-dynamic-subnets and can’t figure out how to allow connections from public subnets to private ones (I am setting up bastion host) - what am I missing?
Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets
This is not a github.com/terraform-aws-module (which this channel is for)
Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets
Sorry, I misread
Better to just use #terraform which is a catch all for everything else
sure
Ya, it’s confusing. :-)
Make sure they are all in the same route table
Make sure your bastion’s security group permits the relative networks
I can ssh to bastion already
RDS security group allows incoming 5432 port (postgres)
but psql connection from bastion on that port times out
RDS security group allows incoming 5432 port (postgres)
via network CIDR ACLs or security group id of the bastion?
and the route table for the vpc has routes for both public and private subnets?
No, I didn’t explicitly create a routing table for vpc
Did I understand you correctly, that by adding routes to the VPC routing table for public and private subnets I’ll make it possible to connect from EC2 instance in public subnet to the RDS in private?
yep, you got it
RDS security group allows incoming 5432 port (postgres) via RDS security group (I didn’t explicitly add ACLs and as far as I can see - they allow everything)
Thank you Erik, I’ll look into the routing table for the VPC
hrmm…
actually, I might have misspoke. it’s been a while since I looked at this module.
looks like it creates both public/private subnets at the same time.
yes
ok, so looksl ike we create a public route table, and a private route table
and we don’t permit routing out of the box between them
Aha
I guess there’s not much profit by keeping them seperate if we at routes to both
yeah, what I actually want is for private nets not to be accessed from the internet but only from pubnets
maybe even from specific EC2 ip
I think what we did was use an ELB/TCP in public
(bastion)
and then deploy bastion in private
ELB was pointed at bastion
I c
Thanks, than my plan is to move bastion from pub to priv net, add elb and point it to the bastion
I’ve just found something else:
Session Manager is a managed service that provides you with one-click secure access to your instances without the need to open inbound ports and manage bastion hosts. You have centralized access control over who can access your instances and full auditing capabilities to ensure compliance with corporate policies.
have you guys run into any issues with this release of terraform-aws-vpc? https://github.com/cloudposse/terraform-aws-vpc/releases/tag/0.8.0
Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc
when we upgraded to TF 12 and applied this module it removed all of our default security group rules, which included rules to allow egress traffic
according to the docs
When Terraform first adopts the Default Security Group, it immediately removes all ingress and egress rules in the Security Group. It then proceeds to create any rules specified in the configuration. This step is required so that only the rules specified in the configuration are created.
I understand best practice is to not use the default security group, but in our case we have to because of legacy infrastructure
seems like this should have been a 1.0.0 release instead of a minor release with some documentation in the readme describing how the new version will remove any SG rules on your existing default SG
We don’t have a single 1.0 module :-)
In semver pre-1.0 has a special meaning
today i learned: https://semver.org/#spec-item-4
Semantic Versioning spec and website
do you guys plan on releasing any 1.0.0 modules? or is that too much effort to maintain for you guys?
With even terraform itself being pre-1.0 there is some debate as to if it is even possible ;)
But it’s largely because they way all this fits together it’s really hard to solve the calculus of when to release 1.0. It’s a judgement call at best. I think it should be somewhat automatic. If we don’t need to change the interface for a module in 12 months, then cut 1.0 or something
Open to discussion
ahhhh okay that makes sense
The change came about because we are working to achieve compliance with CIS Benchmarks
The default group was flagged
that needs to be fixed, maybe by adding default_security_group_enabled
flag
i think a variable will also be needed to allow the user to pass in any ingress and egress rules they need
This resource treats its inline rules as absolute; only the rules defined inline are created, and any additions/removals external to this resource will result in diff shown. For these reasons, this resource is incompatible with the aws_security_group_rule resource.
you mean to the default SG?
correct
maybe better to expose the default SG ID as output
so the user could do whatever they want with it
i don’t think that’ll work based on the docs… where it’s saying the inline rule is treated as the source of truth and aws_default_security_group
is incompatible with aws_security_group_rule
at least that’s how i interpret what they’re trying to say
so yes, I agree. What was done in https://github.com/cloudposse/terraform-aws-vpc/blob/master/main.tf#L22 is not usable w/o inline rules since you can’t use external rules with default SG
Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc
and default SG is created in a VPC
so for that to be usable, we need to provide variables to specify ingress and egress inline rules
we did not catch that (visually and in CI/CD tests) b/c we actually don’t use the default SG in VPCs (always create SGs specific to resources)
that makes sense… ideally we wouldn’t be either but unfortunately we are
On the other hand, that was created specifically for CIS probably to completely restrict the default SG
So adding the inline rules back prob defeats the purpose
enabled flag might be better
oh yeah i agree with that
So TF will not touch the default SG at all if not needed for CIS
with the enabled flag the user can manage the default SG outside of the module and do as they wish
i also don’t think the way terraform implemented this is very intuitive either … this is the only case i’ve seen where they automatically import the resource and remove things from it
it’s better to specify default egress in any SG
Also, reminded me of https://aws.amazon.com/blogs/compute/update-issue-affecting-hashicorp-terraform-resource-deletions-after-the-vpc-improvements-to-aws-lambda/
On September 3, 2019, we announced an exciting update that improves the performance, scale, and efficiency of AWS Lambda functions when working with Amazon VPC networks. You can learn more about the improvements in the original blog post. These improvements represent a significant change in how elastic network interfaces (ENIs) are configured to connect to […]
But I don’t think related
agreed, in our case we had used a previous version of the module so it implicitly created the egress, so when we updated terraform removed it
2019-11-06
Greetings people
I’m new here with a burning question on terraform <> AWS
resource "aws_kinesis_analytics_application" "app" {
name = var.analytics_app_name
tags = local.tags
// TODO: need to make inputs & outputs dynamic -- cater for cases of multiple columns, e.t.c.
// inputs {
// name_prefix = ""
// "schema" {
// "record_columns" {
// name = ""
// sql_type = ""
// }
// "record_format" {}
// }
// }
// outputs {
// name = ""
// "schema" {}
// }
}
Here’s my quandary
@Saichovsky see https://sweetops.slack.com/archives/CB6GHNLG0/p1573144102121000?thread_ts=1573113838.112200&cid=CB6GHNLG0
use dynamic blocks https://www.terraform.io/docs/configuration/expressions.html#dynamic-blocks
number of columns is not fixed
(perhaps try #terraform as it doesn’t seem like this is related to https://github.com/terraform-aws-modules)
Collection of Terraform AWS modules supported by the community - Terraform AWS modules
2019-11-07
Anyone have a good base module for a redshift cluster? i could have sworn cloudposse had one, but i’m not seeing it in their github
no we did not have one
but easy to implement using the rds-cluster module
should be exactly the same except a few new resources https://www.terraform.io/docs/providers/aws/r/redshift_cluster.html
Provides a Redshift Cluster resource.
We recently implemented EMR
2019-11-12
This message was deleted.