#terraform-aws-modules

Discussions related to https://github.com/terraform-aws-modules Archive: https://archive.sweetops.com/terraform-aws-modules/

2019-11-12

SlackBot
09:06:25 PM

This message was deleted.

2019-11-07

Alex Siegman

Anyone have a good base module for a redshift cluster? i could have sworn cloudposse had one, but i’m not seeing it in their github

aknysh

no we did not have one

aknysh

but easy to implement using the rds-cluster module

aknysh

should be exactly the same except a few new resources https://www.terraform.io/docs/providers/aws/r/redshift_cluster.html

AWS: aws_redshift_cluster - Terraform by HashiCorp

Provides a Redshift Cluster resource.

Erik Osterman

We recently implemented EMR

2019-11-06

Saichovsky

Greetings people

Saichovsky

I’m new here with a burning question on terraform <> AWS

Saichovsky
resource "aws_kinesis_analytics_application" "app" {
  name = var.analytics_app_name
  tags = local.tags

  // TODO: need to make inputs & outputs dynamic -- cater for cases of multiple columns, e.t.c.
  //  inputs {
  //    name_prefix = ""
  //    "schema" {
  //      "record_columns" {
  //        name = ""
  //        sql_type = ""
  //      }
  //      "record_format" {}
  //    }
  //  }
  //  outputs {
  //    name = ""
  //    "schema" {}
  //  }
}

Here’s my quandary

Saichovsky

number of columns is not fixed

Erik Osterman

(perhaps try #terraform as it doesn’t seem like this is related to https://github.com/terraform-aws-modules)

Terraform AWS modules

Collection of Terraform AWS modules supported by the community - Terraform AWS modules

1

2019-11-04

Yuriy

Hi! I use this module https://github.com/cloudposse/terraform-aws-dynamic-subnets and can’t figure out how to allow connections from public subnets to private ones (I am setting up bastion host) - what am I missing?

cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

Erik Osterman

This is not a http://github.com/terraform-aws-module (which this channel is for)

cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

Yuriy

Sorry, I misread

Erik Osterman

Better to just use #terraform which is a catch all for everything else

Yuriy

sure

Erik Osterman

Ya, it’s confusing. :-)

Erik Osterman

Make sure they are all in the same route table

Erik Osterman

Make sure your bastion’s security group permits the relative networks

Yuriy

I can ssh to bastion already

Yuriy

RDS security group allows incoming 5432 port (postgres)

Yuriy

but psql connection from bastion on that port times out

Erik Osterman


RDS security group allows incoming 5432 port (postgres)

Erik Osterman

via network CIDR ACLs or security group id of the bastion?

Erik Osterman

and the route table for the vpc has routes for both public and private subnets?

Yuriy

No, I didn’t explicitly create a routing table for vpc

Yuriy

Did I understand you correctly, that by adding routes to the VPC routing table for public and private subnets I’ll make it possible to connect from EC2 instance in public subnet to the RDS in private?

Erik Osterman

yep, you got it

Yuriy

RDS security group allows incoming 5432 port (postgres) via RDS security group (I didn’t explicitly add ACLs and as far as I can see - they allow everything)

Yuriy

Thank you Erik, I’ll look into the routing table for the VPC

Erik Osterman

hrmm…

Erik Osterman

actually, I might have misspoke. it’s been a while since I looked at this module.

Erik Osterman

looks like it creates both public/private subnets at the same time.

Yuriy

yes

Erik Osterman

ok, so looksl ike we create a public route table, and a private route table

Erik Osterman

and we don’t permit routing out of the box between them

Yuriy

Aha

Erik Osterman

I guess there’s not much profit by keeping them seperate if we at routes to both

Yuriy

yeah, what I actually want is for private nets not to be accessed from the internet but only from pubnets

Yuriy

maybe even from specific EC2 ip

Erik Osterman

I think what we did was use an ELB/TCP in public

Yuriy

(bastion)

Erik Osterman

and then deploy bastion in private

Erik Osterman

ELB was pointed at bastion

Yuriy

I c

Yuriy

Thanks, than my plan is to move bastion from pub to priv net, add elb and point it to the bastion

Yuriy

I’ve just found something else:

Yuriy

Session Manager is a managed service that provides you with one-click secure access to your instances without the need to open inbound ports and manage bastion hosts. You have centralized access control over who can access your instances and full auditing capabilities to ensure compliance with corporate policies.

have you guys run into any issues with this release of terraform-aws-vpc? https://github.com/cloudposse/terraform-aws-vpc/releases/tag/0.8.0

cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

when we upgraded to TF 12 and applied this module it removed all of our default security group rules, which included rules to allow egress traffic

according to the docs

When Terraform first adopts the Default Security Group, it immediately removes all ingress and egress rules in the Security Group. It then proceeds to create any rules specified in the configuration. This step is required so that only the rules specified in the configuration are created.

I understand best practice is to not use the default security group, but in our case we have to because of legacy infrastructure

seems like this should have been a 1.0.0 release instead of a minor release with some documentation in the readme describing how the new version will remove any SG rules on your existing default SG

Erik Osterman

We don’t have a single 1.0 module :-)

Erik Osterman

In semver pre-1.0 has a special meaning

Semantic Versioning 2.0.0

Semantic Versioning spec and website

do you guys plan on releasing any 1.0.0 modules? or is that too much effort to maintain for you guys?

Erik Osterman

With even terraform itself being pre-1.0 there is some debate as to if it is even possible ;)

Erik Osterman

But it’s largely because they way all this fits together it’s really hard to solve the calculus of when to release 1.0. It’s a judgement call at best. I think it should be somewhat automatic. If we don’t need to change the interface for a module in 12 months, then cut 1.0 or something

Erik Osterman

Open to discussion

ahhhh okay that makes sense

Erik Osterman

The change came about because we are working to achieve compliance with CIS Benchmarks

Erik Osterman

The default group was flagged

aknysh

that needs to be fixed, maybe by adding default_security_group_enabled flag

2

i think a variable will also be needed to allow the user to pass in any ingress and egress rules they need

This resource treats its inline rules as absolute; only the rules defined inline are created, and any additions/removals external to this resource will result in diff shown. For these reasons, this resource is incompatible with the aws_security_group_rule resource.
2
aknysh

you mean to the default SG?

2

correct

2
aknysh

maybe better to expose the default SG ID as output

2
aknysh

so the user could do whatever they want with it

2

i don’t think that’ll work based on the docs… where it’s saying the inline rule is treated as the source of truth and aws_default_security_group is incompatible with aws_security_group_rule

2

at least that’s how i interpret what they’re trying to say

2
aknysh

so yes, I agree. What was done in https://github.com/cloudposse/terraform-aws-vpc/blob/master/main.tf#L22 is not usable w/o inline rules since you can’t use external rules with default SG

cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

2
aknysh

and default SG is created in a VPC

2
aknysh

so for that to be usable, we need to provide variables to specify ingress and egress inline rules

2
aknysh

we did not catch that (visually and in CI/CD tests) b/c we actually don’t use the default SG in VPCs (always create SGs specific to resources)

2

that makes sense… ideally we wouldn’t be either but unfortunately we are

2
aknysh

On the other hand, that was created specifically for CIS probably to completely restrict the default SG

2
aknysh

So adding the inline rules back prob defeats the purpose

2
aknysh

enabled flag might be better

2

oh yeah i agree with that

2
aknysh

So TF will not touch the default SG at all if not needed for CIS

2

with the enabled flag the user can manage the default SG outside of the module and do as they wish

2

i also don’t think the way terraform implemented this is very intuitive either … this is the only case i’ve seen where they automatically import the resource and remove things from it

aknysh

it’s better to specify default egress in any SG

Erik Osterman
Update: Issue affecting HashiCorp Terraform resource deletions after the VPC Improvements to AWS Lambda | Amazon Web Services

On September 3, 2019, we announced an exciting update that improves the performance, scale, and efficiency of AWS Lambda functions when working with Amazon VPC networks. You can learn more about the improvements in the original blog post. These improvements represent a significant change in how elastic network interfaces (ENIs) are configured to connect to […]

Erik Osterman

But I don’t think related

agreed, in our case we had used a previous version of the module so it implicitly created the egress, so when we updated terraform removed it

2019-11-03

navdeep

Hey guys, I have been using cloudposse for getting our emr cluster up, though I am able to get my cluster up in our vpc , but our autoscaling policy is unable get triggered with the error Failed to provision the AutoScaling policy: Unable to assume IAM role: arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">iam:role/emr-stage-dataorc-emr-ec2-autoscaling , though in the module <https://github.com/cloudposse/terraform-aws-emr-cluster> i can see most of the assume policy requirement given, any direction I can look up or something I am missing

navdeep

i gues i found the issue , below policy requires one more trustee <http://application-autoscaling.amazonaws.com>

data "aws_iam_policy_document" "assume_role_ec2" {
  count = var.enabled ? 1 : 0

  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["<http<i class="em em-//elasticmapreduce.amazonaws.com\|elasticmapreduce.amazonaws.com>","<http"></i>//application-autoscaling.amazonaws.com>"]
    }

    actions = ["sts:AssumeRole"]
  }
}
navdeep

good for a PR ?

aknysh

thanks @navdeep, we’ll review it

2019-11-02

Erik Osterman

Yes see our terraform module for hosting an s3 website

1
Erik Osterman

Oh with cloud front.. we have something for that too

Erik Osterman
cloudposse/terraform-aws-cloudfront-s3-cdn

Terraform module to easily provision CloudFront CDN backed by an S3 origin - cloudposse/terraform-aws-cloudfront-s3-cdn

Erik Osterman

I think that does what you want

Erik Osterman

(Not a http://github.com/terraform-aws-module though which is what this channel is for)

2019-11-01

Sharanya

Hey Guys - Quick Question — if any one has every come across this “Is there any way we can make our S3 bucket Private and then Cloudfront provides - URL , Which can be viewed from this Private S3 Bucket” ?

    keyboard_arrow_up