#packer (2018-11)
Discuss Packer for building AMIs and Docker Images
Archive: https://archive.sweetops.com/packer/
2018-11-30
![Erik Osterman (Cloud Posse) avatar](https://secure.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0023-72.png)
@Erik Osterman (Cloud Posse) has joined the channel
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
@rohit has joined the channel
![tamsky avatar](https://avatars.slack-edge.com/2019-10-31/817094217669_6e765cea39b456597957_72.jpg)
@tamsky has joined the channel
![catdevman avatar](https://secure.gravatar.com/avatar/08dd2e177a984cc989a254eeb5479a4a.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
@catdevman has joined the channel
![rohit.verma avatar](https://secure.gravatar.com/avatar/2fe004659fb5ecb920681f93103d6957.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
@rohit.verma has joined the channel
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
so i am just thinking about creating a custom ami off of redhat ami in aws using packer
![tamsky avatar](https://avatars.slack-edge.com/2019-10-31/817094217669_6e765cea39b456597957_72.jpg)
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
anything i should know before i go down that path ?
![tamsky avatar](https://avatars.slack-edge.com/2019-10-31/817094217669_6e765cea39b456597957_72.jpg)
I’m not a huge fan of humans reading or writing json; I use a python tool that generates ephemeral packer.json
files from version-controlled packer.yaml
https://gist.github.com/tamsky/c7df19684e5605023f2e
biggest additional feature that tool has is: it supports !include
which encourages good re-use of code.
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
what happens when you say !include
?
![davidvasandani avatar](https://avatars.slack-edge.com/2019-10-02/784259469622_7d9e31719822afd94ef8_72.jpg)
@davidvasandani has joined the channel
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
@joshmyers has joined the channel
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
@rohit what are you wanting to accomplish?
![jsanchez avatar](https://secure.gravatar.com/avatar/7cb981d651cde886b5e22343615dfe29.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0025-72.png)
@jsanchez has joined the channel
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
@joshmyers we are planning to encrypt our root volume and by default it is not
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
so we are planning to create a custom ami off of redhat ami and also apply yum updates, install aws cli, chef-client so that when there is an autoscaling event it doesn’t take lot of time
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
so we are planning to solve 2 problems with this solution
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
are going you going build in one account and distribute to others, or build in all accounts and regions?
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Have done before and it was a royal PITA (namely because of some of the IAM policies we had in place)
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
AMIs are region and account specific as are the KMS keys that are used
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
At the time we had to write a wrapper script with packer to: provision new AMI and bootstrap, take that underlying EBS volume and create a new AMI out of that
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
with a custom KMS key to allow for cross account policies
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
(you couldn’t share a default KMS key across accounts AFAICR)
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
I believe that this has been built natively into packer now though, not digged to see what it is doing under the covers
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Note that shunting AMIs around is slooooowww
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
are you planning on doing this for each Chef role you have? There is overhead in managing all those AMIs (pruning script or the like)
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
how do your cookbooks end up on the instances in the ASG?
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
this will be for one aws account but in multiple regions
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
we would only install chef-client but the cookbook will end up later when the ami is used in a launch template
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
makes sense ?
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Yeah, you want to block for the entire first chef run on a node before it is provisioned
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
You are going to have to shunt AMIs around regions
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
How long does your Chef bootstrap run take?
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
chef bootstrap currently takes somewhere aroung 8-10 mins
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
because the user data includes yum updates, instaling awscli, installing chef-client
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
by creating the custom ami we are planning to reduce this time
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
now that you know my usecase, is there an example that you could recommend ?
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
well, if you are going down this route, may as well do a first chef run during packer build, right?
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Even if you don’t do it for each role you have, there is a lot common base across all I’m sure.
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Can take an initial chef run from 10mins to 1 min or so
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Basically updates and anything on top of base
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
no, we don’t want to do that because then we will have to create an AMI for each app
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
how much do you care about it being the exact same artefact in different regions/accounts vs building in each
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
each app or each role?
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
even if you do base, the common denominator between all your roles will shave some time off that first run
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
Also not suggesting baking apps into the AMI
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
or do you use Chef runs for app deployments?
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
we use chef run for app deployments
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
You may want to split that out.
![rohit avatar](https://secure.gravatar.com/avatar/96545ffc5c19a46414f41c76b28d2944.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
we use chef to pull/extract artifact from artifactory
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
OK, I’d look into benefits of splitting out that deployment process
![joshmyers avatar](https://avatars.slack-edge.com/2018-11-20/483958217281_8117d6f6c62807ce9912_72.jpg)
do a base role during packer build and have cloud-init pull in the extra role to add on?
![tamsky avatar](https://avatars.slack-edge.com/2019-10-31/817094217669_6e765cea39b456597957_72.jpg)
do a base role during packer build and have cloud-init pull in the extra role to add on?
I’ve typically followed this pattern as well, @rohit – a base image, also built by packer, that other app-specific images use as their source_ami
when they get built in packer.