#release-engineering (2020-09)

my company is doing a small PoC with them. Their pricing is per pod deployed

Ah yah that matches up with what I saw on a rando blog post … which means the annual cost is \(\)$

yes. we currently have a license for 100 pods. I don’t know exactly how much it is but it is on the order of several thousand dollars per year

“several” thousand would be ok, the pricing I saw suggested it would be more like $60k for 150-200 EC2 instances

which would get me laughed out of the room by my VP

Is that 200 permanent EC2 instances for $60K/year? Or do they scale up to 200 as needed? Because a single c5.large is $438/year if you pay upfront, so $60K/year sounds reasonable? Or does their $60K only include their pricing and not the compute cost itself?

they don’t provide the infra

Oh wow. Yeah that’s a different story then.

I dunno if its accurate, but just some googling the other day I found a suggestion that it was priced at $25/node/month

so for 200 nodes that would be 60k annual. Seems a bit absurd to price that way though if they aren’t actually running or managing anything on the node itself

which is why I was hoping someone might have actual real world info (before I reach out and have to sit through a 2 hour sale pitch)

the reason to go with something like harness is needing an enterprise-level centralized deployment strategy, with governance and auditability. If you just want cool deployments something like ArgoCD is probably better to go with

yah thats fine, I’m more looking for “small tier centralized deployment”

have you looked at Codefresh’s CD capabilities?

briefly, but since they lock SSO and > 10 users on enterprise….

Lol. I guess the size of my company has rewired my brain. something that costs ~$10,000 is a rounding error in the contracts we do

If I yank myself back to before when I was at a company with 10 employees I see exactly what you are talking about

yah…. our monthly aws spend is like 16-17k

Might bite the bullet and see how bad spinnaker is to set up

Never done it myself, but I’ve heard its a 900 lb gorilla

Thats what I’ve heard too … sooooo maybe too much for what i need.

@Zach fwiw i think codefresh can bundle SSO into a non-enterprise plan. at least that was the case this time last year

hm ok, I’m just going off their pricing/feature page

its not free still, you just don’t need to be enterprise

Yah I understood. I just mean on their site that SSO is labeled as an enterprise feature.

pricing model per pod sounds atrocious though. its not like harness is doing anything differently deploying a helm chart with 1 pod or 100 pods

I’d rather have that than per user pricing

At least this way it is correlated to usage

One of the reasons we are considering codefresh over cloudbees is cloudbees is per user, and we have like 4000 users

Codefresh is per concurrent pipeline

we have like 4000 pods

but yeah same, one of the pros to chosing codefresh was because they didnt charge us per user. i think that has changed though?

Enterprise is still unlimited users

Looks like pro is 10 users max now

We run Spinnaker with 2 sre for 2 years and it is heavyweight but dunno what else would use. K8s is straight forward but I haven’t been able to use EC2 how I’d want after couple attempts so…

K8s is straight forward but I haven’t been able to use EC2 how I’d want after couple attempts so Do you mean running spinnaker on ec2 vs K8, or using spinnaker to manage ec2 deploys?

We run Spinnaker on k8s and deploy k8s is good but bake deploy ec2 still not done yet.

TL;DR - how are you successfully CICD CloudFormation with multi-account?

sounds about right… approach depends on the app i suppose, and whether the resources utilized can be configured more than once in an account. i.e. you could deploy a complete new instance of the app using the PROD version in the TEST account, then apply the updates from the PR to that stack

probably can’t deploy them more than once in the same account/region. but I could deploy them to a region we’re not using and then destroy

but then I’d get into the whole game of what regions that we’re not using, support everything we’re deploying within that stack

it may also be worth separating portions of the app that can only be deployed once in the account into a different template/stack

for example, enabling config, or setting the iam account alias

yeah, I like that

anything else could be refactored to support this kind of multi-deployment model perhaps

yeah and I could move resources like S3 Buckets, etc to a sibling Stack

stateful stuff with data can certainly be tricky, needs it’s own lifecycle for sure. it’s nice to at least have a mode for an s3 bucket that creates a unique name, or appends a unique id, so it is easy to create loads of wholly new instances on the fly

CDK essentially does that for us

by unique, i don’t necessarily mean “random”. but could be the branch name, for example. or a pr number. or a feature tracking number.

oh, i see what you mean. yeah those resources (to date) haven’t been the issue. but noted.

if you haven’t played with the aws amplify cli and console at all, it can be worth a spin. has modes to create wholly unique instances and backends per branch that are built and deployed as you push. while i’m not using amplify actively right now, working with it helped me reframe some of my ideas and misconceptions and approaches to managing deployments (particularly around cloudformation)

you get support

and the additional cloudbees plugins that they are pretty good

if you are using any of their plugins and you go opensource

then you can’t use them anymore

and the opensource ones do not have the same features

like advance permission plugin they have

Ah got it. Thank you!

np

Is licensed Jenkins the same as CloudBees CI?

mmm I do not know…

I thought the only company that offered jenkins support was cloudbees

It’s all right. I just haven’t found mention specifically of Jenkins license or Enterprrise Jenkins on CloudBee’s website. I only see things like CloudBees CI and CloudBees CD for their products.

They have a product called cloudbees core which I believe is what usually gets talked about when saying licensed Jenkins. In my opinion the biggest advantage to cloudbees core is user management and governance of lots of distributed Jenkins Masters that different teams use. If you have 10 different development teams all running Jenkins themselves however they feel like doing it you can bring all that under one umbrella with cloudbees core.

Ok. got it.

Thanks a lot everyone for your insight.

it would be interesting to see the cost, as I haven’t come across anything that cant be achieved with the open source version of Jenkins. with matrix access and folder based access i have achieved multi tenancy jenkins deployments for 3rd parties, running within a k8 environment means scalability is not an issue, the downside to multiple masters i see is more overhead from a management perspective.. I understand there are probably advantages… just curios to know if this outweighs the cost.

u can try https://github.com/antonbabenko/pre-commit-terraform

Checkov has an exception mechanism - you can add them as comments in the TF files. Is that no good?

Shameless plug - https://www.youtube.com/watch?v=KdH-7_vpFag . I see rather little value in checkov myself, because I need to disable more than I want. Plus some key features are not supported yet (eg, tfvars files, tf.json, modules).