#office-hours (2024-04)

cc @Yonatan Koren

It’s important to know what the assumptions are as well • What flavor of Kubernetes (e.g. EKS, GKE, AKS, etc) • Are they running a current version of Kubernetes • Is deploying an ALB even viable? E.g. they might already have ALB controller deployed • Is using only helm a requirement? Maybe helmfile will simplify things

I used to work at Codefresh and we had to solve similar issues for delivering on on-prem installation to customers.

You can take a look at the documentation for the on-prem codefresh helm chart: https://artifacthub.io/packages/helm/codefresh-onprem/codefresh

My devil’s advocate response to wanting to package all of your low-level app dependencies for customers is, what happens when you have two customers with different underlying cluster requirements?

For example, when I worked at Codefresh we had to support both aws-load-balancer-controller and nginx-ingress-controller. So we saved these as open ended for the customer to choose, and these served as pre-requisites for our app. Yes it’s less automated and we needed support engineers to help new customers, but it was a complicated architecture and the expectation was that setup needs some manual preparation anyways. So we focused more on ensuring compatibility with different ingress controllers, different public cloud providers, even added support for openshift customers. This was more important to us than to automate everything from end to end only for one type of customer.

So in my opinion, take #1 and #2 out of the equation. This needs to be handled by the customer. This way you can onboard more types of customers who have different k8s environments.

If you can automate #3 and #4 using a Kubernetes resource instead of a script, you definitely should do that. Have you looked into https://external-secrets.io/latest/provider/hashicorp-vault/ ? If External Secrets Operator can sync that vault secret you need for the service account, you don’t need to manually intervene with a script. It will eventually sync and your application will come up. You should also give your customers different options to get this secret into their cluster. Maybe they have it in AWS Secrets Manager. You can instruct them how to set up ESO to sync that secret as well. This ESO resource probably won’t be inside your helm templates, but you will give an option to change the name of the secret inside the SA template. Then they deploy it either in templates/ with your chart as a sub-chart, or just deploy it manually beforehand.

We can discuss this on the office hours call today if you’d like

EKS addons https://github.com/aws-ia/terraform-aws-eks-blueprints-addon

Sorry and thank you for the replies. Our first customer is running kubernetes on VMware TKGI (kind cluster behinds the scenes). So I cannot take advantage of any cloud managed k8s

@Yonatan Koren i am reading your replies. thank you!

Our first customer is running kubernetes on VMware TKGI Hah! Exactly the challenge

So many ways to run k8s. Other considerations that we ran into were exposing web interfaces over TLS without self-signed certs, which required Let’s Encrypt and public DNS names.

As another example, take a look at the Teleport project and how they support their helm installation for different scenarios.

https://goteleport.com/docs/deploy-a-cluster/helm-deployments/

They have a single helm chart but support EKS, GKE, AKS, and others. There are semi-manual steps involved, such as setting up IAM, setting up TLS, setting up DynamoDB, etc. Their helm chart only installs their app, but they walk you through how to install it based on your scenario.

IMO it’s much better to keep new potential customers happy by giving and documenting different options for them to use. One of the worse things that can happen is if a customer wants to install your product on their cluster and you say that they have to use cert-manager instead of AWS Certificate Manager. Or that they have to use aws-load-balancer-controller instead of nginx-ingress-controller. In my experience this leaves them with a bad taste in their mouth, because it makes them feel like they need to do a lot of changes to install your app, and that your app isn’t engineered in a way that had compatibility in mind.

Make sure. you click the link from the invite

I clicked the one in my gmail/calendar. Still says waiting for host to start the meeting. I guess no OH for me today.

Others mentioned that you might have the old invite link

In september 2023, we changed meeting

Aha, Public "Office Hours" is definitely the old invite

That post should be deleted, is actually not accurate

https://x.com/johnrowleyster1/status/1775715773534867577?s=46&t=2r8dMaTQPp5tQwJLhNtqvw

That post should be deleted, is actually not accurate the InfoWorld post or this Slack post?

not yours, the infoworld post

or it should be corrected

we should discuss it in office hours

Will do!

https://www.linkedin.com/posts/opentofuorg_opentofu-project-was-recently-made-aware-a[…]?utm_source=combined_share_message&utm_medium=member_android

https://sweetops.slack.com/archives/C063TG2DYTC/p1712851827367829

Redacted Cease & Desist letter https://opentofu.github.io/legal-documents/2024-04-03%20HashiCorp%20C%26D/OpenTofu%20C&D%20-%20Redacted.pdf

This quote is from OpenTofu: “In the future, if you should have any concerns or questions about how source code in OpenTofu is developed, we would ask that you contact us first. Immediately issuing DMCA takedown notices and igniting salacious negative press articles is not the most helpful path to resolving concerns like this.”

That’s such a reasonable request by OpenTofu and how reasonable parties should operate on both sides.

Yes this is exactly along the lines of what I’m looking for. Thanks for sharing!!

You might also enjoy this other recent thread. It’s not entirely related, but talks about some of the challenges we’ve encountered managed ECS tasks purely with Terraform. https://sweetops.slack.com/archives/CCT1E7JJY/p1712686050689659