SweetOps #kubernetes for November, 2024

Archive: https://archive.sweetops.com/kubernetes/

2024-11-13

miko

Hello, has anyone here managed to deploy Apache Kafka using Strimzi operator in AWS EKS? I’ve managed to deploy a cluster but I need to expose the consumer’s port to the outside world but I can’t find an example that I could follow

venkata.mutyala

03:35:23 AM

I’ve used it before. I don’t recall using it for that particular use case though. You should be able to expose the service externally. Do you not see an option with their provided CRDs?

venkata.mutyala

03:36:43 AM

Does this not work? https://doc.crds.dev/github.com/strimzi/strimzi-kafka-operator/kafka.strimzi.io/Kafka/[email protected]#spec-kafka-listeners-type

Kafka.kafka.strimzi.io/v1beta2

Automatic documentation for your CustomResourceDefinitions.

miko

04:08:09 AM

Hi @venkata.mutyala, could I see your configs on how you exposed it and how you enabled the secured connection (like having scram-sha-512 password based Auth because sasl_plaintext is not supported) please? I have been trying to look for guides but I’m having a really tough time looking for one :( I feel overwhelmed by their documentation

venkata.mutyala

04:19:39 AM

I didn’t expose it secured like you are trying too. We used it internally within the clsuter only.

miko

04:23:26 AM

Ohh so there’s no password of any kind and simply connect to it directly within the cluster? If the Devs need access they would use port forward?

venkata.mutyala

04:24:25 AM

Our use case was primarily to replicate data between databases via kafka + kafka connect + debuzium connectors. If you are publicly exposing you will definitely want to limit access

venkata.mutyala

04:24:50 AM

Even in our use case we probably should have had auth but we didn’t.

venkata.mutyala

04:26:34 AM

here is something that i was able to find off github for your situation: https://github.com/ibm-mas/ansible-devops/blob/1792e98b77b32185c486da66a31c4c89fb1[…]bfcf/ibm/mas_devops/roles/kafka/templates/redhat/masuser.yml.j2

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: maskafka-credentials
  namespace: "{{ kafka_namespace }}"
data:
  username: "{{ kafka_user_name | b64encode }}"
  password: "{{ kafka_user_password | b64encode }}"
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: "{{ kafka_user_name }}"
  labels:
    strimzi.io/cluster: "{{ kafka_cluster_name }}"
  namespace: "{{ kafka_namespace }}"
spec:
  authentication:
    type: scram-sha-512
    password:
      valueFrom:
        secretKeyRef:
          name: maskafka-credentials
          key: password

venkata.mutyala

04:26:38 AM

Let me know how it goes.

venkata.mutyala

04:27:18 AM

looks like they committed that file in the past 3 months so decent shot it could be a working example

miko

04:28:26 AM

Thanks man! Strimzi also have this resource User (forgot the exact name haha), I think it’s connected to enabling the security

#kubernetes (2024-11)

2024-11-13

2024-11-14