Configuring Kafka with External Auth and TLS

#kubernetes #kafka #jotthatdown

Age: 28

Profession: Engineer

Location: 🇺🇸

Requirements

Kubernetes Cluster
cert-manager
1 DNS Entry

How To

Create a ClusterIssuer for LetsEncrypt

### issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: letsencrypt-prod
    # Add a single challenge solver, HTTP01 using nginx
    solvers:
    - http01:
        ingress:
          class: nginx

Create a secret containing a password to protect your jks keystore

kubectl create secret generic jks-password-secret --from-literal=password=SOME_PASSWORD -n kafka

Use the following definition to request a certificate from LetsEncrypt

^{If the certificate is successfully issued, this step will create a secret kafka-cert-secret containing your cert in the kafka namespace}

### certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kafka-cert
  namespace: kafka
spec:
  secretName: kafka-cert-secret
  dnsNames:
    - kafka-0.blah.com
    - kafka-1.blah.com
    - kafka-2.blah.com
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  keystores:
    jks:
      create: true
      passwordSecretRef: # Password used to encrypt the keystore
        key: password
        name: jks-password-secret

Set the following settings in your values.yaml for the kafka helm chart:

^{Full values.yaml can be found here}

^{Listener Configuration}

listeners:
  external:
    containerPort: 9095
    protocol: SASL_SSL
    name: EXTERNAL
    sslClientAuth: "none" #No mTLS

^{User Configuration}

auth:
  client:
    users:
      - user
    passwords: SOME_PASSWORD

^{Secret Configuration}

tls:
  type: JKS
  pemChainIncluded: false
  existingSecret: kafka-cert-secret
  autoGenerated: false
  passwordsSecret: jks-password-secret
  passwordsSecretKeystoreKey: password
  passwordsSecretTruststoreKey: password
  passwordsSecretPemPasswordKey: password
  keystorePassword: ""
  truststorePassword: ""
  keyPassword: ""
  jksKeystoreKey: keystore.jks
  jksTruststoreSecret: ""
  jksTruststoreKey: keystore.jks
  endpointIdentificationAlgorithm: https

^{Ingress Configuration}

externalAccess:
  enabled: true
  autoDiscovery:
    enabled: false
      loadBalancerNames:
        - kafka-0.blah.com
        - kafka-1.blah.com
        - kafka-2.blah.com
      loadBalancerAnnotations:
        - external-dns.alpha.kubernetes.io/hostname: kafka-0.blah.com.
        - external-dns.alpha.kubernetes.io/hostname: kafka-1.blah.com.
        - external-dns.alpha.kubernetes.io/hostname: kafka-2.blah.com.

Use the documentation here to deploy Kafka packaged by Bitnami