5. Configuration and Secrets Management

Hard-coding configuration and credentials into container images is inflexible and insecure — you would rebuild the image to change a setting, and secrets would leak into your registry. Kubernetes separates configuration from code with ConfigMaps (non-sensitive config) and Secrets (sensitive data), plus environment variables and the Downward API for injecting values at runtime. This chapter shows how to externalize all configuration cleanly.

---

5.1 ConfigMaps

A ConfigMap stores non-confidential configuration as key-value pairs, decoupled from your Pods so the same image runs in dev, staging, and prod with different settings. This subchapter covers creating and consuming them.

---

Creating ConfigMaps from literals, files, and directories

Theory

A ConfigMap is an API object holding non-sensitive configuration data as key-value pairs (and larger text blobs). Its purpose is the separation of configuration from application code: the same container image can be deployed everywhere, with environment-specific values supplied by a ConfigMap. This follows the twelve-factor app principle of storing config in the environment.

You can create ConfigMaps several ways:

• From literals: --from-literal=key=value for individual values.
• From files: --from-file=path — each file becomes a key (filename) with the file contents as the value. Great for whole config files (app.conf, nginx.conf).
• From directories: --from-file=dir/ — every file in the directory becomes a key.
• Declaratively: a YAML manifest with a data map (and binaryData for binary values).

ConfigMaps have a 1 MiB size limit (they are stored in etcd), so they are for configuration, not bulk data.

Example

# From literals:
kubectl create configmap app-config \
  --from-literal=LOG_LEVEL=info \
  --from-literal=MAX_CONNECTIONS=100

# From a file (key = filename, value = contents):
kubectl create configmap nginx-config --from-file=nginx.conf

# From a whole directory (each file becomes a key):
kubectl create configmap settings --from-file=./config-dir/

# Declarative equivalent:
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  LOG_LEVEL: "info"
  MAX_CONNECTIONS: "100"
  app.properties: |          # a multi-line value (a whole config file)
    timeout=30
    retries=3

Exercises

1. (Beginner) What is the primary purpose of a ConfigMap?
2. (Beginner) When you create a ConfigMap with --from-file=nginx.conf, what becomes the key and what becomes the value?
3. (Intermediate) Why are ConfigMaps unsuitable for storing a 5 MB data file, and what is the limit?
4. (Interview) How does using ConfigMaps support the goal of building a container image once and deploying it to multiple environments? (Hint: config in the environment, not the image.)

Answers

1. To store non-sensitive configuration separately from application code/images, so configuration can change without rebuilding the image and the same image runs across environments.
2. The key is the filename (nginx.conf) and the value is the file's contents.
3. ConfigMaps are stored in etcd and capped at 1 MiB. They are meant for configuration, not bulk data; a 5 MB file exceeds the limit and would also bloat etcd. Use a volume, object storage, or an init container to fetch large data instead.
4. The image contains only code; all environment-specific values (URLs, log levels, feature flags) live in ConfigMaps that are injected at runtime as env vars or mounted files. You build and test one immutable image, then deploy it to dev/staging/prod with different ConfigMaps — eliminating per-environment image variants and "config baked into the image" drift, consistent with twelve-factor config.

---

Consuming ConfigMaps as environment variables

Theory

The simplest way to use a ConfigMap is to inject its values as environment variables in a container. Two approaches:

• Individual keys via valueFrom.configMapKeyRef: map one ConfigMap key to one env var (with control over the env var's name).
• All keys at once via envFrom.configMapRef: import every key in the ConfigMap as env vars (the keys become the variable names, optionally with a prefix).

The important caveat: environment variables are read once at container startup. If the ConfigMap changes later, the running container's env vars do not update — the Pod must be restarted to pick up new values. This is a key difference from volume-mounted ConfigMaps (next topic). Env-var injection is ideal for simple, rarely-changing settings.

Example

spec:
  containers:
  - name: app
    image: myapp:1.0
    env:
    - name: LOG_LEVEL                 # pick one specific key
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: LOG_LEVEL
    envFrom:                          # OR import ALL keys at once
    - configMapRef:
        name: app-config
        # prefix: APP_                # optional prefix for imported var names

Exercises

1. (Beginner) What is the difference between configMapKeyRef and envFrom.configMapRef?
2. (Beginner) If you update a ConfigMap, do a running container's environment variables change?
3. (Intermediate) You imported a ConfigMap via envFrom but one key is named max-connections (with a dash). Why might it not appear as an env var?
4. (Interview) Given that env-var injection is read-once at startup, how do teams typically force Pods to pick up changed ConfigMap values? (Hint: rollout restart, config hash annotations.)

Answers

1. configMapKeyRef injects a single chosen key as one env var (you control the env var name). envFrom.configMapRef imports all keys in the ConfigMap as env vars at once (key names become variable names, optionally prefixed).
2. No. Env vars are set at container start and do not update when the ConfigMap changes; the Pod/container must be restarted to see new values.
3. Env var names must be valid C-identifiers (letters, digits, underscores; not starting with a digit). A key like max-connections contains a dash, which is not a valid env var name, so envFrom skips it (it is reported as a skipped/invalid key). Use a valid key name or map it explicitly with configMapKeyRef to a valid env var name.
4. Because env vars are read-once, teams trigger a rollout to restart Pods: e.g., kubectl rollout restart deployment/<name>, or — to make it automatic — they add a checksum/hash of the ConfigMap as a Pod-template annotation (common in Helm, checksum/config). When the ConfigMap changes, the hash changes, the Pod template changes, and the Deployment performs a rolling update so new Pods read the updated values.

---

Consuming ConfigMaps as volumes

Theory

The alternative to env vars is mounting a ConfigMap as a volume: each key becomes a file in a directory, with the key's value as the file's contents. This is the natural fit for whole configuration files (e.g., mounting an nginx.conf or application.yaml into the path the app reads).

The major advantage over env vars: volume-mounted ConfigMaps update automatically. The kubelet periodically syncs the mounted files when the ConfigMap changes (with a propagation delay of up to ~1 minute, depending on kubelet sync period and cache TTL). The catch is that the application must re-read the file to use the new value — Kubernetes updates the file on disk, but it cannot make your app reload its config. You can mount specific keys to specific paths with items, and use subPath to mount a single file (though subPath mounts do not receive updates).

Example

spec:
  containers:
  - name: web
    image: nginx:1.25
    volumeMounts:
    - name: cfg
      mountPath: /etc/nginx/conf.d     # each ConfigMap key becomes a file here
  volumes:
  - name: cfg
    configMap:
      name: nginx-config
      items:                           # optionally map specific keys -> paths
      - key: nginx.conf
        path: default.conf

Exercises

1. (Beginner) When a ConfigMap is mounted as a volume, what does each key become?
2. (Beginner) Which consumption method (env var or volume) reflects ConfigMap updates without a Pod restart?
3. (Intermediate) You mount a ConfigMap key with subPath. Why do later updates to the ConfigMap not appear in the container?
4. (Interview) Volume-mounted ConfigMaps update the file on disk automatically, yet the application may still use stale config. Why, and how is this typically solved? (Hint: the app must reload; file watchers or SIGHUP.)

Answers

1. Each key becomes a file in the mounted directory, with the key's value as that file's contents.
2. Volume-mounted ConfigMaps (the kubelet syncs updated files into the mount; env vars do not update live).
3. subPath mounts copy the file into the container at mount time and are not kept in sync by the kubelet — the projection that enables live updates only applies to the whole-volume mount, not individual subPath entries. So changes to the ConfigMap won't propagate to a subPath-mounted file.
4. Kubernetes updates the file on disk, but it cannot make a running process re-read its configuration — the app keeps using whatever it loaded at startup. Solutions: have the app watch the file for changes (inotify) and reload, send a reload signal (e.g., SIGHUP) to the process, use a sidecar that detects changes and triggers a reload, or simply roll the Deployment so Pods restart and re-read config.

---

ConfigMap update propagation

Theory

Bringing the previous two topics together: how do configuration changes actually reach your running workloads? It depends entirely on how the ConfigMap is consumed:

Consumption method	Live update?	How to apply changes
Env var (configMapKeyRef/envFrom)	No	Restart the Pod (rollout restart).
Volume mount (whole volume)	Yes (~up to 1 min delay)	Kubelet syncs files; app must reload them.
Volume mount with subPath	No	Restart the Pod.

Two important details: (1) ConfigMaps marked immutable: true cannot be updated at all (you must create a new one) — this improves performance (the kubelet stops watching them) and prevents accidental changes. (2) The common production pattern to make any config change trigger a safe rolling update is to embed a hash of the ConfigMap in the Pod template annotations; when the config changes, the template changes, and the Deployment rolls automatically.

Example

# Immutable ConfigMap: cannot be edited; must be replaced.
apiVersion: v1
kind: ConfigMap
metadata: { name: app-config-v2 }
immutable: true
data:
  LOG_LEVEL: "warn"

# Pattern: config hash annotation forces a rollout when config changes
spec:
  template:
    metadata:
      annotations:
        checksum/config: "9f86d081..."   # hash of the ConfigMap content

kubectl rollout restart deployment/web   # force Pods to re-read env-var config

Exercises

1. (Beginner) Which consumption method requires a Pod restart to apply a ConfigMap change?
2. (Beginner) What does marking a ConfigMap immutable: true prevent, and what is one benefit?
3. (Intermediate) Describe the "config hash annotation" pattern and what problem it solves.
4. (Interview) Your team mounts config as a volume expecting "live reload," but changes don't take effect for over a minute and sometimes not at all. Diagnose the likely causes. (Hint: kubelet sync delay, subPath, app not reloading, immutable.)

Answers

1. Environment-variable consumption (and subPath volume mounts) — both are read-once and require a Pod restart.
2. It prevents any updates to the ConfigMap's data (you must create a new ConfigMap instead). Benefits: protects against accidental changes and improves performance/scalability because the kubelet/API server no longer needs to watch it for updates.
3. You compute a hash/checksum of the ConfigMap's contents and place it as an annotation in the Deployment's Pod template. When the ConfigMap changes, the hash changes, which changes the Pod template, which causes the Deployment to perform a rolling update — ensuring Pods (even those using env vars) pick up the new config safely and automatically. It solves the "env vars don't auto-update" problem deterministically.
4. Likely causes: (a) the up-to-~1-minute kubelet sync/cache delay explains the lag for whole-volume mounts; (b) keys mounted via subPath never update — they require a restart; (c) the application reads its config only at startup and never re-reads the file, so the file updates but behavior doesn't (needs a reload/watch); (d) the ConfigMap may be immutable or a different object than the Pod mounts; (e) env-var consumption was mistaken for volume consumption. Verify the mount type, check the file actually changes in the container, and confirm the app reloads.

---

5.2 Secrets

Secrets hold sensitive data — passwords, tokens, keys — with slightly different handling from ConfigMaps. This subchapter covers their types, creation, consumption, and the critical topic of protecting them at rest.

---

Secret types and use cases

Theory

A Secret is like a ConfigMap but intended for sensitive data: passwords, API tokens, TLS certificates, SSH keys. Functionally they are similar (key-value, 1 MiB limit, consumed as env vars or volumes), but Secrets get special handling: they are only distributed to nodes that need them, can be encrypted at rest, and Kubernetes avoids writing them to logs.

A common misconception: Secret values are stored base64-encoded, not encrypted, by default. Base64 is encoding, not security — anyone with read access can decode it trivially. Real protection requires enabling encryption at rest and tight RBAC (covered ahead).

Secrets have typed uses, signaled by the type field:

Type	Purpose
Opaque	Arbitrary user-defined data (the default).
kubernetes.io/tls	TLS cert + key (tls.crt, tls.key) — used by Ingress.
kubernetes.io/dockerconfigjson	Registry credentials for pulling private images.
kubernetes.io/basic-auth, ssh-auth	Basic-auth and SSH credentials.
kubernetes.io/service-account-token	ServiceAccount token (legacy/long-lived).

Example

# Generic (Opaque) secret from literals (kubectl base64-encodes for you):
kubectl create secret generic db-creds \
  --from-literal=username=admin \
  --from-literal=password='S3cr3t!'

# TLS secret for an Ingress:
kubectl create secret tls web-tls --cert=tls.crt --key=tls.key

# Image pull secret for a private registry:
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com --docker-username=u --docker-password=p

Exercises

1. (Beginner) How are Secret values stored by default — encrypted or encoded?
2. (Beginner) Which Secret type is used to store a TLS certificate and key for an Ingress?
3. (Intermediate) What is the purpose of a kubernetes.io/dockerconfigjson Secret?
4. (Interview) Explain why "Secrets are base64-encoded" is a security pitfall and what must be done to actually protect Secret data. (Hint: encoding vs. encryption, encryption at rest, RBAC.)

Answers

1. Encoded (base64) by default, not encrypted — unless you enable encryption at rest.
2. kubernetes.io/tls (holding tls.crt and tls.key).
3. It stores container registry credentials so the kubelet can authenticate and pull images from a private registry; it is referenced by imagePullSecrets in a Pod or ServiceAccount.
4. Base64 is reversible encoding with no secrecy — anyone who can read the Secret object (or etcd) can decode the value instantly. So storing secrets as base64 alone provides no protection. To actually secure them you must: enable encryption at rest in the API server (so etcd holds ciphertext, ideally via a KMS), enforce strict RBAC so few identities can read Secrets, restrict etcd access, and consider external secret managers. Encoding is for safe transport of binary data, not confidentiality.

---

Creating and managing Secrets

Theory

Secrets can be created imperatively (kubectl create secret) or declaratively (YAML). A subtlety with declarative YAML: the data field requires base64-encoded values, while the convenience field stringData lets you write plaintext that Kubernetes encodes for you (and which takes precedence). Never commit plaintext Secret manifests to Git in the clear.

Managing Secrets safely means addressing the GitOps problem: you want everything in Git, but you cannot store raw secrets there. Solutions include:

• Sealed Secrets (Bitnami): encrypt a Secret into a SealedSecret CRD that is safe to commit; a controller decrypts it in-cluster.
• External Secrets Operator / CSI Secrets Store: pull values from an external manager (Vault, AWS/GCP/Azure secret managers) at runtime so the real secret never lives in Git or etcd long-term.
• SOPS with age/KMS: encrypt secret files for Git, decrypt in the pipeline.

Example

apiVersion: v1
kind: Secret
metadata: { name: db-creds }
type: Opaque
stringData:                 # plaintext here; Kubernetes base64-encodes it
  username: admin
  password: "S3cr3t!"
# data:                     # alternative: values must be pre-base64-encoded
#   username: YWRtaW4=

# Read a secret back (note: it returns base64; decode to view):
kubectl get secret db-creds -o jsonpath='{.data.password}' | base64 -d

Exercises

1. (Beginner) What is the difference between the data and stringData fields in a Secret manifest?
2. (Beginner) Why should you avoid committing a plain Secret YAML (with base64 values) to a public Git repo?
3. (Intermediate) Name two tools/approaches that let you store secrets safely in a GitOps workflow.
4. (Interview) Describe how Sealed Secrets makes it safe to store a secret in Git, and where the decryption actually happens. (Hint: asymmetric encryption, in-cluster controller.)

Answers

1. data requires values already base64-encoded; stringData accepts plaintext that Kubernetes encodes into data on creation (and stringData takes precedence if both set a key). stringData is a write-only convenience.
2. Base64 is trivially reversible, so committing such a manifest is equivalent to publishing the plaintext secret. Anyone reading the repo can decode it.
3. Any two: Sealed Secrets (encrypt to a SealedSecret CRD that's safe to commit), External Secrets Operator or CSI Secrets Store (sync from Vault/cloud secret managers at runtime), SOPS with age/KMS (encrypt files for Git).
4. With Sealed Secrets, you encrypt your Secret using the cluster controller's public key (via kubeseal), producing a SealedSecret resource that only the controller's private key can decrypt. Because only ciphertext is stored, the SealedSecret is safe to commit to Git. The in-cluster Sealed Secrets controller watches for SealedSecrets, decrypts them with its private key, and creates the corresponding normal Secret — so decryption happens only inside the cluster, never in the repo or in transit.

---

Consuming Secrets in pods

Theory

Secrets are consumed in Pods almost exactly like ConfigMaps — as environment variables (secretKeyRef / envFrom.secretRef) or as volume mounts (each key becomes a file). The same trade-offs apply: env vars are read-once; volume mounts update (with delay). For Secrets, volume mounts are generally preferred for sensitive data because:

• Secret files mounted from volumes are stored in a tmpfs (in-memory) filesystem on the node, never written to disk.
• Environment variables are more prone to accidental leakage — they can show up in crash dumps, logs, child-process environments, and kubectl describe/exec output.

Additionally, registry credentials are consumed not by your container but by the kubelet, via imagePullSecrets on the Pod or ServiceAccount. Always restrict who can read Secrets with RBAC, regardless of consumption method.

Example

spec:
  imagePullSecrets:                 # used by the kubelet to pull private images
  - name: regcred
  containers:
  - name: app
    image: registry.example.com/myapp:1.0
    env:
    - name: DB_PASSWORD             # as an env var (read-once)
      valueFrom:
        secretKeyRef:
          name: db-creds
          key: password
    volumeMounts:
    - name: creds                   # as files (preferred for sensitive data)
      mountPath: /etc/creds
      readOnly: true
  volumes:
  - name: creds
    secret:
      secretName: db-creds

Exercises

1. (Beginner) What are the two main ways a Pod consumes a Secret?
2. (Beginner) Which field tells the kubelet which Secret to use for pulling private images?
3. (Intermediate) Why are volume-mounted Secrets often preferred over environment variables for sensitive values?
4. (Interview) List three ways an environment-variable secret can accidentally leak that a tmpfs-mounted secret file is less exposed to. (Hint: logs, describe/exec, child processes, crash dumps.)

Answers

1. As environment variables (secretKeyRef or envFrom.secretRef) and as volume-mounted files (secret volume).
2. imagePullSecrets (on the Pod spec or the ServiceAccount).
3. Volume-mounted Secrets are stored in node memory (tmpfs), never persisted to disk, can be updated, and are less prone to incidental exposure. Env vars are read-once and tend to leak into logs, process listings, and diagnostic output.
4. Examples: (a) logs — apps or frameworks often dump the full environment on startup or error; (b) kubectl describe pod / exec / process listing — env vars are visible to anyone who can inspect the Pod or run printenv/read /proc/<pid>/environ; (c) child processes inherit the parent's environment, spreading the secret to subprocesses; (d) crash/core dumps capture the process environment. A tmpfs-mounted file is only readable at a known path with appropriate permissions and isn't broadcast through the environment, reducing these incidental-exposure vectors.

---

Encryption at rest for Secrets

Theory

By default, Secrets are stored in etcd as base64 (effectively plaintext). Anyone who can read etcd — through a backup file, a disk, or direct access — can read every Secret. Encryption at rest fixes this by having the API server encrypt Secret data before writing it to etcd, using an EncryptionConfiguration.

You configure one or more providers in priority order; the first listed is used for writing, and all are tried for reading (so you can rotate keys). Provider options include aescbc/aesgcm (encrypt with a local key) and, best practice, kms (a KMS plugin that uses an external Key Management Service like AWS KMS, GCP KMS, or HashiCorp Vault — so the data-encryption keys themselves are protected by a master key you control and can rotate/audit). Never put identity (no encryption) first if you want encryption active.

Example

# EncryptionConfiguration passed to the API server via --encryption-provider-config
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources: ["secrets"]
  providers:
  - kms:                       # preferred: external KMS protects the keys
      name: myKmsPlugin
      endpoint: unix:///var/run/kms.sock
  - aescbc:                    # fallback local-key provider
      keys:
      - name: key1
        secret: <base64-32-byte-key>
  - identity: {}               # plaintext — only for reading old data during migration

# After enabling, re-encrypt existing Secrets by rewriting them:
kubectl get secrets --all-namespaces -o json | kubectl replace -f -

Exercises

1. (Beginner) How are Secrets stored in etcd by default, and what risk does this create?
2. (Beginner) What configuration object enables encryption at rest?
3. (Intermediate) In an EncryptionConfiguration with multiple providers, which provider is used to encrypt new writes, and why list more than one?
4. (Interview) Why is a KMS provider considered best practice over a static local aescbc key, and what does it protect against that the local key does not? (Hint: key custody, rotation, auditing, etcd backup theft.)

Answers

1. Base64-encoded (effectively plaintext). Anyone with access to etcd, its disk, or its backups can read every Secret.
2. An EncryptionConfiguration (provided to the API server via --encryption-provider-config).
3. The first provider listed is used to encrypt new writes; all listed providers are tried in order when decrypting reads. Listing multiple providers enables key rotation and migration (e.g., read data written by an old key while writing with a new one, or include identity last to read still-unencrypted data during rollout).
4. With a static local aescbc key, the encryption key sits in the API server's config on disk — if that host or config is compromised, so are all Secrets. A KMS provider keeps the key-encryption key in an external, hardened, auditable service: the data keys are wrapped by a master key you never expose, you can rotate and revoke it centrally, and access is logged. This protects against etcd/backup theft (the data is ciphertext whose key isn't co-located) and gives proper key custody, rotation, and audit — capabilities a static on-disk key lacks.

---

External secret management (Vault, AWS Secrets Manager)

Theory

Even with encryption at rest, native Secrets live in the cluster and are managed with Kubernetes tooling. Many organizations prefer a dedicated, centralized secret manager — HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault — as the single source of truth for secrets across all systems, with features Kubernetes lacks: dynamic/short-lived credentials, fine-grained access policies, automatic rotation, leasing, and rich audit logs.

Two integration patterns dominate:

• External Secrets Operator (ESO): a controller that reads from the external manager and syncs values into native Kubernetes Secrets, which Pods consume normally. Simple, but the secret is materialized in etcd.
• Secrets Store CSI Driver: mounts secrets directly from the external manager into the Pod as a volume at runtime, so the secret need not be stored as a Kubernetes Secret at all (optionally it can also sync one).

These also enable dynamic secrets (e.g., Vault issuing a database credential valid for 1 hour), drastically reducing the blast radius of a leak.

Example

# External Secrets Operator: declaratively sync a value from AWS Secrets Manager
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata: { name: db-creds }
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-creds            # the K8s Secret ESO will create/keep in sync
  data:
  - secretKey: password
    remoteRef:
      key: prod/db
      property: password

Exercises

1. (Beginner) Name two external secret managers commonly integrated with Kubernetes.
2. (Beginner) What does the External Secrets Operator do at a high level?
3. (Intermediate) Contrast the External Secrets Operator with the Secrets Store CSI Driver in terms of where the secret ends up.
4. (Interview) What is a "dynamic secret," and how does using one (e.g., a Vault-issued 1-hour DB credential) reduce risk compared to a long-lived static Secret? (Hint: short TTL, automatic rotation, blast radius.)

Answers

1. Any two: HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault.
2. It is a controller that fetches secret values from an external manager and synchronizes them into native Kubernetes Secrets (keeping them refreshed), so Pods consume them through the normal Secret mechanism.
3. ESO materializes the secret as a Kubernetes Secret in etcd (Pods read that Secret). The CSI Secrets Store driver mounts the secret value directly into the Pod as a volume at runtime, so it need not be stored as a Kubernetes Secret at all (optionally it can also sync one). CSI keeps the secret out of etcd; ESO is simpler but stores it.
4. A dynamic secret is a credential generated on demand with a short lease/TTL and automatically revoked when it expires (e.g., Vault creates a unique DB username/password valid for 1 hour). Because each credential is short-lived, unique per consumer, and auto-rotated, a leaked credential is useless after expiry and can be tied to a specific consumer for auditing/revocation — dramatically shrinking the blast radius compared to a static, long-lived Secret that, if leaked, grants indefinite access and must be manually rotated everywhere.

---

5.3 Environment Variables and Downward API

Beyond ConfigMaps and Secrets, Pods can receive configuration from static values and — uniquely — from information about themselves via the Downward API. This subchapter ties together how environment is assembled.

---

Setting static environment variables

Theory

The most basic configuration is a static environment variable defined inline in the container spec with env[].value. These are literal values baked into the Pod manifest — useful for simple, non-sensitive, fixed settings (a feature flag, a fixed port, an app mode).

A useful and sometimes surprising feature is variable expansion: within env values you can reference earlier-defined variables using $(VAR_NAME) syntax, letting you compose values. (To emit a literal $, write $$.) Order matters — only variables defined earlier in the list can be referenced. While handy, prefer ConfigMaps/Secrets for anything that varies by environment; reserve inline value for truly constant settings.

Example

spec:
  containers:
  - name: app
    image: myapp:1.0
    env:
    - name: APP_MODE
      value: "production"            # static literal
    - name: PORT
      value: "8080"
    - name: BASE_URL                 # composed from earlier vars via $(...)
      value: "http://localhost:$(PORT)"
    - name: PRICE_LABEL
      value: "Cost is $$5"           # $$ -> literal $  => "Cost is $5"

Exercises

1. (Beginner) How do you define a static environment variable directly in a container spec?
2. (Beginner) What is the syntax to reference another environment variable's value within an env entry?
3. (Intermediate) Why does the order of env entries matter when using $(VAR) expansion?
4. (Interview) When should you use an inline static env value versus pulling the value from a ConfigMap or Secret? (Hint: constancy, sensitivity, environment variance.)

Answers

1. With an env entry that has a name and a literal value (e.g., - name: APP_MODE / value: "production").
2. $(VAR_NAME) — and $$ produces a literal $.
3. Expansion can only reference variables that are already defined earlier in the same env list; a $(VAR) referencing a variable defined later (or not at all) is left unexpanded. So you must define a variable before composing other values from it.
4. Use inline static value only for truly constant, non-sensitive settings that don't change between environments (e.g., a fixed mode or port). Use a ConfigMap for non-sensitive values that differ per environment (URLs, log levels), and a Secret for sensitive values (passwords, tokens). This keeps the image/manifest environment-agnostic and keeps secrets out of plain manifests.

---

Referencing ConfigMap and Secret keys

Theory

This topic consolidates the injection mechanisms covered earlier into one mental model. A container's environment can be assembled from four sources, all under env/envFrom:

1. Static literals — env[].value.
2. ConfigMap keys — env[].valueFrom.configMapKeyRef (one key) or envFrom.configMapRef (all keys).
3. Secret keys — env[].valueFrom.secretKeyRef (one key) or envFrom.secretRef (all keys).
4. Downward API — env[].valueFrom.fieldRef / resourceFieldRef (covered next).

Precedence and behavior to remember: envFrom imports bulk keys, then explicit env entries can override individual names; if a referenced ConfigMap/Secret or key is missing, the container fails to start unless you mark the reference optional: true. This unified valueFrom model is how Kubernetes lets one Pod template draw configuration from many typed sources cleanly.

Example

spec:
  containers:
  - name: app
    image: myapp:1.0
    envFrom:
    - configMapRef: { name: app-config }      # bulk import all ConfigMap keys
    - secretRef: { name: db-creds }           # bulk import all Secret keys
    env:
    - name: DB_PASSWORD                        # specific Secret key -> chosen name
      valueFrom:
        secretKeyRef: { name: db-creds, key: password }
    - name: FEATURE_X                          # specific ConfigMap key, optional
      valueFrom:
        configMapKeyRef: { name: app-config, key: feature_x, optional: true }

Exercises

1. (Beginner) What are the four sources an environment variable's value can come from?
2. (Beginner) What happens if a container references a ConfigMap key that does not exist?
3. (Intermediate) How do you make a missing key reference non-fatal so the container still starts?
4. (Interview) Explain how envFrom and explicit env entries interact when both set a variable of the same name. (Hint: explicit env overrides bulk import.)

Answers

1. Static literal (value), ConfigMap key (configMapKeyRef/envFrom.configMapRef), Secret key (secretKeyRef/envFrom.secretRef), and the Downward API (fieldRef/resourceFieldRef).
2. The container fails to start (it cannot be created) because the required reference can't be resolved — unless the reference is marked optional: true.
3. Set optional: true on the configMapKeyRef/secretKeyRef (or on the configMapRef/secretRef in envFrom). The missing key/object is then skipped instead of blocking startup.
4. envFrom performs a bulk import of all keys as env vars first; then explicit env entries are applied and override any same-named variable from the bulk import. So an explicit env entry takes precedence over a value brought in by envFrom with the same name, letting you import many values but selectively override or rename specific ones.

---

Downward API: exposing pod metadata

Theory

Sometimes an application needs to know things about itself and its environment that are only determined at runtime: its own Pod name, namespace, node, IP, labels, or its resource limits. Hard-coding these is impossible (the Pod name is generated; the node is chosen by the scheduler). The Downward API exposes this Pod/container metadata to the container, either as environment variables or as files in a volume, without the app needing to call the Kubernetes API.

It is called "downward" because information flows down from the platform into the workload. Common uses: tagging logs/metrics with the Pod name and node, constructing a unique instance ID, reporting which namespace the app runs in, or making an app aware of its own memory limit so it can size internal caches/heaps accordingly.

Example

spec:
  containers:
  - name: app
    image: myapp:1.0
    env:
    - name: POD_NAME
      valueFrom:
        fieldRef: { fieldPath: metadata.name }
    - name: POD_NAMESPACE
      valueFrom:
        fieldRef: { fieldPath: metadata.namespace }
    - name: NODE_NAME
      valueFrom:
        fieldRef: { fieldPath: spec.nodeName }
    volumeMounts:
    - name: podinfo
      mountPath: /etc/podinfo
  volumes:
  - name: podinfo
    downwardAPI:
      items:
      - path: "labels"               # writes the Pod's labels to /etc/podinfo/labels
        fieldRef: { fieldPath: metadata.labels }

Exercises

1. (Beginner) What does the Downward API let a container learn about?
2. (Beginner) In which two forms can Downward API data be exposed to a container?
3. (Intermediate) Why can't you just hard-code the Pod name and node name in the manifest instead of using the Downward API?
4. (Interview) Give a concrete scenario where exposing the Pod's memory limit to the app via the Downward API materially improves the application's behavior. (Hint: JVM heap / cache sizing inside a container.)

Answers

1. Metadata about the Pod and container itself and its placement — e.g., name, namespace, UID, labels/annotations, node name, Pod IP, service account, and the container's resource requests/limits.
2. As environment variables (fieldRef/resourceFieldRef) and as files in a downwardAPI volume.
3. Many of these values are not known until runtime and are assigned by the platform: the Pod name may be generated (e.g., with a random suffix), the node is chosen by the scheduler, and the Pod IP is allocated at creation. They differ per replica and per scheduling, so they cannot be statically written in the manifest — the Downward API supplies them dynamically.
4. A JVM (or any runtime with its own heap/cache) running in a container may, by default, size its heap based on the host's total memory rather than the container's cgroup limit, leading to OOMKills. By exposing the container's memory limit via resourceFieldRef (e.g., limits.memory) as an env var, the app/startup script can set the max heap or cache size proportionally to its actual limit — avoiding OOM kills and using its allotted memory efficiently. (Modern JVMs are container-aware, but explicit sizing via the Downward API remains a robust, general technique for any runtime.)

---

fieldRef and resourceFieldRef

Theory

The Downward API has two distinct selectors, and knowing which to use is the practical crux of this subchapter:

• fieldRef exposes Pod-level metadata and spec fields — things like metadata.name, metadata.namespace, metadata.uid, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, and status.podIP/status.hostIP.
• resourceFieldRef exposes a container's resource requests and limits — requests.cpu, limits.cpu, requests.memory, limits.memory, and ephemeral-storage equivalents. You can specify a divisor to scale the output (e.g., express memory in Mi).

A few constraints: not every field is available in both env-var and volume form (e.g., status.podIP and resourceFieldRef work as env vars; labels/annotations are typically exposed via volume files because they are maps). For resourceFieldRef, if a limit is not set, it defaults to the node's allocatable capacity.

Example

spec:
  containers:
  - name: app
    image: myapp:1.0
    resources:
      requests: { cpu: "250m", memory: "256Mi" }
      limits:   { cpu: "1",    memory: "512Mi" }
    env:
    - name: POD_IP
      valueFrom:
        fieldRef: { fieldPath: status.podIP }          # fieldRef: pod metadata
    - name: MEM_LIMIT_MI
      valueFrom:
        resourceFieldRef:                              # resourceFieldRef: resources
          containerName: app
          resource: limits.memory
          divisor: "1Mi"                               # output in MiB -> "512"
    - name: CPU_REQUEST
      valueFrom:
        resourceFieldRef:
          containerName: app
          resource: requests.cpu                       # "250m"

Exercises

1. (Beginner) What category of information does fieldRef expose versus resourceFieldRef?
2. (Beginner) Which selector would you use to expose a container's CPU limit to itself?
3. (Intermediate) What does the divisor field do in a resourceFieldRef, and give an example output.
4. (Interview) If a container does not specify a memory limit but uses resourceFieldRef on limits.memory, what value does it receive, and why could that be surprising? (Hint: defaults to node allocatable.)

Answers

1. fieldRef exposes Pod-level metadata/spec/status fields (name, namespace, uid, labels, annotations, nodeName, podIP, etc.). resourceFieldRef exposes a container's resource requests and limits (cpu/memory/ephemeral-storage).
2. resourceFieldRef with resource: limits.cpu (and the appropriate containerName).
3. divisor scales/normalizes the reported resource value into the desired unit. For example, with resource: limits.memory and divisor: "1Mi", a 512Mi limit is reported as 512; with resource: limits.cpu and divisor: "1m", a 1 CPU limit reports 1000.
4. It receives the node's allocatable amount for that resource (the effective upper bound when no limit is set), not zero or an error. This is surprising because the app may believe it has, say, the whole node's memory as its "limit" and size buffers accordingly, risking OOM if the node is shared — which is exactly why setting explicit limits (and reading them via the Downward API) is recommended.

---