Kubeflow Security Testing: Unauthenticated Pipeline API, Notebook Server Privilege Escalation, and MinIO Credential Exposure

Kubeflow is the leading open-source machine learning platform for Kubernetes, used by enterprises to orchestrate ML workflows at scale. Its security surface is broad: the Kubeflow Pipelines API server on port 8888 is accessible without authentication on default single-user installations — any client that can reach it can list pipelines, run arbitrary ML jobs, and read artifact outputs; Jupyter notebook servers spawned by Kubeflow often run as root within their pod namespace and inherit the namespace's default service account, which may have broad Kubernetes RBAC permissions enabling cluster-level privilege escalation; MinIO is deployed as the default artifact store with default credentials (minio/minio123) that are often not rotated, exposing all ML model files, training data samples, and feature stores; Kubeflow's OIDC proxy mode trusts the kubeflow-userid HTTP header from internal network clients — header injection by an attacker with network access bypasses authentication; and Kubeflow's multi-user mode relies on Istio authorization policies that are commonly misconfigured to allow cross-namespace traffic. This guide covers systematic Kubeflow security assessment.

Table of Contents

  1. Kubeflow Discovery and API Testing
  2. MinIO Default Credentials and Artifact Exposure
  3. Notebook Server Privilege Escalation
  4. OIDC Header Injection Authentication Bypass
  5. Pipeline API Unauthenticated Access
  6. Kubeflow Security Hardening

MinIO Default Credentials and Artifact Exposure

# Kubeflow deploys MinIO as artifact storage with default credentials
# Default: minio/minio123 — frequently not changed after deployment

# Access MinIO console (usually port 9001 or exposed via Istio gateway)
curl -s -o /dev/null -w "%{http_code}" \
  http://kubeflow.example.com/minio/ 2>/dev/null

# Test MinIO default credentials via API
curl -s -u "minio:minio123" \
  "http://kubeflow.example.com:9000/minio/health/ready" 2>/dev/null | head -3

# List all MinIO buckets (if default creds work)
curl -s -u "minio:minio123" \
  "http://kubeflow.example.com:9000/" 2>/dev/null | \
  grep -oE "[^<]+" | head -10
# Common buckets: mlpipeline, katib (hyperparameter tuning results)

# Get the MinIO credentials from Kubeflow ConfigMap (if K8s access available)
kubectl get secret mlpipeline-minio-artifact -n kubeflow \
  -o jsonpath='{.data.accesskey}' 2>/dev/null | base64 -d
kubectl get secret mlpipeline-minio-artifact -n kubeflow \
  -o jsonpath='{.data.secretkey}' 2>/dev/null | base64 -d

Kubeflow Security Hardening

Kubeflow Security Hardening Checklist:
Security TestMethodRisk
MinIO default credentials (minio/minio123)curl -u minio:minio123 http://host:9000/ — lists all ML artifacts and modelsCritical
Kubeflow Pipelines API unauthenticated accessGET /pipeline/apis/v1beta1/pipelines — lists all pipelines without auth in single-user modeCritical
Notebook server runs as root with broad RBACexec into notebook pod: id; kubectl get pods --all-namespaces — root + broad SA = cluster accessHigh
OIDC proxy header injection authentication bypassSend request with kubeflow-userid: admin@example.com header from internal network clientHigh
Cross-namespace traffic via Istio misconfigurationSend requests from user namespace to system namespace — if Istio policies don't block, lateral movement is possibleHigh
Katib hyperparameter tuning results exposureAccess MinIO katib bucket — may contain model performance data and proprietary training configurationsMedium

Automate Kubeflow Security Testing

Ironimo tests Kubeflow deployments for MinIO default credential exposure giving access to all ML artifacts and model files, Kubeflow Pipelines API server accessible without authentication in single-user mode, notebook server pods running as root with overly broad Kubernetes RBAC service accounts, OIDC proxy header injection bypassing authentication from internal network positions, Istio authorization policy misconfiguration allowing cross-namespace lateral movement, and pipeline component images pulled from unrestricted registries.

Start free scan