Deploy Kubeflow pipelines on Mac
This post covers deploying of kubeflow pipelines on your Mac and play with it(if you’re new to it) before creating/deploying any pipelines to your production system.
Creating Kubernetes cluster : Before creating kubeflow pipeline, kubernetes cluster should get created. If Kubernetes cluster is already available in your environment, please ignore the below step. Though kind, K3s, K3ai supports creating kubernetes cluster, this guide shows to create cluster using kind tool.
kind is a tool for creating kubernetes cluster using Docker container nodes. There are numerous places where cluster created using kind is used for local development. kind can get installed using the following command in Mac:
brew install kind
Once the tool is installed, kubernetes cluster can be created using the following command:
kind create cluster
You can validate this by running commands using like kubectl version, kubectl cluster-info..etc
Note: Cluster name will be created with name kind by default. To change this, assign a name using -name flag.
Deploying Kubeflow Pipelines : For deploying kubeflow pipeline, kubectl commands should get executed with the respective kubeflow version. We are using Kubeflow version of 1.3.0 in our production. So, I’m installing the same version locally to play with it. Based on your requirements, please update the kubeflow pipeline version. apply command in kubectl applies a configuration to a resource by filename or stdin whereas wait command waits for a specific condition on one or many resources.
export PIPELINE_VERSION=1.3.0
kubectl apply -k “github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION”
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
kubectl apply -k “github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION”
Once all the aforementioned steps gets completed, please wait for few minutes for kubeflow pipelines deployment. Check whether all the deployments are up and running within kubeflow namespace using below command.
kubectl -n kubeflow get deployment
Once all the services are running without issues, do a port forward and start using kubeflow pipeline.
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
Thank you for going through the post. Kindly comment for any other questions.
