Charmed Kubeflow 1.6: what’s new?

Kubeflow 1.6 was released on September 7, and Charmed Kubeflow 1.6 (Canonical’s distribution) came shortly after, as it follows the same roadmap. Charmed Kubeflow introduces a new version of Kubeflow pipelines as well as model training enhancements.  Read our official press release.

Kubeflow pipelines: a better user experience

Kubeflow pipelines are an end-to-end orchestration platform that helps users build and deploy reusable multi-step ML workflows. The alpha release of the functionality (KFP v2) represents the biggest improvement, which brings a better user experience and new features that help you save time and improve efficiency.

Metadata is a project that is used to better track and manage machine learning workflows. It provides information about runs, models, datasets and data artefacts, enabling users to monitor and understand their artificial intelligence projects. However, in the previous versions of Kubeflow, machine learning engineers had to manually configure it to benefit from this feature, which was often challenging. Moreover, they could not log additional metadata or use any metadata in downstream components. Kubeflow 1.6 changes the asynchronous process implementation that the metadata project had. It offers more assurance that the metadata is captured and recorded, regardless of the deployment step. The metadata is now sourced from the pipeline execution cache. KFP concepts are used to capture the metadata instead of the Pod spec. 

The latest version of Kubeflow improved the correlation between the input and output as well. It makes it more intuitive for those unaware of the rules they need to follow when writing their own pipelines. Lastly, changes have been made to the authoring component, allowing engineers and data scientists to develop faster. The YAML components will be supported in the future, but some parts of the code will need to change, such as ContainerOp. 

Watch our livestream and learn more about Kubeflow Pipelines!

Hyperparameter support in Katib

Katib is a Kubernetes-native project dedicated to automated machine learning (AutoML).  Katib supports hyperparameter tuning. It is important to have this feature available for data scientists who want to control a parameter in the learning process. Katib is agnostic of the AI framework and allows developers to write their programming language of choice. Population-based training (PBT) provides optimised modelling and ease of production fit for models and is available in the latest Kubeflow version.  Kubeflow’s distributed training operator combines PBT with various frameworks such as Tensorflow, PyTorch or MPI operator.  The model serving was also part of Kubeflow’s roadmap. In the latest version, a new Model Spec was introduced to the inference service, aiming to specify new models.

CI/CD for Charmed Kubeflow

Charmed Kubeflow is Canonical’s enterprise-ready distribution of Kubeflow, an open-source machine learning toolkit designed for use with Kubernetes. Charmed Kubeflow is composed of a bundle of charms, which are Kubernetes operators that automate maintenance and security operations. 

One of Canonical’s objectives was to align Charmed Kubeflow releases with the upstream release. Thus, the engineering team invested time in automating the CI/CD pipelines to enable faster operations. This gives the user the chance to use the latest stable release, but also the latest edge, allowing them to see the latest technical updates that the team does.

Learn more about Charmed Kubeflow

Upgrade to the latest version of Charmed Kubeflow by following our guide and contact us if you have any questions.

Follow our tutorials and have fun with Charmed Kubeflow.

Read our whitepaper and get started with AI.