12 docs tagged with "orchestration"

Orchestrate ML training pipelines with Airflow - data quality gates, KubernetesPodOperator training, champion/challenger evaluation, and conditional deployment.

Deep dive into Apache Airflow - DAGs, Scheduler internals, Executors, Operators, XCom, and production patterns for reliable pipeline orchestration.

Learn how to use Apache Airflow to orchestrate production ML pipelines - DAG authoring, executors, XCom patterns, and avoiding the most common Airflow pitfalls.

A decision framework for selecting the right ML pipeline orchestrator - comparing Airflow, Prefect, Kubeflow Pipelines, Metaflow, ZenML, and Dagster across team size, maturity, and infrastructure requirements.

Building, compiling, and running production ML pipelines on Kubernetes using Kubeflow Pipelines v2 with MLMD metadata tracking and automatic retraining triggers.

Building scalable, reproducible ML workflows with Netflix's Metaflow - the flow-step model, cloud compute with @batch and @kubernetes, and Cards for documentation.

Understand the fundamental concepts behind ML pipeline orchestration - DAGs, dependency management, idempotency, and why cron jobs are a silent disaster for production ML.

Master the tools and patterns for orchestrating reliable, production-grade ML pipelines using Airflow, Prefect, Kubeflow, ZenML, and beyond.

Module overview for Pipeline Orchestration - turning ad-hoc scripts into reliable, observable, recoverable production data pipelines.

Building and deploying production ML workflows using Prefect 2.x/3.x - flows, tasks, deployments, work pools, and observability.

Prefect orchestration deep dive - flows, tasks, deployments, work pools, automations, and a direct comparison with Apache Airflow.

Building portable, stack-agnostic MLOps pipelines with ZenML - stacks, steps, materializers, and seamless local-to-cloud migration with MLflow and Vertex AI.