Awesome MLOps

Generals

Articles

• Neptune.ai - MLOps Landscape in 2024: Top Tools and Platforms 🌟 (Recommended)
• Neptune.ai - MLOps Principles and How to Implement Them 🌟 (Recommended)
• Google Cloud - MLOps: Continuous delivery and automation pipelines in machine learning
• Datacamp - 25 Top MLOps Tools You Need to Know in 2025
• Youtube - GPU Instance Selection: AI & LLM Inference Benchmarking 🌟 (Recommended)
• Blog - LLM Inference Speed Benchmarks

Awesome Repositories

• awesome-mlops: A curated list of references for MLOps
• AI-Infra: init to record my learning path of AI Infra, especially on inference.
• AI-Infra-from-Zero-to-Hero: 🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice
• Awesome-LLMOps: An awesome & curated list of best LLMOps tools for developers
• Awesome-LLM-Inference: 📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

Blogs

• Machine Learning Operations: Provide an end-to-end machine learning development process to design, build and manage reproducible, testable, and evolvable ML-powered software
• Neptune.ai - MLOps Learning Hub: Strategies, tools, practical insights, and example projects on MLOps 🌟 (Recommended)
• Medium - Marvelous MLOps

Code Example / Crash Course

• MLflow Model Registry Webhooks REST API Example: Define webhook MLflow and trigger MLflow engine used Restful API to handle request
• MLOpsVN Courses: The Course about MLOps was written in Vietnamese

Landscape

• Awesome LLMOps
• LF AI & Data Foundation Interactive Landscape 🌟 (Recommended)

Topics

• AutoML
• LLMOps
• MLOps
• Model Serving

MLOps Tools (Curious Version 🔭)

MLOps Toolkits

• KitOps: An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.
• polyaxon: MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle

Model Management & Deployment

• KServe: A standard Model Inference Platform on Kubernetes, built for highly scalable use cases.
• MLServer: An open source inference server for your machine learning models.
• gpustack: Simple, scalable AI model deployment on GPU clusters
• openvino: OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

LLM Gateway

• PortKey: A blazing fast AI Gateway with integrated guardrails
• higress: 🤖 AI Gateway | AI Native API Gateway

MLOps Tools

Compute Engine & Serving

• Ray: an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. 🌟 (Recommended)
• vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
• onnxruntime: cross-platform, high performance ML inferencing and training accelerator
• dynamo: A Datacenter Scale Distributed Inference Serving Framework, By Nvidia
• OpenLLM: Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
• BentoML: The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
• kserve: Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
• gpustack: A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
• ollama: Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
• llama.cpp: LLM inference in C/C++
• sglang: SGLang is a high-performance serving framework for large language models and multimodal models.

Model Management & Deployment

• MLFlow: Open source platform for the machine learning lifecycle 🌟 (Recommended)
• Kubeflow Pipelines: a platform for building and deploying portable and scalable machine learning (ML) workflows using containers on Kubernetes-based systems.
• metaflow: Build, Manage and Deploy AI/ML Systems

LLM Gateway

• litellm: Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format 🌟 (Recommended)
• Kong AI Gateway: Connectivity and governance layer for modern AI-native applications
• WSO2 AI Gateway: AI-ready, GitOps-driven API platform for full lifecycle management across cloud, hybrid, and on-prem.

VectorDB

• Chroma: The AI-native open-source vector database (Opensource) 🌟 (Recommended)
• Milvus: A high-performance, highly scalable vector database that runs efficiently across a wide range of environments, from a laptop to large-scale distributed systems 🌟 (Recommended)