Awesome MLOps

Generals

Articles

• Neptune.ai - MLOps Landscape in 2024: Top Tools and Platforms 🌟 (Recommended)
• Neptune.ai - MLOps Principles and How to Implement Them 🌟 (Recommended)
• Google Cloud - MLOps: Continuous delivery and automation pipelines in machine learning
• Datacamp - 25 Top MLOps Tools You Need to Know in 2025

Awesome Repositories

• awesome-mlops: A curated list of references for MLOps
• AI-Infra: init to record my learning path of AI Infra, especially on inference.
• AI-Infra-from-Zero-to-Hero: 🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice
• Awesome-LLMOps: An awesome & curated list of best LLMOps tools for developers

Blogs

• Machine Learning Operations: Provide an end-to-end machine learning development process to design, build and manage reproducible, testable, and evolvable ML-powered software
• Neptune.ai - MLOps Learning Hub: Strategies, tools, practical insights, and example projects on MLOps 🌟 (Recommended)
• Medium - Marvelous MLOps

Code Example / Crash Course

• MLflow Model Registry Webhooks REST API Example: Define webhook MLflow and trigger MLflow engine used Restful API to handle request
• MLOpsVN Courses: The Course about MLOps was written in Vietnamese

Landscape

• Awesome LLMOps

Topics

• AutoML
• LLMOps
• MLOps
• Model Serving

MLOps Tools (Curious Version 🔭)

MLOps Toolkits

• KitOps: An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.
• polyaxon: MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle

Model Management & Deployment

• KServe: A standard Model Inference Platform on Kubernetes, built for highly scalable use cases.
• MLServer: An open source inference server for your machine learning models.
• gpustack: Simple, scalable AI model deployment on GPU clusters
• openvino: OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

LLM Gateway

• PortKey: A blazing fast AI Gateway with integrated guardrails
• higress: 🤖 AI Gateway | AI Native API Gateway

MLOps Tools

Compute Engine & Serving

• Ray: an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. 🌟 (Recommended)
• vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
• onnxruntime: cross-platform, high performance ML inferencing and training accelerator

Model Management & Deployment

• MLFlow: Open source platform for the machine learning lifecycle 🌟 (Recommended)

LLM Gateway

• litellm: Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format 🌟 (Recommended)

VectorDB

• Chroma: The AI-native open-source vector database (Opensource) 🌟 (Recommended)
• Milvus: A high-performance, highly scalable vector database that runs efficiently across a wide range of environments, from a laptop to large-scale distributed systems 🌟 (Recommended)