Awesome AI, ML and Data Engineer

Machine Learning

Articles

• Nanonets - Tesseract OCR in Python with Pytesseract & OpenCV
• Milvus - Deploy a Milvus Cluster on EKS
• Medium - Understanding Milvus: Key Concepts and Potential Applications
• Medium - Why You Shouldn’t Invest In Vector Databases?
• Datacamp - The Top 7 Vector Databases in 2025
• Medium - 30+ MCP Ideas with Complete Source Code
• Medium - The Open-Source Stack for AI Agents

Awesome Repositories

• applied-ml: 📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
• awesome-deep-learning : A curated list of awesome Deep Learning tutorials, projects and communities.
• awesome-llm-apps: Collection of awesome LLM apps with AI Agents and RAG using OpenAI, Anthropic, Gemini and opensource models.
• Awesome-LLMs-for-Video-Understanding: 🔥🔥🔥Latest Papers, Codes and Datasets on Vid-LLMs.
• awesome-local-ai : An awesome repository of local AI tools
• awesome-mcp-servers: A collection of MCP servers.
• awesome-model-quantization: A list of papers, docs, codes about model quantization
• awesome-production-machine-learning: A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning. Website
• best-of-ml-python: 🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.

Blogs

• DigitalOcean - AI/ML Topics: Articles and Community about AI/ML 🌟 (Recommended)
• Machine Learning Blog: ML@CMU | Carnegie Mellon University
• Machine Learning cơ bản: Vietnamese Forum and Community about ML 🌟 (Recommended)
• Machine Learning Mastery: The best resources to approaching ML
• MarkTechPost: ML and AI Tech News
• Mì AI - Học AI theo cách mì ăn liền!: Learn about AI in Vietnamese Community
• Neptune.ai: Learn from AI/ML engineers, researchers, and folks building foundation models: best practices, tool reviews, and real-world examples. 🌟 (Recommended)
• PyImageSearch: A brand new Computer Vision, Deep Learning, and OpenCV
• Medium - Gao Dalie (高達烈): Learn AI Agent, LLMs, RAG & Generative AI 🌟 (Recommended)
• Medium - Samar Singh: Build Scalable AI Solutions 🚀 🌟 (Recommended)

Landscape

• LF AI & Data Foundation Interactive Landscape 🌟 (Recommended)
• The 2024 MAD (Machine Learning, AI and Data) Landscape 🌟 (Recommended)

Organization

• DeepSeek: LLM Model for Prompting
• Machine Learning Tooling: Provide multiple collections about ML
• MLFoundations: Community for implementing ML Model
• OpenAI: OpenAI Github Community
• Qwen: Alibaba Cloud’s general-purpose AI models
• Seldon: Machine Learning Deployment for Kubernetes
• Triton Inference Server: provides a cloud and edge inferencing solution optimized for both CPUs and GPUs.

Page

• AI Agents Directory: Browse our AI agents list and build your digital workforce in minutes, not months 🌟 (Recommended)
• AI Tools Directory: Access the largest list of top-quality AI tools available on the web 🌟 (Recommended)
• DeepWiki: AI Documentation for any repository 🌟 (Recommended)
• Feature Stores for ML: Collection pages about Feature stores for ML
• Hugging Face : The Opensource AI community 🌟 (Recommended)
• Kaggle: Your Machine Learning and Data Science Community 🌟 (Recommended)
• LF AI & DATA Projects: Linux Foundation project about Open Source Innovation in Artificial Intelligence and Data 🌟 (Recommended)
• Made With ML: Learning how to responsibly deliver value with ML!
• NGC Catalog: NGC Catalog - GPU Accelerated AI models and SDKs that help you infuse AI into your applications at speed of light (NVIDIA)
• Papers With Code: The latest in Machine Learning 🌟 (Recommended)
• TensorFlow Hub: A repository of trained machine learning models.
• There’s An AI For That: Update the new about tools and technologies about AI

Papers

• Advances and Challenges in Foundation Agents
• CNCF - CNCF Cloud Native AI White Paper

Topic

• Artificial Intelligence
• Inference
• Machine Learning
• RAG
• Stream Processing
• LLMs
• NLP
• Computer Vision

Youtube Channel

• NeuralNine : Educational brand focusing on programming, machine learning and computer science
• sentdex : Funny guy who teach you about build cool stuff with python like AI
• MLOps.community : The MLOps Community fills the swiftly growing need to share real-world Machine Learning Operations best practices from engineers in the field 🌟 (Recommended)

Data Engineer

Articles

• Medium - Data Pipeline Development with MinIO, Iceberg, Nessie, Polars, StarRocks, Mage, and Docker
• Medium - ETL and ELT
• Medium - ELT with Fabric, Azure and Databricks
• Medium - The Change Data Capture (CDC) Design Pattern
• Medium - Apache Airflow Overview
• Blog - Data Lake vs Data Warehouse
• ML4Devs - Scalable Efficient Big Data Pipeline Architecture
• MonteCarlo - Data Pipeline Architecture Explained: 6 Diagrams and Best Practices
• Medium - Data Engineering Best Practices: How Big Tech & FAANG Firms Manage and Optimize Apache Kafka
• Medium - 5 Steps to Build Efficient Data Pipelines with Apache Airflow

Awesome Repositories

• awesome-apache-airflow: Curated list of resources about Apache Airflow
• awesome-bigdata: A curated list of awesome big data frameworks, ressources and other awesomeness.
• awesome-datascience: 📝 An awesome Data Science repository to learn and apply for real world problems.
• awesome-etl: A curated list of awesome ETL frameworks, libraries, and software.
• awesome-open-source-data-engineering: A curated list of open source tools used in analytics platforms and data engineering ecosystem
• data-engineering-roadmap: A comprehensive roadmap tailored for data engineering professionals at all levels
• data-engineer-handbook: a repo with links to everything you’d ever want to learn about data engineering

Blogs

• Medium - Vu Trinh: Tech blogger who cover a lot information about databases and data engineers 🌟 (Recommended)
• Medium - Data Engineers Things 🌟 (Recommended)
• Datacamp 🌟 (Recommended)
• LakeFS 🌟 (Recommended)
• Practical Data Engineering 🌟 (Recommended)
• Medium - Kai Waehner: echnology Evangelist — www.kai-waehner.de → Big Data Analytics, Data Streaming, Apache Kafka, Middleware, Microservices
• r/dataengineering

Organization

• Big Data Europe: Integrating Big Data, software & communicaties for addressing Europe’s societal challenge
• DataTalksClub: The place to talk about data

Landscape

• LakeFS - The State of Data Engineering 2024 🌟 (Recommended)
• Practicle Data Engineering - Open Source Data Engineering Landscape 2024
• Practicle Data Engineering - Open Source Data Engineering Landscape 2025 🌟 (Recommended)
• Blog - The Ultimate Data Engineering Roadmap (2025) – Built by a Data Engineer
• Portable - 100+ Best ETL Tools List & Software (As Of February 2025)

Topic

• Data
• Data Engineering
• Dataflow
• Data Processing
• Data Science
• Data Warehouse
• Data Lake
• LakeHouse
• ETL
• ELT

Youtube

• Netflix - Data Engineering Open Forum 2025

MLOps

Articles

• Neptune.ai - MLOps Landscape in 2024: Top Tools and Platforms
• Neptune.ai - MLOps Principles and How to Implement Them
• Google Cloud - MLOps: Continuous delivery and automation pipelines in machine learning
• Datacamp - 25 Top MLOps Tools You Need to Know in 2025

Awesome Repositories

• awesome-mlops: A curated list of references for MLOps

Blogs

• Machine Learning Operations: Provide an end-to-end machine learning development process to design, build and manage reproducible, testable, and evolvable ML-powered software
• Neptune.ai - MLOps Learning Hub: Strategies, tools, practical insights, and example projects on MLOps 🌟 (Recommended)
• Medium - Marvelous MLOps

Code Example / Crash Course

• MLflow Model Registry Webhooks REST API Example: Define webhook MLflow and trigger MLflow engine used Restful API to handle request
• MLOpsVN Courses: The Course about MLOps was written in Vietnamese

Topics

• AutoML
• LLMOps
• MLOps
• Model Serving

AI/ML/Data/MLOps Tools

Big Data

• Spark: A multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
  • spark-docker: Official Dockerfile for Apache Spark
  • spark-standalone-cluster-on-docker: Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker. ⚡

Computer Vision

• opencv: Open Source Computer Vision Library 🌟 (Recommended)

Data Orchestration Workflow

• airbyte: The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted. 🌟 (Recommended)

• airflow: A platform to programmatically author, schedule, and monitor workflows 🌟 (Recommended)

  • Astronomer Registry: Building Blocks for your Apache Airflow Data Pipelines.
  • Astro: A fully-managed SaaS application for data orchestration that helps teams write and run data pipelines with Apache Airflow

• kestra: ⚡ Workflow Automation Platform

• prefect: A workflow orchestration framework for building resilient data pipelines in Python.

Labeling and Annotation

• Argilla: a collaboration tool for AI engineers and domain experts to build high-quality datasets 🌟 (Recommended)

UI Builder

• chainlit: Build Conversational AI in minutes ⚡️ 🌟 (Recommended)
• streamlit: A faster way to build and share data apps 🌟 (Recommended)

LLM

• langfuse: Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets 🌟 (Recommended)
• litellm: Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format 🌟 (Recommended)
• TaskingAI: A BaaS (Backend as a Service) platform for LLM-based Agent Development and Deployment

MLOps

• KitOps: An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.
• polyaxon: MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle

Model Management and Serving

• KServe: A standard Model Inference Platform on Kubernetes, built for highly scalable use cases.
• MLFlow: Open source platform for the machine learning lifecycle 🌟 (Recommended)
• MLServer: An open source inference server for your machine learning models.
• openvino: OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
• Ray: an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. 🌟 (Recommended)

VectorDB

• Chroma: The AI-native open-source vector database (Opensource) 🌟 (Recommended)
• Milvus: A high-performance, highly scalable vector database that runs efficiently across a wide range of environments, from a laptop to large-scale distributed systems 🌟 (Recommended)