xeusnguyen.xyz

Search

SearchSearch

Recent Notes

See 129 more โ†’

DevOps & System & Tech collections

Jan 05, 2025, 25 min read

Awesome Tech Repository

Info

Include repository, blog about technologies, self-hosted, devops, design and system design, โ€ฆ

Repository

  • awesome: ๐Ÿ˜Ž Awesome lists about all kinds of interesting topics. Website
  • awesome-selfhosted : About A list of Free Software network services and web applications which can be hosted on your own servers
  • awesome-scalability : The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
  • awesome-devops : A curated list of awesome DevOps platforms, tools, practices and resources
  • awesome-for-beginners : A list of awesome beginners-friendly projects.
  • system-design-primer: Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
  • awesome-tf: Curated list of resources on HashiCorpโ€™s Terraform and OpenTofu
  • awesome-design: Curated design resources from all over the world.
  • awesome-sysadmin: A curated list of amazingly awesome open-source sysadmin resources.
  • awesome-ebpf : A curated list of awesome projects related to eBPF.
  • awesome-cloudnative-trainings: Awesome Trainings from Cloud Native Computing Foundation Projects and Kubernetes related software
  • web-servers.md : Each of these commands will run an ad hoc http static server in your current (or specified) directory
  • awesome-devsecops: Curating the best DevSecOps resources and tooling.
  • self-hosting-guide : Learn all about locally hosting
  • static-analysis: A curated list of static analysis (SAST) tools and linters
  • dynamic-analysis: A curated list of dynamic analysis tools and linters
  • system-design-101: Explain complex systems using visuals and simple terms. Help you prepare for system design interviews.
  • startpack: Selfhosted tech starter pack for development of new project or startup
  • awesome-console-services: A curated list of awesome console services (reachable via HTTP, HTTPS and other network protocols)
  • awesome-test-automation: A curated list of awesome test automation frameworks, tools, libraries, and software for different programming languages
  • awesome-embedded: A curated list of awesome embedded programming.
  • awesome-iam: ๐Ÿ‘ค Identity and Access Management knowledge for cloud platforms
  • gitlab-explore: Find and explore tab of GitLab
  • awesome-webhooks: A curated list about real-time webhooks.
  • DevOps-Bash-tools: ย DevOps Bash Scripts Collection
  • to be continuousโ€ฆ: GitLab CI templates
  • DevOps-Roadmap: DevOps Roadmap with learning resources. contents by Milan (Legit Recommend)
  • devops-exercises: For who want to start with DevOps journey

Page

  • HelloGitHub : Share interesting, entry-level open source projects on GitHub. EN
  • Awesome Software Architecture : Curated list of awesome articles and resources to learn and practice software architecture, patterns and principles. this repository will be updated continuously, keep yourself up to date.
  • Nubenetes : Awesome Kubernetes & Cloud
  • StackExchange : StackOverFlow Ecosystem
  • Ansible Galaxy : Ansible template and external playbook
  • Collection about kind of YAML : Collection about yaml in wide technologies
  • free-for-dev : A list of SaaS, PaaS and IaaS offerings that have free tiers of interest to devops and infradev. Web
  • khuedoanโ€™s homelab : Fully automated homelab from empty disk to running services with a single command - Repo belong A. Khue Doan. Web Version
  • awesome-compilers: Curated list of awesome resources on Compilers, Interpreters and Runtimes

Topic

Organization

  • Spotify: Spotify Github Repositories
  • Netflix: Netflix Open Source Platform
  • Facebook: Meta - build community through open source technology.
  • FacebookResearch: Meta Research OpenSource Platform
  • Linkedin: linkedin Open Source Platform
  • Microsoft: Open source projects and samples from Microsoft
  • Alibaba: Alibaba Open Source
  • Apache: The Apache Software Foundation
  • Red Hat Developer: Github home of the Red Hat Developer program.
  • Container Storage Interface : Container Storage Interface (CSI) Specification.
  • Airbnb: Airbnb Open Source Community
  • Google: Google โค๏ธ Open Source
  • Tencent : Tencent Open Source Community
  • Fossasia: Open Technologies developed in Asia and Around the Globe
  • GoogleContainerTools : Tools in-house of Google about Container
  • Ory: Open Source Identity Platform For Everyone. Website
  • Cloudflare: CloudFlare Open Source Community
  • Ansible: The Community whose stand behind Ansible
  • Ansible Community: Collection of community-driven Ansible repositories
  • Kong: The Cloud Connectivity Company. Community Driven & Enterprise Adopted.
  • Dyne.org: Think &do tank and free software foundry
  • Puppet: Puppet Community
  • Vox Pupuli: Modules and tooling maintained by and for the Puppet community
  • LXC - Linux Containers: Linux container projects
  • Hashicorp: Consistent workflows to provision, secure, connect, and run any infrastructure for any application.
  • PingCAP: The team behind TiDB TiKV, an open source MySQL compatible HTAP database
  • Leaning Technologies: Enterprise-grade WebAssembly solutions
  • Gruntwork: Organization who makes Terragrunt and couple of things related to entire infrastructure
  • Authzed: Creators of SpiceDB, the open source fine-grained permissions database. YC W21.
  • Fluent: a cloud native logging solution to unify data collection and consumption.
  • GoogleChrome: GoogleChrome Extension and couple useful things
  • Bytedance Inc: ByteDanceโ€™s open source projects hub for innovation. Website, Who stand behind KubeWharf

Landscape

Innovation

  • LFX: a hub for collaboration and sustainability in open source

Blog DevOps and Tech

Note

Useful page for finding the target and gain more experience in DevOps, System and moreover in technology field

Tech Blog

  • Trang chแปง ยป AnonyViet - Vietnamese Community who teaching multiple techniques on Computer
  • Digital Ocean Community - Community offers thousands of tutorials, videos, and answers to questions on a wide range of topics
  • StackExchange - StackOverFlow Ecosystem
  • XDA Forums - Valuable resource for people who want to make the most of their mobile devices, from customizing the look and feel to adding new functionality.
  • How-To Geek - Community who explain Technologies
  • Hackaday.io - The worldโ€™s largest collaborative hardware development community.
  • Hackaday - Fresh Hacks Every Day
  • Opensource.com | Opensource.com - Community talk about Opensource - powered by Red Hat
  • 3os Project - Collocation of technical documentation and guides for devops, developers, pentesters, systems administrators and other IT professionals.
  • Medium - ITNext - Community in Medium where talk about lots of technological thing
  • Medium - Mr.PlanB - Write about Kubernetes, Promox, Homelab topics
  • Medium - Jake Page - Kubernetes Deployment Specialist @ Glasskube
  • News Ycombinator - Multiple discussion around tech articles in the internet

DevOps Blog

Architecture Blog

EBPF Blog

  • Brendan Gregg - Cool guy with pleasant contents about ebpf and kernel technologies
  • Isovalent Blog - More about ebpf and kernel technologies, especially cilium and sub-projects of cilium

Youtube Channel

Tech Articles

Note

Collections articles about DevOps, System Design and moreover about technologies and opensource

General

Technologies

Techniques

System and Cloud Design

Opensource Finder

Note

Place to finding the cool stuff about Opensource and priorities software around the world

Tech Research Sheet

Research pages

Note

Place where you can find multiple documentation and papers about technologies in multiple topics from DevOps, Tech and AI

  • Springer: Search for research articles, academic books and more
  • IT eBooks: Download IT ebooks for free
  • arXiv.org: e-Print archive
  • ResearchGate: Papers about researching around the world
  • MDPI: A pioneer in scholarly, open access publishing, MDPI has supported academic communities
  • RFC Editor : The RFC Series (ISSN 2070-1721) contains technical and organizational documents about the Internet
  • The Pattern Languages of Programs archives: Hundreds of patterns are generated yearly in conferences alone, not to mention books and software engineering websites.

Documentation

Papers

Articles

Note

Topics which I concern about and do more inspect and research about them to provide knowledge for doing something around them in future

Quantum Computing

Repository

  • openqasm: Quantum assembly language for extended quantum circuits
  • qiskit: Qiskit is an open-source SDK for working with quantum computers at the level of extended quantum circuits, operators, and primitives.
  • QuantumKatas: Tutorials and programming exercises for learning Q# and quantum computing
  • pennylane: a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
  • open-quantum-safe: Software for prototyping quantum-resistant cryptography

Articles

DevOps & Useful Tools

Note

Collections a supper cool and powerful tools for multiple purpose attaching with DevOps

Automation Build Platforms

  • Expo: Expo is a framework that makes developing Android and iOS apps easier. Our framework provides file-based routing, a standard library of native modules, and much more
  • Buildpacks.io: Cloud Native Buildpacks (CNBs) transform your application source code intoย container imagesย that can run on any cloud.
  • lerna: ๐Ÿ‰ Lerna is a fast, modern build system for managing and publishing multiple JavaScript/TypeScript packages from the same repository.
  • nx: Smart Monorepos ยท Fast CI
  • just: a handy way to save and run project-specific commands.
  • electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
  • CMake: Mirror of CMake upstream repository. Plus on Make
  • harness: Harness Open Source is an end-to-end developer platform with Source Control Management, CI/CD Pipelines, Hosted Developer Environments, and Artifact Registries.

Automation Deployment Platforms

  • gitpod: The developer platform for on-demand cloud development environments to create software faster and more securely.
  • prefect: Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
  • dapr: Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge. Website

Cloud Infrastructure Designer

  • Brainboard : An AI driven platform to visually design, generate terraform code and manage cloud infrastructure, collaboratively.
  • Cloudairy : Cloudchartย is a cloud focused diagram tool designed to help you easily map out cloud infrastructures
  • Excalidraw: Hand-drawn look & feel - Collaborative - Secure

Configuration as Code

  • Configu: Open-source ConfigOps infrastructure โš™๏ธ

eBPF

  • ecapture: Capturing SSL/TLS plaintext without a CA certificate using eBPF. Supported on Linux/Android kernels for amd64/arm64. Documentation

Feature Flag

  • flagsmith: Open Source Feature Flagging and Remote Config Service. Host on-prem or use our hosted version atย https://flagsmith.com/
  • unleash: Open-source feature management solution built for developers.
  • flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solution

Infrastructure as Code (Machine + Configuration)

  • Vagrant : Vagrant is the command line utility for managing the lifecycle of virtual machines.
  • Ansible : Ansible provides open-source automation that reduces complexity and runs everywhere, especially VM
  • awx: AWX provides a web-based user interface, REST API, and task engine built on top of Ansible.

Infrastructure as Code (Cloud)

  • Pulumi Documentations : An open source infrastructure as code tool for creating, deploying, and managing cloud infrastructure.
  • Pulumi Registry: Package, SDK of Pulumi (Example: AWS, Azure, GCP)
  • Terraform Documentation: An infrastructure as code tool that lets you build, change, and version infrastructure safely and efficiently
  • Terraform Registry: Terraform providers that power all of Terraformโ€™s resource types, or find modules for quickly deploying common infrastructure configurations.
  • Terraform Best Practices: An attempt to systematically describe best practices using Terraform and provide recommendations for the most frequent problems Terraform users experience
  • OpenTofu: An infrastructure as code tool that lets you define infrastructure resources in human-readable configuration files that you can version, reuse, and share
  • Spacelift: Spacelift is a sophisticated, continuous integration and deployment (CI/CD) platform forย infrastructure-as-code
  • terraformer: CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code
  • inframap: Read your tfstate or HCL to generate a graph specific for each provider, showing only the resources that are most important/relevant.

Internal Developer Platform (IDP)

Identity Provider (SSO, OAuth, MFA, โ€ฆ)

  • authentik : ย An IdP (Identity Provider) and SSO (single sign on) that is built with security at the forefront of every piece of code, every feature, with an emphasis on flexibility and versatility.
  • FusionAuth : FusionAuth is a modern platform for Customer Identity and Access Management (CIAM). FusionAuth provides APIs and a responsive web user interface to support login, registration, localized email, multi-factor authentication, reporting and much more.
  • Authelia: Authelia is an open-sourceย authenticationย andย authorizationย server and portal fulfilling the identity and access management (IAM) role of information security in providingย multi-factor authenticationย and single sign-on (SSO) for your applications via a web portal. It acts as a companion forย common reverse proxies.
  • tailscale: The easiest, most secure way to use WireGuard and 2FA.
  • zitadel: Identity infrastructure, simplified forย you.
  • casdoor: An open-source UI-first Identity and Access Management (IAM) / Single-Sign-On (SSO) platform with web UI supporting
  • keycloak: Open Source Identity and Access Management For Modern Applications and Services. Website
  • spicedb: Open Source, Google Zanzibar-inspired database for scalably storing and querying fine-grained authorization data. Documentation

Generated + Validate Configuration

  • moz://a SSL Configuration Generator : SSL Generation for multiple webserver
  • YAML Multiline Strings : Find the right syntax for YAML file
  • terraform-docs : A utility to generate documentation from Terraform modules in various output formats.
  • .gitignore.io: Create useful .gitignore files for your project
  • diagrams: ๐ŸŽจ Diagram as Code for prototyping cloud system architectures
  • crontab guru: The quick and simple editor for cron schedule expressions by Cronitor (SHELL/Pipeline)
  • checkov: Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages
  • Enable CORS: enable cross-origin resource sharing

Gitops

  • Atlantis - Terraform Pull Request Automation : Running Terraform Workflows with Ease
  • scalr : ย Remote operations backend for Terraform and OpenTofu (OTF)
  • digger : open-source CI/CD orchestrator for Terraform
  • env0:ย Automate and manage any Infrastructure-as-Code, from Terraform and OpenTofu to Pulumi, CloudFormation, Terragrunt, Kubernetes, and more.
  • bytebase : The GitHub/GitLab for database DevOps.
  • Semaphore UI : Elegant web interface for executing Ansible playbooks, Terraform, OpenTofu, Pulumi code and Bash scripts.
  • act: Run your GitHub Actions locally ๐Ÿš€

Git Hosting

  • gitea: Painless self-hosted all-in-one software development service, including Git hosting, code review, team collaboration, package registry and CI/CD. Doc
  • gitlab : Download, install and maintain your own GitLab instance.
  • gogs: a painless self-hosted Git service. Doc

Local Cloud Self-hosted

Monitoring and observability

  • Netdata : The open-source observability platform everyone needs!

  • ELK: Combination from four main components. Guide, Helm and Docker

    1. Elasticsearch: a distributedย RESTย search engine which stores all of the collected data.
    2. Logstash: the data processing component of the Elastic Stack which sends incoming data to Elasticsearch.
    3. Kibana: a web interface for searching and visualizing logs.
    4. Beats: lightweight, single-purpose data shippers that can send data from hundreds or thousands of machines to either Logstash or Elasticsearch.

  • Grafana Stack: Ecosystem of Grafana for providing monitor methodology for your application, container, nodes and moreover.

  • Zabbix: A software that monitors numerous parameters of a network and the health and integrity of servers, virtual machines, applications, services, databases, websites, the cloud and more

  • Checkmk: Offers comprehensive and specialized solutions for dealing with the very diverse environments of IT infrastructures.

  • signoz: An open-source observability platform native to OpenTelemetry with logs, traces and metrics in a single application

  • VictoriaMetrics: fast, cost-effective monitoring solution and time series database. Documentation, helm

  • highlight: The open source, full-stack monitoring platform. Error monitoring, session replay, logging, distributed tracing, and more.

  • dcgm: Manage and Monitor GPUs in Cluster Environments

  • Log Viewer (Log4j, NLog): a high performance realtime log viewer via UDP (Chainsaw/NLogViewer) or text file.

  • vector: A high-performance observability data pipeline.

  • coroot: an open-source APM & Observability tool

  • sloth: ๐Ÿฆฅ Easy and simple Prometheus SLO (service level objectives) generator

  • OpenSLO: Open specification for defining and expressing service level objectives (SLO)

  • pyrra: Making SLOs with Prometheus manageable, accessible, and easy to use for everyone!

  • cloudprober: An active monitoring software to detect failures before your customers do.

Orchestration Framework

  • luigi: a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
  • camunda: Process Orchestration Framework. Website
  • prefect: A workflow orchestration framework for building resilient data pipelines in Python.

Product Error Analytics and Monitoring

  • openreplay: Session replay and analytics tool you can self-host. Ideal for reproducing issues, co-browsing with users and optimizing your product.
  • opentelemetry: OpenTelemetry, also known as OTel, is a vendor-neutral open sourceย observabilityย framework for instrumenting, generating, collecting, and exporting telemetry data such asย traces,ย metrics, andย logs.
  • Sentry: Developer-first error tracking and performance monitoring. Website
  • GlitchTip: Simple, open source error tracking

Queue Services

  • AutoMQ : AutoMQ re-engineers Kafka for the cloud by decoupling storage to object storage. While maintainingย 100% compatibilityย with Apache Kafkaยฎ, it offers users up toย 10 times cost efficiencyย andย 100 times elasticity.
  • Apache Kafka: an open-source, distributed event streaming platform with message communication and storage capabilities (superb)
  • RabbitMQ: An open-source message broker for asynchronous messaging, queueing and delivery acknowledgement.
  • Apache ActiveMQ : An open-source Java-based message queue that can be accessed by clients written in Javascript, C, C++, Python and .NET.
  • Amazon Simple Queue System (SQS): A fully managed message queue built for distributed and serverless platforms
  • Microsoft Azure Service Bus: A reliable, fully managed Cloud service for delivering messages via queues or topics.https://www.serverless.com/framework/docs
  • Celery: a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system.

Serverless Self-hosted

  • fn: The container native, cloud agnostic serverless platform.
  • knative: An Open-Source Enterprise-level solution to build Serverless and Event Driven Applications. Documentation
  • serverless framework: โšก Serverless Framework โ€“ Effortlessly build apps that auto-scale, incur zero costs when idle, and require minimal maintenance using AWS Lambda and other managed cloud services. Documentation
  • faas: OpenFaaS - Serverless Functions Made Simple

Secrets management

  • dotenvx : Loads environment variables from .env for nodejs projects.
  • dotenv : The secrets manager for .env files โ€“ from the same people that pioneered dotenv.
  • sops : Simple and flexible tool for managing secrets
  • Hashicorp Vault: Vault is an identity-based secret and encryption management system. This documentation covers the main concepts of Vault, what problems it can solve, and contains a quick start for using Vault.
  • Ansible Vault: Ansible Vault is a feature of ansible that allows you to keep sensitive data such as passwords or keys in encrypted files, rather than as plaintext in playbooks or roles. These vault files can then be distributed or placed in source control.
  • trufflehog: Find, verify, and analyze leaked credentials.
  • infisical: the open-source secret management platform: Sync secrets across your team/infrastructure, prevent secret leaks, and manage internal PKI
  • teller: Cloud native secrets management for developers - never leave your command line for secrets.
  • doppler: The official CLI for interacting with your Doppler secrets and configuration. Documentation

Storage Platforms

  • MinIO: MinIO is an object storage solution that provides an Amazon Web Services S3-compatible API and supports all core S3 features.
  • Ceph: Ceph can be used to provideย Ceph Object Storageย toย Cloud Platformsย and Ceph can be used to provideย Ceph Block Deviceย services toย Cloud Platforms. Ceph can be used to deploy aย Ceph File System. Allย Ceph Storage Clusterย deployments begin with setting up eachย Ceph Nodeย and then setting up the network.
  • seaweedfs : a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering.
  • CubeFS : A next-generation cloud-native storage product that is currently an incubating open-source project hosted by theย Cloud Native Computing Foundationopen in new windowย (CNCF).
  • rclone: โ€œrsync for cloud storageโ€ - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

Service Discovery

  • nacos: An easy-to-use dynamic service discovery, configuration and service management platform for building cloud native applications.
  • consul: A multi-networking tool that offers a fully-featured service mesh solution

Service Proxy

  • envoy : Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures
  • traefik proxy : Traefik is anย open-sourceย Edge Routerย that makes publishing your services a fun and easy experience
  • HAProxy : HAProxy is a free, very fast and reliable reverse-proxy offeringย high availability,ย load balancing, and proxying for TCP and HTTP-based applications

SSL + DNS Utilities

  • certbot: Certbot is EFFโ€™s tool to obtain certs from Letโ€™s Encrypt and (optionally) auto-enable HTTPS on your server.
  • acme.sh: A pure Unix shell script implementing ACME client protocol
  • ddns-updater: Container to update DNS records periodically with WebUI for many DNS providers
  • cloudflare-ddns: ๐ŸŽ‰๐ŸŒฉ๏ธ Dynamic DNS (DDNS) service based on Cloudflare! Access your home network remotely via a custom domain name without a static IP!

Static Site Generator

  • gatsby: The best React-based framework with performance, scalability and security built in.
  • quartz: ๐ŸŒฑ a fast, batteries-included static-site generator that transforms Markdown content into fully functional websites
  • hugo: The worldโ€™s fastest framework for building websites.

Utilities Terminal Tools

  • tldr: ๐Ÿ“š Collaborative cheatsheets for console commands

  • bat: A cat(1) clone with wings.

  • warp: The terminal reimagined with AI and collaborative tools for better productivity.

  • fzf: ๐ŸŒธ A command-line fuzzy finder

  • bpytop: Linux/OSX/FreeBSD resource monitor

  • entr: Run arbitrary commands when files change

  • pet: Simple command-line snippet manager

  • bcal: ๐Ÿ”ข Bits, bytes and address calculator

  • fd : A simple, fast and user-friendly alternative to โ€˜findโ€™

  • ohmyzsh: Framework for managing your zsh configuration

  • duf: Disk Usage/Free Utility - a better โ€˜dfโ€™ alternative

  • dust : A more intuitive version of du in rust

  • thefuck : Magnificent app which corrects your previous console command.

  • ripgrep : ripgrep recursively searches directories for a regex pattern while respecting your gitignore

  • exa: A modern replacement for โ€˜lsโ€™.

  • skopeo: Work with remote images registries - retrieving information, images, signing content

  • pre-commit: A framework for managing and maintaining multi-language pre-commit hooks.

    • pre-commit-hooks: Some out-of-the-box hooks for pre-commitSome out-of-the box hooks for pre-commit
    • husky: Git hooks made easy ๐Ÿถ woof!

  • auto-unlocker: Unlocker for VMWare macOS

  • darling: Darwin/macOS emulation layer for Linux

  • httpie: modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more. Documentation

Utilities and Fun Tools

  • Melty: The open source AI code editor for 10x engineers
  • spacedrive: Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.
  • localsend: An open-source cross-platform alternative to AirDrop (Local Vietnamese Guy ๐Ÿ––)
  • nhost: The Open Source Firebase Alternative with GraphQL. Documentation

Virtual Machine

  • webvm: Virtual Machine for the Web
  • firecracker: Secure and fast microVMs for serverless computing.

Web Analytics

  • Plausible: Simple, open source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics.
  • umami: Umami is a simple, fast, privacy-focused alternative to Google Analytics.

Web Testing and Automation

  • playwright: Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API. Documentation
  • selenium: A browser automation framework and ecosystem. Documentation
  • jest: A delightful JavaScript Testing Framework with a focus on simplicity.

Graph View

Backlinks