xeusnguyen.xyz

Search

SearchSearch

Recent Notes

See 120 more โ†’

DevOps & System & Tech collections

Nov 17, 2024, 24 min read

Awesome Tech Repository

Info

Include repository, blog about technologies, self-hosted, devops, design and system design, โ€ฆ

Repository

  • awesome: ๐Ÿ˜Ž Awesome lists about all kinds of interesting topics. Website
  • awesome-selfhosted : About A list of Free Software network services and web applications which can be hosted on your own servers
  • awesome-scalability : The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
  • awesome-devops : A curated list of awesome DevOps platforms, tools, practices and resources
  • awesome-for-beginners : A list of awesome beginners-friendly projects.
  • system-design-primer: Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
  • awesome-tf: Curated list of resources on HashiCorpโ€™s Terraform and OpenTofu
  • awesome-design: Curated design resources from all over the world.
  • awesome-sysadmin: A curated list of amazingly awesome open-source sysadmin resources.
  • awesome-ebpf : A curated list of awesome projects related to eBPF.
  • web-servers.md : Each of these commands will run an ad hoc http static server in your current (or specified) directory
  • awesome-devsecops: Curating the best DevSecOps resources and tooling.
  • self-hosting-guide : Learn all about locally hosting
  • static-analysis: A curated list of static analysis (SAST) tools and linters
  • dynamic-analysis: A curated list of dynamic analysis tools and linters
  • system-design-101: Explain complex systems using visuals and simple terms. Help you prepare for system design interviews.
  • startpack: Selfhosted tech starter pack for development of new project or startup
  • awesome-console-services: A curated list of awesome console services (reachable via HTTP, HTTPS and other network protocols)
  • awesome-test-automation: A curated list of awesome test automation frameworks, tools, libraries, and software for different programming languages
  • awesome-embedded: A curated list of awesome embedded programming.
  • awesome-iam: ๐Ÿ‘ค Identity and Access Management knowledge for cloud platforms
  • gitlab-explore: Find and explore tab of GitLab
  • awesome-webhooks: A curated list about real-time webhooks.
  • DevOps-Bash-tools: ย DevOps Bash Scripts Collection
  • to be continuousโ€ฆ: GitLab CI templates

Page

Topic

Organization

  • Spotify: Spotify Github Repositories
  • Netflix: Netflix Open Source Platform
  • Facebook: Meta - build community through open source technology.
  • FacebookResearch: Meta Research OpenSource Platform
  • Linkedin: linkedin Open Source Platform
  • Microsoft: Open source projects and samples from Microsoft
  • Alibaba: Alibaba Open Source
  • Apache: The Apache Software Foundation
  • Red Hat Developer: Github home of the Red Hat Developer program.
  • Container Storage Interface : Container Storage Interface (CSI) Specification.
  • Airbnb: Airbnb Open Source Community
  • Google: Google โค๏ธ Open Source
  • Tencent : Tencent Open Source Community
  • Fossasia: Open Technologies developed in Asia and Around the Globe
  • GoogleContainerTools : Tools in-house of Google about Container
  • Ory: Open Source Identity Platform For Everyone. Website
  • Cloudflare: CloudFlare Open Source Community
  • Ansible: The Community whose stand behind Ansible
  • Ansible Community: Collection of community-driven Ansible repositories
  • Kong: The Cloud Connectivity Company. Community Driven & Enterprise Adopted.
  • Dyne.org: Think &do tank and free software foundry
  • Puppet: Puppet Community
  • Vox Pupuli: Modules and tooling maintained by and for the Puppet community
  • LXC - Linux Containers: Linux container projects
  • Hashicorp: Consistent workflows to provision, secure, connect, and run any infrastructure for any application.
  • PingCAP: The team behind TiDB TiKV, an open source MySQL compatible HTAP database

Landscape

Blog DevOps and Tech

Note

Useful page for finding the target and gain more experience in DevOps, System and moreover in technology field

Tech Blog

  • Trang chแปง ยป AnonyViet - Vietnamese Community who teaching multiple techniques on Computer
  • Digital Ocean Community - Community offers thousands of tutorials, videos, and answers to questions on a wide range of topics
  • StackExchange - StackOverFlow Ecosystem
  • XDA Forums - Valuable resource for people who want to make the most of their mobile devices, from customizing the look and feel to adding new functionality.
  • How-To Geek - Community who explain Technologies
  • Hackaday.io - The worldโ€™s largest collaborative hardware development community.
  • Hackaday - Fresh Hacks Every Day
  • Opensource.com | Opensource.com - Community talk about Opensource - powered by Red Hat
  • 3os Project - Collocation of technical documentation and guides for devops, developers, pentesters, systems administrators and other IT professionals.
  • Medium - ITNext - Community in Medium where talk about lots of technological thing
  • Medium - Mr.PlanB - Write about Kubernetes, Promox, Homelab topics
  • Medium - Jake Page - Kubernetes Deployment Specialist @ Glasskube

DevOps Blog

Architecture Blog

EBPF Blog

  • Brendan Gregg - Cool guy with pleasant contents about ebpf and kernel technologies
  • Isovalent Blog - More about ebpf and kernel technologies, especially cilium and sub-projects of cilium

Tech Articles

Note

Collections articles about DevOps, System Design and moreover about technologies and opensource

General

Technologies

Techniques

System and Cloud Design

DevOps Youtube Channel

Research Sheet

Note

Topics which I concern about and do more inspect and research about them to provide knowledge for doing something around them in future

Opensource Finder

Note

Place to finding the cool stuff about Opensource and priorities software around the world

Documentations and Papers

Note

Place where you can find multiple documentation and papers about technologies in multiple topics from DevOps, Tech and AI

Quantum Computing

  • openqasm: Quantum assembly language for extended quantum circuits
  • qiskit: Qiskit is an open-source SDK for working with quantum computers at the level of extended quantum circuits, operators, and primitives.
  • QuantumKatas: Tutorials and programming exercises for learning Q# and quantum computing
  • pennylane: a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
  • open-quantum-safe: Software for prototyping quantum-resistant cryptography

New Programming Language

  • zig: General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software. Documentation
  • mojo: The Mojo Programming Language. Documentation
  • cue: An open-source data validation language with its roots in logic programming. Documentation

DevOps & Useful Tools

Note

Collections a supper cool and powerful tools for multiple purpose attaching with DevOps

API Gateway

  • Kong: API gateway built for hybrid and multi-cloud, optimized for microservices and distributed architectures

Automation Build Platforms

  • Expo: Expo is a framework that makes developing Android and iOS apps easier. Our framework provides file-based routing, a standard library of native modules, and much more
  • Buildpacks.io: Cloud Native Buildpacks (CNBs) transform your application source code intoย container imagesย that can run on any cloud.
  • lerna: ๐Ÿ‰ Lerna is a fast, modern build system for managing and publishing multiple JavaScript/TypeScript packages from the same repository.
  • nx: Smart Monorepos ยท Fast CI
  • just: a handy way to save and run project-specific commands.
  • electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
  • CMake: Mirror of CMake upstream repository. Plus on Make
  • harness: Harness Open Source is an end-to-end developer platform with Source Control Management, CI/CD Pipelines, Hosted Developer Environments, and Artifact Registries.

Automation Deployment Platforms

  • gitpod: The developer platform for on-demand cloud development environments to create software faster and more securely.
  • prefect: Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
  • werf: A solution for implementing efficient and consistent software delivery to Kubernetes facilitating best practices.
  • nelm: Nelm is a Helm 3 alternative and werf deployment engine
  • dapr: Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge. Website

Cloud Infrastructure Designer

  • Brainboard : An AI driven platform to visually design, generate terraform code and manage cloud infrastructure, collaboratively.
  • Cloudairy : Cloudchartย is a cloud focused diagram tool designed to help you easily map out cloud infrastructures
  • Excalidraw: Hand-drawn look & feel - Collaborative - Secure

Data integration platform

  • airbyte: The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

eBPF

  • ecapture: Capturing SSL/TLS plaintext without a CA certificate using eBPF. Supported on Linux/Android kernels for amd64/arm64. Documentation

Feature Flag

  • flagsmith: Open Source Feature Flagging and Remote Config Service. Host on-prem or use our hosted version atย https://flagsmith.com/
  • unleash: Open-source feature management solution built for developers.
  • flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solution

Infrastructure as Code (Machine + Configuration)

  • Vagrant : Vagrant is the command line utility for managing the lifecycle of virtual machines.
  • Ansible : Ansible provides open-source automation that reduces complexity and runs everywhere, especially VM
  • awx: AWX provides a web-based user interface, REST API, and task engine built on top of Ansible.

Infrastructure as Code (Cloud)

  • Pulumi Documentations : An open source infrastructure as code tool for creating, deploying, and managing cloud infrastructure.
  • Pulumi Registry: Package, SDK of Pulumi (Example: AWS, Azure, GCP)
  • Terraform Documentation: An infrastructure as code tool that lets you build, change, and version infrastructure safely and efficiently
  • Terraform Registry: Terraform providers that power all of Terraformโ€™s resource types, or find modules for quickly deploying common infrastructure configurations.
  • Terraform Best Practices: An attempt to systematically describe best practices using Terraform and provide recommendations for the most frequent problems Terraform users experience
  • OpenTofu: An infrastructure as code tool that lets you define infrastructure resources in human-readable configuration files that you can version, reuse, and share
  • Spacelift: Spacelift is a sophisticated, continuous integration and deployment (CI/CD) platform forย infrastructure-as-code
  • terraformer: CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code
  • inframap: Read your tfstate or HCL to generate a graph specific for each provider, showing only the resources that are most important/relevant.

Internal Developer Platform (IDP)

Identity Provider (SSO, OAuth, MFA, โ€ฆ)

  • authentik : ย An IdP (Identity Provider) and SSO (single sign on) that is built with security at the forefront of every piece of code, every feature, with an emphasis on flexibility and versatility.
  • FusionAuth : FusionAuth is a modern platform for Customer Identity and Access Management (CIAM). FusionAuth provides APIs and a responsive web user interface to support login, registration, localized email, multi-factor authentication, reporting and much more.
  • Authelia: Authelia is an open-sourceย authenticationย andย authorizationย server and portal fulfilling the identity and access management (IAM) role of information security in providingย multi-factor authenticationย and single sign-on (SSO) for your applications via a web portal. It acts as a companion forย common reverse proxies.
  • tailscale: The easiest, most secure way to use WireGuard and 2FA.
  • zitadel: Identity infrastructure, simplified forย you.
  • casdoor: An open-source UI-first Identity and Access Management (IAM) / Single-Sign-On (SSO) platform with web UI supporting
  • keycloak: Open Source Identity and Access Management For Modern Applications and Services. Website

Orchestration Framework

  • luigi: a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
  • camunda: Process Orchestration Framework. Website

Generated + Validate Configuration

  • moz://a SSL Configuration Generator : SSL Generation for multiple webserver
  • YAML Multiline Strings : Find the right syntax for YAML file
  • terraform-docs : A utility to generate documentation from Terraform modules in various output formats.
  • .gitignore.io: Create useful .gitignore files for your project
  • diagrams: ๐ŸŽจ Diagram as Code for prototyping cloud system architectures
  • crontab guru: The quick and simple editor for cron schedule expressions by Cronitor (SHELL/Pipeline)
  • checkov: Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages
  • Enable CORS: enable cross-origin resource sharing

Gitops

  • Atlantis - Terraform Pull Request Automation : Running Terraform Workflows with Ease
  • scalr : ย Remote operations backend for Terraform and OpenTofu (OTF)
  • digger : open-source CI/CD orchestrator for Terraform
  • env0:ย Automate and manage any Infrastructure-as-Code, from Terraform and OpenTofu to Pulumi, CloudFormation, Terragrunt, Kubernetes, and more.
  • bytebase : The GitHub/GitLab for database DevOps.
  • argo-cd : ย A declarative, GitOps continuous delivery tool for Kubernetes.
  • fluxcd : Tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.
  • Semaphore UI : Elegant web interface for executing Ansible playbooks, Terraform, OpenTofu, Pulumi code and Bash scripts.
  • act: Run your GitHub Actions locally ๐Ÿš€

Git Hosting

  • gitea: Painless self-hosted all-in-one software development service, including Git hosting, code review, team collaboration, package registry and CI/CD. Doc
  • gitlab : Download, install and maintain your own GitLab instance.
  • gogs: a painless self-hosted Git service. Doc

Local Cloud Self-hosted

Monitoring and observability

  • Netdata : The open-source observability platfrom everyone needs!

  • ELK: Combination from four main components. Guide, Helm and Docker

    1. Elasticsearch: a distributedย RESTfulย search engine which stores all of the collected data.
    2. Logstash: the data processing component of the Elastic Stack which sends incoming data to Elasticsearch.
    3. Kibana: a web interface for searching and visualizing logs.
    4. Beats: lightweight, single-purpose data shippers that can send data from hundreds or thousands of machines to either Logstash or Elasticsearch.

  • Grafana Stack: Ecosystem of Grafana for providing monitor methodology for your application, container, nodes and moreover. Helm, Docker and Guide

  • Zabbix : A software that monitors numerous parameters of a network and the health and integrity of servers, virtual machines, applications, services, databases, websites, the cloud and more

  • Checkmk : Offers comprehensive and specialized solutions for dealing with the very diverse environments of IT infrastructures.

  • gitana: Gitana is a lightweight dashboard sync

  • FlameGraph: Stack trace visualizer (Use for Pyroscope)

  • highlight: The open source, full-stack monitoring platform. Error monitoring, session replay, logging, distributed tracing, and more.

  • VictoriaMetrics: VictoriaMetrics: fast, cost-effective monitoring solution and time series database. Documentation

  • Log Viewer (Log4j, NLog): a high performance realtime log viewer via UDP (Chainsaw/NLogViewer) or text file.

  • vector: A high-performance observability data pipeline.

Serverless Self-hosted

  • fn: The container native, cloud agnostic serverless platform.
  • knative: An Open-Source Enterprise-level solution to build Serverless and Event Driven Applications. Documentation
  • serverless framework: โšก Serverless Framework โ€“ Effortlessly build apps that auto-scale, incur zero costs when idle, and require minimal maintenance using AWS Lambda and other managed cloud services.
  • faas: OpenFaaS - Serverless Functions Made Simple

Secrets management

  • dotenvx : Loads environment variables from .env for nodejs projects.
  • dotenv : The secrets manager for .env files โ€“ from the same people that pioneered dotenv.
  • sops : Simple and flexible tool for managing secrets
  • Hashicorp Vault: Vault is an identity-based secret and encryption management system. This documentation covers the main concepts of Vault, what problems it can solve, and contains a quick start for using Vault.
  • Ansible Vault: Ansible Vault is a feature of ansible that allows you to keep sensitive data such as passwords or keys in encrypted files, rather than as plaintext in playbooks or roles. These vault files can then be distributed or placed in source control.
  • trufflehog: Find, verify, and analyze leaked credentials.
  • infisical: the open-source secret management platform: Sync secrets across your team/infrastructure, prevent secret leaks, and manage internal PKI
  • teller: Cloud native secrets management for developers - never leave your command line for secrets.
  • doppler: The official CLI for interacting with your Doppler secrets and configuration. Documentation

Threat Intelligence

  • openappsec: A machine learning security engine that preemptively and automatically prevents threats against Web Application & APIs.
  • teleport: The easiest, and most secure way to access and protect all of your infrastructure.

Storage Platforms

  • MinIO: MinIO is an object storage solution that provides an Amazon Web Services S3-compatible API and supports all core S3 features.
  • Ceph: Ceph can be used to provideย Ceph Object Storageย toย Cloud Platformsย and Ceph can be used to provideย Ceph Block Deviceย services toย Cloud Platforms. Ceph can be used to deploy aย Ceph File System. Allย Ceph Storage Clusterย deployments begin with setting up eachย Ceph Nodeย and then setting up the network.
  • seaweedfs : a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering.
  • CubeFS : A next-generation cloud-native storage product that is currently an incubating open-source project hosted by theย Cloud Native Computing Foundationopen in new windowย (CNCF).
  • rclone: โ€œrsync for cloud storageโ€ - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

Service Discovery

  • nacos: An easy-to-use dynamic service discovery, configuration and service management platform for building cloud native applications.
  • consul: A multi-networking tool that offers a fully-featured service mesh solution

Service Proxy

  • envoy : Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures
  • traefik proxy : Traefik is anย open-sourceย Edge Routerย that makes publishing your services a fun and easy experience
  • HAProxy : HAProxy is a free, very fast and reliable reverse-proxy offeringย high availability,ย load balancing, and proxying for TCP and HTTP-based applications

Queue Services

  • AutoMQ : AutoMQ re-engineers Kafka for the cloud by decoupling storage to object storage. While maintainingย 100% compatibilityย with Apache Kafkaยฎ, it offers users up toย 10 times cost efficiencyย andย 100 times elasticity.
  • Apache Kafka: an open-source, distributed event streaming platform with message communication and storage capabilities (superb)
  • RabbitMQ: An open-source message broker for asynchronous messaging, queueing and delivery acknowledgement.
  • Apache ActiveMQ : An open-source Java-based message queue that can be accessed by clients written in Javascript, C, C++, Python and .NET.
  • Amazon Simple Queue System (SQS): A fully managed message queue built for distributed and serverless platforms
  • Microsoft Azure Service Bus: A reliable, fully managed Cloud service for delivering messages via queues or topics.
  • Celery: a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system.

SSL + DNS Utilities

  • certbot: Certbot is EFFโ€™s tool to obtain certs from Letโ€™s Encrypt and (optionally) auto-enable HTTPS on your server.
  • acme.sh: A pure Unix shell script implementing ACME client protocol
  • kcert: A Simple Letโ€™s Encrypt Manager for Kubernetes
  • ddns-updater: Container to update DNS records periodically with WebUI for many DNS providers
  • cloudflare-ddns: ๐ŸŽ‰๐ŸŒฉ๏ธ Dynamic DNS (DDNS) service based on Cloudflare! Access your home network remotely via a custom domain name without a static IP!

Utilities Terminal Tools

  • tldr: ๐Ÿ“š Collaborative cheatsheets for console commands

  • bat: A cat(1) clone with wings.

  • warp: The terminal reimagined with AI and collaborative tools for better productivity.

  • fzf: ๐ŸŒธ A command-line fuzzy finder

  • bpytop: Linux/OSX/FreeBSD resource monitor

  • entr: Run arbitrary commands when files change

  • pet: Simple command-line snippet manager

  • bcal: ๐Ÿ”ข Bits, bytes and address calculator

  • fd : A simple, fast and user-friendly alternative to โ€˜findโ€™

  • ohmyzsh: Framework for managing your zsh configuration

  • duf: Disk Usage/Free Utility - a better โ€˜dfโ€™ alternative

  • dust : A more intuitive version of du in rust

  • thefuck : Magnificent app which corrects your previous console command.

  • ripgrep : ripgrep recursively searches directories for a regex pattern while respecting your gitignore

  • exa: A modern replacement for โ€˜lsโ€™.

  • skopeo: Work with remote images registries - retrieving information, images, signing content

  • pre-commit: A framework for managing and maintaining multi-language pre-commit hooks.

    • pre-commit-hooks: Some out-of-the-box hooks for pre-commitSome out-of-the box hooks for pre-commit
    • husky: Git hooks made easy ๐Ÿถ woof!

  • auto-unlocker: Unlocker for VMWare macOS

  • darling: Darwin/macOS emulation layer for Linux

  • httpie: modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more. Documentation

Utilities and Fun Tools

  • Melty: The open source AI code editor for 10x engineers
  • spacedrive: Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.
  • localsend: An open-source cross-platform alternative to AirDrop (Local Vietnamese Guy ๐Ÿ––)
  • nhost: The Open Source Firebase Alternative with GraphQL. Documentation

Static Site Generator

  • gatsby: The best React-based framework with performance, scalability and security built in.
  • quartz: ๐ŸŒฑ a fast, batteries-included static-site generator that transforms Markdown content into fully functional websites
  • hugo: The worldโ€™s fastest framework for building websites.

Product Error Analytics and Monitoring

  • openreplay: Session replay and analytics tool you can self-host. Ideal for reproducing issues, co-browsing with users and optimizing your product.
  • opentelemetry: OpenTelemetry, also known as OTel, is a vendor-neutral open sourceย observabilityย framework for instrumenting, generating, collecting, and exporting telemetry data such asย traces,ย metrics, andย logs.
  • Sentry: Developer-first error tracking and performance monitoring. Website
  • GlitchTip: Simple, open source error tracking

Web server

  • traefik proxy : Traefik is anย open-sourceย Edge Routerย that makes publishing your services a fun and easy experience
  • Caddy server: Caddy is a powerful, extensible platform to serve your sites, services, and apps, written in Go
  • HAProxy : HAProxy is a free, very fast and reliable reverse-proxy offeringย high availability,ย load balancing, and proxying for TCP and HTTP-based applications
  • httpd - Apache2 : The Apache HTTP Server Project is an effort to develop and maintain an open-source HTTP server for modern operating systems including UNIX and Windows.
  • IIS: Window webserver
  • Nginx: nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server. (Most powerful)

Web Analytics

  • Plausible: Simple, open source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics.
  • umami: Umami is a simple, fast, privacy-focused alternative to Google Analytics.

Web Testing and Automation

  • playwright: Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API. Documentation
  • selenium: A browser automation framework and ecosystem. Documentation
  • jest: A delightful JavaScript Testing Framework with a focus on simplicity.

Graph View

Backlinks