Cloud Native
Cloud native application architecture and technology stack
List of free software network services and web applications that can be hosted on your own server.
Kubernetes - Kubernetes is an open source system for managing containerized applications on multiple hosts; provides basic mechanisms for application deployment, maintenance and expansion
Netdata - Netdata can perform distributed, real-time performance and health monitoring for systems or applications
The GitHub Statistics Generator is a simple yet powerful tool that allows you to display dynamically generated GitHub statistics in your README file. It offers a variety of styles and customization options, making it easy to embed into your GitHub projects and bringing them to life with added appeal.
Grafana - A tool for monitoring, metric analysis and dashboards for Graphite, InfluxDB and Prometheus, etc.
Prometheus - CNCF project, used to monitor other systems or services. It collects metrics from the target at a given time interval, evaluates them according to rules, displays the results, and can also trigger alarms if certain monitoring conditions are met
MinIO - MinIO is a high-performance object storage server compatible with Amazon S3 API
A DNS black hole, which can be set up on your own Linux hardware, enables network-wide ad blocking. It effectively shields your devices from unwanted ads and tracking content without the need to install any client software. Pi-hole is a robust tool that enhances your browsing experience by minimizing the negative impact of ads on network performance.
etcd is a reliable distributed key-value storage system used to store critical data in a distributed environment.
An end-to-end developer platform built on Docker. It provides a containerized microservice library that can be applied to web, mobile, and backend. Appwrite simplifies the API writing process through a visual interface, creating an efficient development environment for developers while ensuring software security.
Terraform - Terraform allows you to create, change and improve infrastructure in a safe and predictable manner. It is an open-source tool that encodes APIs as declarative profiles, which can be shared among team members just like code, edited, reviewed and version controlled.
A free big data analysis database management system (DBMS) designed for handling massive amounts of data. It provides powerful analytical functions that can be used for real-time queries and analysis of large-scale data sets, helping users quickly extract valuable information from massive data.
An open-source distributed NewSQL database that can elegantly replace traditional database middleware, database sharding, and other sharding schemes. TiDB has core features such as horizontal scalability, high availability, high concurrency, and strong consistency, making it suitable for various large-scale data processing and real-time analysis scenarios. It simplifies database operations and maintenance and improves data processing performance.
Istio - Istio is an open platform for providing a unified way to integrate microservices, manage traffic across microservices, implement policies and aggregate telemetry data
A free and open-source similarity search engine for massive feature vectors. Compared to operator libraries such as Faiss and SPTAG, Milvus provides a complete framework for vector data updates, indexing, and queries. Milvus utilizes GPU for index acceleration and query acceleration, significantly improving single-machine performance. Its deployment and usage are simple, reducing the difficulty of AI application implementation.
Nacos - A dynamic service discovery, configuration management and service management platform that is easier to build cloud-native applications.
An open source and free personal cloud computer tool, providing rich functions, ultimate experience and highly personalized customization, which can be used as a network disk, development environment, remote desktop, etc., and supports self-hosting to protect privacy.
An open-source cloud storage platform with rich features suitable for building a company's internal cloud storage platform. The platform provides file storage, synchronization, sharing, collaboration and more functions, enabling teams to efficiently manage and share files. Through Nextcloud, users can build their own secure and controllable cloud services to meet various needs of enterprises for cloud storage.
SurrealDB is a cloud-native, end-to-end database designed for modern applications, including web, mobile, serverless, and traditional systems. It simplifies development by combining database, API, and authentication layers into one platform, reducing the need for server-side components. Key features include real-time collaboration, support for SQL, GraphQL, and WebSocket connections, granular row-level access control, and multi-model data storage (tables, documents, and graph). SurrealDB supports structured and unstructured data, full-text indexing, geospatial querying, and embedded JavaScript functions for custom logic. It can be deployed as a single node or in a distributed cluster, ensuring scalability and high availability. With easy installation and integration across multiple platforms, SurrealDB accelerates secure, performant app development.
Consul - Consul is a distributed, highly available and datacenter aware solution for connecting and configuring applications across dynamic distributed infrastructures
Helm - a Kubernetes package manager, CNCF project
Envoy - Envoy is an open-source service proxy designed for cloud applications
Harbor - Harbor is an open source cloud native registry for storing, signing and scanning container images to find vulnerabilities
Loki - Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus
A portable event-driven runtime designed for building distributed applications across cloud and edge. Dapr provides modular building blocks that enable developers to easily create reliable and scalable microservice applications. It supports multiple programming languages and platforms, making it an ideal tool for developing distributed systems.
SkyWalking - APM tool (Application performance monitor) under distributed systems, designed for microservices, cloud-native and container-based (Docker, K8s, Mesos) architectures.
A set of Go microservice frameworks from Bilibili, including a large number of microservice-related frameworks and tools
A big data platform specifically designed and optimized for industries such as the Internet of Things (IoT) and application monitoring. Its database insertion and query operations are 10 times faster than other databases! It also consumes very low costs compared to other typical solutions in this category. TDengine only requires less than 1/5 of computing resources, and it provides interfaces for development in Java, C/C++, Python, Go, RESTful API, etc. Are you still worried about the performance of data writing, reading, and computing? With it, your hair survival rate will definitely increase significantly.
Pulumi - Pulumi's Infrastructure as Code SDK is the simplest way to create and deploy cloud software using containers, serverless functions, managed services, and infrastructure on any cloud.
An open-source public cloud file system that supports multiple cloud storage drivers, client direct upload and download speed limit, file management, user permission allocation, etc.
A relatively practical Kubernetes IDE, supporting several major desktop platforms such as Windows, Linux and macOS
Sentinel - (A lightweight Java library for traffic control, circuit breaker, and degradation)
RocketMQ - a distributed message and stream platform under Apache, with low latency, high performance, high reliability, trillions of capacity, and dynamic scalability.
A relatively practical open source tool, as an open source alternative to DataDog and NewRelic, which can help developers monitor and analyze project problems in real time
Cilium - Cilium provides and transparently protects network connectivity and load balancing between application workloads such as application containers or processes
Jaeger - Inspired by Dapper and OpenZipkin, Jaeger is an open source distributed tracing system released by Uber Technologies
The open source book "Advanced Programming in Go" covers advanced topics such as CGO, Go assembly language, RPC implementation, Protobuf plugin implementation, Web framework implementation, and distributed systems.
Vitess - Vitess is a database cluster system for horizontal scaling of MySQL
An open-source IoT platform written in Java, which can be used for data collection, processing, visualization and device management, etc.
Make front-end development simple and friendly, with a large number of reusable materials, and desktop tools that can build front-end applications quickly, improving efficiency by 100%.
containerd - an open and reliable container runtime
A framework for quickly developing API services, providing fully automated APIs for simple add, delete, modify and query operations as well as complex queries and simple transaction operations. With APIJSON, users no longer need to write interfaces and documents, greatly improving development efficiency.
kops - Kubernetes Operations (kops) - Production-grade K8s installation, upgrade and management tool
TiKV - a distributed transactional key-value database, initially created to complement TiDB
Open source learning resources on GitHub: "DevOps 2022 Technology Roadmap", which will help you quickly understand the latest DevOps technology stack. It includes various DevOps-related learning materials such as Git, common programming languages, Linux, network security, containers, IaC, CI/CD, etc.
OpenObserve is a cloud-native visualization monitoring platform specifically designed for logs, metrics, tracing, and analytics, engineered for PB-scale. It offers 10 times simplicity, 140 times lower storage costs, high performance, and an Elasticsearch/Splunk/Datadog alternative for PB-scale (logs, metrics, tracing).
SaltStack - SaltStack is a large-scale complex system management software that can be easily run within minutes, with scalability sufficient to manage tens of thousands of servers and speed sufficient to complete communication with them in seconds.
A tool for starting a development environment that can be used in any infrastructure. Similar to Terraform, but focused on code management for development environments. It supports any IDE and allows use on any cloud platform, Kubernetes cluster, or local host. Provides flexibility and customization, enabling developers to quickly start and manage development environments, thereby improving development efficiency.
Thanos - Thanos is a set of components that can be combined to form a highly available system with infinite storage capacity, which can seamlessly add to an existing Prometheus.
A distributed and scalable visualization DAG workflow task scheduling system. It is committed to solving the complex dependencies in data processing processes, making the scheduling system ready-to-use in data processing processes. DolphinScheduler provides rich task scheduling functions, supporting the visualization design and management of task flows.
Akka - Building highly concurrent, distributed and elastic message-driven applications on the JVM
Fluentd - Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop, etc.
CoreDNS - CoreDNS (written in Go) is a linkable plugin DNS server, each plugin performs DNS functions.
Rook - Rook is an open source cloud native storage orchestrator for Kubernetes that provides a platform, framework and support for various storage solutions to integrate locally with the cloud native environment
The latest learning route guide for data engineers in 2020, which contains: CS foundation, database foundation, relational database, cluster computing foundation, data processing, monitoring data pipeline, data security and privacy, etc.