A unified management architecture leveraging modern abstract design patterns and microservices architecture to provide the most flexible and solid foundation for running and managing data services.
Architecture Overview
UPM implements a classic “hub-agent” architectural pattern, consisting of two primary layers: the management platform layer and the execution engine layer. This architecture effectively supports Kubernetes multi-cluster orchestration while maintaining a powerful, modular, and scalable platform. Moving away from decentralized design and imperative architecture, UPM embraces the “hub-agent” pattern that aligns with Kubernetes’ native “hub-kubelet” architecture. In this design, the management platform layer functions as the “Hub,” while each managed cluster operates as an “agent.” Let’s explore the two core components of UPM:
UPM Platform: Serves as the central control platform, providing external API interfaces and managing Kubernetes API resource objects (CRs) through workflows. Built on a microservices architecture, it scales efficiently to support various database and middleware operational requirements.
UPM Engine: Functions as the execution engine, built using the Kubernetes Operator framework. This Kubernetes-based extension suite primarily focuses on stateful service automation and advanced operational automation for data applications. It comprises two custom-developed Operators and features a generalized workload design to accommodate diverse database types and multi-database cluster architectures.
The architectural advantages enable us to decouple multi-cluster operations into two distinct aspects:
- Computation/Decision-making
- Execution
This separation allows the UPM Engine to handle actual cluster execution tasks, while the UPM Platform focuses on task orchestration and interface management through workflows. Instead of direct execution, the UPM Engine maintains cluster state declaratively through CRDs. The UPM Platform actively monitors CRD status from Kubernetes and orchestrates subsequent tasks accordingly.
This design significantly reduces the operational burden on the UPM Platform by:
- Eliminating the need to handle event floods from Kubernetes clusters
- Reducing direct request volume to individual Kubernetes clusters
To put this in perspective: imagine managing a 5,000-node Kubernetes cluster without kubelets, where the control plane directly operates container daemons - such centralized control would be extremely challenging. UPM addresses this scalability challenge by distributing execution across individual agents, enabling the UPM Platform to efficiently manage multiple clusters.
UPM Platform Architecture
The UPM Platform serves as the core control center, providing external API interfaces and managing execution workflows. Built on a microservices architecture, it offers exceptional scalability and flexibility.
Architecture
UPM Platform Overview
The UPM Platform serves as the core control center, providing external API interfaces and managing execution workflows. Built on a microservices architecture, it offers exceptional scalability and flexibility.
Core Components
The UPM Platform consists of three main components:
UPM UI: A unified management interface that provides administrators with comprehensive control and monitoring capabilities through an intuitive web-based dashboard.
UPM API-Server: The management core, built on the SpringCloud microservices framework. It delivers:
- RESTful API endpoints for management operations
- MySQL-based data persistence
- Redis-powered caching for optimized query performance
Helix: An extensible platform component that enhances database and middleware service capabilities. Currently supports MySQL user and database management functionalities.
Microservices Architecture
The UPM API-Server plays a pivotal role as the management core. Its microservices architecture is specifically designed to meet both current operational requirements and emerging service management needs from the open-source community.
Key architectural benefits:
- Decomposed functionality into focused, cohesive services
- Loose coupling between services
- Independent databases per service
- API-based communication
- Individual service scalability
- Independent development, deployment, testing, and scaling capabilities
List of microservices modules
| Items | Name | Description |
|---|---|---|
| 1 | Gateway API | The Gateway API module is responsible for receiving external requests and routing them to the corresponding microservice module |
| 2 | Auth | The Auth module is responsible for user authentication and permission management |
| 3 | Resource | The Resource module provides management of system resources, such as projects, Kubernetes clusters, nodes, storage classes, software, etc. |
| 4 | User | The User module is responsible for the management and operation of user information. |
| 5 | OperatorLog | The OperatorLog module records system operation logs for tracking system operations. |
| 6 | MySQL service | The MySQL service module provides complete operation and maintenance workflow control and management functions for MySQL databases. |
| 7 | Redis service | The Redis service module provides complete operation and maintenance workflow control and management functions for Redis cache. |
| Redis-Cluster service | Redis-Cluster service module provides complete operation and maintenance workflow control and management functions for Redis cluster cache. | |
| 8 | Kafka service | The Kafka service module provides complete operation and maintenance workflow control and management functions for Kafka event flow. |
| 9 | Zookeeper service | The Zookeeper service module provides complete operation and maintenance workflow control and management functions for Zookeeper Service Discovery. |
| 10 | Elasticsearch service | The Elasticsearch service module provides complete operational workflow control and management capabilities for the Elasticsearch search engine. |
UPM Engine Overview
The UPM Engine functions as the execution layer, built using the Kubernetes Operator framework. This Kubernetes-based extension suite specializes in stateful service automation and advanced operational automation for data applications.
Architecture
Core Components
The UPM Engine comprises two primary components:
Unit Operator (code name: Tesseract Cube): A general-purpose workload operator that enhances database and middleware stateful services through:
- Unified workload management (Units and UnitSets)
- Template-based service definitions
- Extensible configurations for specific service types and versions
Compose Operator (code name: Kauntlet): An advanced operations operator that extends complex operational capabilities through Custom Resource Definitions (CRDs).
Operator Design Philosophy
Our operator design is based on the control loop concept from robotics and automation. In Kubernetes environments:
- Controllers continuously monitor cluster state
- The system handles constant change gracefully
- Multiple controllers manage specific aspects of cluster state
- Simple, focused controllers provide better reliability than complex, interconnected ones
Unit Operator Details
Unit Operator (code name: Tesseract Cube) unifies database and middleware workload management through:
Unit Management
Units represent complete service instances, encompassing:
- Pod definitions
- Service configurations
- Persistent storage
- Lifecycle management
Unit Management Architecture
UnitSet Operations
UnitSets manage stateful database instance collections, providing:
- Deployment and scaling capabilities
- Persistent storage management
- Configuration templating
- Security features (encryption, authentication)
- Rolling update policies
UniSet Management Architecture
Compose Operator Capabilities
Compose Operator (code name: Kauntlet) extends Kubernetes’ database management capabilities through specialized CRDs:
Supported Features
MySQL Replication Management
- Leader/follower topology management
- Automated failover
- Read/write separation
- Replication monitoring
Redis Replication Management
- Leader/follower configuration
- Sentinel architecture support
- Automated failover
- State monitoring
Redis Cluster Management
- Automated cluster configuration
- Dynamic slot allocation
- High availability features
- Performance optimization
ProxySQL Synchronization
- Topology monitoring
- MySQL server configuration
- User synchronization
- Load balancing
- Failover management
Compose Operator significantly simplifies database cluster management while ensuring high availability and operational efficiency. Its extensible design allows for custom capabilities to meet specific operational requirements.
Note: Please check High Availability section for further information.
