English | 简体中文
Azkaban Azkaban is an open source batch workflow task scheduler by LinkedIn that runs a set of jobs and processes in a specific order within a workflow. It helps users manage and track workflows by defining dependencies between tasks and providing a web user interface. Azkaban is suitable for scheduling various types of jobs, such as Command, Hadoop MapReduce, Hive, Spark, and Pig, and supports custom plugins.
Core Features:
- Batch Workflow Scheduling: Azkaban allows users to define and execute workflows with complex dependencies.
- Web User Interface: Provides an easy-to-use web interface for creating, maintaining, monitoring, and tracking workflows.
- Task Dependencies: Azkaban uses job configuration files to define dependencies between tasks, ensuring jobs are executed in the correct order.
- Support for Multiple Job Types: Can schedule various types of jobs, such as Command, Hadoop MapReduce, Hive, Spark, and Pig.
- Pluggable Plugin Mechanism: Supports custom plugins to extend Azkaban's functionality, such as supporting new job types or integrating with other systems.
- Distributed Execution: Azkaban can improve scheduling capacity and reliability through multiple executors.
- Error Handling and Retry: Supports a retry mechanism to automatically retry tasks when they fail.
- Security Management: Provides authentication and authorization mechanisms to ensure only authorized users can access and operate workflows.
- History and Auditing: Records job execution history for easy auditing and troubleshooting.
This project offers pre-configured Azkaban-Task Scheduler，images with Azkaban and its runtime environment pre-installed, along with deployment templates. Follow the guide to enjoy an "out-of-the-box" experience.
Architecture Design:
System Requirements:
- CPU: 4vCPUs or higher
- RAM: 16GB or more
- Disk: At least 50GB
Register a Huawei account and activate Huawei Cloud
| Image Version | Description | Notes |
|---|---|---|
| Azkaban24.1.3-arm-v1.0 | Deployed on Kunpeng servers with Huawei Cloud EulerOS 2.0 64bit |
- Submit an issue
- Contact Huawei Cloud Marketplace product support
- Fork this repository and submit a merge request.
- Update README.md synchronously based on your open-source mirror information.