Azkaban嵌套流的依赖
在Azkaban工作流的定义文件中可以添加子工作流
查看官方文档
https://azkaban.readthedocs.io/en/latest/createFlows.html
给出了
Embedded Flows
定义嵌套工作流的实例
nodes:
- name: jobC
type: noop
dependsOn:
- embedded_flow
- name: embedded_flow
type: flow
config:
prop: value
nodes:
- name: jobB
type: noop
dependsOn:
- jobA
- name: jobA
type: command
config:
command: pwd
结构如下
嵌套工作流之间的依赖
为了进一步理解,对上述示例进一步加深
目录中文件结构如下
|groupA
|--ga_1.sh
|--gn_2.sh
|groupB
|--gb_1.sh
|--gb_2.sh
|embedded.flow
|embedded.project
embedded.flow
工作流定义如下
nodes:
- name: jobA
type: noop
dependsOn:
- groupA
- name: groupA
type: flow
config:
prop: value
dependsOn:
- groupB
nodes:
- name: jobA1
type: command
config:
command: sh ./groupA/ga_1.sh
- name: jobA2
type: command
config:
command: sh ./groupA/ga_2.sh
- name: groupB
type: flow
config:
prop: value
nodes:
- name: jobB1
type: command
config:
command: sh ./groupB/gb_1.sh
- name: jobB2
type: command
config:
command: sh ./groupB/gb_2.sh
打包上产到Azkaban的project,可以查看工作流的结构如下
实现了不同嵌套流之间的依赖