yystopf
/
piflow
forked from opensci/piflow
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
piflow/piflow使用文档-v0.8.md

591 lines
128 KiB
Markdown
Raw Permalink Normal View History

Add files via upload
2020-09-29 19:19:34 +08:00
# 大数据流水线系统PiFlow 使用说明书v0.8
# 引言
## 编写目的
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
该文档主要用于介绍大数据流水线系统PiFlow 的使用
Add files via upload
2020-09-29 19:19:34 +08:00
## 建设范围
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
PiFlow server 及PiFlow web的使用说明
Add files via upload
2020-09-29 19:19:34 +08:00
## 术语
- PiFlow ：大数据流水线系统；
- Flow：大数据流水线；
- Stop：大数据流水线数据处理组件；
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- Path: 每个大数据流水线数据处理组件之间的连接线；
Add files via upload
2020-09-29 19:19:34 +08:00
- Group：大数据流水线组，支持流水线/流水线组的顺序调度；
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- Template：大数据流水线模板，支持将流水线/流水线组保存成模板、下载、上传和加载；
Add files via upload
2020-09-29 19:19:34 +08:00
- DataSource：数据源，支持FTP、JDBC、ElasticSearch、Hive等数据源注册，支持自定义数据源；
- Schedule：大数据流水线调度，支持流水线/流水线组的调度及定时调度
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- StopsHub：组件热插拔，支持用户开发自定义组件一键上传
Add files via upload
2020-09-29 19:19:34 +08:00
# 项目概述
大数据流水线系统PiFlow 主要是针对大数据的ETL工具，它具有如下特性：
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 简单易用
Add files via upload
2020-09-29 19:19:34 +08:00
- 提供所见即所得页面配置流水线
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 监控流水线状态
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 查看流水线日志
Add files via upload
2020-09-29 19:19:34 +08:00
- 检查点功能
- 调度功能
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 组件热插拔功能
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 可扩展性
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 支持用户自定义开发组件
Add files via upload
2020-09-29 19:19:34 +08:00
- 性能优越
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 基于分布式计算引擎Spark开发
Add files via upload
2020-09-29 19:19:34 +08:00
- 功能强大
- 提供100+数据处理组件
- 包括
spark、mllib、hadoop、hive、hbase、solr、redis、memcache、elasticSearch、jdbc、mongodb、http、ftp、xml、csv、json等.
# 使用说明
## 3.1 界面说明
### 3.1.1 注册
Update piflow使用文档-v0.8.md
2020-09-29 20:48:48 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/1.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.2 登录
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/2.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.4 首页
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
首页展示了资源使用情况，包括CPU、内存和磁盘。同时，展示了流水线Flow的总体情况、Group的总体情况、调度Schedule的总体情况、数据源DataSource的总体情况、数据处理组件的基本情况。其中Processor为运行态流水线/流水线组，状态可分为Started开始、完成Completed、失败Failed、Killed杀死、其他Other状态。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/3.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
同时，支持了国际化，如下图所示。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/4.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.4 流水线Flow
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.1 流水线列表
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/5.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 可点击进入流水线配置页面按钮，对流水线进行配置。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/6.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 可编辑流水线信息
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/7.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 可运行流水线
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/8.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 可以debug模式运行流水线
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/9.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 可删除流水线
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/10.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 可对流水线保存模板
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/11.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.2 创建流水线
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
用户点击创建按钮，创建流水线。需要输入流水线名称及描述信息，同时可设置流水线需要的资源。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/12.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.3 配置流水线
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 用户可通过拖拽方式进行流水线的配置，方式类似visio，如下图所示。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/13.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 画布左边栏显示组件组和组件，可按关键字搜索。用户选择好组件后可拖至画布中央。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/14.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 画布右侧显示流水线基本信息，包括流水线名称及描述。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/15.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 画布中央选择任一数据处理组件，右侧显示该数据处理组件的基本信息，包括名称，描述，作者等信息。选择AttributeInfo
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
Tab，显示该数据处理组件的属性信息，用户可根据实际需求进行配置。鼠标浮动到问号上会出现对应属性的说明，同时可以选择已设置好的数据源进行属性填充。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
数据处理组件基本信息如下图所示，点击StopName可对数据处理组件进行改名。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/16.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
数据处理组件属性信息设置如下图所示。“问号”按钮提示该属性描述信息，“红星”表示必填项。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/17.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
数据处理组件属性样例信息如下图所示：
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/18.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
数据处理组件数据源填充如下图所示。已选择数据源相关数据会自动填充到所选数据处理组件中。数据源变更后，相应组件的属性也会随之更新。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/19.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.4 运行流水线
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 用户配置好流水线后，可点击运行按钮运行流水线。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/20.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.5 流水线监控
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
加载完成之后，进入流水线监控页面。监控页面会显示整条流水线的执行状况，包括运行状态、执行进度、执行时间等。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/21.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
点击具体数据处理组件，显示该数据处理组件的运行状况，包括运行状态、执行时间。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/22.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.6 流水线日志
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:58:21 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/23.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.4.7 调试流水线
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 可以以Debug模式运行流水线，运行后可查看流经每条线上的数据信息，实现数据可溯源
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/24.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/25.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/26.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.4.8 检查点
- 流水线可设置检查点，再次运行时可选择从检查点运行
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/27.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.5 流水线组Group
#### 3.1.5.1 流水线组列表
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 流水线组支持流水线的顺序调度功能，组嵌套功能。列表功能与流水线列表功能一致。列表支持进入、编辑、运行/停止、删除、保存模板功能。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/28.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.5.2 新建流水线组
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 点击创建按钮，输入流水线组名称和基本信息可创建流水线组Group。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/29.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.5.3 配置流水线组
##### 3.1.5.3.1 创建group
- 拖动左侧group图标
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/30.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.2 创建flow
- 拖动flow图标创建流水线flow
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/31.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.3 创建label
- 拖动Label可添加标签，用于备注说明
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/32.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.4 创建调度关系
- 连线实现调度顺序
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/33.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.5 创建子group
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- Group可双击进入，配置组内流水线组group、流水线flow、以及之间的调度顺序。下部有导航栏，可退出该级目录，返回上一级。同时可以返回根目录。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/34.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.6 配置流水线flow
- 双击flow图标，可进入具体流水线的配置界面
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/35.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.7 导入流水线flow
- 可导入flow列表中已配置的流水线
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/36.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
##### 3.1.5.3.8 更换图标
- 右键group或flow，可支持更换图标
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/37.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- Group图标列表，支持用户上传
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/38.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- Flow图标列表，支持用户上传
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/39.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.5.4 运行流水线组
- 运行
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/40.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 可右键运行单个group或flow
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/41.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/42.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.5.5 监控流水线组
- 默认显示流水线组监控信息
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/43.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 单击group或flow，显示点击组件的监控信息
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/44.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
可双击进入group或flow，查看进一步监控信息
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.5.6 流水线组日志
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/45.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.6 运行态流水线Process
已运行流水线组和流水线会显示在Process
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
List中，包括开始时间、结束时间、进度、状态等。同时可对已运行流水线进行查看，在运行，停止，和删除操作。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/46.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.7 模板Template
流水线组和流水线可保存成模板
#### 3.1.7.1 模板保存
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 流水线保存模板
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/47.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 流水线组保存模板
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/48.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.7.2 模板列表
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/49.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.7.3 模板下载
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/50.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.7.4 模板上传
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/51.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.7.5 模板加载
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 流水线模板加载
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/52.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
- 流水线组模板加载
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/53.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
#### 3.1.7.6 模板删除
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/54.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
### 3.1.8 数据源
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.8.1 创建数据源
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
- 支持JDBC、ElasticSearch、等数据源的创建。同时支持自定义数据源（other）
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/55.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
#### 3.1.8.1 使用数据源
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
在流水线配置页面，选择某个组件，设置该组件属性时，可从已配置数据源填充相关属性。同时数据源变更时，该组件属性也随之变更。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/56.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
### 3.1.9 定时调度
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
支持对已配置流水线/流水线组进行定时调度，调度采用Cron表达式进行设置，支持设置调度的开始时间和结束时间。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
调度列表如下图所示，支持进入流水线/流水线组进行配置，支持编辑调度信息，启动调度、停止调度和删除调度。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/57.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
新建调度如下图所示。需选择调度的类型（Flow/FlowGroup），Cron表达式，调度开始时间，调度结束时间，以及被调度的流水线/流水线组。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/58.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
被调度成功的流水线会在Processor列表页显示。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/59.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
### 3.1.10 组件热插拔
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 20:33:44 +08:00
支持上传自定义开发组件jar包，mount成功后流水线配置页面会自动显示自定义开发组件。Unmount后，自定义开发组件会消失。
Add files via upload
2020-09-29 19:19:34 +08:00
Update piflow使用文档-v0.8.md
2020-09-29 21:03:37 +08:00
![](https://github.com/cas-bigdatalab/piflow/blob/master/doc/V0.8/60.png?raw=true)
Add files via upload
2020-09-29 19:19:34 +08:00
## 3.2 Restful API
接口采用REST设计风格，目前需求如下接口：
### 3.2.1 getAllGroups
| **基本信息** | | | |
|----------------|--------------------------------|------------------------------------------|----------|
| 接口名称 | getAllGroups | | |
| 接口描述 | 获取所有数据处理组件Stop所在组 | | |
| 接口URL | GET /stop/ groups | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| 无 | | | |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回所有组信息 | 200 | {“groups”:”Common,Hive,Http,…”} | |
| | 500 | “getGroup Method Not Implemented Error!” | |
### 3.2.2 getAllStops
| **基本信息** | | | |
|----------------|----------------------|---------------------------------------------|----------|
| 接口名称 | getAllStops | | |
| 接口描述 | 获取所有数据处理组件 | | |
| 接口URL | GET /stop/list | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| 无 | | | |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回所有Stop | 200 | {“stops”:”cn.PiFlow .bundle.Common.Fork,…”} | |
| | 500 | “Can not found stop !” | |
### 3.2.3 getStopInfo
| **基本信息** | | | |
|------------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
| 接口名称 | getStopInfo | | |
| 接口描述 | 获取数据处理组件Stop的详细信息 | | |
| 接口URL | GET /stop/info?bundle=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| bundle | Stop的类名 | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Stop详细信息 | 200 | {"name":"LoadFromFtp","bundle":"cn.PiFlow .bundle.ftp.LoadFromFtp",“groups”:"ftp,load","description":"load data from ftp server","properties":[{"property":{"name":"url_str","displayName":"URL","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}},{"property":{"name":"port","displayName":"PORT","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}},{"property":{"name":"username","displayName":"USER_NAME","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}},{"property":{"name":"password","displayName":"PASSWORD","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}},{"property":{"name":"ftpFile","displayName":"FTP_File","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}},{"property":{"name":"localPath","displayName":"Local_Path","description":null,"defaultValue":"","allowableValues":"","required":"true","sensitive":"false"}}]} | |
| | 500 | “get PropertyDescriptor or getIcon Method Not Implemented Error!” | |
### 3.2.4 startFlow
| **基本信息** | | |
|----------------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 接口名称 | startFlow | |
| 接口描述 | 运行流水线 | |
| 接口URL | POST /flow/start | |
| **参数说明** | | |
| 描述 | 类型 | 实例 |
| Flow的json配置字符串 | String | {"flow":{"name":"test","uuid":"1234","stops":[{"uuid":"1111","name":"XmlParser","bundle":"cn.PiFlow .bundle.xml.XmlParser","properties":{"xmlpath":"hdfs://10.0.86.89:9000/xjzhu/dblp.mini.xml","rowTag":"phdthesis"}},{"uuid":"2222","name":"SelectField","bundle":"cn.PiFlow .bundle.common.SelectField","properties":{"schema":"title,author,pages"}},{"uuid":"3333","name":"PutHiveStreaming","bundle":"cn.PiFlow .bundle.hive.PutHiveStreaming","properties":{"database":"sparktest","table":"dblp_phdthesis"}},{"uuid":"4444","name":"CsvParser","bundle":"cn.PiFlow .bundle.csv.CsvParser","properties":{"csvPath":"hdfs://10.0.86.89:9000/xjzhu/phdthesis.csv","header":"false","delimiter":",","schema":"title,author,pages"}},{"uuid":"555","name":"Merge","bundle":"cn.PiFlow .bundle.common.Merge","properties":{}},{"uuid":"666","name":"Fork","bundle":"cn.PiFlow .bundle.common.Fork","properties":{"outports":["out1","out2","out3"]}},{"uuid":"777","name":"JsonSave","bundle":"cn.PiFlow .bundle.json.JsonSave","properties":{"jsonSavePath":"hdfs://10.0.86.89:9000/xjzhu/phdthesis.json"}},{"uuid":"888","name":"CsvSave","bundle":"cn.PiFlow .bundle.csv.CsvSave","properties":{"csvSavePath":"hdfs://10.0.86.89:9000/xjzhu/phdthesis_result.csv","header":"true","delimiter":","}}],"paths":[{"from":"XmlParser","outport":"","inport":"","to":"SelectField"},{"from":"SelectField","outport":"","inport":"data1","to":"Merge"},{"from":"CsvParser","outport":"","inport":"data2","to":"Merge"},{"from":"Merge","outport":"","inport":"","to":"Fork"},{"from":"Fork","outport":"out1","inport":"","to":"PutHiveStreaming"},{"from":"Fork","outport":"out2","inport":"","to":"JsonSave"},{"from":"Fork","outport":"out3","inport":"","to":"CsvSave"}]} |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回flow的appId | 200 | {“flow”:{“id”:”\*\*\*”,”pid”:””\*\*\*}} |
| | 500 | “Can not start flow!” |
### 3.2.5 stopFlow
| **基本信息** | | |
|----------------|-----------------|--------------------------------|
| 接口名称 | stopFlow | |
| 接口描述 | 停止流水线 | |
| 接口URL | POST /flow/stop | |
| **参数说明** | | |
| 描述 | 类型 | 实例 |
| Flow的appID | String | {“appID”:”\*\*\*”} |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回执行状态 | 200 | “ok” |
| | 500 | “Can not found process Error!” |
### 3.2.6 getFlowInfo
| **基本信息** | | | |
|------------------|-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
| 接口名称 | getFlowInfo | | |
| 接口描述 | 获取流水线Flow的信息 | | |
| 接口URL | GET /flow/info?appID=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| appID | Flow的Id | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Flow详细信息 | 200 | {"flow":{"id":"application_1540442049798_0297","pid":"process_372bd7da-a53e-46b4-8c44-edc0463064f5_1","name":"xml,csv-merge-fork-hive,json,csv","state":"COMPLETED","startTime":"Tue Nov 27 14:37:03 CST 2018","endTime":"Tue Nov 27 14:37:28 CST 2018","stops":[{"stop":{"name":"JsonSave","state":"COMPLETED","startTime":"Tue Nov 27 14:37:24 CST 2018","endTime":"Tue Nov 27 14:37:28 CST 2018"}},{"stop":{"name":"CsvSave","state":"COMPLETED","startTime":"Tue Nov 27 14:37:20 CST 2018","endTime":"Tue Nov 27 14:37:24 CST 2018"}},{"stop":{"name":"PutHiveStreaming","state":"COMPLETED","startTime":"Tue Nov 27 14:37:13 CST 2018","endTime":"Tue Nov 27 14:37:20 CST 2018"}},{"stop":{"name":"Fork","state":"COMPLETED","startTime":"Tue Nov 27 14:37:13 CST 2018","endTime":"Tue Nov 27 14:37:13 CST 2018"}},{"stop":{"name":"Merge","state":"COMPLETED","startTime":"Tue Nov 27 14:37:11 CST 2018","endTime":"Tue Nov 27 14:37:11 CST 2018"}},{"stop":{"name":"SelectField","state":"COMPLETED","startTime":"Tue Nov 27 14:37:11 CST 2018","endTime":"Tue Nov 27 14:37:11 CST 2018"}},{"stop":{"name":"XmlParser","state":"COMPLETED","startTime":"Tue Nov 27 14:37:09 CST 2018","endTime":"Tue Nov 27 14:37:11 CST 2018"}},{"stop":{"name":"CsvParser","state":"COMPLETED","startTime":"Tue Nov 27 14:37:03 CST 2018","endTime":"Tue Nov 27 14:37:09 CST 2018"}}]}} | |
| | 500 | “appID is null or flow run failed!” | |
### 3.2.7 getFlowProgress
| **基本信息** | | | |
|----------------|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|----------|
| 接口名称 | getFlowProgress | | |
| 接口描述 | 获取流水线Flow的执行进度 | | |
| 接口URL | GET /flow/progress?appID=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| appID | Flow的Id | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Flow的进度 | 200 | {"flow":{"appId":"application_1540442049798_0297","name":"xml,csv-merge-fork-hive,json,csv","state":"COMPLETED","progress":"100%"}} | |
| | 500 | “appId is null or flow run failed!” | |
### 3.2.8 getFlowLog
| **基本信息** | | | |
|---------------------|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
| 接口名称 | getFlowProgress | | |
| 接口描述 | 获取流水线Flow的执行进度 | | |
| 接口URL | GET /flow/log?appID=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| appID | Flow的Id | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Flow的log的地址 | 200 | {"app":{"id":"application_1540442049798_0297","user":"root","name":"xml,csv-merge-fork-hive,json,csv","queue":"default","state":"FINISHED","finalStatus":"SUCCEEDED","progress":100.0,"trackingUI":"History","trackingUrl":"http://master:8088/proxy/application_1540442049798_0297/A","diagnostics":"","clusterId":1540442049798,"applicationType":"SPARK","applicationTags":"","startedTime":1543300611067,"finishedTime":1543300648590,"elapsedTime":37523,"amContainerLogs":"http://master:8042/node/containerlogs/container_1540442049798_0297_01_000001/root","amHostHttpAddress":"master:8042","allocatedMB":-1,"allocatedVCores":-1,"runningContainers":-1,"memorySeconds":217375,"vcoreSeconds":105,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0}} | |
| | 500 | “appID is null or flow does not exist!” | |
### 3.2.9 getFlowCheckPoints
| **基本信息** | | | |
|-----------------------|------------------------------------|-----------------------------------------|----------|
| 接口名称 | getFlowCheckPoints | | |
| 接口描述 | 获取流水线Flow的checkPoints | | |
| 接口URL | GET /flow/checkpoints?appID=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| appID | Flow的appID | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Flow的checkpoints | 200 | {"checkpoints":"Merge,Fork"} | |
| | 500 | “appID is null or flow does not exist!” | |
### 3.2.10 getFlowDebugData
| **基本信息** | | | |
|----------------------------------------------|----------------------------------|-----------------------------------------|----------|
| 接口名称 | getFlowDebugData | | |
| 接口描述 | 获取流水线Flow的调试数据 | | |
| 接口URL | GET /flow/debugData?appID=\*\*\* | | |
| **参数说明** | | | |
| 名称 | 描述 | 类型 | 数据类型 |
| appID | Flow的appID | Query | String |
| stopName | stop的名称 | Query | String |
| Port | Stop的端口名 | Query | String |
| **返回值说明** | | | |
| 描述 | 返回代码 | 实例 | |
| 返回Flow的指定stop和端口的调试数据的hdfs路径 | 200 | {"schema":””, “debugDataPath”:" "} | |
| | 500 | “appID is null or flow does not exist!” | |
### 3.2.11 startFlowGroup
| **基本信息** | | |
|---------------------------|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 接口名称 | startFlowGroup | |
| 接口描述 | 运行流水线组 | |
| 接口URL | POST /group/start | |
| **参数说明** | | |
| 描述 | 类型 | 实例 |
| FlowGroup的json配置字符串 | String | {"group":{"flows":[{"flow":{"executorNumber":"1","driverMemory":"1g","executorMemory":"1g","executorCores":"1","paths":[{"inport":"","from":"MockData","to":"ShowData","outport":""}],"name":"f4","stops":[{"customizedProperties":{},"name":"MockData","uuid":"8a80d63f720cdd2301723b7745b72649","bundle":"cn.PiFlow .bundle.common.MockData","properties":{"schema":"title:String,author:String,age:Int","count":"10"}},{"customizedProperties":{},"name":"ShowData","uuid":"8a80d63f720cdd2301723b7745b72647","bundle":"cn.PiFlow .bundle.external.ShowData","properties":{"showNumber":"5"}}],"uuid":"8a80d63f720cdd2301723b7745b62645"}},{"flow":{"executorNumber":"1","driverMemory":"1g","executorMemory":"1g","executorCores":"1","paths":[{"inport":"","from":"MockData","to":"ShowData","outport":""}],"name":"f3","stops":[{"customizedProperties":{},"name":"MockData","uuid":"8a80d63f720cdd2301723b7745b9265d","bundle":"cn.PiFlow .bundle.common.MockData","properties":{"schema":"title:String,author:String,age:Int","count":"10"}},{"customizedProperties":{},"name":"ShowData","uuid":"8a80d63f720cdd2301723b7745b9265b","bundle":"cn.PiFlow .bundle.external.ShowData","properties":{"showNumber":"5"}}],"uuid":"8a80d63f720cdd2301723b7745b82659"}}],"name":"SimpleGroup","groups":[{"group":{"flows":[{"flow":{"executorNumber":"1","driverMemory":"1g","executorMemory":"1g","executorCores":"1","paths":[{"inport":"","from":"MockData","to":"ShowData","outport":""}],"name":"MockData","stops":[{"customizedProperties":{},"name":"MockData","uuid":"8a80d63f720cdd2301723b7745b4261a","bundle":"cn.PiFlow .bundle.common.MockData","properties":{"schema":"title:String,author:String,age:Int","count":"10"}},{"customizedProperties":{},"name":"ShowData","uuid":"8a80d63f720cdd2301723b7745b32618","bundle":"cn.PiFlow .bundle.external.ShowData","properties":{"showNumber":"5"}}],"uuid":"8a80d63f720cdd2301723b7745b32616"}},{"flow":{"executorNumber":"1","driverMemory":"1g","executorMemory":"1g","executorCores":"1","paths":[{"inport":"","from":"MockData","to":"ShowData","outport":""}],"name":"MockData","stops":[{"customizedProperties":{},"name":"MockData","uuid":"8a80d63f720cdd2301723b7745b5262e","bundle":"cn.PiFlow .bundle.common.MockData","properties":{"schema":"title:String,author:String,age:Int","count":"10"}},{"customizedProperties":{},"name":"ShowData","uuid":"8a80d63f720cdd2301723b7745b5262c","bundle":"cn.PiFlow .bundle.external.ShowData","properties":{"showNumber":"5"}}],"uuid":"8a80d63f720cdd2301723b7745b4262a"}}],"name":"g1","uuid":"8a80d63f720cdd2301723b7745b22615"}}],"conditions":[{"entry":"f4","after":"g1"},{"entry":"f3","after":"g1"}],"uuid":"8a80d63f720cdd2301723b7745b22614"}} |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回flowGroup的Id | 200 | {"group":{"id":"group_fc1cb223-9c44-467f-a063-e959ffb6bcd8"}} |
| | 500 | “Can not start group!” |
### 3.2.12 stopFlowGroup
| **基本信息** | | |
|---------------------------|------------------|----------------------------------|
| 接口名称 | stopFlowGroup | |
| 接口描述 | 停止流水线组 | |
| 接口URL | POST /group/stop | |
| **参数说明** | | |
| 描述 | 类型 | 实例 |
| FlowGroup的json配置字符串 | String | {“groupId”:”\*\*\*”} |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回停止操作的状态 | 200 | “Stop FlowGroup OK!!!” |
| | 500 | “Can not found FlowGroup Error!” |
### 3.2.13 getFlowGroupInfo
| **基本信息** | | |
|----------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 接口名称 | getFlowGroupInfo | |
| 接口描述 | 获取流水线组信息 | |
| 接口URL | GET /group/info?groupID=\*\*\* | |
| **参数说明** | | |
| 名称 | 描述 | 类型 |
| groupId | GroupId | String |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回group信息 | 200 | {"group":{"name":"SimpleGroup","startTime":"FriMay2918:10:50CST2020","state":"STARTED","flows":[],"groups":[{"group":{"name":"g1","startTime":"FriMay2918:10:50CST2020","state":"COMPLETED","flows":[{"flow":{"name":"MockData","startTime":"FriMay2918:11:03CST2020","state":"COMPLETED","endTime":"FriMay2918:11:07CST2020","id":"application_1589249052248_0440","pid":"process_b3c96bf0-c9b4-41b1-b0e0-06fb2d5e4be5_1","progress":"100","stops":[{"stop":{"name":"ShowData","state":"COMPLETED","startTime":"FriMay2918:11:07CST2020","endTime":"FriMay2918:11:07CST2020"}},{"stop":{"name":"MockData","state":"COMPLETED","startTime":"FriMay2918:11:03CST2020","endTime":"FriMay2918:11:07CST2020"}}]}}],"groups":[],"endTime":"FriMay2918:11:20CST2020","id":"group_2322a41d-7b69-4fe7-9a87-a78c50f26e09"}}],"endTime":"","id":"group_0a7abbd3-9c9a-4dfa-9a0b-7f77fdacf3d4"}} |
| | 500 | “Can not found FlowGroup Error!” |
### 3.2.14 getFlowGroupProgress
| **基本信息** | | |
|----------------|------------------------------------|----------------------------------------------------|
| 接口名称 | getFlowGroupProgress | |
| 接口描述 | 获取流水线group的执行进度 | |
| 接口URL | GET /group/progress?groupId=\*\*\* | |
| **参数说明** | | |
| 名称 | 描述 | 类型 |
| groupId | Group的id | String |
| **返回值说明** | | |
| 描述 | 返回代码 | 实例 |
| 返回执行进度 | 200 | “100” |
| | 500 | “groupId is null or flowGroup progress exception!” |