judy0131
|
dd11c444a2
|
1.implement checkpoint
|
2018-11-20 16:46:47 +08:00 |
|
judy0131
|
4ab88005d7
|
add flow resource configuration
|
2018-11-07 16:49:22 +08:00 |
|
judy0131
|
7c65a7510a
|
return flow info when the flow's state waiting
|
2018-10-29 15:14:23 +08:00 |
|
judy0131
|
8f93a619cd
|
update flow example
|
2018-10-29 14:59:19 +08:00 |
|
judy0131
|
7f459ee008
|
add server log
|
2018-10-26 17:26:45 +08:00 |
|
judy0131
|
6b6a1588ec
|
1.add h2 port
2.fix bug: multi server
|
2018-10-25 13:18:14 +08:00 |
|
judy0131
|
c6db0d1acb
|
add getFlowProgress and getFlowLog API
|
2018-10-24 18:13:40 +08:00 |
|
judy0131
|
80e3e874d6
|
monitor flow info and stop info
|
2018-10-24 17:33:46 +08:00 |
|
judy0131
|
c329a8459f
|
fix bug: merge and fork
|
2018-10-22 15:13:14 +08:00 |
|
judy0131
|
554cd25dc4
|
fix checkpoint bug
|
2018-10-22 12:22:55 +08:00 |
|
judy0131
|
93812c5783
|
fix stop flow bug
|
2018-10-19 15:52:19 +08:00 |
|
judy0131
|
416594d594
|
fix image bug
|
2018-10-19 14:37:17 +08:00 |
|
judy0131
|
50885f291d
|
fix bug:can not receive the whole json str
|
2018-10-18 17:53:28 +08:00 |
|
judy0131
|
0d98554e13
|
fix bug
|
2018-10-17 18:13:37 +08:00 |
|
judy0131
|
0942318d80
|
print exception
|
2018-10-16 17:30:28 +08:00 |
|
judy0131
|
0639ee8716
|
1.fix find configurableStop in classpath bug
2.get all stop with groups
|
2018-10-16 17:27:02 +08:00 |
|
judy0131
|
97aed5b250
|
add get all stops api
|
2018-10-15 12:42:25 +08:00 |
|
judy0131
|
ab040dad25
|
add logs for application
|
2018-10-09 10:36:08 +08:00 |
|
judy0131
|
f374264dac
|
fix bug: getGroups(user defined stop must implement interface "getGroup")
|
2018-10-08 14:10:41 +08:00 |
|
judy0131
|
f2510a4285
|
set appname
|
2018-10-08 13:46:12 +08:00 |
|
judy0131
|
f577b6ad79
|
run application on yarn by SparkLauncher(Now checkpoint.path is fixed, we can not read this config in config.properties, because yarn will find config.properties in path /opt/hadoop-2.6.0/tmp/nm-local-dir/usercache/root/appcache/appId/containerId/conf/config.properties)
|
2018-10-08 13:26:00 +08:00 |
|
judy0131
|
8b5fb4e619
|
fix bug
|
2018-09-30 16:38:04 +08:00 |
|
judy0131
|
b34dec8110
|
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems)
|
2018-09-29 18:03:18 +08:00 |
|
judy0131
|
1825eb07ff
|
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems)
|
2018-09-29 17:53:30 +08:00 |
|
judy0131
|
6804e91a59
|
fix bug: can not find user defined Configurable Stop
|
2018-09-27 11:23:25 +08:00 |
|
judy0131
|
fd2ef07acd
|
add stop info api
|
2018-09-25 18:07:34 +08:00 |
|
judy0131
|
4c3385ea9f
|
add stop groups api
|
2018-09-25 17:35:11 +08:00 |
|
judy0131
|
08079e1985
|
add wait process terminate Runnable
|
2018-09-17 15:13:36 +08:00 |
|
judy0131
|
c142c03529
|
add get stop properties api
|
2018-09-14 12:58:50 +08:00 |
|
judy0131
|
dd06ac6a43
|
add comment
|
2018-09-13 10:35:41 +08:00 |
|
judy0131
|
11a594d20c
|
add yarn.jars config to solve the problem -->"Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME",which makes the spark on yarn job run very slow!
|
2018-09-13 10:16:22 +08:00 |
|
judy0131
|
6b72da8518
|
1.return immediately when start flow, do not wait
2.fix stop flow bug
|
2018-09-12 16:42:33 +08:00 |
|
judy0131
|
06f9a772dc
|
remove core-site.xml hdfs-site.xml hive-site.xml yarn-site.xml
|
2018-09-11 14:51:24 +08:00 |
|
judy0131
|
674de714c6
|
return applicationID when start the flow
|
2018-09-10 16:26:55 +08:00 |
|
judy0131
|
60e4f3310e
|
run on yarn
|
2018-09-10 15:26:51 +08:00 |
|
judy0131
|
da88183668
|
get flow info
|
2018-09-10 10:58:42 +08:00 |
|
judy0131
|
2ab05fccdd
|
1.modify piflow module,delete piflow-conf 2.deploy jar on server
|
2018-09-05 17:54:21 +08:00 |
|
judy0131
|
508f0639a9
|
rename piflow-api to piflow-server;add getFlowInfo Api
|
2018-09-04 09:53:19 +08:00 |