yystopf
/
piflow
forked from opensci/piflow
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
422 Commits 16 Branches 0 Tags 88 MiB
Commit Graph

57 Commits

Author SHA1 Message Date
judy0131 da63af8562 add FlowLauncher 2019-05-08 11:01:49 +08:00
judy0131 7deb8946c7 support run project API 2019-05-05 15:34:35 +08:00
judy0131 1754a233cd 1.add json util 
2. fix bug: launch spark application for each flow in FlowGroup
 2019-04-30 16:29:22 +08:00
judy0131 76fa9280c2 support run FlowGroup API 2019-04-26 18:02:54 +08:00
Airzihao e75514a3a3 do nothing. 2019-04-11 09:19:35 +08:00
judy0131 7d9133c025 1.modify scala-maven-plugin for both java and scala file 
2.fixbug
 2019-04-09 15:20:22 +08:00
judy0131 a0764ceb68 1. modify show data function 
2. fix bug: when more than 5 requests of starting flow are posted, piflow server will get exception:No element passed in the last 1 minute
 2019-03-27 15:09:34 +08:00
judy0131 8afaf376a3 modify the icon of ftp and graphx 2019-03-16 22:47:16 +08:00
judy0131 2e9ad38fca 1.modify streaming stop's property descriptor 
2.modify TextFileStreaming example
 2019-03-07 17:09:01 +08:00
judy0131 0936d4b495 1.verify flume streaming example 
2.add debugData api
 2019-03-07 15:59:10 +08:00
judy0131 3f46331945 fix TextFileStream bug 2019-03-01 14:44:54 +08:00
judy0131 91561ad010 support flume streaming 2019-03-01 13:34:47 +08:00
judy0131 87a3b002e4 fix bug 2019-02-28 14:19:47 +08:00
judy0131 c28fee4c1e support kafka streaming 2019-02-28 11:00:21 +08:00
judy0131 b48ff0bfc2 show flow debug data by path 2019-01-23 11:27:05 +08:00
judy0131 1fe9f009e1 fix bug: when stop flow, the end time do not show 2019-01-15 15:23:29 +08:00
judy0131 212f60479a fix stop flow bug 2018-12-14 18:12:21 +08:00
judy0131 98be6dca8d modify stop group description 2018-12-14 17:03:45 +08:00
judy0131 808a8e1693 fix checkpoint bug 2018-11-28 15:02:52 +08:00
judy0131 dd11c444a2 1.implement checkpoint 2018-11-20 16:46:47 +08:00
judy0131 4ab88005d7 add flow resource configuration 2018-11-07 16:49:22 +08:00
judy0131 7c65a7510a return flow info when the flow's state waiting 2018-10-29 15:14:23 +08:00
judy0131 8f93a619cd update flow example 2018-10-29 14:59:19 +08:00
judy0131 7f459ee008 add server log 2018-10-26 17:26:45 +08:00
judy0131 6b6a1588ec 1.add h2 port 
2.fix bug: multi server
 2018-10-25 13:18:14 +08:00
judy0131 c6db0d1acb add getFlowProgress and getFlowLog API 2018-10-24 18:13:40 +08:00
judy0131 80e3e874d6 monitor flow info and stop info 2018-10-24 17:33:46 +08:00
judy0131 c329a8459f fix bug: merge and fork 2018-10-22 15:13:14 +08:00
judy0131 554cd25dc4 fix checkpoint bug 2018-10-22 12:22:55 +08:00
judy0131 93812c5783 fix stop flow bug 2018-10-19 15:52:19 +08:00
judy0131 416594d594 fix image bug 2018-10-19 14:37:17 +08:00
judy0131 50885f291d fix bug:can not receive the whole json str 2018-10-18 17:53:28 +08:00
judy0131 0d98554e13 fix bug 2018-10-17 18:13:37 +08:00
judy0131 0942318d80 print exception 2018-10-16 17:30:28 +08:00
judy0131 0639ee8716 1.fix find configurableStop in classpath bug 
2.get all stop with groups
 2018-10-16 17:27:02 +08:00
judy0131 97aed5b250 add get all stops api 2018-10-15 12:42:25 +08:00
judy0131 ab040dad25 add logs for application 2018-10-09 10:36:08 +08:00
judy0131 f374264dac fix bug: getGroups(user defined stop must implement interface "getGroup") 2018-10-08 14:10:41 +08:00
judy0131 f2510a4285 set appname 2018-10-08 13:46:12 +08:00
judy0131 f577b6ad79 run application on yarn by SparkLauncher(Now checkpoint.path is fixed, we can not read this config in config.properties, because yarn will find config.properties in path /opt/hadoop-2.6.0/tmp/nm-local-dir/usercache/root/appcache/appId/containerId/conf/config.properties) 2018-10-08 13:26:00 +08:00
judy0131 8b5fb4e619 fix bug 2018-09-30 16:38:04 +08:00
judy0131 b34dec8110 support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems) 2018-09-29 18:03:18 +08:00
judy0131 1825eb07ff support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems) 2018-09-29 17:53:30 +08:00
judy0131 6804e91a59 fix bug: can not find user defined Configurable Stop 2018-09-27 11:23:25 +08:00
judy0131 fd2ef07acd add stop info api 2018-09-25 18:07:34 +08:00
judy0131 4c3385ea9f add stop groups api 2018-09-25 17:35:11 +08:00
judy0131 08079e1985 add wait process terminate Runnable 2018-09-17 15:13:36 +08:00
judy0131 c142c03529 add get stop properties api 2018-09-14 12:58:50 +08:00
judy0131 dd06ac6a43 add comment 2018-09-13 10:35:41 +08:00
judy0131 11a594d20c add yarn.jars config to solve the problem -->"Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME",which makes the spark on yarn job run very slow! 2018-09-13 10:16:22 +08:00