judy0131
|
f577b6ad79
|
run application on yarn by SparkLauncher(Now checkpoint.path is fixed, we can not read this config in config.properties, because yarn will find config.properties in path /opt/hadoop-2.6.0/tmp/nm-local-dir/usercache/root/appcache/appId/containerId/conf/config.properties)
|
2018-10-08 13:26:00 +08:00 |
|
judy0131
|
8b5fb4e619
|
fix bug
|
2018-09-30 16:38:04 +08:00 |
|
judy0131
|
3ebd4e1758
|
Merge remote-tracking branch 'origin/master'
|
2018-09-29 18:03:32 +08:00 |
|
judy0131
|
b34dec8110
|
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems)
|
2018-09-29 18:03:18 +08:00 |
|
judy0131
|
1825eb07ff
|
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems)
|
2018-09-29 17:53:30 +08:00 |
xiaoxiao
|
a987093467
|
fix the bug with data consumption from kafka
|
2018-09-28 16:14:04 +08:00 |
|
xiaoxiao
|
1b63b70c4d
|
alter getGroup method and add authorEmail for stops
|
2018-09-28 13:31:27 +08:00 |
|
xiaoxiao
|
952f0cefea
|
add ftp group(LoadFromFtp and UploadToFtp)
|
2018-09-28 13:24:32 +08:00 |
|
xiaoxiao
|
97e2600fb8
|
fix a bug for ReadFromKafka Stop
|
2018-09-28 12:47:07 +08:00 |
|
xiaoxiao
|
caf919bae5
|
fix a bug
|
2018-09-28 10:50:37 +08:00 |
|
xiaoxiao
|
38cd7de567
|
fix a bug for ReadFromKafka stop
|
2018-09-28 10:40:57 +08:00 |
|
xiaoxiao
|
54b7a8509b
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-09-27 12:23:48 +08:00 |
|
xiaoxiao
|
9fd6688dd7
|
add redis and kafka group
|
2018-09-27 12:20:07 +08:00 |
|
xiaoxiao
|
7ae7e6a888
|
add redis group(WriteToRedis and ReadFromRedis)
|
2018-09-27 12:16:24 +08:00 |
|
judy0131
|
dcd8777558
|
add bundle for getStopInfo
|
2018-09-27 11:31:24 +08:00 |
|
judy0131
|
6804e91a59
|
fix bug: can not find user defined Configurable Stop
|
2018-09-27 11:23:25 +08:00 |
|
judy0131
|
9102bf97bb
|
fix bug: can not find user defined Configurable Stop
|
2018-09-26 17:49:32 +08:00 |
|
judy0131
|
fd2ef07acd
|
add stop info api
|
2018-09-25 18:07:34 +08:00 |
|
judy0131
|
4c3385ea9f
|
add stop groups api
|
2018-09-25 17:35:11 +08:00 |
|
judy0131
|
e2b2df0418
|
1.modify ConfigurableStop getGroup api
2.modify findAllConfigurableStop
3.add findAllGroups
|
2018-09-25 16:31:55 +08:00 |
|
judy0131
|
08079e1985
|
add wait process terminate Runnable
|
2018-09-17 15:13:36 +08:00 |
|
judy0131
|
e6c4b22e5a
|
add stop's author
|
2018-09-17 14:10:51 +08:00 |
|
judy0131
|
c142c03529
|
add get stop properties api
|
2018-09-14 12:58:50 +08:00 |
|
judy0131
|
dd06ac6a43
|
add comment
|
2018-09-13 10:35:41 +08:00 |
|
judy0131
|
11a594d20c
|
add yarn.jars config to solve the problem -->"Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME",which makes the spark on yarn job run very slow!
|
2018-09-13 10:16:22 +08:00 |
|
judy0131
|
6b72da8518
|
1.return immediately when start flow, do not wait
2.fix stop flow bug
|
2018-09-12 16:42:33 +08:00 |
|
judy0131
|
66b3175ef7
|
fix bug
|
2018-09-11 17:13:05 +08:00 |
|
judy0131
|
ac8ad1a92e
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-09-11 14:53:54 +08:00 |
|
judy0131
|
06f9a772dc
|
remove core-site.xml hdfs-site.xml hive-site.xml yarn-site.xml
|
2018-09-11 14:51:24 +08:00 |
|
xiaoxiao
|
bb2ddc60d3
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-09-11 14:24:50 +08:00 |
|
xiaoxiao
|
3732d0bcef
|
add ftp group(LoadFromFtp and UploadToFtp)
|
2018-09-11 14:11:46 +08:00 |
|
judy0131
|
674de714c6
|
return applicationID when start the flow
|
2018-09-10 16:26:55 +08:00 |
|
judy0131
|
60e4f3310e
|
run on yarn
|
2018-09-10 15:26:51 +08:00 |
|
judy0131
|
da88183668
|
get flow info
|
2018-09-10 10:58:42 +08:00 |
|
judy0131
|
88471237ad
|
fix bug: can not run flow when package code into piflow.jar
|
2018-09-06 17:09:35 +08:00 |
|
judy0131
|
2ab05fccdd
|
1.modify piflow module,delete piflow-conf 2.deploy jar on server
|
2018-09-05 17:54:21 +08:00 |
lj044500
|
e4d51735cc
|
XmlTesst
|
2018-09-04 10:48:35 +08:00 |
|
lj044500
|
ed1c4e47fa
|
add folder xml file dispose
|
2018-09-04 10:46:01 +08:00 |
|
lj044500
|
1250dee09f
|
文件夹中所有得.xml文件处理
|
2018-09-04 10:40:06 +08:00 |
|
judy0131
|
508f0639a9
|
rename piflow-api to piflow-server;add getFlowInfo Api
|
2018-09-04 09:53:19 +08:00 |
|
judy0131
|
c347710084
|
fix bug
|
2018-09-03 14:50:55 +08:00 |
|
judy0131
|
cf32e770dc
|
clean code
|
2018-09-03 13:14:33 +08:00 |
|
judy0131
|
6db6d54e56
|
add httpClient
|
2018-08-31 13:07:44 +08:00 |
|
xiaoxiao
|
3285bf602b
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-08-30 15:21:34 +08:00 |
|
xiaoxiao
|
6aecf64633
|
add clean group
|
2018-08-30 15:19:44 +08:00 |
|
judy0131
|
83d575f084
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-08-30 15:09:14 +08:00 |
|
judy0131
|
c1b362bf02
|
modify akka version
|
2018-08-30 14:39:06 +08:00 |
|
xiaoxiao
|
14ba86908e
|
8.28
|
2018-08-28 11:39:22 +08:00 |
|
xiaoxiao
|
590cba100e
|
Merge branch 'master' of https://github.com/cas-bigdatalab/piflow
|
2018-08-28 11:34:45 +08:00 |
|
xiaoxiao
|
7d7d4e53a8
|
8.28
|
2018-08-28 11:34:38 +08:00 |