piflow/conf/config.properties

server.ip=10.0.86.98
server.port=8001

#spark.master=spark://10.0.86.89:7077
#spark.master=spark://10.0.86.191:7077
spark.master=yarn
spark.deploy.mode=cluster
yarn.resourcemanager.hostname=10.0.86.191
yarn.resourcemanager.address=10.0.86.191:8032
yarn.access.namenode=hdfs://10.0.86.191:9000
yarn.stagingDir=hdfs://10.0.86.191:9000/tmp/
yarn.jars=hdfs://10.0.86.191:9000/user/spark/share/lib/*.jar

hive.metastore.uris=thrift://10.0.86.191:9083

#piflow.bundle=/opt/project/piflow/out/artifacts/piflow_bundle/piflow-bundle.jar
piflow.bundle=/opt/project/piflow/piflow-server/target/piflow-server-0.9.jar

yarn.url=http://10.0.86.191:8088/ws/v1/cluster/apps/
checkpoint.path=hdfs://10.0.86.89:9000/xjzhu/piflow/checkpoints/

log.path=/opt/project/piflow/logs
add startFlow & stopFlow Api 2018-08-23 17:16:28 +08:00			`server.ip=10.0.86.98`
modify akka version 2018-08-30 14:39:06 +08:00			`server.port=8001`

fix bug 2018-09-30 16:38:04 +08:00			`#spark.master=spark://10.0.86.89:7077`
run on yarn 2018-09-10 15:26:51 +08:00			`#spark.master=spark://10.0.86.191:7077`
fix bug 2018-09-30 16:38:04 +08:00			`spark.master=yarn`
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems) 2018-09-29 17:53:30 +08:00			`spark.deploy.mode=cluster`
remove core-site.xml hdfs-site.xml hive-site.xml yarn-site.xml 2018-09-11 14:51:24 +08:00			`yarn.resourcemanager.hostname=10.0.86.191`
			`yarn.resourcemanager.address=10.0.86.191:8032`
			`yarn.access.namenode=hdfs://10.0.86.191:9000`
			`yarn.stagingDir=hdfs://10.0.86.191:9000/tmp/`
add yarn.jars config to solve the problem -->"Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME",which makes the spark on yarn job run very slow! 2018-09-13 10:16:22 +08:00			`yarn.jars=hdfs://10.0.86.191:9000/user/spark/share/lib/*.jar`
get flow info 2018-09-10 10:58:42 +08:00
remove core-site.xml hdfs-site.xml hive-site.xml yarn-site.xml 2018-09-11 14:51:24 +08:00			`hive.metastore.uris=thrift://10.0.86.191:9083`
rename piflow-api to piflow-server;add getFlowInfo Api 2018-09-04 09:53:19 +08:00
support multi sparkContext to run more than one flow at the same time(but only in standalone mode, yarn still have problems) 2018-09-29 17:53:30 +08:00			`#piflow.bundle=/opt/project/piflow/out/artifacts/piflow_bundle/piflow-bundle.jar`
			`piflow.bundle=/opt/project/piflow/piflow-server/target/piflow-server-0.9.jar`
1.modify piflow module,delete piflow-conf 2.deploy jar on server 2018-09-05 17:54:21 +08:00
run on yarn 2018-09-10 15:26:51 +08:00			`yarn.url=http://10.0.86.191:8088/ws/v1/cluster/apps/`
add logs for application 2018-10-09 10:36:08 +08:00			`checkpoint.path=hdfs://10.0.86.89:9000/xjzhu/piflow/checkpoints/`

			`log.path=/opt/project/piflow/logs`