yystopf
/
piflow
forked from opensci/piflow
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
587 Commits 16 Branches 0 Tags 88 MiB
Commit Graph

376 Commits

Author SHA1 Message Date
judy0131 a0764ceb68 1. modify show data function 
2. fix bug: when more than 5 requests of starting flow are posted, piflow server will get exception:No element passed in the last 1 minute
 2019-03-27 15:09:34 +08:00
yanfqidong0604 e9f354457f neo4j 
qidong yang
 2019-03-26 15:03:46 +08:00
judy0131 4080568461 fix spelling mistakes 2019-03-21 16:48:33 +08:00
judy0131 30a2b5c936 modify description of stop 2019-03-18 14:39:14 +08:00
judy0131 40e592fa30 remove icons which is unused 2019-03-17 09:28:37 +08:00
judy0131 d7addc00d4 modify the icon of solr, streaming and xml group 2019-03-17 09:25:19 +08:00
judy0131 b80515a267 modify the icon of mongo,neo4j,rdf, redis and script group 2019-03-17 09:14:07 +08:00
judy0131 5ce5475da5 modify the icon of meachine learning related group 2019-03-17 08:41:48 +08:00
judy0131 51b7e2a44f modify the icon of kafka, memcache and microorganism group 2019-03-17 08:27:10 +08:00
judy0131 816ebbd5a5 modify the icon of jdbc and json group 2019-03-17 08:02:28 +08:00
judy0131 76131eaf8c 1.modify the icon of hive, http and internetWorm group 
2.add icons
 2019-03-17 07:52:19 +08:00
judy0131 0f5a0192a4 modify the icon of hbase and hdfs group 2019-03-17 07:34:24 +08:00
judy0131 8afaf376a3 modify the icon of ftp and graphx 2019-03-16 22:47:16 +08:00
judy0131 4b5f7a6d32 modify the icon of elasticsearch, excel and file group 2019-03-16 22:39:00 +08:00
judy0131 447c238636 1.update the icon of clean, common, csv group 
2.fix bug: flow resources is not effective
 2019-03-16 22:26:25 +08:00
judy0131 f1facfa99e modify the icons of clean and common group 2019-03-15 16:43:50 +08:00
yanggang 84a7e75b92 updata describe 2019-03-12 11:04:29 +08:00
yanggang b7b93b9a1f upload kegg/medline stop ,rename microo stop and delete repetition stop 2019-03-12 10:59:08 +08:00
yanfqidong0604 193d5dd04b impala 
qidong yang
 2019-03-11 16:39:48 +08:00
yanfqidong0604 70b2753145 impala 
qidong yang
 2019-03-11 16:24:11 +08:00
yanfqidong0604 0f3f2dafab neo4j 
qidong yang
 2019-03-11 10:47:58 +08:00
judy0131 2e9ad38fca 1.modify streaming stop's property descriptor 
2.modify TextFileStreaming example
 2019-03-07 17:09:01 +08:00
judy0131 0936d4b495 1.verify flume streaming example 
2.add debugData api
 2019-03-07 15:59:10 +08:00
yanggang 375af3a1ab Merge remote-tracking branch 'origin/master' 2019-03-05 16:02:11 +08:00
yanggang 076ea0173b update bioProject 2019-03-05 16:01:27 +08:00
judy0131 ca8abccf0c add streaming window function 2019-03-05 15:59:56 +08:00
judy0131 cd615ad58e set batchDuration from streamingStop 2019-03-01 16:16:25 +08:00
judy0131 e28fdf3159 set icon from streaming group's stops 2019-03-01 15:18:32 +08:00
judy0131 91561ad010 support flume streaming 2019-03-01 13:34:47 +08:00
judy0131 c28fee4c1e support kafka streaming 2019-02-28 11:00:21 +08:00
yanggang 695e61e455 update microorganism stop 2019-02-27 13:47:06 +08:00
judy0131 0bd5c9bcf5 Merge remote-tracking branch 'origin/master' 2019-02-27 10:58:55 +08:00
judy0131 889b83f475 fix getGroups api bug 2019-02-27 10:58:34 +08:00
yanfqidong0604 03c939f1ed QiDong Yang 
Introduction of dependency on FTP server download
 2019-02-25 10:59:34 +08:00
yanfqidong0604 7f48516f27 QiDong Yang 
Introduction of dependency on FTP server download
 2019-02-25 10:57:59 +08:00
yanfqidong0604 880b55a7af QiDong Yang 
Full-scale testing of JSON and final stop code
 2019-02-25 10:36:08 +08:00
judy0131 3d54df81f4 support streaming 2019-02-20 18:37:23 +08:00
yanggang cd0cadeaf0 delete json，updata port 2019-01-31 17:36:49 +08:00
yanggang d64e96f631 Merge remote-tracking branch 'origin/master' 2019-01-30 15:05:43 +08:00
yanggang 0d1608f2b3 修改描述信息，提交微生物stop 2019-01-30 15:02:42 +08:00
judy0131 dd09d8c6d1 fix bug: jdbcRead stop's properties repeat 2019-01-30 10:41:28 +08:00
yanfqidong0604 2a2d029d19 In the HDFS filesystem, select the file you need according to the name of the file 
Whether a parameter of decompression stop must be adjusted
QiDong Yang
 2019-01-29 17:40:44 +08:00
judy0131 b48ff0bfc2 show flow debug data by path 2019-01-23 11:27:05 +08:00
yanfqidong0604 11a15d6ca7 YangQiDong 2019-01-18 16:56:22 +08:00
yanfqidong0604 736fc3592a YangQiDong 2019-01-18 16:55:47 +08:00
yanfqidong0604 553f36760d YangQiDong 2019-01-18 16:32:21 +08:00
yanfqidong0604 6d483b7461 YangQiDong 2019-01-18 16:27:46 +08:00
yanfqidong0604 d027f5e589 YangQiDong 2019-01-18 16:13:22 +08:00
yanfqidong0604 c2ae0447d7 YangQiDong 2019-01-18 15:05:43 +08:00
yanfqidong0604 4bde8b0f23 qidong yang 2019-01-18 14:04:39 +08:00
yanfqidong0604 14f1da58f2 qidong yang 2019-01-16 11:13:33 +08:00
judy0131 9cba0f2e6e show dataFrame when job is completed 2019-01-14 16:54:22 +08:00
yanfqidong0604 762368a2a9 qidong yang 2019-01-11 18:45:42 +08:00
yanfqidong0604 588bc66125 qidong yang 2019-01-11 16:53:25 +08:00
yanfqidong0604 770d46aee1 qidong yang 2019-01-11 16:04:42 +08:00
yanfqidong0604 1211afab88 Image Modification of Microbial Data Stop 
qidong yang
 2019-01-11 15:32:56 +08:00
yanfqidong0604 2e1ecd0087 join stop 
qidong yang
 2019-01-11 15:14:12 +08:00
yanfqidong0604 180ef7a181 join stop 
qidong yang
 2019-01-11 15:02:38 +08:00
yanfqidong0604 c4d8437824 pom.xml 
Yang QiDong
 2019-01-10 10:25:12 +08:00
yanfqidong0604 3332ee704e Parsing of ensembl data 
Yang QiDong
 2019-01-08 20:46:47 +08:00
yanfqidong0604 53cb53f713 Parsing of PAB and Pfam data 
Yang QiDong
 2019-01-07 09:51:43 +08:00
yanfqidong0604 fb5557b62c Download of Swissprot type data 
Yang QiDong
 2019-01-03 09:54:06 +08:00
yanfqidong0604 5fc0bc662f Download of Swissprot type data 
Yang QiDong
 2019-01-03 09:48:59 +08:00
yanfqidong0604 dbcafe0395 Solving the problem of active and passive mode of FTP server 
QiDong Yang
 2018-12-28 17:40:52 +08:00
yanfqidong0604 30488ddfd8 Solving the problem of active and passive mode of FTP server 
QiDong Yang
 2018-12-28 17:38:29 +08:00
yanfqidong0604 2944e024f7 Modification of FTP server download and HDFS file decompression 
QiDong Yang
 2018-12-28 14:40:45 +08:00
yanfqidong0604 fe0cbb3f89 Modification of FTP server file download 
QiDong Yang
 2018-12-27 17:33:23 +08:00
yanfqidong0604 c73b87180a NCBI_Microbe_genome type data parsing 
QiDong Yang
 2018-12-27 17:23:51 +08:00
yanfqidong0604 f6e9573018 Gene type data parsing 
QiDong Yang
 2018-12-26 16:56:15 +08:00
judy0131 eacd2fb50f Merge remote-tracking branch 'origin/master' 2018-12-24 17:25:21 +08:00
judy0131 bf4cce966a fix bug 2018-12-24 17:24:41 +08:00
yanfqidong0604 ceb2239755 Increased functionality for decompressing stop 
QiDong Yang
 2018-12-24 15:58:46 +08:00
yanfqidong0604 2d1685af1a Analysis of microbial data EMBL, Refseq_genome, Refseq_protein and related components. 
FTP server download, HDFS file screening, HDFS file decompression optimization.
QiDong Yang
 2018-12-21 12:20:58 +08:00
judy0131 3d48146b99 fix bug 2018-12-14 17:10:59 +08:00
judy0131 917ead7e7f Merge remote-tracking branch 'origin/master' 2018-12-14 17:03:58 +08:00
judy0131 98be6dca8d modify stop group description 2018-12-14 17:03:45 +08:00
yanggang ddae38d3ea Merge remote-tracking branch 'origin/master' 2018-12-14 14:57:24 +08:00
yanggang c42508d98c update unGz 2018-12-14 14:56:28 +08:00
yanggang 81343172ea update unGz 2018-12-14 14:55:36 +08:00
yanfqidong0604 0a4e2f24e7 Optimization of HDFS file storage 
yang qidong
 2018-12-13 22:48:15 +08:00
yanggang bcd658bb11 Optimization and modification of Oracle 
yang qidong
 2018-12-11 10:52:34 +08:00
judy0131 8b109c6902 fix bug: remove hive-site.xml core-site.xml 2018-11-30 17:07:19 +08:00
yanggang 426bcff6c2 Merge remote-tracking branch 'origin/master' 
# Conflicts:
#	piflow-bundle/src/test/scala/cn/piflow/bundle/UrlTest.scala
 2018-11-29 17:01:59 +08:00
yanggang 499f70cda1 update unGz 2018-11-29 16:59:43 +08:00
judy0131 972f8af248 fix flow progress bug 2018-11-28 18:17:39 +08:00
judy0131 63e3cdda81 remove common-pool2.jar 2018-11-28 15:34:16 +08:00
judy0131 808a8e1693 fix checkpoint bug 2018-11-28 15:02:52 +08:00
yanggang 7921775508 Merge remote-tracking branch 'origin/master' 2018-11-26 15:43:02 +08:00
yanggang c29f9bcba7 bioProject stop 2018-11-26 15:42:48 +08:00
yanfqidong0604 c8a23fdb72 the test for dblp 
yang qidong
 2018-11-26 10:25:21 +08:00
yanfqidong0604 892866fd51 the test for dblp 
yang qidong
 2018-11-26 10:23:26 +08:00
yanfqidong0604 2b46264473 modification of ConvertSchema 
yang qidong
 2018-11-23 12:56:14 +08:00
yanggang cda81eae5c update describe 2018-11-22 19:56:08 +08:00
yanggang ba4b63ae15 Merge remote-tracking branch 'origin/master' 2018-11-22 19:52:33 +08:00
yanggang 04fe22c69c Merge branch 'master' of /workFtp/1119/piflow with conflicts. 2018-11-22 19:52:30 +08:00
yanggang 73ff82783a update describe 2018-11-22 19:49:06 +08:00
yanggang 95316d516d goldData Parse 2018-11-22 19:28:08 +08:00
yanggang f8f34a4cee update describe 2018-11-22 19:27:04 +08:00
xiaoxiao d02f3da4ee alter email and add description for properties 2018-11-22 17:23:29 +08:00
yanfqidong0604 e97fac9d84 Schema substitution for dataframe 
About downloading network files to HDFS
About HDFS file decompression
yang qidong
 2018-11-22 16:52:05 +08:00
yanggang 2e26f7cbe3 photo png(hdfs,hbase,hive) 2018-11-22 16:01:13 +08:00
yanggang 81cfdb8953 goldData Parse 2018-11-22 15:42:14 +08:00
yanggang 4bb8abba9c goldData Parse 2018-11-22 14:46:04 +08:00
yanggang 5099851fe7 png photo 2018-11-22 14:32:04 +08:00
yanggang 5828c2b5bc Merge remote-tracking branch 'origin/master' 2018-11-22 14:10:45 +08:00
yanggang 7a329eb7cb goldData Parse 2018-11-22 14:09:43 +08:00
judy0131 dd11c444a2 1.implement checkpoint 2018-11-20 16:46:47 +08:00
yanfqidong0604 ed8d9faaf2 Standardization of historical stop 
yang qidong
 2018-11-19 22:03:18 +08:00
yanggang 5a3b4d694d Merge remote-tracking branch 'origin/master' 2018-11-19 16:13:26 +08:00
yanggang 9160be777c add maven 2018-11-19 16:13:06 +08:00
xiaoxiao 239786f881 add graphx stops 2018-11-19 15:24:40 +08:00
yanggang 350f74a741 Merge remote-tracking branch 'origin/master' 2018-11-19 15:15:33 +08:00
yanggang a07cca1f8b doMap flatMap executeSql 2018-11-19 15:15:06 +08:00
yanfqidong0604 9fcf843efc Stop name and parameter name adjustment 
yang qidong
 2018-11-19 14:46:10 +08:00
yanfqidong0604 acf2acf3b3 Modification of special types of Oracle 
yang qidong
 2018-11-19 10:24:04 +08:00
yanfqidong0604 becfca6b91 Modification of special types of Oracle 
yang qidong
 2018-11-19 09:43:45 +08:00
yanfqidong0604 7bbfd7d3ee Oracle database read and write and related driver package 
yang qidong
 2018-11-15 10:13:49 +08:00
yanfqidong0604 32a0ef0f0f Merge remote-tracking branch 'origin/master' 2018-11-13 12:36:22 +08:00
yanfqidong0604 32925d3071 Additional stop reading impala, changes to description information of previous Memcache stop, and changes to logo of spider 
yang qidong
 2018-11-13 12:35:52 +08:00
yanggang 7f53e25275 netty-all dependency 2018-11-13 12:12:47 +08:00
yanggang 6994550803 Merge remote-tracking branch 'origin/master' 2018-11-12 18:11:53 +08:00
yanggang fd5cfe6b59 hbase 2018-11-12 18:11:11 +08:00
judy0131 2d80db9585 fix bug 2018-11-12 17:19:56 +08:00
judy0131 88def74fa1 Merge remote-tracking branch 'origin/master' 2018-11-12 15:26:27 +08:00
judy0131 5fcb6eff7d fix bug 2018-11-12 15:26:07 +08:00
yanfqidong0604 1360645156 The group name standardization of stop recently completed, and the image of the crawler. 
yang qidong
 2018-11-12 15:15:04 +08:00
yanfqidong0604 8d198cb392 Maven dependency of Memcache 
yang qidong
 2018-11-12 14:42:50 +08:00
yanfqidong0604 96abf1b61f Read and write mongodb, read and write Memcache, and complement Memcache. 
yang qidong
 2018-11-09 15:10:27 +08:00
judy0131 82c64aabae add flow name in flowImpl 2018-11-06 17:07:13 +08:00
judy0131 a4badc9703 1.modify getFlowProgress api 
2.add maven-install-plugin to install the databricks file
 2018-11-06 13:55:28 +08:00
yanggang 1b956b7cf0 update genbank 2018-11-06 10:01:55 +08:00
yanfqidong0604 32749fa906 The first version of web spider stop 
yang qidong
 2018-11-04 18:31:42 +08:00
yanggang 163acb2525 incremental download from ftpUrl 2018-11-02 14:50:30 +08:00
yanggang 707505979f update Es 2018-11-02 10:40:11 +08:00
yanggang d14b8ebc1f update Es 2018-11-02 10:10:08 +08:00
yanggang f8dd478f4d genbank002 2018-11-01 17:37:49 +08:00
yanggang 1d623d70b7 genbank001 2018-11-01 17:33:40 +08:00
xiaoxiao 06bdb859d2 add gaussion mixture stops 2018-11-01 17:17:21 +08:00
yanggang 778d41744e Merge remote-tracking branch 'origin/master' 2018-11-01 17:13:21 +08:00
yanggang f3f492e75b GenBankParse 2018-11-01 17:13:05 +08:00
xiaoxiao 6b2698b94f add lda stops 2018-11-01 16:48:43 +08:00
yanggang d8ba5012e7 GenBankParse 2018-11-01 16:44:25 +08:00
yanggang dfcc2a997e GenBankParse 2018-11-01 16:33:16 +08:00
yanggang 45319224a5 ungzUtil 2018-10-31 16:08:03 +08:00
yanggang 7ef5206305 New put to es,fetch from es.query from es 2018-10-31 10:40:11 +08:00
xiaoxiao 84768e8abe add Word2Vec stop 2018-10-26 11:21:22 +08:00
yanggang 0af6b10fb3 Merge remote-tracking branch 'origin/master' 2018-10-25 14:09:13 +08:00
yanggang def72cf714 hdfs(delete,get,list,put),http(get,post,invoke) 2018-10-25 14:05:48 +08:00
coco11563 2105731818 fix to String bug 2018-10-24 17:09:03 +08:00
xiaoxiao 8d626a876a add BisectingKMeans stops 2018-10-24 14:41:32 +08:00