opensci
/
gStore
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
722 Commits 29 Branches 12 Tags 60 MiB
Commit Graph

159 Commits

Author SHA1 Message Date
bookug 4b4ccc8385 feat: forbidden the openmp sort to speed up; 
by zengli
 2018-04-24 16:45:35 +08:00
bookug bf22f41bd3 Merge branch 'qinzongyue' into dev 
merge qinzongyue's optimizations for key-value indices
 2018-04-15 15:15:41 +08:00
bookug 66dbee493b feat: merge branch of suxunbin; 
add support for parallel sort using openmp;

by suxunbin
 2018-03-26 19:26:44 +08:00
ZongyueQin 416385c6af Fix: Fix a bug in Strategy.cpp and optimize for gbuild 2018-03-25 15:14:03 +08:00
bookug aa586d5273 fix: bug in Strategy; 
by hulin
 2018-03-22 14:46:58 +08:00
ZongyueQin 98e1d88b08 Add: Replace ISTree with ISArray, and fix some bugs in IVArray 2018-02-24 09:33:21 +08:00
ZongyueQin a6daca4792 Add: Replace IVTree with IVArray 2018-02-13 11:37:54 +08:00
suxunbin 3fb748104c Add: add parallel sort using openmp 2018-02-02 19:03:05 +08:00
ZongyueQin 5698851e43 Merge branch 'hulin' into qinzongyue 2018-01-30 15:57:34 +08:00
hulin12138 2048d38588 Merge branch 'bookug' into hulin 2018-01-30 15:24:44 +08:00
hulin12138 e79cd86e11 modify to pre_handler 2018-01-30 14:59:14 +08:00
bookug 6b9532c815 fix: error with sizeof(int**) in Database.cpp 2018-01-27 22:17:47 +08:00
ZongyueQin 315cf44d23 Add: an update of String Compressing part 2018-01-27 12:49:00 +08:00
ZongyueQin 60e36878e0 Fix: optimize calling of string compressing 2018-01-24 23:31:59 +08:00
ZongyueQin cd42d10888 Add: Compress strings when building database 2018-01-22 21:27:46 +08:00
root d9d14f1320 Modify: modify std::sort to parallelsort using openmp 2017-12-24 14:43:25 +08:00
bookug 75a662e521 docs: add/sub some notations 2017-12-21 16:17:34 +08:00
bookug 45b4a9e660 docs: add test for dbpedia 2017-12-21 13:50:56 +08:00
bookug e5fba910aa fix: bug in IVTree::preLoad insertion/deletion 2017-11-10 18:43:27 +08:00
bookug 3719970514 fix: support 12000 query threads now; 
merge the modifications of chenjiaqi;
it is the opened socket for each connection that reall matters;

by zengli
 2017-10-27 13:58:35 +08:00
bookug 8bc423fb95 feat: add parallism for ghttp; 
use C++11 thread, each for a query;
lock teh whole database when load/build/unload;
lock teh database when running update query;
query cache and the whole KVstore not resolved;
debug_lock should be removed later;
A query thread can open file in Util::memoryLeft once, and may also use Stream which will also open several files, so the limits of system must be increased to support high parallism;

by zengli, bugs to do
 2017-10-26 21:24:44 +08:00
bookug 438c4c48a5 Merge branch 'bookug' into parallel 
get the latest message
 2017-10-19 19:41:48 +08:00
bookug 9629682786 feat: parallism; 
use multiple processes for query processing now;

by zengli
 2017-10-19 19:28:28 +08:00
bookug 160e6b3123 Merge branch 'qinzongyue' 
fix bug in vlist cache with insertions or deletions
 2017-10-09 20:45:49 +08:00
ZongyueQin 4b14337f56 <fix> fix the bug that after gsub and gadd first 3000 triples of LUBM500M.nt, result of q15 is wrong 2017-10-09 20:06:57 +08:00
bookug b45934d073 Merge branch 'chenjiaqi' 
update Query Cache if there are insertions or deletions in the database
 2017-09-24 18:07:03 +08:00
magicpanda0618 2c64f1f954 fix: a bug when use QueryCache 
clear QueryCache after any UPDATE query
by chenjiaqi
 2017-09-24 18:01:46 +08:00
bookug b8d7cd2158 Merge branch 'hulin' 
fix bugs in make
 2017-09-19 13:52:04 +08:00
hulin12138 6e6216d0aa move pre_handler to stragety && optimize pre_handler 2017-09-19 13:43:08 +08:00
bookug de58b9b389 Merge branch 'hulin' 
adjust the structure of filter and optimize the prehandler
 2017-09-19 13:19:10 +08:00
hulin12138 31bcc83fa8 put pre_handler in stragety && optimize the pre_handler 2017-09-19 12:42:15 +08:00
magicpanda0618 1caf4be04b perf: add QueryCache 
determine whether new query and cached query are isomorphism

by cuihao and chenjiaqi
 2017-09-06 14:17:20 +08:00
hulin12138 5116653533 optimize the prehandler 2017-09-06 10:06:57 +08:00
ZongyueQin 05d9d0dc35 Merge branch 'master' into qinzongyue 
Conflicts:
	KVstore/ISTree/ISTree.cpp
	KVstore/IVTree/IVTree.cpp
	KVstore/KVstore.cpp
	KVstore/SITree/SITree.cpp

merge master into the branch
 2017-08-08 15:15:44 +08:00
bookug 1f3fcc966a perf: no backup and limit time in Endpoint 
no need to use backup function when ghttp used as SPARQL Endpoint;
query time is limited to 1 minute in SPARQL Endpoint by default;

by zengli
 2017-08-06 22:15:55 +08:00
ZongyueQin f069bd5bee <fix> <fix a bug in cache of Vlist> 2017-08-02 23:21:36 +08:00
bookug 11edefed54 style: add get_data_time to print query logs 2017-07-31 21:22:45 +08:00
bookug c0119300df Merge branch 'chenjiaqi' into bookug 
merge chenjiaqi's code
 2017-07-29 12:08:19 +08:00
bookug af093913c9 refactor: fix ghttp and add function 2017-07-29 12:00:51 +08:00
bookug e94eafc613 docs: modify the java api for new REST interface; 
bug exists in the HTTP transformation;

by zengli
 2017-07-27 17:54:45 +08:00
magicpanda0618 aac9638541 fix a bug when occuring OPTIONAL_GROUP_PATTERN; optimize the performance of StringIndex; 2017-07-27 15:58:51 +08:00
bookug 7753459dc2 feat: enhance the HTTP server; 
merge lijing's code, support a good sparql endpoint;
bug in download file, and unable to visit by inputing URLs in browser directly;

by lijing and zengli
 2017-07-25 23:19:05 +08:00
bookug 7b2792bbd4 style: remove bug info 2017-07-17 22:58:13 +08:00
bookug 72526284fd Merge branch 'hulin' into bookug 
merge hulin's code to deal with special pre vars structure in Join
 2017-07-17 21:07:47 +08:00
hulin12138 029f8cfdf7 add handler for pre vars 2017-07-17 16:27:41 +08:00
ZongyueQin cc0ea8a803 <fix> <add "pointer = NULL" after delete a pointer> 
The files modified include Database.cpp ISTree.cpp ISStorage.cpp IVTree.cpp IVTree.h IVStorage.cpp KVstore.cppSITree.cpp SIStorage.cpp ClassForVlistCache.h Vlist.cpp
 2017-07-17 14:04:35 +08:00
qinzongyue aa9569a643 <fix> <fix bugs in get_candiate_preID, get_important_subID, get_important_objID> 2017-07-16 15:47:21 +08:00
bookug d0bf0ac6a0 fix: bugs in update queries; 
after abandoning vstree, the calling of vstree in insert/delete must be removed;

by zengli
 2017-07-15 20:03:43 +08:00
qinzongyue 45e87ce393 <feat> <add Vlist's cache in preID2values> 
I added Vlist's cache in preID2values. Notice the members' name, they mey be confusing.
 2017-07-15 15:12:32 +08:00
ZongyueQin b51281dedf Add Vlist's cache to preID2values 
Add functions to load Vlist's cache in p2values
 2017-07-15 14:08:39 +08:00
ZongyueQin 599a6d072c Fix bug 
fix redefinition of ClassForVlistCache
 2017-07-15 11:27:00 +08:00
ZongyueQin c5a3f979fc add cache for Vlist 
Add functions and members to load cache of Vlist
 2017-07-15 10:47:45 +08:00
ZongyueQin a641c2a500 Add cache for Vlist 
Add functions and members to load cache of Vlist
 2017-07-15 10:45:44 +08:00
ZongyueQin 32020bdaca Add cache for Vlist 
Add functions and members used to add cache for Vlist
 2017-07-15 10:38:59 +08:00
bookug 9ca106ca7e refactor: merge code of chenjiaqi; 
implement Bind and GroupBy in SPARQL;

by zengli
 2017-07-14 16:42:47 +08:00
bookug ae5e610acc fix: bug in KVstore and Join; 
mess code in KVstore due to read after request() release the pointer;
lack of empty judgement in Join::pre_handler;

by zengli
 2017-07-12 20:33:44 +08:00
bookug 09a30931d4 refactor: use KVstore as filter instead of VSTree; 
mmeory is saved, as well as stablity;
TODO: deal with pre var,a nd consider multiple same pres;

by hulin
 2017-07-02 21:39:35 +08:00
bookug 670093e42a feat: add monitor for Http server 
not finished, just a start

by zengli, no changes to others
 2017-06-15 21:22:03 +08:00
bookug 38575b16e6 fix: forbid vlist cache 2017-06-13 15:24:00 +08:00
bookug fa147526f1 feat: use multithreading to load; 
fix type bugs in string index;

by zengli
 2017-06-06 18:56:46 +08:00
bookug 9903c3d5cb refactor: adjust vlist and merge signature; 
add cache for VList;
use binding in Signature;

by zengli and qinzongyue
 2017-05-19 23:05:38 +08:00
bookug c11088ee3e feat: merge wlb branch; 
support backup in gserver but not in HttpConnector;

by zengli
 2017-05-18 23:27:58 +08:00
bookug 77499f990e feat: merge http branch 2017-05-18 22:56:41 +08:00
bookug f3202ada40 refactor: merge type and value branch; 
TODO: bug exists with ISTree;

by zengli
 2017-05-17 20:46:02 +08:00
bookug 3bca240151 feat: add http branch 2017-05-16 21:16:40 +08:00
bookug a0e470727c refactor: to use hash instead of sort for candidates in Join 
by zengli
 2017-04-13 20:01:17 +08:00
bookug 2da666757d feat: add recovery function 
if load a db and quit on accidents, you can start the gconsole in native mode on the db server,
and use restore command to recover to teh latest db (backup function will be auto started when you connect to gserver and load
a db)
in addition, gserver stablity is improved to deal with query fails

by wanglibo, no modifications to others
 2017-04-11 15:26:35 +08:00
bookug 9d760ed643 refactor: use different lrucache size for build and query 
by zengli, no influence on others
 2017-04-06 23:48:45 +08:00
bookug 80080d1bca refactor: add VList for IVTree 
also, lower the copy cost in KVstore

by zengli, all changes closed in KVstore, using new/delete for all instead of malloc/free
 2017-04-01 16:03:05 +08:00
bookug 2f7a7a8b69 refactor: move id_tuples to disk in build 
to run large datasets like freebase

by zengli, no changes to others
 2017-03-29 13:48:39 +08:00
bookug d64358b606 perf: add VList for IVTree; lower the copy cost in KVstore 
this work doe not ends

by zengli, maybe conflicates with type branch
 2017-03-28 16:56:16 +08:00
bookug 464d994ba4 use constexpr for c++11 2017-03-26 21:10:37 +08:00
bookug 8e74a29f07 fix bug in DEBUG macros; 
change int type to unsigned type but Query/* may not be fully changed
 2017-03-25 20:08:23 +08:00
bookug 1b939b3a51 change to github to develop 2017-03-24 20:10:43 +08:00
bookug deabdf293f move devGstore to gStore and go on working on github instead of git osc; 
improve the init configure file;
 support empty database and indices:you can remove all triples from a db and insert/query;
 change signature struct and generating method in build;
change FIFO to LRU; fix bug in Signature, the int maybe overflow;
test on shape.db, 0.7B triples with 0.42B entities
 2017-03-23 21:32:41 +08:00
bookug d35d87eb45 support JSON result 2017-02-28 20:32:01 +08:00
bookug e3b09c6529 change memory-disk swap to all-memory in VSTree::LRUCache; 
deal with extra () in sparql;
support long string with multiple lines in sparql;
support str(IRI) in sparql;
 2017-02-26 17:18:21 +08:00
bookug 74598bf66e fix bugs in api and gclient; 
fix bugs in gconsole, pre_var and vstree;
support selected pre vars and parallel edges;
update both en and ch docs
 2017-02-17 17:09:47 +08:00
bookug b7532bcb7d fix bugs in gclient 2017-02-08 16:29:53 +08:00
bookug 5e1f5e7a17 version 0.4.0 2017-01-16 14:12:57 +08:00
bookug ea36218626 print time in build process 2016-10-25 15:10:52 +08:00
bookug 7303e40611 add postProcessing and stringIndex for getFinalResult;add insertion/deletion;fix bugs 2016-09-25 22:14:36 +08:00
bookug 5ecf86c282 Merge branch 'distribute' 
Conflicts:
	Database/Database.cpp
	Database/Database.h
	Database/Join.cpp
	Database/Strategy.cpp
	KVstore/ISTree/storage/Storage.h
	KVstore/KVstore.cpp
	KVstore/KVstore.h
	KVstore/SITree/storage/Storage.h
	KVstore/SSTree/storage/Storage.h
	LICENSE
	Main/gadd.cpp
	Main/gclient.cpp
	Main/gconsole.cpp
	Main/gload.cpp
	Main/gquery.cpp
	Main/gserver.cpp
	Main/gsub.cpp
	Query/BasicQuery.cpp
	Query/IDList.cpp
	Server/Operation.cpp
	Server/Operation.h
	Server/Server.cpp
	Server/Server.h
	Server/Socket.cpp
	Signature/Signature.cpp
	Util/Bstr.cpp
	Util/Util.cpp
	Util/Util.h
	VSTree/EntryBuffer.h
	VSTree/LRUCache.cpp
	VSTree/LRUCache.h
	VSTree/VNode.cpp
	VSTree/VNode.h
	VSTree/VSTree.cpp
	VSTree/VSTree.h
	api/cpp/src/GstoreConnector.cpp
	api/cpp/src/GstoreConnector.h
	api/java/src/jgsc/GstoreConnector.java
	api/php/GstoreConnector.php
	data/small_p2.sql
	data/small_q0.sql
	data/small_q1.sql
	data/small_q2.sql
	data/small_q3.sql
	docs/MAIL.md
	docs/latex/dbpedia2014_comparison.tex
	makefile

	this is to use distributed gstore to produce a new release
	version
 2016-09-18 20:20:15 +08:00
bookug 52b600d44c add distributed gstore by pengpeng 2016-09-18 20:01:57 +08:00
bookug b26105b111 add insertion and deletion;move new version here 2016-09-16 16:56:18 +08:00
bookug 7bd3d5145c move the oldest gStore to the backup branch 2016-09-16 15:19:28 +08:00
bookug 9da8953769 fix bug in preFilter 2016-06-17 16:28:56 +08:00
bookug bafcf4f565 fix bugs in build() 2016-06-15 16:35:50 +08:00
bookug ab233a19e2 fix bugs in Database::s2po... 2016-06-09 01:26:36 +08:00
bookug 5aedbcd3aa remove wasteful so2p;use p2s or p2o instead of s2p s2o sometimes 2016-05-20 21:57:15 +08:00
bookug 02e54e2c17 merge newest devGstore 2016-05-16 03:16:22 +08:00
bookug a91c71574a fix bug in add_literal_candidate 2016-05-05 23:04:04 +08:00
bookug bf1c5614b8 remove almost all debug info;move time and size to end 2016-05-04 15:36:17 +08:00
bookug 8f4fe83b18 improve join; add not-bgp;fix and better stream;add gconsole;use -O3 2016-04-01 20:58:15 +08:00
bookug 78ad6760d5 not using lfs now 2016-03-12 01:52:18 +08:00
bookug ae9f604a11 remove docs and test 2016-03-12 01:50:36 +08:00
zengli 43f5f38aa1 result direction 2016-01-25 01:43:18 -05:00
Bookug Lobert d03c79ad8c help develop 2016-01-20 15:39:36 +08:00
bookug 59d98adaaa merge several classes; 
remove most warnnings;
adjust debug system;
add Stream for ResultSet;
cant add Stream for IDlist and maybe not need;
add redirection for gquery console;
add realpath for soft links but not enable;

author: zengli
 2015-11-06 19:13:39 +08:00
Caesar11 4db4bb651c restore the output information. 
author:hanshuo
 2015-10-20 02:33:15 +00:00