29e744fdbb
Adding warning message.
2020-06-24 19:23:02 -04:00
515b87bcbe
Disabling perfcheck for ninja
2020-06-24 18:45:47 -04:00
5b4acf14ea
Removing space.
2020-06-24 16:51:28 -04:00
5fc6cb15b8
This should make things even more robust. If .git is not found, just disable all git work.
2020-06-24 16:12:19 -04:00
f6e9a8eee4
Making the cmake more verbose so we can figure out what is happening.
2020-06-24 15:44:22 -04:00
cb8a9ef2c0
This removes git as a dependency
2020-06-24 15:13:47 -04:00
1ff55c2729
Replace auto [x,error] with .get() everywhere
2020-06-21 16:26:59 -07:00
6fa5abcd7e
Replace x.get<T>() with x.get(v) or T(x)
2020-06-21 14:36:38 -07:00
9899e5021d
Allow use of document_stream with tie()
2020-06-20 21:15:05 -07:00
a7fc7d4ffb
Switch from get(v,e) to e = get(v)
2020-06-20 17:57:09 -07:00
f336103f63
Convert tools/docs/benchmarks to bool get() idiom
2020-06-20 17:55:46 -07:00
56e2b38048
Add bool result from tie()/get(), get<T>(T&,error_code&)
2020-06-20 17:55:46 -07:00
7339f67dd7
Merge pull request #462 from simdjson/jkeiser/if-backslash
...
Wrap backslash processing in a branch
2020-06-17 07:07:58 -07:00
7ea05d038e
New API traversal tests. ( #931 )
...
Co-authored-by: Daniel Lemire <lemire@gmai.com>
2020-06-15 13:15:52 -04:00
33930ff046
Adding link.
2020-06-15 13:07:53 -04:00
fd44c2a2ff
Merge pull request #927 from simdjson/dlemire/exposingthestringminifier
...
Exposing the string minifier.
2020-06-13 07:47:20 -07:00
a86a82b39c
Rename minify class to minifier so the minify() method is cleared up
2020-06-12 17:05:25 -07:00
d1a54249e7
New API traversal tests.
2020-06-12 17:42:57 -04:00
4dfbf98e4e
Using a worker instead of a thread per batch ( #920 )
...
In the parse_many function, we have one thread doing the stage 1, while the main thread does stage 2. So if stage 1 and stage 2 take half the time, the parse_many could run at twice the speed. It is unlikely to do so. Still, we see benefits of about 40% due to threading.
To achieve this interleaving, we load the data in batches (blocks) of some size. In the current code (master), we create a new thread for each batch. Thread creation is expensive so our approach only works over sizeable batches. This PR improves things and makes parse_many faster when using small batches.
This fixes our parse_stream benchmark which is just busted.
This replaces the one-thread per batch routine by a worker object that reuses the same thread. In benchmarks, this allows us to get the same maximal speed, but with smaller processing blocks. It does not help much with larger blocks because the cost of the thread create gets amortized efficiently.
This PR makes parse_many beneficial over small datasets. It also makes us less dependent on the thread creation time.
Unfortunately, it is going to be difficult to say anything definitive in general. The cost of creating a thread varies widely depending on the OS. On some systems, it might be cheap, in others very expensive. It should be expected that the new code will depend less drastically on the performances of the underlying system, since we create juste one thread.
Co-authored-by: John Keiser <john@johnkeiser.com>
Co-authored-by: Daniel Lemire <lemire@gmai.com>
2020-06-12 16:51:18 -04:00
1b6258ec8c
Added std::minify
2020-06-12 16:37:41 -04:00
7c6723d912
Print progress bar even if there is only one file
2020-06-12 10:01:19 -07:00
a6e4933d93
Exposing the string minifier.
2020-06-11 13:07:18 -04:00
ae6dddfff4
Merge pull request #903 from simdjson/jkeiser/dom-parser-implementation
...
Move parser state to implementation-specific class
2020-06-04 13:09:57 -07:00
1aab4752e2
Store all parser state in the implementation
2020-06-01 12:15:54 -07:00
6a71b24495
Reuse stored buf and len from parser
2020-06-01 12:14:09 -07:00
a3a9bde83e
Move DOM parsing into concrete interface implementation
2020-06-01 12:14:09 -07:00
2fe2dd170b
The "competition tests" are being made portable ( #907 )
...
* More portable competition
* This will enable SIMDJSON_COMPETITION everywhere by default.
* Minor fixes
2020-05-31 20:34:06 -04:00
e6c9dfbd91
Make include files more fine-grained
2020-05-19 14:42:04 -07:00
8927a0561f
Obvious fix. ( #885 )
2020-05-14 20:39:44 -04:00
8c600ca553
Make benchfeatures work again
2020-05-05 09:39:29 -07:00
5312fd30e5
Fix CRT_SECURE warnings in clang
2020-05-04 11:36:00 -07:00
af968c5b44
Merge branch 'master' of github.com:simdjson/simdjson into cmake-flags
2020-05-03 02:12:23 +03:00
293c104cc4
CMake: Separate public and private compilation flags
...
simdjson-internal-flags for macros and warnings
simdjson-flags for pthread, sanitizer, and libcpp
2020-05-02 04:08:47 +03:00
5a3035bb72
Propagate some CMake variables to checkperf
...
Although it passes user-defined options, if the project is build in Debug mode or with Clang (since
CXX defaults to gcc on Linux) results can flactuate
2020-05-02 02:40:13 +03:00
fa4ce6a8bc
There is confusion between gigabytes and gigibytes. Let us standardize throughout. ( #838 )
...
* There is confusion between gigabytes and gigibytes.
* Trying to be consistent.
2020-05-01 12:16:18 -04:00
c3dec1a5ea
Default SIMDJSON_GOOGLE_BENCHMARKS to ON.
2020-04-29 15:21:43 -07:00
f0d5337818
Adding independent benchmarks using Google Benchmark ( #826 )
...
* Adding independent benchmarks using Google Benchmark
2020-04-29 13:53:54 -04:00
4cd9de5c37
This will change the default of the parse benchmark so that it work over hot buffers ( #827 )
...
* This will change the default of the parse benchmark so that it work over hot buffers
by default, thus omitting memory allocation as part of the benchmark.
* Everyone should be using '-H' from now on.
2020-04-29 13:43:27 -04:00
92d7af0881
Don't include benchmark overhead in documents/s
2020-04-28 13:15:01 -07:00
0e6ea76e88
Make checkperf work on Windows ( #799 )
...
* Make command line arguments work for Windows
* Run checkperf on Windows
2020-04-27 14:20:05 -04:00
0d1c574cb1
A few more changes... ( #775 )
...
* More nitpicking.
2020-04-23 11:36:52 -04:00
e030f02776
Merge branch 'master' into jkeiser/wconversion
2020-04-22 22:03:34 -04:00
80dbf9a32a
We should not enable checkperf on anything but Linux. ( #762 )
2020-04-22 21:54:03 -04:00
d4a37f6ef5
Enable conversion warnings on Linux and Windows
2020-04-22 14:21:30 -07:00
3091e2dc0e
Add fallback, westmere and unthreaded checkperf
2020-04-20 11:18:40 -07:00
fc50a36cc5
Make checkperf build master with the same options
2020-04-20 11:14:46 -07:00
9bf9fba2ec
Add checkperf to cmake
2020-04-20 11:14:46 -07:00
289cc3e7a0
Treat warnings as errors during compilation
2020-04-15 19:59:38 -07:00
7480b87e07
Merge pull request #693 from simdjson/jkeiser/cmake-quickstartcpp
...
Add C++11 tests to cmake
2020-04-15 19:53:14 -07:00
befa6423be
This massively improves the performance of tight loops relying on a type() call. ( #721 )
...
* This massively improves the performance of tight loops relying on a type() call.
* Adding a few more benchmarks
2020-04-15 20:45:40 -04:00
Daniel Lemire
Daniel Lemire
John Keiser
Furkan Usta