Commit Graph

124 Commits

Author SHA1 Message Date
Daniel Lemire f0ac55ec0c
testing on freebsd (#768)
* Adding cirrus tests
* Adding cirrus badge.
2020-04-22 21:22:09 -04:00
John Keiser a116e68a47
Merge pull request #729 from simdjson/jkeiser/cmake-amalgamate
Add amalgamation support to cmake
2020-04-22 14:16:10 -07:00
PavelP ffaa292006
Master vs2019 x86 compile fixes (#743)
* Added bitexact implementations of _BitScanForward64 and _BitScanReverse64 for VS2019 32-bit builds

* Added bitexact implementations of _umul128 for VS2019 x86, arm, arm64 builds

* Implement mul_overflow for VS2019 arm64 builds

 + implement mul_overflow using __umulh (msvc/clang results: https://godbolt.org/z/smRwA7)

* Added Win32 for VS2019 to .appveyor.yml

* Update amalgamated headers (fix x86 builds with VS2019)
2020-04-21 14:42:53 -04:00
John Keiser d3e44b1108 Add amalgamation support to cmake 2020-04-20 19:50:51 -07:00
John Keiser f9337a1111 Update amalgamated headers 2020-04-19 11:38:02 -07:00
Daniel Lemire 3cb79e6977
Trying again. (#671) 2020-04-02 19:24:43 -04:00
Daniel Lemire 3116e29d16
Release candidate (#655)
* Release candidate
2020-03-31 17:47:25 -04:00
John Keiser 6ee3318531 Update amalgamated headers 2020-03-29 15:07:35 -07:00
John Keiser 5ad405006c Return document::element from parse, load, parse_many, load_many 2020-03-27 12:24:41 -07:00
Bruce Mitchener c3c43769ae Fix typos. 2020-03-22 09:14:14 -07:00
Daniel Lemire 0e45663ce8
Quickstart and intro (#575)
* Add quickstart

* Fix amalgamation

Co-authored-by: John Keiser <john@johnkeiser.com>
2020-03-20 22:07:52 -04:00
John Keiser 7cf3a7511b Add fallback implementation to CI
- Also add SIMDJSON_IMPLEMENTATION_HASWELL/WESTMERE/ARM64/FALLBACK=1/0 to
enable/disable various implemnentations
2020-03-17 14:59:47 -07:00
John Keiser 1a5d8f1957 Add tests for SIMDJSON_EXCEPTIONS=0, add `tie()` support 2020-03-17 13:54:37 -07:00
John Keiser 3bdfe167de Support cout << error 2020-03-06 15:41:51 -08:00
John Keiser 31e8a12e88 Make error_message(error_code) return C string
- Also move all error message logic to include inline
2020-03-06 15:41:51 -08:00
John Keiser b2220d6157 Fix amalgamation 2020-03-05 11:13:25 -08:00
John Keiser b3ea8c406e Add simdjson.cpp for unified use (#515) 2020-03-04 10:12:27 -08:00
John Keiser 99667f7c55 Create top level simdjson.h (#515)
- Allows everyone to #include the same way, singleheader or not.
2020-03-04 10:12:27 -08:00
John Keiser 560f0742cc Update amalgamated headers 2020-02-22 10:42:13 -08:00
John Keiser 910f272467
Add parser implementation interface and selection API (#501)
* Make architecture implementations virtual functions

- Easier to add new architectures (add implementation to implementation.cpp)
- Easier to add new algorithms / functions to architecture selection
(add to implementation.h, implement)
- Automatically select best implementation in static initialization
- Allow user to explicitly select implementation with a string (i.e.
parameter)
- Allow user to inspect current implementation name/description
- Allow user to list available implementations
- Eliminate architecture enum and architecture-based templating
- Add noexcept in non-inline functions

* Move implementation static methods to their own classes

* Detect best supported implementation on first use

* available_implementationsI() -> available_implementations
2020-02-21 16:34:27 -05:00
John Keiser 0c8f2b9d85
Make "make amalgamate" more automatic (#480)
- automatically include local includes in the right places
2020-02-03 12:51:24 -05:00
Daniel Lemire c924aaede9
Fix issue472: make JsonStream a template. (#473)
* Fix issue472: make JsonStream a template.

* Adding missing include.

* Tweaking headers and some minor formatting.

* Removing file from aggregation.

* Moving jsoncharutils

* Adding new header.

* Trying another header.

* Let us try to route around Visual Studio's nonesense.
2020-01-30 17:16:41 -05:00
Daniel Lemire 28710f8ad5
fix for Issue 467 (#469)
* Fix for issue467

* Updating single-header

* Let us make it so that JsonStream is constructed from a padded_string which will avoid dangerous overruns.

* Fixing parse_stream

* Updating documentation.
2020-01-29 19:00:18 -05:00
Daniel Lemire e695a19d11
Trying to fix issue 465 (#466)
* Trying to fix issue 465

* Actually testing

* Refreshing amal.

* Removing spurious ;
2020-01-27 11:25:23 -05:00
Daniel Lemire ab6d4871d8
Adding haswell amal. tests (#447)
* Adding an extra test.

* Disabling the AVX-accelerated minifier.

* Updating amalgamation.
2020-01-15 19:49:11 -05:00
Daniel Lemire f611b65bc0
This updates the minifier. (#446) 2020-01-15 13:45:32 -05:00
UKABUER 773883c486 Fix #420 (#421) 2020-01-09 09:56:43 -05:00
Daniel Lemire 1d621bba37 Being more explicit about EMPTY errors. 2019-12-18 14:39:48 +00:00
Daniel Lemire 102262c7ab
Fixing issue386 (#396)
* Creating arch-specific bitmanipulation.h files.
* Improving system and compiler portability.
* We want to allow trailing_zeroes on zero inputs.
2019-12-16 19:09:18 -05:00
Daniel Lemire 1211c01ca1
Resolves issue 186 (#383)
* Resolves issue 186
https://github.com/lemire/simdjson/issues/186
2019-12-02 12:23:45 -05:00
piotte13 178a0842fe Refreshing the single header files. 2019-11-26 11:08:32 -05:00
Jeremie Piotte 29fc51522a
Introducing concurrency mode in JsonStream. (#373)
* JsonStream threaded prototype

* JsonStream Threaded version working. Still supporting non-threaded version.

* Fix where invalid files would enter infinite loop.

* SingleHeader update

* I will remove -pthread in cmake for now.

* Attempt at resolving the -pthread issue
2019-11-21 11:22:06 -05:00
Jeremie Piotte bdc2b07339
Streams of JSON documents + Large files (>4GB) (#350) (#364)
* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* Fix for https://github.com/lemire/simdjson/issues/345

* Follow up test and fix for https://github.com/lemire/simdjson/issues/345 (#347)

* Final (?) fix for https://github.com/lemire/simdjson/issues/345

* Verbose basictest

* Being more forgiving of powers of ten.

* Let us zero the tail end.

* add basic fuzzers (#348)

* add basic fuzzing using libFuzzer

* let cmake respect cflags, otherwise the fuzzer flags go unnoticed

also, integrates badly with oss-fuzz

* add new fuzzer for minification, simplify the old one

* add fuzzer for the dump example

* clang format

* adding Paul Dreik

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* Fixing issue 351 (#352)

* Fixing issues 351 and 353

* minor fixes and cleaning.

* removing warnings

* removing some copies

* Fix ARM compile errors on g++ 7.4 (#354)

* Fix ARM compilation errors

* Update singleheader

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* fix integer overflow in subnormal_power10 (#355)

detected by oss-fuzz

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18714

* Adding new test file, following https://github.com/lemire/simdjson/pull/355

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* merging main

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* rough prototype working.  Needs more test and fine tuning.

* minor fixes and cleaning.

* adding jsonstream to amalgamation

* merged main into branch

* Addind a JsonStream Demo to Amalgamation

* merging main

* merging main

* make file fix
2019-11-08 17:39:45 -05:00
John Keiser c97eb41dc6 Fix ARM compile errors on g++ 7.4 (#354)
* Fix ARM compilation errors

* Update singleheader
2019-11-04 10:36:34 -05:00
John Keiser 9bbd6bd874 Move headers to implementation area
- jsoncharutils.h, numberparsing.h, simdprune_tables.h
2019-10-14 11:51:41 -07:00
John Keiser 69caa477fb Use struct for UTF-8 checks, remove templating
- Removes templating from simd_input, utf8_checker, and parse_string
- Make drone gcc run a lot faster
- Make drone clang run a little faster (NOTE:
https://hub.docker.com/r/silkeh/clang helps even more, but I wasn't sure
whether we wanted to trust that)
- Make drone arm run in parallel to get results quicker
2019-10-08 17:58:45 -07:00
John Keiser 441963c84c Add AMD64 build_bitmask 2019-08-26 09:46:49 -07:00
John Keiser da0f1cacea Remove static modifiers 2019-08-26 09:46:48 -07:00
John Keiser 5e5592178d Update amalgamated cpp 2019-08-26 09:46:48 -07:00
John Keiser 585f84a734 Move architecture-specific headers to src/ (#287)
* Use namespaces instead of templates for stage1 impls

* Move stage1 implementation into the src/ directory

* Move architecture-specific code to src/
2019-08-21 07:59:49 -04:00
John Keiser 640283fec6 Reuse jobs in Circle CI 2019-08-15 19:41:30 -07:00
John Keiser 0042d9b406 Move UTF8 checking functions into their own file 2019-08-14 10:34:11 -07:00
Daniel Lemire 2ca574d9e6
Removing windows.h (#273) 2019-08-12 19:40:21 -04:00
Daniel Lemire bd9628df93 Producing a new release 2019-08-04 15:43:47 -04:00
Daniel Lemire ee66fb1c60 Version 0.2.0. 2019-08-01 16:23:30 -04:00
Daniel Lemire eba02dc1b9 Runtime dispatch
* Attempt 1 - fn targeting

GCC won't work with templates with different targets, need to specialize all the way up the call stack.

* Compiles properly with cmake. Does not with the Makefile.

* Compilation works with Makefile

* instruction_set changes to architecture

* some aesthetic changes

* fix amalgation and tests + aesthetic changes

* This now compiles and passes tests under CLANG

* Minor correction.

* Trying to make it work on ARM

* Adding missing namespace

* Missing bracket

* Fixing minor compilation issues.

* Getting parse to use runtime dispatch

* Fixing amalgamation script.

* Making sure that NEON is supported.

* Fixing typo

* Merging https://github.com/lemire/simdjson/pull/229

* Manual merge of
https://github.com/lemire/simdjson/pull/229
by @jkeiser  (second part)

* Trying another way.

* Removing the paral.

* Fixing the make file

* Let us make the practice run long enough.

* Resolved the awful slowness.

* Cleaning the README.md

* With runtime dispatching, we should not need flags anymore.

* Changing isa detection file's name + fixing typos.
2019-07-28 22:46:33 -04:00
Daniel Lemire 467831e3d8 Refreshing the single header files. 2019-07-11 09:40:42 -04:00
Daniel Lemire 98b387aac3
Fixing a messed up interleaved #ifdef/namespace. (#211) 2019-07-09 19:48:20 -04:00
Daniel Lemire 977f57fd37 We need to guard the simdutf8check files. 2019-07-09 16:53:28 -04:00
Daniel Lemire a1ea37c336 Updating the amalgamation. 2019-07-09 15:21:07 -04:00
ioioioio 40c098f78a updating single header 2019-07-02 16:37:05 -04:00
Daniel Lemire 954b89e762 New version (0.1.2). 2019-05-09 20:55:26 -04:00
Daniel Lemire e370a65383
Fix for issues 32, 50, 131, 137
* Improving portability.

* Revisiting faulty logic regarding same-page overruns.

* Disabling same-page overruns under VS.

* Clarifying the documentation

* Fix for issue 131 + being more explicit regarding memory realloc.

* Fix for issue 137.

* removing "using namespace std" throughout. Fix for 50

* Introducing typed malloc/free.

* Introducing a custom class (padded_string) that solves several minor usability issues.

* Updating amalgamation for testing.
2019-05-09 17:59:51 -04:00
Dong Xie 1153778f92 fix a bug in copy constructor of ParsedJson::iterator. (#146) 2019-04-22 10:37:02 -04:00
Daniel Lemire bf9b1b1457 New version (mostly setting the singleheader version in sync). 2019-03-13 21:02:39 -04:00
Daniel Lemire d5a185b13e new release. 2019-03-13 20:02:44 -04:00
myd7349 d2fa086198 Fix C4146 build error on UWP with MSVC (#113)
* Fix C4146 build error on UWP with MSVC

* Regenerate single header version

* Fix typo in parsedjson.h

* Regenerate single header version
2019-03-09 08:46:06 -05:00
Nan Xiao c1dea29501 Fix typo 2019-03-07 13:55:41 +08:00
Georgios Floros d873ee9983 Reuse aligned_malloc (#108) 2019-03-06 10:12:55 -05:00
Geoff Langdale 6628c365c9 Substantial refactor (and clang-format google stype) of stage1_find_marks.cpp 2019-03-06 11:09:50 +11:00
Thomas Navennec 352dd5e7fa Change parse_json return type from bool to int (#82)
* Added simdjerr namespace

* Updated jsonparser files

* updated stage1 and stage2

* removed stage2 inline function

* Added forgotten return statements

* Updated tools and benchmarks

* Corrected parenthesis

* Removed extra =

* Accidentally undid reinterpret_cast

* Better comments, undid a header name fuckup

* Added an errorMsg method, updated readme

* Removed useless header from stage2

* Updated single-header file

* added simdjerr.cpp contents to simdjson.cpp

* Made single header version work

* Updated singleheader test, fixed simdjson.cpp

* Renamed simdjerr namespace and files to simdjson

* Updating the amalgamation.
2019-03-02 17:18:45 -05:00
Daniel Lemire a24e701b4e First release (0.0.1) 2019-02-26 10:14:49 -05:00
Geoff Langdale 105ced3b01 Update single-header version as well as fix c++11 -> c++17 in scripts for this version 2019-02-26 13:30:28 +11:00
Geoff Langdale 527ee8eace Merge branch 'master' of https://github.com/lemire/simdjson 2019-02-23 11:28:48 +11:00
Geoff Langdale 2a3e9b5476 Fix the suggested command line to build to amalgamation_demo; it needs C++17 for string_view 2019-02-23 11:28:09 +11:00
Daniel Lemire 1f22ed98ac Updating amalgamation files. 2019-02-22 19:16:46 -05:00
Daniel Lemire 44f39815d3 Updating amalgamation script. 2019-02-22 15:42:44 -05:00
Thomas Navennec 9606343b2c ParsedJson & ParsedJson::iterator definitions in .cpp files (#47)
* Minor change to benchmark cmake

* Moved ParsedJson and its Iterator to separate .cpp files

* Uncommented functions, that has nothing to do with this pr

* Removed really_inline comments

* Reinstated some inline functions to restore previous performance

* Re-merged iterator in ParsedJson

* Uncommented some WARN_UNUSED
2019-02-22 14:38:35 -05:00
Daniel Lemire 3e74059cc3 Fix the shuffle. 2019-01-04 20:12:55 -05:00
Daniel Lemire a00df9b992 Number parsing fix. 2019-01-04 17:36:52 -05:00
Daniel Lemire bad32be5f6 Merge branch 'stage12unified_attempt2' 2018-12-31 17:33:01 -05:00
Daniel Lemire 3ce1dd8087 Cleaning. 2018-12-31 17:13:32 -05:00
Daniel Lemire 0317287071 Minor fix 2018-12-31 11:59:12 -05:00
Daniel Lemire 3803c84452 Single-header version. 2018-12-30 21:50:10 -05:00