Commit Graph

40 Commits

Author SHA1 Message Date
Daniel Lemire 8dbb598057 Adding another test 2020-01-02 14:22:43 -05:00
Daniel Lemire ba9dc12164 Adding tests motivated by https://github.com/lemire/simdjson/pull/430 2020-01-02 14:20:51 -05:00
Daniel Lemire 2caac2b218
Adding two test files from https://github.com/lemire/simdjson/pull/404/files (#415) 2019-12-24 14:04:11 -05:00
Jeremie Piotte bdc2b07339
Streams of JSON documents + Large files (>4GB) (#350) (#364)
* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* Fix for https://github.com/lemire/simdjson/issues/345

* Follow up test and fix for https://github.com/lemire/simdjson/issues/345 (#347)

* Final (?) fix for https://github.com/lemire/simdjson/issues/345

* Verbose basictest

* Being more forgiving of powers of ten.

* Let us zero the tail end.

* add basic fuzzers (#348)

* add basic fuzzing using libFuzzer

* let cmake respect cflags, otherwise the fuzzer flags go unnoticed

also, integrates badly with oss-fuzz

* add new fuzzer for minification, simplify the old one

* add fuzzer for the dump example

* clang format

* adding Paul Dreik

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* Fixing issue 351 (#352)

* Fixing issues 351 and 353

* minor fixes and cleaning.

* removing warnings

* removing some copies

* Fix ARM compile errors on g++ 7.4 (#354)

* Fix ARM compilation errors

* Update singleheader

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* fix integer overflow in subnormal_power10 (#355)

detected by oss-fuzz

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18714

* Adding new test file, following https://github.com/lemire/simdjson/pull/355

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* merging main

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* rough prototype working.  Needs more test and fine tuning.

* minor fixes and cleaning.

* adding jsonstream to amalgamation

* merged main into branch

* Addind a JsonStream Demo to Amalgamation

* merging main

* merging main

* make file fix
2019-11-08 17:39:45 -05:00
Daniel Lemire 52640518d3 Adding new test file, following https://github.com/lemire/simdjson/pull/355 2019-11-05 08:58:41 -05:00
Daniel Lemire b1224a77db
Fixing issue 351 (#352)
* Fixing issues 351 and 353
2019-11-01 16:05:28 -04:00
Daniel Lemire da1c35d04b Final (?) fix for https://github.com/lemire/simdjson/issues/345 2019-10-24 16:33:37 -04:00
Daniel Lemire aa45fe7359
Adding check for subnormal parsing. (#328) 2019-10-07 11:01:34 -04:00
Daniel Lemire 92334a8e28 Better tests. 2019-09-02 12:32:44 -04:00
saka1 c1f27fb848 Accept large unsigned integers (#295)
* handle uint64 value in JSON
* Add integer_tests
* Add get_unsigned_integer() on  ParsedJson::BasicIterator
* Write 'u' to tape when the value seems unsigned
* Add to handle 'u' element
* Brush up integer_tests.cpp
* Append tests/integer_tests in .gitignore
* Add comments to is_integer and is_unsigned_integer
2019-09-02 10:50:24 -04:00
Daniel Lemire dd07212f02 Adding more test files. 2019-07-29 13:34:13 -04:00
Daniel Lemire a53d95099c
Intrinsic-based flatten (#234)
* Providing a flatten function with intrinsics (for Visual Studio).
2019-07-29 13:28:02 -04:00
Daniel Lemire f76ee5e5ef Fixes issue 221 (#222)
https://github.com/lemire/simdjson/issues/221
2019-07-29 10:07:07 -04:00
Daniel Lemire d7f7f1b200
Fixing issue. (#193) 2019-06-20 18:49:47 -04:00
Daniel Lemire 3db8c5a0eb
Fixing the issue (#191) 2019-06-12 16:32:46 -04:00
Daniel Lemire 59194dcf4d
Issue182: fixed (#183)
* Verifying issue 182.

* Fixing the corresponding bug.
2019-06-05 18:51:29 -04:00
Daniel Lemire f75280ac9c
Fix for issue 150 (#162)
* Checks for issue 150. We run through the test files with sanitizers on.

* Fix for issue 150: the remaining issues were an overrun on the depth capacity and an "off-by-1" overrun on tape capacity.

* Improving makefile.

* Safer git submodule command.

* Getting get 'git' on circleci
2019-05-09 20:51:33 -04:00
Daniel Lemire e370a65383
Fix for issues 32, 50, 131, 137
* Improving portability.

* Revisiting faulty logic regarding same-page overruns.

* Disabling same-page overruns under VS.

* Clarifying the documentation

* Fix for issue 131 + being more explicit regarding memory realloc.

* Fix for issue 137.

* removing "using namespace std" throughout. Fix for 50

* Introducing typed malloc/free.

* Introducing a custom class (padded_string) that solves several minor usability issues.

* Updating amalgamation for testing.
2019-05-09 17:59:51 -04:00
Heinz N. Gies c5a3f9ccd4 Add failing test for a json with content zero (#134)
* Add failing test for a json with content zero

* Mark 0 byte as false in structural_or_whitespace_or_exponent_or_decimal_negated
2019-05-09 12:24:22 -04:00
Heinz N. Gies c1975166a0 False atom fix (#156)
* Add failing test for falsy atom

* Fix false atom parsing
2019-05-09 10:45:42 -04:00
technateNG 6f0d350f2c Fix to issue #148. (#151)
* Issue #148 fix.

* Test cases for issue #148.
2019-05-07 20:56:36 -04:00
Daniel Lemire 0d81fd287e
With this commit we can do all tests with full sanitizers on, and get no warning (#132)
* Making sure we can run with the sanitizers on.
* Minor code simplification in the number parsing.
* Following @EmilGedda 's recommendations regarding the makefile.
* Reference to blog post.
* Adding link to https://johnnylee-sde.github.io/Fast-numeric-string-to-int/
* Better hex parsing.
2019-04-24 17:31:47 -04:00
Daniel Lemire 609e96b5d1 Fix for https://github.com/lemire/simdjson/issues/119 2019-03-13 11:01:31 -04:00
Geoff Langdale b3aa5f131e Add another check for bogus codepoints. 2019-02-26 13:25:41 +11:00
Geoff Langdale 5289bf3eeb Fixing Utf8 validation question #72 2019-02-26 13:17:29 +11:00
Daniel Lemire 671527c3ba Tracking https://github.com/lemire/simdjson/issues/71 2019-02-25 14:10:06 -05:00
Daniel Lemire c16ecd5138 A 2019-01-18 15:06:23 -05:00
Daniel Lemire e90c0f4296 More tests. 2019-01-04 17:47:56 -05:00
Daniel Lemire 59b26dd64d Adding more tests 2019-01-04 17:37:44 -05:00
Daniel Lemire 779ce184fb Getting ready to document the tape format. 2018-12-18 14:21:22 -05:00
Daniel Lemire 0769c39e27 Ok. Looks complete. 2018-12-14 21:32:42 -05:00
Daniel Lemire e4703a383b Even safer. 2018-12-10 20:54:31 -05:00
Daniel Lemire 8a2281269c Cleaning memset amd adding more tests. 2018-12-10 19:20:52 -05:00
Daniel Lemire 8615760331 Should now pass. 2018-12-10 15:16:31 -05:00
Daniel Lemire 176d2ccda4 Tweaking. 2018-12-10 14:25:49 -05:00
Daniel Lemire 18633e02d2 Added more thorough testing. 2018-10-23 20:19:33 -04:00
Daniel Lemire 9fc8d8444b We want to allow more than just arrays and objects, as per the JSON spec. 2018-10-17 13:57:42 -04:00
Daniel Lemire 57d7544b52 Adding... 2018-10-17 13:23:45 -04:00
Daniel Lemire 70c122074f Tests. 2018-10-08 14:41:36 -04:00
Daniel Lemire 93f4e1d9b4 Adding validating step. 2018-07-25 22:59:40 -04:00