Commit Graph

576 Commits

Author SHA1 Message Date
John Keiser 55faf4c5bc
Recommend simdjson::ondemand over simdjson::builtin::ondemand (#1380)
Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-01-14 17:33:49 -05:00
John Keiser 3279c2f15b Remove unnecessary "try_get_XXX" methods
Also don't distinguish between & and && (instead, advance the first time
you encounter a scalar no matter what)
2021-01-11 15:19:26 -08:00
John Keiser 38da15b501 Rename checkpoint() -> position(), add token_position type 2021-01-11 15:19:24 -08:00
John Keiser 0f515785c6 Support reading scalars out of order 2021-01-11 15:17:46 -08:00
John Keiser 6fed8d2a26 Use active implementation for stage 1 on demand 2021-01-11 14:57:52 -08:00
John Keiser 0039c5b981 Disallow parser.iterate("1"_padded), as it won't work 2021-01-01 19:18:00 -08:00
John Keiser a1cf588d5f Fix GCC 7 warning when inlining does its job 2020-12-21 09:19:07 -08:00
John Keiser e7e09e444c Use find_field in benchmark 2020-12-20 11:39:56 -08:00
John Keiser 195acc3e45 Add find_field / find_field_unordered to object 2020-12-20 11:39:50 -08:00
John Keiser b8426584fc Make field lookup order-insensitive. 2020-12-19 13:57:29 -08:00
Daniel Lemire 85001c55fb
Fixing UTF-8 validation under PPC64 (#1346)
* Entering a new UTF-8 test

* Maybe *I* had a bug in the tests.

* Replacing nulls with 1s.

* Let us try to be more verbose.

* Return 0.

* Fixing issue.

* Adding puzzler scenario.

* Fixing PPC64

Co-authored-by: Daniel Lemire <dlemire@rcs-power9-talos>
2020-12-19 10:42:27 -05:00
John Keiser f785f76d98
Merge pull request #1342 from simdjson/jkeiser/iter-safety
Safety: assert in debug mode when on demand values are used out of order
2020-12-16 15:58:29 -08:00
John Keiser d670906ff3 Don't keep a separate at_start boolean in object 2020-12-15 11:29:31 -08:00
Daniel Lemire c578f63f34
Fixing issue 1337 (#1338)
* Fixing issue 1337

* This test should always run.

* Removing bad attribute access.
2020-12-14 16:20:36 -05:00
Daniel Lemire 9a404bdcdc
Reenabling the optimized kernels (main branch). (#1344)
* Reenabling the optimized kernels (main branch).

* Defining SIMDJSON_CAN_ALWAYS_RUN_PPC64 and SIMDJSON_CAN_ALWAYS_RUN_ARM64

* Adding the bad UTF8 string from the fuzzer.

* Taking into account John's comments.

* Bumping the lib version.

* Update CMakeLists.txt
2020-12-14 14:28:45 -05:00
John Keiser f9c6dedca1 Add test for out-of-order parse asserts 2020-12-13 13:39:47 -08:00
John Keiser 73c7b7db3d Safety: check index of scalar values before use 2020-12-11 11:57:09 -08:00
John Keiser 6b02b06581
Merge pull request #1330 from simdjson/jkeiser/depth-tracking
Permit partial iteration in On Demand
2020-12-09 12:40:58 -08:00
John Keiser 806cb39103 Clean up some more benchmark/test code 2020-12-07 13:44:58 -08:00
John Keiser 3baba73cf5 Don't call functions in assume (VS2019 Clang doesn't like it) 2020-12-07 11:17:24 -08:00
John Keiser db6bf15361 Allow all array/object/etc. to be copied 2020-12-06 16:29:53 -08:00
John Keiser 3aa3175378 Add tests for recovering from partial child iteration 2020-12-06 16:19:06 -08:00
John Keiser 4c63956624 Enable object["x"]["y"] 2020-12-06 15:23:52 -08:00
John Keiser c89647af9e Make all values use same underlying iterator 2020-12-06 15:23:52 -08:00
John Keiser 1303a88769 Unconditionally track depth 2020-12-06 15:23:52 -08:00
Daniel Lemire cbacec0760 Releasing 0.7.0. 2020-12-04 18:56:15 -05:00
Daniel Lemire aaa12c8fb6
more thread testing (#1324)
* Adding a few tests.

* More tests.

* Trailing.

* More links/documentations.
2020-12-02 11:33:36 -05:00
Daniel Lemire 9b5150daef
Fixing issue 1247 (#1325) 2020-12-01 19:09:55 -05:00
Daniel Lemire bc4087ac96
Hunting for data races in document_stream (#1323)
* Adding matching tests.

* Actually adding thread testing actions.
2020-12-01 12:40:15 -05:00
Daniel Lemire dc69bc28ae
Trying to verify recent document stream issues. (#1318)
* Trying to verify recent document stream issues.

* Adding another one.

* More thorough tests.

* Removing trailing spaces.

* Working toward exposing some issues.

* Tweaking.
2020-11-27 17:04:10 -05:00
Daniel Lemire 53577f11e1 Explicit worker disposal. 2020-11-27 15:39:43 -05:00
Daniel Lemire 1abb64f6f6
This fixes issue 1302 (#1303)
* This verifies issue 1302

* I believe that this fixes the issue.

* Let us do it differently.

* Better comments.
2020-11-27 14:19:36 -05:00
Mikhail Matrosov b442b29ccf
Mark comparison operators as const (#1320)
Co-authored-by: Mikhail Matrosov <mikhail.matrosov@aimtech.com>
2020-11-27 14:18:42 -05:00
Daniel Lemire 5609851747
This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU. (#1319)
* This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU.
2020-11-27 09:37:54 -05:00
Paul Dreik 68a8004518
don't memcpy after failed alloc (#1315)
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=27675
2020-11-21 07:52:24 +01:00
Paul Dreik 0b39e3a6cf
add fuzzer for padded_string (#1312)
This also fixes an overflow problem.
2020-11-19 16:51:56 +01:00
Daniel Lemire 268f26a84f Tweak of the documentation. 2020-11-07 16:33:12 -05:00
Paul Dreik d9456b3030
fix numeric_limits usage problem on windows caused by the max macro (#1293)
See https://stackoverflow.com/questions/2789481/problem-calling-stdmax/2789509#2789509
2020-11-06 06:55:27 +01:00
Daniel Lemire 218c274090
Updating main branch for legacy libc++ support (#1288)
* Updating main branch for legacy libc++ support

* Adopting

* Removing unnecessary math header.

* Updating the single-header files so we can pass the new tests.

* Portable infinite-value detection is hard.

* Working toward disabling boost json selectively.

* Selectively disabling Boost JSON

* More work toward selectively disabling boost json.
2020-11-04 12:24:42 -05:00
Tyler Kennedy 921c79f26f
Add method to get the total number of slots used by an array in the t… (#1253)
* Add method to get the total number of slots used by an array in the tape.

* Whoops
2020-11-04 10:35:54 -05:00
Paul Dreik af4db55e66
remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Convery e6258377d9
Additional define to silence the 32-bit portability warning. (#1279)
Under compilers like MSVC the `#pragma message` about portability will be issued for each translation module; regardless of whether or not `SIMDJSON_PORTABILITY_H` has already been defined. As such a new define `SIMDJSON_NO_PORTABILITY_WARNING`, can be defined prior to the inclusion of `<simdjson.h>` to silence it.
2020-11-02 08:37:13 -05:00
Paul Dreik 54ffbbe7db
remove asserts from compute_float_64 (#1276) 2020-11-01 18:34:42 +01:00
Paul Dreik 0b82f07115
fix segfault in numberparsing #1273 (#1274)
This was a read overflow.
2020-11-01 18:27:21 +01:00
Paul Dreik 265db2e533
fix non ascii sources (#1275)
Master does not build because of non-ascii sources, merging without waiting for CI.
2020-11-01 11:14:01 +01:00
Paul Dreik f93fb21c95
optionally disable deprecated apis (#1271)
Introduce cmake option SIMDJSON_DISABLE_DEPRECATED_API (default Off)
which turns off deprecated simdjson api functions by setting the macro
 SIMDJSON_DISABLE_DEPRECATED_API.

For non-cmake users, users will have to set SIMDJSON_DISABLE_DEPRECATED_API
by some other means to disable the api.

Closes #1264
2020-11-01 06:38:52 +01:00
Daniel Lemire b1444b4dfb
Mostly tiny changes, with one optimization to fallback for number parsing. (#1265)
* Mostly tiny changes, with one optimization to fallback for number parsing.

* Missed an update.
2020-10-29 11:18:11 -04:00
Danila Kutenin f46a0f64f2
PPC64 support (#1254)
* Initial PPC64 support

* Add travis CI

* Fix outdated cmake version for travis

* Fix indendtation

* Try another workaround for outdated cmake in travis

* Try beta cmake

* Add dash before beta

* Use builtin snaps

* Use cmake as rocksdb

* Test cmake on bionic

* Remove unnecessary things from travis

* Remove unnecessary things from travis

* Another try of compiler install

* Add all major compilers

* Add all major compilers

* Add all major compilers

* Tweak travis a bit

* Typo

* More robust travis

* Typos typos typos

* Add fewer compilers, add non specific build for clang and gcc, should be the final config

* CMAKE_FLAGS is in incorrect place

* Remove default implementation

* Limit build thread number

* Fall back prefix_xor to a usual implementation, no performance boost is noticed

* Test for power9 as it is the main architecture for OpenPOWER right now

* Add to documentation to build with power9 as the implementation is compatible but compiler optimizations is not

* Replace ARM with PPC in the comment
2020-10-27 18:43:39 -04:00
Daniel Lemire a75c07065f
Fix for issue 1246. We document the relationship between parser instances and elements (#1250)
* Fix for issue 1246.

* Adopting John's wording.
2020-10-26 08:40:45 -04:00
Daniel Lemire 6a86ef5a7d Issue release 2020-10-23 09:32:25 -04:00