Commit Graph

586 Commits

Author SHA1 Message Date
Daniel Lemire 06643fc9f5
Additional tests and document tuning (#1684)
* Additional example.

* Adds more tests.

* Actually using the variable.
2021-08-02 16:35:02 -04:00
Daniel Lemire 47a62db559
Isolated jkeiser fix for issue 1632: make it so that INCORRECT_TYPE is a recoverable condition in On Demand (#1663) 2021-07-23 11:32:26 -04:00
Daniel Lemire ea3d4e7ce5
Fuzzing timeout (bug fix) (#1650)
* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-06 14:36:38 -04:00
Daniel Lemire bea1483cde
Fixing minor issue with document stream (DOM). (#1648)
* Fixing minor issue with document stream (DOM).

* Porting over the fix.
2021-07-05 17:40:04 -04:00
Amos Bird 8df32cea33
Return err when alloc failure (#1567) 2021-05-14 22:51:07 -04:00
Daniel Lemire 85b910814e
Under ARM, it is slightly better to reverse the word once and then extract the bits. (#1545)
* Under ARM, it is slightly better to reverse the word once and then extract the bits.

* Guarding the zero_leading_bit call to avoid sanitizer warnings.
2021-04-30 18:34:21 -04:00
friendlyanon 5ec85197f8
CMake refactor stage1 (#1512)
* Remove CMP0025 policy

This policy is already set to NEW by the minimum required version.

* Use HOMEPAGE_URL in the project call

* Use VERSION in the project call

* Detect if this is the top project

* Port simdjson-user-cmakecache to a CMake script

* Create a developer mode

The SIMDJSON_DEVELOPER_MODE option set to ON will enable targets that
are only useful for developers of simdjson.

* Consolidate root CML commands into logical sections

* Warn about intended use of developer mode

* Prettify the just_ascii test

* Remove redundant CMake variables

* Inline CML contents from include and src

* Raise minimum CMake requirement to 3.14

* Define proper install rules

* Restore thread support variable

* Add BUILD_SHARED_LIBS as a top level only option

* Force developer mode to be on in CI

* Include flags earlier in developer mode

* Set CMAKE_BUILD_TYPE conditionally

CMAKE_BUILD_TYPE is used only by single configuration generators and is
otherwise completely ignored.

* Remove useless static/shared options

simdjson now uses the CMake builtin BUILD_SHARED_LIBS to switch the
built artifact's type.

* Remove unused CMAKE_MODULE_PATH variable

* Refactor implementation switching into a module

* Factor exception option out into a module

* Reformat simdjson-flags.cmake

* Rename simdjson-flags to developer-options

* Accumulate properties into an include module

This is done this way to avoid using utility targets that must be
exported and installed, which could potentially be misused by users of
the library.

* Port impl definitions to props

* Port exception options to props

* Lift normal options to the top

* Port developer options to props

* Remove simdjson-flags from benchmark

* Document the developer mode in HACKING

* Fix include path in installed config file

* Fix formatting of prop commands

* Fix tests that include .cpp files

* Change GCC AVX fixes back to compile options

* Deprecate SIMDJSON_BUILD_STATIC

* Always link fuzz targets to simdjson

* Install CMake from simdjson's debian repo

* Add gnupg for apt-key

* Make sure ASan link flags come first

* Pass CI env variable to cmake invocation

* Install package for apt-add-repository

* Remove return() from flush macro

* Use directory level commands instead of props

* Restore the github repository variable

* Set developer mode unconditionally for checkperf

The CI env variable is only set in the CI and this target is always run
in developer mode.

* Attempt to fix ODR violation in parsing checks

These tests were compiling the simdjson.cpp file again and linking to
the simdjson library target causes ODR violations.

Instead of linking to the target, just inherit its props.

* Move variables before the source dir

* Mark props to be flushed after adding more

* Use props for every command for the library

* Use keyword form for linking libs

* Handle deprecation of SIMDJSON_JUST_LIBRARY

* Handle deprecations in a separate module

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2021-04-23 09:24:56 -04:00
Daniel Lemire 8eed8f5155
Document stream: truncate final unfinished document and give access to the number of truncated bytes. (#1534)
* Truncate final unclosed string.

* Adding more precise remarks.

* Better documentation and more robust code.

* ARM + PPC corrections.

* Patching ARM implementation with new stage1_mode parameter.

* Fixed most problems.

* Correcting white spaces and adding a remark.

* This adds the truncated_bytes() method to the stream instances.
2021-04-23 09:24:00 -04:00
Daniel Lemire 6ca6ee5a6f
Correcting what seems to be a typo and adding some extra logging. (#1535) 2021-04-04 12:25:05 -04:00
Daniel Lemire d0821adf0e
This implements string serialization for On Demand instances. (#1527)
* This implementations string serialization for On Demand instances.

* Adding more documentation.

* Another remark.

* Marking the new functions as inline.

* casts apparently do not work.

* Upgrading the API.

* Making the code really free from exceptions.

* At another fix for exceptionless.

* Modify to_chars so that it does not pad integers with '.0'.

* Negative 0 cannot be expressed as an integer.

* Again, accomodating exceptionless usage.

* Using x <= -0 does not allow you to determine the sign since 0 <= -0. I am not sure where
this bug comes from.
2021-04-01 11:25:00 -04:00
John Keiser a987192040 Remove SIMDJSON_CPP 2021-03-09 09:10:27 -08:00
John Keiser 985dfab2c4 Don't use TARGET unless the target options are *not* specified
This eliminates the possibility of inlining target failures for ondemand

Also makes it so we always compile common architectures needed by simdjson.cpp in simdjson.h, since amalgamation has no way to reason about whether to include / exclude it.
2021-03-08 13:49:09 -08:00
John Keiser f51d50399c Only include builtin implementation from header 2021-03-08 13:48:53 -08:00
John Keiser f55893807a
Merge pull request #1473 from simdjson/jkeiser/promise-padded
Support user-provided buffers in On Demand
2021-03-04 20:56:34 -08:00
Daniel Lemire dc2f767171
Ensuring that stage 1's json_block and cie are never copied and using explicit constructors (issue 1475) (#1478) 2021-03-04 07:59:36 -05:00
John Keiser f0e92e3bdd Pass "capacity" straight to iterate, support std::string 2021-03-03 12:51:00 -08:00
Daniel Lemire 0c199cffc3
Removing mallocs in main library. (#1468) 2021-03-02 13:09:22 -05:00
Daniel Lemire 9577c54999
Provide the CMake install the necessarily information (and flags) to hand Windows DLL and add Windows installation tests (#1457)
* This gives the CMake install the necessarily information (and flags) to know
whether we have a Windows DLL and in such cases how to handle the linkage.
2021-02-26 16:17:05 -05:00
John Keiser b2de2dfd1b
Merge pull request #1416 from simdjson/jkeiser/safe-iterators-2
Add safety checks for out of order array/object iteration+indexing
2021-02-05 09:47:03 -08:00
Daniel Lemire 26b5b19f88 Unneeded. 2021-02-02 17:58:10 -05:00
Tibbel 5613d30e97
partly replacement old-style-cast to c++ *_cast (#1403)
Co-authored-by: Tibbel <tibbel@ma-gi.de>
2021-01-27 13:33:48 -05:00
John Keiser 22b3ea93a8 Emit an error if user tries to iterate arrays out of order 2021-01-26 20:00:19 -08:00
Daniel Lemire 1005c62e90
It seems that we export too many targets. (#1385)
* It seems that we export too many targets.

* Adding missing word.

* Let us try this.

* Restoring dead line.

* Some fixes.

* Update src/CMakeLists.txt

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Update cmake/simdjson-config.nothread.cmake.in

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Update cmake/simdjson-config.cmake.in

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Update CMakeLists.txt

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Update CMakeLists.txt

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Removing useless file

* Simplifying the PR somewhat.

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>
2021-01-20 13:20:49 -05:00
Daniel Lemire 7dbe4caf3f
Taking a float and adding hundreds of zeros, you may get a truncated value that is just one over the desired value (bug fix) (#1388)
* Found a bug where if take some float and add many zeros, you may get a truncated value that is just one over the desired value.
2021-01-11 17:07:25 -05:00
John Keiser 158a3c53d8
Merge pull request #1351 from simdjson/jkeiser/unordered-lookup
Make `object["field"]` order-insensitive in On Demand
2020-12-24 13:33:09 -08:00
John Keiser 041d59cc17 Create acceptance_tests, all_tests, etc. make targets
And use them for mingw build and test
2020-12-23 09:14:45 -08:00
Daniel Lemire 6a2435ab32 Merge branch 'master' into dlemire/prune_validate_utf8 2020-12-19 10:46:14 -05:00
Daniel Lemire 85001c55fb
Fixing UTF-8 validation under PPC64 (#1346)
* Entering a new UTF-8 test

* Maybe *I* had a bug in the tests.

* Replacing nulls with 1s.

* Let us try to be more verbose.

* Return 0.

* Fixing issue.

* Adding puzzler scenario.

* Fixing PPC64

Co-authored-by: Daniel Lemire <dlemire@rcs-power9-talos>
2020-12-19 10:42:27 -05:00
Daniel Lemire 8c4cf7757f I meant to remove these files a while ago. 2020-12-18 15:29:08 -05:00
Daniel Lemire d9c4191e8a
This should fix an issue we have with unclosed strings with document_stream (#1321)
* This should fix an issue we have with unclosed strings.

* Patching the fallback kernel.

* Better guarding the code.

* Patching the fallback.
2020-11-30 13:43:57 -05:00
Daniel Lemire 218c274090
Updating main branch for legacy libc++ support (#1288)
* Updating main branch for legacy libc++ support

* Adopting

* Removing unnecessary math header.

* Updating the single-header files so we can pass the new tests.

* Portable infinite-value detection is hard.

* Working toward disabling boost json selectively.

* Selectively disabling Boost JSON

* More work toward selectively disabling boost json.
2020-11-04 12:24:42 -05:00
Paul Dreik af4db55e66
remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Daniel Lemire 6321df078a
This fixes very minor "mistakes". (#1281) 2020-11-02 20:04:04 -05:00
Daniel Lemire a8bf10ea5a Minor patch. 2020-10-30 14:51:50 -04:00
Daniel Lemire 562d4f4f58 Fixing a minor logical error. 2020-10-29 16:42:50 -04:00
Danila Kutenin f46a0f64f2
PPC64 support (#1254)
* Initial PPC64 support

* Add travis CI

* Fix outdated cmake version for travis

* Fix indendtation

* Try another workaround for outdated cmake in travis

* Try beta cmake

* Add dash before beta

* Use builtin snaps

* Use cmake as rocksdb

* Test cmake on bionic

* Remove unnecessary things from travis

* Remove unnecessary things from travis

* Another try of compiler install

* Add all major compilers

* Add all major compilers

* Add all major compilers

* Tweak travis a bit

* Typo

* More robust travis

* Typos typos typos

* Add fewer compilers, add non specific build for clang and gcc, should be the final config

* CMAKE_FLAGS is in incorrect place

* Remove default implementation

* Limit build thread number

* Fall back prefix_xor to a usual implementation, no performance boost is noticed

* Test for power9 as it is the main architecture for OpenPOWER right now

* Add to documentation to build with power9 as the implementation is compatible but compiler optimizations is not

* Replace ARM with PPC in the comment
2020-10-27 18:43:39 -04:00
Daniel Lemire 07a6e098c8
This would allow users to find out what builtin is. (#1227)
* This would allow users to find out what builtin is.

* Trying another approach.

* Added instructions.

* Cleaning up the printout.

* Let us be less invasive.

* Adding a comment.
2020-10-15 21:58:42 -04:00
Daniel Lemire bb2bc98a22
Fix issue https://github.com/simdjson/simdjson/issues/1127 (#1224) 2020-10-13 09:18:54 -04:00
Daniel Lemire 43da4f7ccc Corrected number 2020-10-12 17:59:13 -04:00
Daniel Lemire 37e6d1e9c7
new number parsing (#1222)
* Remove our dependency on strtod_l by bundling our own slow path.

* Ok. Let us drop strtod entirely.

* Trimming down the powers to -342.

* Removing useless line.

* Many more comments.

* Adding some DLL exports.

* Let the gods help those who rely on windows+gcc.

* Marking the subnormals as unlikely. This is pretty much "performance neutral", but it might help just a bit with twitter.json.
2020-10-10 12:47:49 -04:00
John Keiser 8b978e6aea Don't call functions max() (conflicts with Windows macro) 2020-10-04 12:47:30 -07:00
John Keiser a700848bae Move ondemand implementation to include/ 2020-10-04 12:47:30 -07:00
John Keiser 9bbfd5804e Put haswell/westmere/fallback/amd64 in simdjson namespace 2020-10-04 12:47:30 -07:00
John Keiser b234d74f43 Remove unnamed namespace from ondemand 2020-10-04 12:47:30 -07:00
John Keiser cd49ff330d Fix g++-7 quickstart errors 2020-10-04 12:47:30 -07:00
John Keiser 49faf7af1a Make simdjson_result implementation-specific 2020-10-04 12:47:30 -07:00
John Keiser c892b83c93 Make ondemand classes usable from simdjson_result 2020-10-04 12:47:30 -07:00
John Keiser 8fd0cdc732 Iterate value without going through indirection
Avoids issues with value being released early
2020-10-04 12:47:30 -07:00
John Keiser fe7a4d42d3 Fix top level values 2020-10-04 12:47:30 -07:00
John Keiser 7c2072789c Fix issue with strings not including their first character 2020-10-04 12:47:30 -07:00