Commit Graph

1946 Commits

Author SHA1 Message Date
friendlyanon 669e5b91b4
Improve CI workflows (#1291)
* Fix Cirrus CI

CMake is not installed at this point yet.

* Add caching in Circle CI

* Add caching to the MinGW Github workflows

* Fix Circle CI config

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-05 08:42:00 -05:00
Daniel Lemire a06e0958ca
We do not want the amalgamate.py script to run *ever* if you do not have at least Python 3. (#1290) 2020-11-04 13:52:52 -05:00
friendlyanon a56149e79a
Cache CMake dependencies in CI (#1282)
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-04 13:35:33 -05:00
friendlyanon c805fc28a4
Remove git modules (#1258)
* Bump minimum CMake version

* Remove unnecessary git checks

* Move benchmark options where they are used

* Declare helper functions for dependencies

The custom solution here is tailored for fast configure times, but only
works for dependencies on Github.

* Import dependencies using the declared commands

* Remove git submodules

* Call target_link_libraries properly

target_link_libraries must not be called without a requirement
specifier.

* Fix includes for competition

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-04 13:34:29 -05:00
Daniel Lemire 218c274090
Updating main branch for legacy libc++ support (#1288)
* Updating main branch for legacy libc++ support

* Adopting

* Removing unnecessary math header.

* Updating the single-header files so we can pass the new tests.

* Portable infinite-value detection is hard.

* Working toward disabling boost json selectively.

* Selectively disabling Boost JSON

* More work toward selectively disabling boost json.
2020-11-04 12:24:42 -05:00
Tyler Kennedy 921c79f26f
Add method to get the total number of slots used by an array in the t… (#1253)
* Add method to get the total number of slots used by an array in the tape.

* Whoops
2020-11-04 10:35:54 -05:00
Paul Dreik af4db55e66
remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Paul Dreik 9f78559cc8
detect trailing whitespace in a CI job (#1268) 2020-11-03 20:41:19 +01:00
Paul Dreik 23b4bc93aa
move amalgamate from bash to python (#1278)
This is much faster (from 3.5 to 0.14 seconds)
2020-11-03 07:35:16 +01:00
Daniel Lemire 6321df078a
This fixes very minor "mistakes". (#1281) 2020-11-02 20:04:04 -05:00
Convery e6258377d9
Additional define to silence the 32-bit portability warning. (#1279)
Under compilers like MSVC the `#pragma message` about portability will be issued for each translation module; regardless of whether or not `SIMDJSON_PORTABILITY_H` has already been defined. As such a new define `SIMDJSON_NO_PORTABILITY_WARNING`, can be defined prior to the inclusion of `<simdjson.h>` to silence it.
2020-11-02 08:37:13 -05:00
Jeremy Ong b3a26fd0cc
Do not enable -fPIC when compiling on Windows (#1280)
Regardless of the compiler, Windows targets do not support -fPIC,
as position independent code is already implicitly enabled. Compiling
simdjson with Clang on Windows will error because -fPIC is an
unsupported option for target 'x86_64-pc-windows-msvc'.
2020-11-02 08:18:04 -05:00
Paul Dreik 47669566da
fix unintended early return in ondemand unit test loops (#1263)
fix uninteded early return in ondemand unit test loops

go through and fix warnings appearing in qtcreator,

qualify with std::, add const, abort on error

get rid of ulp_distance, not needed anymore when parsing is exact
2020-11-01 19:08:40 +01:00
Paul Dreik 03a1bb0a5b
detect if amalgamation is needed (#1269)
output amalgamation date to be locale and timezone independent
2020-11-01 19:07:15 +01:00
Paul Dreik 54ffbbe7db
remove asserts from compute_float_64 (#1276) 2020-11-01 18:34:42 +01:00
Paul Dreik 0b82f07115
fix segfault in numberparsing #1273 (#1274)
This was a read overflow.
2020-11-01 18:27:21 +01:00
Paul Dreik 265db2e533
fix non ascii sources (#1275)
Master does not build because of non-ascii sources, merging without waiting for CI.
2020-11-01 11:14:01 +01:00
Paul Dreik 2b2abf4895 reamalgamate 2020-11-01 07:03:17 +01:00
Paul Dreik f93fb21c95
optionally disable deprecated apis (#1271)
Introduce cmake option SIMDJSON_DISABLE_DEPRECATED_API (default Off)
which turns off deprecated simdjson api functions by setting the macro
 SIMDJSON_DISABLE_DEPRECATED_API.

For non-cmake users, users will have to set SIMDJSON_DISABLE_DEPRECATED_API
by some other means to disable the api.

Closes #1264
2020-11-01 06:38:52 +01:00
Paul Dreik b7fe764e6c
fuzz more of generic/numberparsing.h (#1272)
make the ondemand fuzzer use more of the api
2020-11-01 06:29:16 +01:00
Paul Dreik 500e4c3572
fuzz with the intended clang version (#1267)
This builds the CI fuzzers with the intended clang version. It also allows users to set the clang version locally,
in case they need to.

It also switches the CI fuzzers to use an optimized sanitizer build, to do something oss-fuzz doesn't and get more done in the short time the CI fuzzer runs.
2020-10-31 08:22:49 +01:00
friendlyanon 55281c01fb
Fix unintended Drone CI break (#1266)
* Use 'set -e' as it is portable

* Install git on the i386 images

* Remove CMake PPA from arm64 images

CMake's PPA doesn't actually distribute arm64 binary packages. See
related issues:
https://gitlab.kitware.com/cmake/cmake/-/issues/17923
https://gitlab.kitware.com/cmake/cmake/-/issues/20122

* Use Debian Buster instead of Ubuntu Bionic Beaver

Buster has all the packages in apt that is needed, which is convenient

* Install Clang 6.0 explicitly

Make sure that what gets installed is clang 6.0 as the step name
suggests.
https://packages.debian.org/buster/clang-6.0

* Install CMake from buster-backports

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-10-30 22:47:40 -04:00
Daniel Lemire a8bf10ea5a Minor patch. 2020-10-30 14:51:50 -04:00
friendlyanon 9f60093b41
Update CMake to at least 3.16 in Drone CI (#1261)
* Add script for CMake PPA

* Call the CMake PPA script in Drone CI

"apt-get update -qq" can be omitted, as that command is already called
by the script to pull in necessary packages for the CMake GPG keys.

* Remove sudo calls in the CMake PPA script

This script is intended to be run in Docker images, where the default
user is already root.

* Use echo instead of printf

* Use /etc/os-release instead of lsb_release

lsd_release could be installed, but os-release is just more convenient
to grab the version code from at this point.

* On Debian images grab CMake from buster-backports

It's not wise to mix Ubuntu PPAs with Debian and buster-backports has
CMake 3.16, which is recent enough for our purposes.

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-10-30 13:55:35 -04:00
Daniel Lemire 562d4f4f58 Fixing a minor logical error. 2020-10-29 16:42:50 -04:00
Paul Dreik ac87437588
fuzz the on demand api (#1220) 2020-10-29 19:14:44 +01:00
Daniel Lemire b1444b4dfb
Mostly tiny changes, with one optimization to fallback for number parsing. (#1265)
* Mostly tiny changes, with one optimization to fallback for number parsing.

* Missed an update.
2020-10-29 11:18:11 -04:00
friendlyanon ed21875083
Remove old CMake version (#1259)
The images used by github already come with a sufficiently recent CMake
version instead of a very outdated one. See:
https://github.com/actions/virtual-environments

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-10-28 09:44:13 -04:00
Daniel Lemire 0249b5b311
Update CONTRIBUTORS 2020-10-27 18:44:28 -04:00
Danila Kutenin f46a0f64f2
PPC64 support (#1254)
* Initial PPC64 support

* Add travis CI

* Fix outdated cmake version for travis

* Fix indendtation

* Try another workaround for outdated cmake in travis

* Try beta cmake

* Add dash before beta

* Use builtin snaps

* Use cmake as rocksdb

* Test cmake on bionic

* Remove unnecessary things from travis

* Remove unnecessary things from travis

* Another try of compiler install

* Add all major compilers

* Add all major compilers

* Add all major compilers

* Tweak travis a bit

* Typo

* More robust travis

* Typos typos typos

* Add fewer compilers, add non specific build for clang and gcc, should be the final config

* CMAKE_FLAGS is in incorrect place

* Remove default implementation

* Limit build thread number

* Fall back prefix_xor to a usual implementation, no performance boost is noticed

* Test for power9 as it is the main architecture for OpenPOWER right now

* Add to documentation to build with power9 as the implementation is compatible but compiler optimizations is not

* Replace ARM with PPC in the comment
2020-10-27 18:43:39 -04:00
Jonathan Wakely 1fd0447dbb
Remove repeated words (#1252) 2020-10-26 20:41:01 -04:00
Jeremy Ong 9856201f5c
Disable exceptions when compiling with MSVC (#1256)
Projects that link simdjson from MSVC with exceptions off will
include simdjson headers which transitively include STL headers.
The MSVC STL stipulates that _HAS_EXCEPTIONS=0 be defined or code
requiring exceptions will be enabled. This change adds a new job
to the appveyor build matrix to verify the build and tests with
exceptions disabled, and disables exceptions at the compiler level
when SIMDJSON_EXCEPTIONS is specified to OFF.
2020-10-26 14:11:25 -04:00
Daniel Lemire a75c07065f
Fix for issue 1246. We document the relationship between parser instances and elements (#1250)
* Fix for issue 1246.

* Adopting John's wording.
2020-10-26 08:40:45 -04:00
Daniel Lemire 6a86ef5a7d Issue release 2020-10-23 09:32:25 -04:00
Daniel Lemire 500e5d6759
Update README.md 2020-10-23 09:21:16 -04:00
Daniel Lemire e3f2c9f29a
Update README.md 2020-10-23 09:18:47 -04:00
Daniel Lemire 3458e6248b
Update README.md 2020-10-23 09:15:04 -04:00
Daniel Lemire 14039d05a9
Adding a new benchmark for ondemand: distinct user id (#1239)
* Adding a distinct user id benchmark

* reenabling everything

* Removing an unnecessary "value()".

* Better tests of the examples and some fixes.

* Guarding exception code.
2020-10-23 08:47:01 -04:00
Daniel Lemire c592da4937
Adds yyjson to our internal benchmarks. (#1244) 2020-10-21 16:23:20 -04:00
Daniel Lemire 61fb4244f2
There are a couple of extra value() that should not be there (#1242) 2020-10-21 12:37:22 -04:00
Daniel Lemire a503e022d8
Update README.md 2020-10-21 11:43:36 -04:00
Daniel Lemire 0942dc0764
This fixes a typo and makes the types more explicit (#1241) 2020-10-20 17:41:37 -04:00
Daniel Lemire 0d6919dd99
Reenable the on-demand tests and allows us to convert a raw string into a C++ string. (#1232)
* Reenable the on-demand tests and allows us to convert a raw string into a C++ string.

* Fixing a 1-byte buffer overrun.

* More documentation.

* Adding more tests.

* Enabling the new tests

* Committing a nicer example.

* Not yet happy but this should fix our failures.

* Duh.

* Ok. Making it easier to get string_view instances from field instances.

* It is a struct.

* Trying to satisfy VS.

* Adopting John's name.
2020-10-19 20:22:24 -04:00
Daniel Lemire 3e8e797bc2 Typo. 2020-10-19 17:30:52 -04:00
Daniel Lemire 0a907ec694
Tweaking further the documentation. (#1237)
* Tweaking further the documentation.

* More details.

* Another sentence.

* Saving.

* Tweaking more
2020-10-19 16:51:04 -04:00
Paul Dreik f1b4a54991
add fuzz element (#1204)
* add definitions for is_number and tie (by lemire)
* add fuzzer for element
* update fuzz documentation
* fix UB in creating an empty padded string
* don't bother null terminating padded_string, it is done by the std::memset already
*  refactor fuzz data splitting into a separate class
2020-10-17 05:48:50 +02:00
Paul Dreik 58e7106df1
remove unused function parse_unsigned 2020-10-16 22:17:11 +02:00
Paul Dreik 7bf391c54a
fix potential use of uninitialized value warning, avoid casting away const
This fixes a "potentially use of uninitialized value" warning, as well as a cstyle cast to non-const.
2020-10-16 22:14:42 +02:00
Daniel Lemire 07a6e098c8
This would allow users to find out what builtin is. (#1227)
* This would allow users to find out what builtin is.

* Trying another approach.

* Added instructions.

* Cleaning up the printout.

* Let us be less invasive.

* Adding a comment.
2020-10-15 21:58:42 -04:00
Daniel Lemire e4897d6b54
We have hardcoded 32 (#1236) 2020-10-15 21:57:10 -04:00