* Remove CMP0025 policy
This policy is already set to NEW by the minimum required version.
* Use HOMEPAGE_URL in the project call
* Use VERSION in the project call
* Detect if this is the top project
* Port simdjson-user-cmakecache to a CMake script
* Create a developer mode
The SIMDJSON_DEVELOPER_MODE option set to ON will enable targets that
are only useful for developers of simdjson.
* Consolidate root CML commands into logical sections
* Warn about intended use of developer mode
* Prettify the just_ascii test
* Remove redundant CMake variables
* Inline CML contents from include and src
* Raise minimum CMake requirement to 3.14
* Define proper install rules
* Restore thread support variable
* Add BUILD_SHARED_LIBS as a top level only option
* Force developer mode to be on in CI
* Include flags earlier in developer mode
* Set CMAKE_BUILD_TYPE conditionally
CMAKE_BUILD_TYPE is used only by single configuration generators and is
otherwise completely ignored.
* Remove useless static/shared options
simdjson now uses the CMake builtin BUILD_SHARED_LIBS to switch the
built artifact's type.
* Remove unused CMAKE_MODULE_PATH variable
* Refactor implementation switching into a module
* Factor exception option out into a module
* Reformat simdjson-flags.cmake
* Rename simdjson-flags to developer-options
* Accumulate properties into an include module
This is done this way to avoid using utility targets that must be
exported and installed, which could potentially be misused by users of
the library.
* Port impl definitions to props
* Port exception options to props
* Lift normal options to the top
* Port developer options to props
* Remove simdjson-flags from benchmark
* Document the developer mode in HACKING
* Fix include path in installed config file
* Fix formatting of prop commands
* Fix tests that include .cpp files
* Change GCC AVX fixes back to compile options
* Deprecate SIMDJSON_BUILD_STATIC
* Always link fuzz targets to simdjson
* Install CMake from simdjson's debian repo
* Add gnupg for apt-key
* Make sure ASan link flags come first
* Pass CI env variable to cmake invocation
* Install package for apt-add-repository
* Remove return() from flush macro
* Use directory level commands instead of props
* Restore the github repository variable
* Set developer mode unconditionally for checkperf
The CI env variable is only set in the CI and this target is always run
in developer mode.
* Attempt to fix ODR violation in parsing checks
These tests were compiling the simdjson.cpp file again and linking to
the simdjson library target causes ODR violations.
Instead of linking to the target, just inherit its props.
* Move variables before the source dir
* Mark props to be flushed after adding more
* Use props for every command for the library
* Use keyword form for linking libs
* Handle deprecation of SIMDJSON_JUST_LIBRARY
* Handle deprecations in a separate module
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
* This gives the CMake install the necessarily information (and flags) to know
whether we have a Windows DLL and in such cases how to handle the linkage.
* This adds a little test to see if we can compiler with very strict flags.
* Trimming a leftover old-style cast.
* More cleaning.
* A few more pedantic casts.
* first try
* use ubuntu 20.04, do the fuzzing
* new try at power fuzz
* hard code clang version
* setting env variables does not seem to work
* use fuzzer-no-link
* switch to Debian Buster for power fuzz
* use non-sanitizer build for power
* me not like yaml
* fix bad syntax
* add ndjson fuzzer
* reproduce #1310 in the newly added unit test
Had to replace the input, because:
1)
the fuzzer uses the first part of the input to determine
the batch_size to use, so that has to be cut off
2)
the master now protects against low values of batch_size
I also made the test not return early, so the error is triggered.
Forks that would like to contribute via PRs from feature branches
needlessly run CI on those branches on top of the PRs.
This is a waste of resources.
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
* Fix Cirrus CI
CMake is not installed at this point yet.
* Add caching in Circle CI
* Add caching to the MinGW Github workflows
* Fix Circle CI config
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
This builds the CI fuzzers with the intended clang version. It also allows users to set the clang version locally,
in case they need to.
It also switches the CI fuzzers to use an optimized sanitizer build, to do something oss-fuzz doesn't and get more done in the short time the CI fuzzer runs.
The images used by github already come with a sufficiently recent CMake
version instead of a very outdated one. See:
https://github.com/actions/virtual-environments
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
* add definitions for is_number and tie (by lemire)
* add fuzzer for element
* update fuzz documentation
* fix UB in creating an empty padded string
* don't bother null terminating padded_string, it is done by the std::memset already
* refactor fuzz data splitting into a separate class
This refactors the dynamic check of which implementations are supported at runtime.
It also reduces duplicated effort in the CI fuzzing job, the differential fuzzers don't need to run with different values of SIMDJSON_FORCE_IMPLEMENTATION.
There is also a convenience script to run the fuzzers locally, to quickly check that the fuzzers still build, run and no easy to find bugs are there. It should be handy not only when developing the fuzzers, but also when modifying simdjson.
This adds a minifier fuzzer. There is also an utf-8 fuzzer, but it is disabled until #1187 is fixed.
Run all fuzzers bug the utf-8 one in the github CI fuzz.
This adds a fuzzer for at_pointer() which recently had a bug.
The #1142 bug had been found with this fuzzer
Also, it polishes the github action job:
cross pollinate the fuzzer corpora (lets fuzzers reuse results from other fuzzers)
use github action syntax instead of bash checks
only run on push if on master
This adds a fuzzer which parses the same input using all the available implementations (haswell, westmere, fallback on x64).
This should get the otherwise uncovered sourcefiles (mostly fallback) to show up in the fuzz coverage.
For instance, the fallback directory has only one line covered.
As of the 20200909 report, 1866 lines are covered out of 4478.
Also, it will detect if the implementations behave differently:
by making sure they all succeed, or all error
turning the parsed data into text again, should produce equal results
While at it, I corrected some minor things:
clean up building too many variants, run with forced implementation (closes#815 )
always store crashes as artefacts, good in case the fuzzer finds something
return value of the fuzzer function should always be 0
reduce log spam
introduce max size for the seed corpus and the CI fuzzer
Daniel Lemire
friendlyanon
Paul Dreik
John Keiser