* This gives the CMake install the necessarily information (and flags) to know
whether we have a Windows DLL and in such cases how to handle the linkage.
* This adds a little test to see if we can compiler with very strict flags.
* Trimming a leftover old-style cast.
* More cleaning.
* A few more pedantic casts.
* first try
* use ubuntu 20.04, do the fuzzing
* new try at power fuzz
* hard code clang version
* setting env variables does not seem to work
* use fuzzer-no-link
* switch to Debian Buster for power fuzz
* use non-sanitizer build for power
* me not like yaml
* fix bad syntax
* add ndjson fuzzer
* reproduce #1310 in the newly added unit test
Had to replace the input, because:
1)
the fuzzer uses the first part of the input to determine
the batch_size to use, so that has to be cut off
2)
the master now protects against low values of batch_size
I also made the test not return early, so the error is triggered.
Forks that would like to contribute via PRs from feature branches
needlessly run CI on those branches on top of the PRs.
This is a waste of resources.
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
* Fix Cirrus CI
CMake is not installed at this point yet.
* Add caching in Circle CI
* Add caching to the MinGW Github workflows
* Fix Circle CI config
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
This builds the CI fuzzers with the intended clang version. It also allows users to set the clang version locally,
in case they need to.
It also switches the CI fuzzers to use an optimized sanitizer build, to do something oss-fuzz doesn't and get more done in the short time the CI fuzzer runs.
The images used by github already come with a sufficiently recent CMake
version instead of a very outdated one. See:
https://github.com/actions/virtual-environments
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
* add definitions for is_number and tie (by lemire)
* add fuzzer for element
* update fuzz documentation
* fix UB in creating an empty padded string
* don't bother null terminating padded_string, it is done by the std::memset already
* refactor fuzz data splitting into a separate class
This refactors the dynamic check of which implementations are supported at runtime.
It also reduces duplicated effort in the CI fuzzing job, the differential fuzzers don't need to run with different values of SIMDJSON_FORCE_IMPLEMENTATION.
There is also a convenience script to run the fuzzers locally, to quickly check that the fuzzers still build, run and no easy to find bugs are there. It should be handy not only when developing the fuzzers, but also when modifying simdjson.
This adds a minifier fuzzer. There is also an utf-8 fuzzer, but it is disabled until #1187 is fixed.
Run all fuzzers bug the utf-8 one in the github CI fuzz.
This adds a fuzzer for at_pointer() which recently had a bug.
The #1142 bug had been found with this fuzzer
Also, it polishes the github action job:
cross pollinate the fuzzer corpora (lets fuzzers reuse results from other fuzzers)
use github action syntax instead of bash checks
only run on push if on master
This adds a fuzzer which parses the same input using all the available implementations (haswell, westmere, fallback on x64).
This should get the otherwise uncovered sourcefiles (mostly fallback) to show up in the fuzz coverage.
For instance, the fallback directory has only one line covered.
As of the 20200909 report, 1866 lines are covered out of 4478.
Also, it will detect if the implementations behave differently:
by making sure they all succeed, or all error
turning the parsed data into text again, should produce equal results
While at it, I corrected some minor things:
clean up building too many variants, run with forced implementation (closes#815 )
always store crashes as artefacts, good in case the fuzzer finds something
return value of the fuzzer function should always be 0
reduce log spam
introduce max size for the seed corpus and the CI fuzzer
The jobs were executed in powershell using the globally installed cmake.
This makes things actually run in a MSYS2 shell.
This also removes the msys/cygwin job because it doesn't build
(it complains about undeclared posix_memalign)
Daniel Lemire
friendlyanon
John Keiser
Paul Dreik
Christoph Reiter