Commit Graph

27 Commits

Author SHA1 Message Date
John Keiser a7fc7d4ffb Switch from get(v,e) to e = get(v) 2020-06-20 17:57:09 -07:00
Daniel Lemire c009e4a57d
Fuzzing should not require loading lots of complicated dependencies (#879)
* I don't think we need google benchmark as part of the fuzzer

* I don't think we should load the "competition" for fuzzing.
2020-05-13 08:30:09 -04:00
Furkan Usta 064eb0b24f CMake: Make simdjson-internal-flags subsume simdjson-flags 2020-05-03 02:48:29 +03:00
Furkan Usta 940c8fcb5e Fix: simdjson-private-flags to simdjson-internal-flags 2020-05-02 05:23:53 +03:00
Furkan Usta 71e0148eb4 CMake: Link simdjson-fuzzer to simdjson-source as before
simdjson target adds extra definitions which MSVC doesn't like
2020-05-02 05:21:57 +03:00
Furkan Usta 293c104cc4 CMake: Separate public and private compilation flags
simdjson-internal-flags for macros and warnings
simdjson-flags for pthread, sanitizer, and libcpp
2020-05-02 04:08:47 +03:00
John Keiser 6ac47734c0
Only build fuzzers when fuzzing. (#822) 2020-04-27 16:02:19 -04:00
John Keiser db314bc381 Make fuzzing work without exceptions 2020-04-21 17:33:47 -07:00
John Keiser 09cf18a646 Add C++11 tests to cmake
- Add simdjson-flags target so callers don't have flags forced on them
2020-04-15 17:26:25 -07:00
Daniel Lemire 6d7c77ddc1
Let us try to check with the exceptions disabled. (#707)
* Tweaking code so that we can run all tests with exceptions off.
* Removing SIMDJSON_DISABLE_EXCEPTIONS
2020-04-15 16:45:36 -04:00
Paul Dreik 92c34f7f38
do not use deprecated apis in the fuzzers (#705)
* move from deprecated interface in fuzz dump raw tape

* update fuzz_dump to the non deprecated replacement

* replace use of deprecated api

* hopefully fix windows build
2020-04-14 07:45:30 +02:00
Paul Dreik 93328c8d6d
improve the fuzzer documentation, refresh links (#697) 2020-04-12 17:49:40 -04:00
Paul Dreik 5710ec13d4
switch to clang 9 in the github action (#695)
The upstream convenience script from llvm does not support installing clang 8 anymore.
2020-04-12 20:02:45 +02:00
Paul Dreik fa637fcecb
Enable the minifier fuzzer (fixes #530) (#694)
This enables the minify fuzzer, which has been disabled because it did not pass the oss-fuzz instrumentation test. Now it does, after changes in simdjson (https://github.com/lemire/simdjson/issues/186).

 * get minify running (api change)
 * disable benchmarks when compiling fuzzers
 * catch exceptions from the minify fuzzer
 * enable repeated corpus creation without recursive inclusion of zip
 * remove leftover comment
2020-04-12 18:47:24 +02:00
John Keiser 54b7291c34 Reference simdjson by name, don't specify include files individually 2020-04-08 14:52:55 -07:00
John Keiser a0bce440a6 Remove document_iterator, document::iterator, ParsedJsonIterator
Keep ParsedJson::Iterator only, without template, in same form as
it was in 0.2
2020-03-25 18:26:51 -07:00
Daniel Lemire 5af0dfb031
Putting back the fuzzer badge and extending the documentation. (#587) 2020-03-24 10:50:27 +01:00
John Keiser 7cf3a7511b Add fallback implementation to CI
- Also add SIMDJSON_IMPLEMENTATION_HASWELL/WESTMERE/ARM64/FALLBACK=1/0 to
enable/disable various implemnentations
2020-03-17 14:59:47 -07:00
John Keiser 1a5d8f1957 Add tests for SIMDJSON_EXCEPTIONS=0, add `tie()` support 2020-03-17 13:54:37 -07:00
John Keiser b3ea8c406e Add simdjson.cpp for unified use (#515) 2020-03-04 10:12:27 -08:00
John Keiser 99667f7c55 Create top level simdjson.h (#515)
- Allows everyone to #include the same way, singleheader or not.
2020-03-04 10:12:27 -08:00
John Keiser 8e7d1a5f09
Separate document state from ParsedJson
This creates a "document" class with only user-facing document state (no parser internals).

- document: user-facing document state
- document::iterator: iterator (equivalent of ParsedJsonIterator)
- document::parser: parser state plus a "docked" document we parse into (equivalent of ParsedJson)

Usage:

```c++
auto doc = simdjson::document::parse(buf, len); // less efficient but simplest
```

```c++
simdjson::document::parser parser; // reusable parser
parser.allocate_capacity(len);
simdjson::document* doc = parser.parse(buf, len); // pointer to doc inside parser
doc = parser.parse(buf2, len); // reuses all buffers and overwrites doc; more efficient
```
2020-02-07 10:02:36 -08:00
Paul Dreik 4da06830f1 add new fuzzers for print_json and dump_raw_tape (#416)
after looking at the coverage report available at
https://storage.googleapis.com/oss-fuzz-coverage/simdjson/reports/20191222/linux/src/simdjson/report.html
2019-12-27 13:42:44 -05:00
Paul Dreik 2704b73399
Add fuzzer badge and improve fuzzer documentation (#367)
* Update Fuzzing.md

* add oss-fuzz badge
2019-11-13 16:57:20 +01:00
Paul Dreik 3fd1c3b64a run short fuzzing and valgrind in github action 2019-11-11 22:17:32 +01:00
Paul Dreik 8ae818e17c add ossfuzz support (#362)
* initial oss-fuzz friendly build

parts taken from libfmt, which I wrote and have the copyright to

* fix build error

* add script for building a corpus zip

see https://google.github.io/oss-fuzz/getting-started/new-project-guide/#seed-corpus

* fix zip command

* drop setting the C++ standard

* disable the minify fuzzer, does not pass oss-fuzz check-build test

* fix integer overflow in subnormal_power10

detected by oss-fuzz

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18714

* invoke the build like oss fuzz does

* document what the scripts are for and how to use them

* add a page about fuzzing
2019-11-08 10:32:43 -05:00
Paul Dreik 9442c9e1f4 add basic fuzzers (#348)
* add basic fuzzing using libFuzzer

* let cmake respect cflags, otherwise the fuzzer flags go unnoticed

also, integrates badly with oss-fuzz

* add new fuzzer for minification, simplify the old one

* add fuzzer for the dump example

* clang format
2019-10-28 14:46:57 -04:00