Commit Graph

2299 Commits

Author SHA1 Message Date
Paul Dreik d3f0e2afb3
[no ci] remove references to bintray (#1702)
* download fuzz corpus from www.pauldreik.se

* remove reference to bintray
2021-08-19 08:39:07 -04:00
Daniel Lemire eac3832ec0
[no ci] removing space 2021-08-17 08:35:21 -04:00
Daniel Lemire db19c7675f
[no ci] tweaking documentation 2021-08-15 10:32:11 -04:00
strager d036fdf919
Reduce #include bloat (<iostream>) (#1697)
Including <iostream> has two problems:

* Compile times are worse because of over-inclusion
* Binary sizes are worse when statically linking libstdc++ because
  iostreams cannot be dead-code-stripped

simdjson only needs std::ostream. Include the header declaring only what
we need (<ostream>), omitting stuff we don't need (std::cout and its
initialization, for example).

This commit should not change behavior, but it might break users who
assume that including <simdjson/simdjson.h> will make std::cout
available (such as many of simdjson's own files).
2021-08-13 11:24:36 -04:00
Daniel Lemire fc0934755a [no ci] Adding new "include <iostream>" lines in the documentation following the recent removal of iostream from our headers. 2021-08-13 11:23:16 -04:00
Daniel Lemire de4deb8c4e
Makes it possible to cast a document to a value. (#1690)
* Makes it possible to cast a document to a value.
2021-08-11 20:02:30 -04:00
Daniel Lemire ba46616cbc
Small test for document_reference usage. (#1694) 2021-08-10 21:08:59 -04:00
Daniel Lemire a552638373 removing trailing spaces 2021-08-10 15:12:05 -04:00
Daniel Lemire e30123d58f
[no ci] 2021-08-10 15:02:51 -04:00
Daniel Lemire b7dee3e9c8
[no ci] remark 2021-08-09 13:04:20 -04:00
Daniel Lemire 717d0c24a6
[no ci] tuning documentation 2021-08-09 12:09:40 -04:00
Daniel Lemire 40813752f4
This will update the amalgamate_demo.cpp file to use On Demand. (#1689)
* This will update the amalgamate_demo.cpp file to use On Demand.

* Making the demo exceptionless.
2021-08-07 12:43:40 -04:00
Daniel Lemire 19902abaf8
Guarding first/second access. (#1688)
* Guarding first/second access.

* Correcting our own usage.

* Adding more documentation.
2021-08-06 20:25:05 -04:00
Daniel Lemire 06643fc9f5
Additional tests and document tuning (#1684)
* Additional example.

* Adds more tests.

* Actually using the variable.
2021-08-02 16:35:02 -04:00
Daniel Lemire 0fa68d8930
Fixing noexcept on operator << with simdjson_result. (#1678)
* Additional tests.

* Finishing touch.

* Extending to IO.
2021-07-31 17:54:27 -04:00
Daniel Lemire cc98358453
Adding error handing examples to the documentation (#1679)
* Adding error handing examples.

* Guarding the exception-throwing test.
2021-07-31 14:31:48 -04:00
Dirk Stolle 5f0b35b2c5
update string_view lite to version 1.6.0 (#1677)
This is the header as seen in the current master branch,
commit d27d7b5081406a35b41cb16b321be8833b4cd811, on
<https://github.com/martinmoene/string-view-lite>.
2021-07-31 11:19:51 -04:00
Daniel Lemire bd0f5e9322
[no ci] details 2021-07-30 18:20:37 -04:00
Daniel Lemire cb0e4b23c7
[no ci] clearing headings 2021-07-30 18:12:06 -04:00
Daniel Lemire d83e69d977
Fix an issue with truncated-byte function. (#1674) 2021-07-30 13:12:42 -04:00
Daniel Lemire f657516a7e Pruning some white spaces. 2021-07-27 13:16:18 -04:00
Daniel Lemire 8d29c5f931
[no ci] README update 2021-07-27 11:10:56 -04:00
Daniel Lemire eeac91579c
[no ci] additional examples 2021-07-27 11:08:24 -04:00
Daniel Lemire e5a4b8b81b
[no ci] minor indentation issue 2021-07-27 11:05:15 -04:00
Daniel Lemire 7b8f775893
Update basics.md 2021-07-27 11:04:01 -04:00
Daniel Lemire c6ef2105ab Minor tweak. 2021-07-27 10:56:05 -04:00
Daniel Lemire e681234e45
[no ci] update to power usage section 2021-07-27 10:30:19 -04:00
Daniel Lemire eb93b98d6a
verify and fix issue 1668 (#1673)
* Adding test.

* Verifies and fix issue 1668. This commit updates the previous behavior of the
On Demand stream support by return a value type (document_reference) instead
of a reference to a document. This allows us to bridge with the usually simdjson
error system, with its simdjson_result types.

* Minor reformat.

* Adds a test with initial tests passing.

* Adding an example.
2021-07-27 08:51:07 -04:00
Nicolas Boyer 7d887fdc1e
Parse numbers inside strings (#1667)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Naive implementation for doubles in string.

* Add double from string in atom doc.

* Simplification (removed all *_from_string())

* Add int and uint parsing in string.

* Make duplicates instead.

* Make tests exceptionless.

* Add missing declarations.

* Add more tests (errors, JSON pointer).

* Add crypto json tests.

* Update doc.

* Update doc after review.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-27 08:50:44 -04:00
Daniel Lemire 9d405a5df4
[skip ci] made requirements more precise 2021-07-26 08:47:21 -04:00
Daniel Lemire 18c003e8bf
Adding CI for old LLVM in GitHub Actions (#1672)
* Adding CI for old LLVM

* Simplifying.
2021-07-26 08:37:29 -04:00
Daniel Lemire ec4316aa84
Adding macos testing. (#1670) 2021-07-25 12:25:31 -04:00
Daniel Lemire b79261eebc
This cleans a bit the current code, especially with respect to EOF guards. (#1669)
* Upgrading the GitHub Actions.

* Upgrading appveyor

* Upgrading circle ci.

* Cleaning.
2021-07-25 10:36:22 -04:00
Daniel Lemire 47a62db559
Isolated jkeiser fix for issue 1632: make it so that INCORRECT_TYPE is a recoverable condition in On Demand (#1663) 2021-07-23 11:32:26 -04:00
Daniel Lemire 01645cbd69
Update bug_report.md 2021-07-22 13:22:49 -04:00
Daniel Lemire 3dbecab494 Trimming the documentation a bit. 2021-07-20 15:05:39 -04:00
Nicolas Boyer 5c590b8434
Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire 2dac3705d2
renames 'to_string' to 'to_json_string' and makes it ridiculously fast (#1642)
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.

* Moving to a fast string_view model

* Making it exception-safe.

* Tweaking.

* Workaround for exceptions.

* more robust to_json_string (#1651)

* WIP.

* Fuzzing timeout  (bug fix) (#1650)

* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

* WIP.

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

Co-authored-by: Paul Dreik <github@pauldreik.se>

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-19 10:24:36 -04:00
Daniel Lemire 774999ee95 Adds some benchmarks for the minifier. 2021-07-16 11:54:55 -04:00
Daniel Lemire c906f864d7
Update basics.md 2021-07-16 09:33:43 -04:00
Daniel Lemire 68e3c4f984
Update basics.md 2021-07-16 09:27:34 -04:00
Daniel Lemire b08818fab9
Update basics.md 2021-07-15 12:31:53 -04:00
Daniel Lemire 33f73b577c
Another attempt at producing problems with threads (more tests) (#1655)
* Another attempt at producing problems with threads.

* Fixing code

* Trying to please visual studio
2021-07-13 17:30:29 -04:00
Daniel Lemire 30422cfdd3
I am getting tired of conflicts in settings.json. (#1652) 2021-07-13 09:28:49 -04:00
Daniel Lemire b085b56e32
This solves a minor issue with our legacy benchmark tools. (#1653)
* This solves a minor issue with our legacy benchmark tools.

* Slightly better code.

* Removing bad typo.
2021-07-13 09:18:58 -04:00
Daniel Lemire ea3d4e7ce5
Fuzzing timeout (bug fix) (#1650)
* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-06 14:36:38 -04:00
Daniel Lemire bea1483cde
Fixing minor issue with document stream (DOM). (#1648)
* Fixing minor issue with document stream (DOM).

* Porting over the fix.
2021-07-05 17:40:04 -04:00
Paul Dreik 90efd79055
Get fuzzing working again (#1646)
* upload corpus to https://www.pauldreik.se/ from the x64 github action job (keep the github action cache)
* drop the github action cache (which was not working anyway) for power fuzzer and download the fuzz corpus from https://www.pauldreik.se/ instead
* resurrect arm64 fuzzing on drone CI, downloading the fuzz corpus from https://www.pauldreik.se/
* update the fuzzing documentation
2021-07-05 09:42:57 +02:00
Daniel Lemire 7e646efd0f
This should print out (once) some instructions to interpret the logging traces. (#1637)
* This should print out (once) some instructions to interpret the logging traces.

* More details.
2021-06-26 11:38:38 -04:00
Nicolas Boyer eb849662c0
Update basic.md to document JSON pointer for On Demand. (#1618)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-06-26 11:38:17 -04:00