simdjson

Commit Graph

Author	SHA1	Message	Date
Daniel Lemire	eb93b98d6a	verify and fix issue 1668 (#1673 ) * Adding test. * Verifies and fix issue 1668. This commit updates the previous behavior of the On Demand stream support by return a value type (document_reference) instead of a reference to a document. This allows us to bridge with the usually simdjson error system, with its simdjson_result types. * Minor reformat. * Adds a test with initial tests passing. * Adding an example.	2021-07-27 08:51:07 -04:00
Nicolas Boyer	5c590b8434	Bringing ndjson(document_stream) to On Demand (#1643 ) * Update basic.md to document JSON pointer for On Demand. * Add automatic rewind for at_pointer * Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding. * Review * Add test * Add document_stream constructors and iterate_many * Attempt to implement streaming. * Kind of fixed next() for getting next document * Temporary save. * Putting in working order. * Add working doc_index and add function next_document() * Attempt to implement streaming. * Re-anchoring json_iterator after a call to stage 1 * I am convinced it should be a 'while'. * Add source() with test. * Add truncated_bytes(). * Fix casting issues. * Fix old style cast. * Fix privacy issue. * Fix privacy issues. * Again * . * Add more tests. Add error() for iterator class. * Fix source() to not included whitespaces between documents. * Fixing CI. * Fix source() for multiple batches. Add new tests. * Fix batch_start when document has leading spaces. Add new tests for that. * Add new tests. * Temporary save. * Working hacky multithread version. * Small fix in header files. * Correct version (not working). * Adding a move assignment to ondemand::parser. * Fix attempt by changing std::swap. * Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE. * Update doc and readme tests. * Update basics.md * Update readme_examples tests. * Fix exceptions in test. * Partial setup for amazon_cellphones. * Benchmark with vectors. * Benchmark with maps * With vectors again. * Fix for weighted average. * DOM benchmark. * Fix typos. Add On Demand benchmark. * Add large amazon_cellphones benchmark for DOM * Add benchmark for On demand. * Fix broken read_me test. * Add parser.threaded to enable/disable thread usage. Co-authored-by: Daniel Lemire <lemire@gmail.com>	2021-07-20 14:17:23 -04:00

Author

SHA1

Message

Date

Daniel Lemire

eb93b98d6a

verify and fix issue 1668 (#1673 )

* Adding test.

* Verifies and fix issue 1668. This commit updates the previous behavior of the
On Demand stream support by return a value type (document_reference) instead
of a reference to a document. This allows us to bridge with the usually simdjson
error system, with its simdjson_result types.

* Minor reformat.

* Adds a test with initial tests passing.

* Adding an example.

2021-07-27 08:51:07 -04:00

Nicolas Boyer

5c590b8434

Bringing ndjson(document_stream) to On Demand (#1643 )

* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>

2021-07-20 14:17:23 -04:00

2 Commits