Commit Graph

14 Commits

Author SHA1 Message Date
Daniel Lemire cd11838e5f
Update parse_many.md 2021-08-20 14:52:43 -04:00
Daniel Lemire 419b7b2ce8
Update parse_many.md 2021-08-20 14:09:54 -04:00
Nicolas Boyer 5c590b8434
Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire eb0ae041e3
Verification and bug fix of issue 1511 (#1602)
* Verification and bug fix.

* Removing comment.

* Removing spaces.

* Guarding exceptions.

* Tweaking the test
2021-06-06 17:55:33 -04:00
Daniel Lemire 8eed8f5155
Document stream: truncate final unfinished document and give access to the number of truncated bytes. (#1534)
* Truncate final unclosed string.

* Adding more precise remarks.

* Better documentation and more robust code.

* ARM + PPC corrections.

* Patching ARM implementation with new stage1_mode parameter.

* Fixed most problems.

* Correcting white spaces and adding a remark.

* This adds the truncated_bytes() method to the stream instances.
2021-04-23 09:24:00 -04:00
Daniel Lemire 9304d88920
Prototype test for issue 1299: using parse_many, find the location of the end of the last document (#1301)
* Prototype test for issue 1299.

* This improves the documentation.

* Removing trailing white spaces.

* Removing trailing spaces

* Trailing.
2020-12-01 15:59:20 -05:00
Daniel Lemire 1f41cc2030
Making it clearer that parse_many is meant for *small* documents. (#1205)
* Making it clearer that parse_many is meant for *small* documents.

* Update parse_many.md
2020-10-06 17:19:34 -04:00
Daniel Lemire 4582a13360 Final steps. 2020-06-26 20:31:24 -04:00
Daniel Lemire 6d3e33d440
Update parse_many.md 2020-06-24 20:41:38 -04:00
Daniel Lemire 3f00e79bcb
Merge branch 'master' into dlemire/better_doxygen_home_page 2020-06-17 16:02:49 -04:00
Daniel Lemire b5ea504ad2 Tweaks doxygen so that we have a better main page. 2020-06-17 11:07:21 -04:00
Daniel Lemire 23fbd9d004 Some tweaks. 2020-06-14 18:28:09 -04:00
Daniel Lemire 74d9b41b7d
Minor fixes to our documentation regarding thread safety. (#683)
* Minor fixes to our documentation regarding thread safety.

* A bit more pessimistic.
2020-04-08 16:41:08 -04:00
John Keiser b5a1017afa Update JsonStream.md -> parse_many to new API 2020-03-30 13:44:03 -07:00