amalgamation.sh shouldn't change contents of src/simdjson.cpp by forcing dmalloc.h that didn't exist in non-amalgamated version and shouldn't change order of includes by placing simdjson.h at the top
fixes#739
* Added bitexact implementations of _BitScanForward64 and _BitScanReverse64 for VS2019 32-bit builds
* Added bitexact implementations of _umul128 for VS2019 x86, arm, arm64 builds
* Implement mul_overflow for VS2019 arm64 builds
+ implement mul_overflow using __umulh (msvc/clang results: https://godbolt.org/z/smRwA7)
* Added Win32 for VS2019 to .appveyor.yml
* Update amalgamated headers (fix x86 builds with VS2019)
* This is an implementation of "size()" for arrays and objects.
* Adding benchmark
* Adding a size() remark in the documentation.
* Extending size() to result types.
* Fallback should use our scalar code.
* parse should have a nicer error message.
* Making it so that "minify" can use different architectures.
* Let us change the minifier competition so that it tests all implementations.
* Documenting the untaken optimization opportunity.
Co-authored-by: John Keiser <john@johnkeiser.com>
* Make architecture implementations virtual functions
- Easier to add new architectures (add implementation to implementation.cpp)
- Easier to add new algorithms / functions to architecture selection
(add to implementation.h, implement)
- Automatically select best implementation in static initialization
- Allow user to explicitly select implementation with a string (i.e.
parameter)
- Allow user to inspect current implementation name/description
- Allow user to list available implementations
- Eliminate architecture enum and architecture-based templating
- Add noexcept in non-inline functions
* Move implementation static methods to their own classes
* Detect best supported implementation on first use
* available_implementationsI() -> available_implementations
This creates a "document" class with only user-facing document state (no parser internals).
- document: user-facing document state
- document::iterator: iterator (equivalent of ParsedJsonIterator)
- document::parser: parser state plus a "docked" document we parse into (equivalent of ParsedJson)
Usage:
```c++
auto doc = simdjson::document::parse(buf, len); // less efficient but simplest
```
```c++
simdjson::document::parser parser; // reusable parser
parser.allocate_capacity(len);
simdjson::document* doc = parser.parse(buf, len); // pointer to doc inside parser
doc = parser.parse(buf2, len); // reuses all buffers and overwrites doc; more efficient
```
* Fix issue472: make JsonStream a template.
* Adding missing include.
* Tweaking headers and some minor formatting.
* Removing file from aggregation.
* Moving jsoncharutils
* Adding new header.
* Trying another header.
* Let us try to route around Visual Studio's nonesense.
* Fix for issue467
* Updating single-header
* Let us make it so that JsonStream is constructed from a padded_string which will avoid dangerous overruns.
* Fixing parse_stream
* Updating documentation.
* dirent portable latest version
* improved
std::string argument passed by const reference
ctor added with std::string_view argument
`allocate_padded_buffer()` moved here with **optional** check on `length < 1`
* allocate_padded_buffer moved to padded_string.h
It's only a coincidence that it works in current uses: it doesn't do
what the name says. Particularly, if the high bit is 1 it will yield
0 even if the lower 4 bits would yield something else.
Only the simdjson library should optionally depend on threads,
the executables that link to simdjson will get the dependency
indirectly.
* add option for controlling threads (default is on)
* add CI testing with threading on/off for msvc, gcc and clang
* fix an unrelated copy paste comment error in the cirlce ci build conf
* JsonStream threaded prototype
* JsonStream Threaded version working. Still supporting non-threaded version.
* Fix where invalid files would enter infinite loop.
* SingleHeader update
* I will remove -pthread in cmake for now.
* Attempt at resolving the -pthread issue
* rough prototype working. Needs more test and fine tuning.
* prototype working on large files.
* prototype working on large files.
* Adding benchmarks
* jsonstream API adjustment
* type
* minor fixes and cleaning.
* minor fixes and cleaning.
* removing warnings
* removing some copies
* runtime dispatch error fix
* makefile linking src/jsonstream.cpp
* fixing arm stage 1 headers
* fixing stage 2 headers
* fixing stage 1 arm header
* making jsonstream portable
* cleaning imports
* including <algorithms> for windows compiler
* cleaning benchmark imports
* adding jsonstream to amalgamation
* merged main into branch
* bug fix where JsonStream would bug on rare cases.
* Addind a JsonStream Demo to Amalgamation
* Fix for https://github.com/lemire/simdjson/issues/345
* Follow up test and fix for https://github.com/lemire/simdjson/issues/345 (#347)
* Final (?) fix for https://github.com/lemire/simdjson/issues/345
* Verbose basictest
* Being more forgiving of powers of ten.
* Let us zero the tail end.
* add basic fuzzers (#348)
* add basic fuzzing using libFuzzer
* let cmake respect cflags, otherwise the fuzzer flags go unnoticed
also, integrates badly with oss-fuzz
* add new fuzzer for minification, simplify the old one
* add fuzzer for the dump example
* clang format
* adding Paul Dreik
* rough prototype working. Needs more test and fine tuning.
* prototype working on large files.
* prototype working on large files.
* Adding benchmarks
* jsonstream API adjustment
* type
* minor fixes and cleaning.
* Fixing issue 351 (#352)
* Fixing issues 351 and 353
* minor fixes and cleaning.
* removing warnings
* removing some copies
* Fix ARM compile errors on g++ 7.4 (#354)
* Fix ARM compilation errors
* Update singleheader
* runtime dispatch error fix
* makefile linking src/jsonstream.cpp
* fixing arm stage 1 headers
* fixing stage 2 headers
* fixing stage 1 arm header
* fix integer overflow in subnormal_power10 (#355)
detected by oss-fuzz
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18714
* Adding new test file, following https://github.com/lemire/simdjson/pull/355
* making jsonstream portable
* cleaning imports
* including <algorithms> for windows compiler
* cleaning benchmark imports
* adding jsonstream to amalgamation
* merged main into branch
* bug fix where JsonStream would bug on rare cases.
* Addind a JsonStream Demo to Amalgamation
* merging main
* rough prototype working. Needs more test and fine tuning.
* prototype working on large files.
* prototype working on large files.
* Adding benchmarks
* jsonstream API adjustment
* minor fixes and cleaning.
* minor fixes and cleaning.
* removing warnings
* removing some copies
* runtime dispatch error fix
* makefile linking src/jsonstream.cpp
* fixing arm stage 1 headers
* fixing stage 2 headers
* fixing stage 1 arm header
* making jsonstream portable
* cleaning imports
* including <algorithms> for windows compiler
* cleaning benchmark imports
* adding jsonstream to amalgamation
* bug fix where JsonStream would bug on rare cases.
* Addind a JsonStream Demo to Amalgamation
* rough prototype working. Needs more test and fine tuning.
* minor fixes and cleaning.
* adding jsonstream to amalgamation
* merged main into branch
* Addind a JsonStream Demo to Amalgamation
* merging main
* merging main
* make file fix
- Removes templating from simd_input, utf8_checker, and parse_string
- Make drone gcc run a lot faster
- Make drone clang run a little faster (NOTE:
https://hub.docker.com/r/silkeh/clang helps even more, but I wasn't sure
whether we wanted to trust that)
- Make drone arm run in parallel to get results quicker
* Allow -f
* Support parse -s (force sse)
* Simplify flatten_bits
- Add directly to base instead of storing variable
- Don't modify base_ptr after beginning of function
- Eliminate base variable and increment base_ptr instead
* De-unroll the flatten_bits loops
* Decrease dependencies in stage 1
- Do all finalize_structurals work before computing the quote mask; mask
out the quote mask later
- Join find_whitespace_and_structurals and finalize_structurals into
single find_structurals call, to reduce variable leakage
- Rework pseudo_pred algorithm to refer to "primitive" for clarity and some
dependency reduction
- Rename quote_mask to in_string to describe what we're trying to
achieve ("mask" could mean many things)
- Break up find_quote_mask_and_bits into find_quote_mask and
invalid_string_bytes to reduce data leakage (i.e. don't expose quote bits
or odd_ends at all to find_structural_bits)
- Genericize overflow methods "follows" and "follows_odd_sequence" for
descriptiveness and possible lifting into a generic simd parsing library
* Mark branches as likely/unlikely
* Reorder and unroll+interleave stage 1 loop
* Nest the cnt > 16 branch inside cnt > 8
* Use generic each/reduce in simdutf8check
* Remove macros from generic simd_input uses
* Use array instead of members to store simd registers
* Default local checkperf to clone from .
* handle uint64 value in JSON
* Add integer_tests
* Add get_unsigned_integer() on ParsedJson::BasicIterator
* Write 'u' to tape when the value seems unsigned
* Add to handle 'u' element
* Brush up integer_tests.cpp
* Append tests/integer_tests in .gitignore
* Add comments to is_integer and is_unsigned_integer
* Get rid of dynamic allocation in ParsedJson::Iterator.
* Implement copy assignment operator for ParsedJson::Iterator.
* ParsedJson::Iterator is now a template class.
* stage1 compiles without macros
* cleaning
* amalgation is weird but works
* macros are removed from stringparsing
* amalgation fixed
* Huge macros are removed.
* clang-format
* Hiding the runtime dispatch pointer in a source file so it is not an exported symbol
* Disabling hard failure on style check.
* Fixes https://github.com/lemire/simdjson/issues/250
* Attempt 1 - fn targeting
GCC won't work with templates with different targets, need to specialize all the way up the call stack.
* Compiles properly with cmake. Does not with the Makefile.
* Compilation works with Makefile
* instruction_set changes to architecture
* some aesthetic changes
* fix amalgation and tests + aesthetic changes
* This now compiles and passes tests under CLANG
* Minor correction.
* Trying to make it work on ARM
* Adding missing namespace
* Missing bracket
* Fixing minor compilation issues.
* Getting parse to use runtime dispatch
* Fixing amalgamation script.
* Making sure that NEON is supported.
* Fixing typo
* Merging https://github.com/lemire/simdjson/pull/229
* Manual merge of
https://github.com/lemire/simdjson/pull/229
by @jkeiser (second part)
* Trying another way.
* Removing the paral.
* Fixing the make file
* Let us make the practice run long enough.
* Resolved the awful slowness.
* Cleaning the README.md
* With runtime dispatching, we should not need flags anymore.
* Changing isa detection file's name + fixing typos.
* Checks for issue 150. We run through the test files with sanitizers on.
* Fix for issue 150: the remaining issues were an overrun on the depth capacity and an "off-by-1" overrun on tape capacity.
* Improving makefile.
* Safer git submodule command.
* Getting get 'git' on circleci
* Improving portability.
* Revisiting faulty logic regarding same-page overruns.
* Disabling same-page overruns under VS.
* Clarifying the documentation
* Fix for issue 131 + being more explicit regarding memory realloc.
* Fix for issue 137.
* removing "using namespace std" throughout. Fix for 50
* Introducing typed malloc/free.
* Introducing a custom class (padded_string) that solves several minor usability issues.
* Updating amalgamation for testing.
* Minor change to benchmark cmake
* Moved ParsedJson and its Iterator to separate .cpp files
* Uncommented functions, that has nothing to do with this pr
* Removed really_inline comments
* Reinstated some inline functions to restore previous performance
* Re-merged iterator in ParsedJson
* Uncommented some WARN_UNUSED
John Keiser
Daniel Lemire
Pavel P
Furkan Usta
ostri
dbj
Paul Dreik
mswilson
Jeremie Piotte
Juho Lauri
Opemipo
saka1
Vitaly Baranov
ioioioio
Heinz N. Gies
technateNG
Dong Xie
Geoff Langdale
Tyler Kennedy
Georgios Floros
myd7349
Thomas Navennec
greedengineer
Carmot
M. Zhou
Kai Wolf
Wojciech Muła