simdjson/benchmark/large_amazon_cellphones/simdjson_dom.h

45 lines
1.1 KiB
C
Raw Normal View History

Bringing ndjson(document_stream) to On Demand (#1643) * Update basic.md to document JSON pointer for On Demand. * Add automatic rewind for at_pointer * Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding. * Review * Add test * Add document_stream constructors and iterate_many * Attempt to implement streaming. * Kind of fixed next() for getting next document * Temporary save. * Putting in working order. * Add working doc_index and add function next_document() * Attempt to implement streaming. * Re-anchoring json_iterator after a call to stage 1 * I am convinced it should be a 'while'. * Add source() with test. * Add truncated_bytes(). * Fix casting issues. * Fix old style cast. * Fix privacy issue. * Fix privacy issues. * Again * . * Add more tests. Add error() for iterator class. * Fix source() to not included whitespaces between documents. * Fixing CI. * Fix source() for multiple batches. Add new tests. * Fix batch_start when document has leading spaces. Add new tests for that. * Add new tests. * Temporary save. * Working hacky multithread version. * Small fix in header files. * Correct version (not working). * Adding a move assignment to ondemand::parser. * Fix attempt by changing std::swap. * Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE. * Update doc and readme tests. * Update basics.md * Update readme_examples tests. * Fix exceptions in test. * Partial setup for amazon_cellphones. * Benchmark with vectors. * Benchmark with maps * With vectors again. * Fix for weighted average. * DOM benchmark. * Fix typos. Add On Demand benchmark. * Add large amazon_cellphones benchmark for DOM * Add benchmark for On demand. * Fix broken read_me test. * Add parser.threaded to enable/disable thread usage. Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-21 02:17:23 +08:00
#pragma once
#if SIMDJSON_EXCEPTIONS
#include "large_amazon_cellphones.h"
#include <algorithm>
namespace large_amazon_cellphones {
using namespace simdjson;
struct simdjson_dom {
using StringType = std::string;
dom::parser parser{};
bool run(simdjson::padded_string &json, std::map<StringType, brand> &result) {
auto stream = parser.parse_many(json);
auto i = stream.begin();
++i; // Skip first line
for (;i != stream.end(); ++i) {
auto doc = *i;
StringType copy(std::string_view(doc.at(1)));
auto x = result.find(copy);
if (x == result.end()) { // If key not found, add new key
result.emplace(copy, large_amazon_cellphones::brand{
double(doc.at(5)) * uint64_t(doc.at(7)),
uint64_t(doc.at(7))
});
} else { // Otherwise, update key data
x->second.cumulative_rating += double(doc.at(5)) * uint64_t(doc.at(7));
x->second.reviews_count += uint64_t(doc.at(7));
}
}
return true;
}
};
BENCHMARK_TEMPLATE(large_amazon_cellphones, simdjson_dom)->UseManualTime();
} // namespace large_amazon_cellphones
#endif // SIMDJSON_EXCEPTIONS