Commit Graph

195 Commits

Author SHA1 Message Date
Zacharya 25becd2d43
feat(pubsub): implement pubsub command close #90 (#175)
* feat(pubsub): implement pubsub command

* fix(pubsub): code review

* fix(pubsub): code review

* fix(pubsub): code review
2022-07-04 15:57:00 +03:00
Roman Gershman c8fe7ba28b fix(server): Fix serialization logic when returning an empty array.
1. Fix SendStringArr logic. Consolidate both string_view and string variants to using the same code.
2. Tighten the test framework so that it will test that the parser consumed the whole response.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-28 19:51:25 +03:00
Roman Gershman 448d065647
fix(server): Fix a crash when running "client list" command. (#179)
Also, enhance release script to apply linker optimizations when building the binaries.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-26 17:25:42 +03:00
Roman Gershman 605b1fd217
chore(server): Improve the implementation of SendStringArr. (#178)
1. Make it use vectorized send instead of concatenating everything into a single string.
2. vectorized SendStringArr could fail sending large arrays for lengths higher than 512 (returned EMSGSIZE).
   We improved the implementation so it would send those arrays in chunks of 256 items.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-26 06:12:53 +03:00
Roman Gershman 6b7d2a22df
chore(strings): Simplify Set flow (#164)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-19 23:19:42 +03:00
Roman Gershman da3ae760d5 feat(stream): implement rdb save support for streams
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-18 20:08:09 +03:00
Roman Gershman d1d64eb014 feat(streams): implement rdb load for streams
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-17 10:27:46 +03:00
Roman Gershman 07b92841fe fix(zmalloc): fix memory accounting of redis types
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-17 06:38:07 +03:00
Roman Gershman 5505ad00f0
fix(parser): Raise parser limit for array length from 8K to 64K (#158)
Add indication of parser errors to info stats.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-16 22:01:40 +03:00
Roman Gershman 32f47be034
fix(lua): Fix few lua-related bugs (#157)
1. Correctly parse keys for EVAL command.
2. Support PUBLISH within lua.
3. Remove spurious failure in debug-mode with the increased verbosity level.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-16 13:41:09 +03:00
Roman Gershman a1c3d8e33d chore(server): Usability improvements
1. Relax processor requirements for Dragonfly. Fixes #124.
2. Introduce cmake option DF_USE_SSL to allow building DF without SSL support. Fixes #128.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-15 19:20:03 +03:00
Roman Gershman 4ec2538204
fix(transaction): Fix rare deadlock case - fixes #150. (#155)
In rare cases a scheduled transaction is not scheduled correctly and we need
to remove it from the tx-queue in order to re-schedule. When we pull it from tx-queue
and it has been located at the head, we must poll-execute the next txs in the queue.

1. Fix the bug.
2. Improve verbosity loggings to make it easier to follow up on tx flow in release mode.
3. Introduce /txz handler that shows currently pending transactions in the queue.
4. Fix a typo in xdel() function.
5. Add a py-script that reproduces the bug.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-15 16:53:27 +03:00
Roman Gershman d37a0bbc29 fix(zset): Properly convert non-c strings to floats.
The old code relied on c strtod function that expected some sort of delimiter
at the end. Coincidently my unit-testing code always passed proper c strings so
strod worked as expected. EVAL passes slices to non-c string and this is why the bug was
discovered via eval call. Fixes #148.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-15 00:43:39 +03:00
Roman Gershman ebd0d89dba chore(zset): add additional logs for parse errors 2022-06-14 22:45:01 +03:00
Roman Gershman c23dc1c94c fix(lua): Avoid infinite loop and fix sha compatibility with redis.
Fixes #146 and fixes #147.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-13 23:32:19 +03:00
Acheev Bhagat 8809713139
feat(server): Add support for CPU to Info family of commands (fixes #18) (#143)
Signed-off-by: Acheev Bhagat <acheevbhagat@hotmail.com>
2022-06-12 21:25:09 +03:00
Roman Gershman d5cd317a13
implement partial streams API (#141)
* feat(stream): implement xrevrange, xrange, xlen, xinfo and xgroup, xdel.
1. add tests that cover xrange,xrevrange and various error states for xadd.
2. Implement 8 stream commands overall. Fixes #59.

* feat(stream): support xadd/maxlen option


Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-12 17:41:01 +03:00
Roman Gershman 15fd451487
fix(parser): Fix wrong parsing of nested arrays. (#140)
Relevant only for client-side parsing. Thus it did not affect production code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-06-12 15:49:31 +03:00
Ansore f7d3f4d640
fix(zset): fixed zrevrange support of WITHSCORES.
Added zrevrange test case zset_family_test.cc.
Fixes #106
2022-06-12 11:21:18 +03:00
Ali-Akber Saifee e82a378354
Reduce scope of HELLO implementation (#129)
* Reduce scope of HELLO implementation

Only accept protover=2 as a valid argument as that is
the only thing that is supported at the moment. For all
other arguments degrade to 'unknown command'

The previous implementation creates issues for clients
expecting the presence of the HELLO command to also signal
the presence of RESP3 (as technically the command appeared in
redis at the same time as support for RESP3).

It also did not raise any errors when unsupported (or invalid)
arguments were passed to the command (such as AUTH, SETNAME)

* Fix error in test

* Remove unnecessary context in assertion

* Fix construction of unknown command error string

* Fix casing of error string
2022-06-09 21:53:30 +03:00
Roman Gershman a9c0fa8ea4 Implement basic XADD, XRANGE, XLEN commands.
Covers most funvtionality for #59.
2022-06-08 22:04:49 +03:00
Roman Gershman 819b2f0716 Introduce the initial skeleton code for stream routines
1. Take redis stream code from Redis 7.0 branch (d2b5a579d).
2. Introduce stream_family class and test.
2022-06-08 20:28:52 +03:00
Roman Gershman e646209da4 Stability fixes to PUBSUB code 2022-06-08 08:46:48 +03:00
Ali-Akber Saifee 746c04fb9d
Use sum instead of individual counts for channels/patterns (#119) 2022-06-08 05:26:02 +03:00
Ali-Akber Saifee 02bc4425ae
Support Pubsub (P)UNSUBSCRIBE commands without arguments (#117)
* Add tests demonstrating expectations of (p)unsubscribe

- When there are subscriptions the return should contain the count of
  remaining channels or patterns subscribed to
- When there are no subscriptions and no arguments are provided
  a single response containing null for channel / pattern and count = 0
  should be returned
- When there are subscriptions and no arguments are provided
  a response per unsubscribed channel or pattern should be returned

* Update arity of (p)unsubscribe commands

* Lint test

* Handle (p)unsubscribe without arguments

Side effects:
- Extract response construction for (p)sububscribe, (p)unsubscribe
  into private method
- Use UnsubscribeAll, PunsubscribeAll in cleanup

* Fix linting errors

* Replace ternary with explicit if/else

* Simplify setup for (p)unsubscribe tests
2022-06-08 05:09:10 +03:00
Ali-Akber Saifee 2caa2b0df2
Update response to HELLO command (#102)
* Update response to HELLO command

Update response from HELLO command to match the api of redis.

Notes:
- For any protocol version other than 2 an error matching what redis
  returns for !2,3 is returned

* Add test for redis HELLO command

* Extract version building to GetVersion function

* Update contributors file

* Add HELLO command to README readiness matrix

* Revert "Add HELLO command to README readiness matrix"

This reverts commit 069f590ad0.

* Add HELLO command to api_status document
2022-06-06 18:17:40 +03:00
Zacharya 5ff2a43a1f
Add Promehtues metrics for Grafana close #61 (#85)
* Add Promehtues metrics for Grafana close #61

* remove debug code

* Add doc and Grafana dashboard

* PR comments fixes

* Remove macros
2022-06-05 21:47:56 +03:00
Ryan Russell 4a8644559a
Fix Various Naming Conventions around `Subscriber` (#87)
* Fix `SubscribeMap`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* Fix `subscriber_arr`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* Fix `CopySubscribers`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* Fix `SubscriberMessagesLen`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* Fix remaining `subscribe` variants

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* Add Ryan Russell to Contributors

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-06-04 05:07:13 +03:00
Roman Gershman 89baa5bfa0
Fix bugs related to concurrency when saving multiple databases under … (#82)
Fix bugs related to concurrency when saving multiple databases under write load
2022-06-03 00:52:38 +03:00
Roman Gershman ec9754150f Implement PSUBSCRIBE/PUNSUBSCRIBE commands.
Add minimal tests.
2022-06-02 22:45:56 +03:00
Roman Gershman d3f598eb88 More printings in rdb_test in order to catch the snapshotting bug 2022-06-02 12:32:10 +03:00
Ryan Russell a049128ab6
Improving Readability r3 (#75)
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-06-01 19:31:36 +03:00
Roman Gershman d5f2c23922
Fixes #66. (#70)
* Fixes #66.

1. Introduce `hz` flag to control the frequency of periodically running tasks.
2. Update helio dependency.

* Fix CI caching for PRs.
2022-06-01 13:19:41 +03:00
Zacharya 1758503e97
Add basic Prometheus support and metrics (#67)
* Add basic Prometheus support and metrics

* Remove human metrics

* Pull AppendMetric outside

* Fix typo
2022-06-01 12:34:37 +03:00
Roman Gershman 398b2b00bf Fix a crash in debug mode when working with huge strings.
Seems like a mimalloc bug.
Opened https://github.com/microsoft/mimalloc/issues/587 for this.
2022-06-01 09:12:02 +03:00
Ryan Russell eae5c08e76
Improve Comment Readability (#69)
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-05-31 23:07:00 +03:00
Roman Gershman f641ba83b9 Add a debug assertion in order to chaise a bug when serialization finishes 2022-05-31 17:56:31 +03:00
Zacharya 4ca9dc71b3
Fix info command to use git version (#49) 2022-05-30 15:17:52 +03:00
Roman Gershman c488baec3c Add a DCHECK to catch a bug 2022-05-30 14:53:42 +03:00
Roman Gershman e239fc68b4 Added glue code to expose various http handlers from dragonfly 2022-05-30 11:53:39 +03:00
Roman Gershman 5af2fe0145 docker-release pipeline now uses release assets for building dockers 2022-05-30 11:24:56 +03:00
Roman Gershman 4d276c7ef0 Integrate version metadata into build. Fixes #46. 2022-05-30 08:49:19 +03:00
Roman Gershman bb7b205bff Switch to using absl flags 2022-05-30 06:45:09 +03:00
Roman Gershman 6dd1552506
Some clean-ups in rdb_save code. Add verbosity printings for CI build (#47) 2022-05-29 20:23:05 +03:00
Roman Gershman ccf051742d Execute commands from the replicated log 2022-05-28 23:20:52 +03:00
Roman Gershman 1490eb5c3c Implement parsing of replication stream from redis 2022-05-28 21:24:07 +03:00
Roman Gershman 114e8bec5d Fixes #41.
1. Found dangling transaction pointers that where left in the watch queue. Fix the state machine there.
2. Improved transaction code a bit, merged duplicated code into RunInShard function, got rid of RunNoop.
3. Improved BPopper::Run flow.
4. Added 'DEBUG WATCH' command. Also 'DEBUG OBJECT' now returns shard id and the lock status of the object.
2022-05-27 12:20:01 +03:00
Roman Gershman 4a2e84b975 In order to support Debian/Bullseye, we know require minimal kernel version - 5.10 2022-05-26 20:04:33 +03:00
Roman Gershman 8bed96d20b Memory and persistence fixes.
1. Fix memory counting stats. Use memory_usable_size(ptr) for counting used memory.
2. Fix crash when there is I/O error when writing snapshot. Now we return error message.
2022-05-23 16:17:13 +03:00
Roman Gershman c087f7c61b Update helio version. Add more gc related stats 2022-05-23 08:52:04 +03:00