Commit Graph

483 Commits

Author SHA1 Message Date
Guilherme Gervasio ce964f103a
chore(tests): adds support for ioredis integration tests + instructio… (#394)
chore(tests): adds support for ioredis integration tests + instructions in docs

Signed-off-by: Guilherme <gilairmay@gmail.com>
2022-10-17 10:15:53 +03:00
Match_yc 8a2de6a1fc
feat(server): Implement GETEX command #385 (#387)
feat(server): Implement GETEX command

Signed-off-by: matchyc <dawnlight.yc@protonmail.com>
2022-10-16 14:51:30 +03:00
Nemo 15725f49b9
feat(server): implement LMOVE #369 (#391)
Signed-off-by: chenyuxuan.allen <dev.allenchen@gmail.com>

Co-authored-by: chenyuxuan.allen <dev.allenchen@gmail.com>
2022-10-16 10:06:01 +03:00
iko1 fcb95bec6e
feat(server): implement json.arrindex command (#104) (#376)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-10-15 19:36:47 +03:00
Roman Gershman ffc25e6ac2
chore(server): Refactor expire functionality and move it to DbSlice (#388)
This is preparation for https://github.com/dragonflydb/dragonfly/pull/387

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-10-14 13:14:09 +03:00
RedhaL 2e875c81c7
feat(server): Implement ZPOPMIN and ZPOPMAX #358 #359 (#378)
* Implements ZPOPMIN and ZPOPMAX commands
2022-10-13 15:01:59 +03:00
iko1 28706715dc
feat(server): implement json.arrinsert command (#104) (#372)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-10-13 12:20:27 +03:00
Boaz Sade ad014ebb86
fix(server): support for dump command, API update #343 (#371)
feat(server): adding support for the dump command (#343)

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-10-13 11:32:45 +03:00
Elle Y 0e2f918f58
feat(server): Implement LPOS command #368 (#379)
Signed-off-by: Elle Y
2022-10-13 07:26:15 +03:00
SALEH 45a5f30cdd
feat(server): Implement SMISMEMBER command (#377)
* feat(server): Implement SMISMEMBER command (#361)

Signed-off-by: ATM SALEH <saleh.cse08@gmail.com>
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2022-10-13 05:21:27 +03:00
iko1 4833f93366
feat(server): implement json.arrtrim command (#104) (#373)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-10-12 09:53:45 +03:00
Ran Ribenzaft f1e4e6f08c
build(ci): add discord message on docker release (#375)
Signed-off-by: Ran Ribenzaft <ranrib@users.noreply.github.com>
2022-10-12 08:03:43 +03:00
Roman Gershman 3901786316
chore(server): Improve malloc-stats output (#370)
Now it will print deduplicated lines, shortening the table
dramatically. Also it's now easier to import it into excel.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-10-10 15:03:54 +03:00
romange 8f4ea2232c chore(helm-chart): update to v0.9.1 2022-10-07 15:35:55 +00:00
Elle Y ef1f0a9efc
feat(server): Add GETDEL command (#365)
* Added GETDEL command

* fix(server): Implement Del inside OpGet

* fix(server): Swapped result for CHECK macro

* docs(contributors): Added myself to Contributors
2022-10-07 15:39:33 +03:00
iko1 3bf08b803c
feat(server): implement json.arrpop command (#104) (#362)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-10-07 14:33:38 +03:00
Logan Raarup 29fb499d9b
feat(server): Implement ZMSCORE command #357 (#367)
* feat(server): Implement ZMSCORE command (#357)
Signed-off-by: Logan Raarup <logan@logan.dk>
2022-10-07 14:21:06 +03:00
Roman Gershman 608a80411a
fix(epoll): pull newest helio dependency (#366)
This should fix the deadlock bugs that happen in some rare cases when using epoll
proactor.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-10-07 10:35:43 +03:00
Roman Gershman fbdfe5885a
chore(test): Allow running tests with epoll proactor (#364) 2022-10-06 19:09:52 +03:00
Roman Gershman d93eb8a5b5
feat(server): Implement MEMORY MALLOC-STATS command (#363)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-10-06 16:45:06 +03:00
romange 6765271148 chore(helm-chart): update to v0.9.0 2022-10-04 15:59:47 +00:00
Boaz Sade 8f5500af16
feat(server): support for persist command #345 (#354)
fix(server): support for persist command #345

Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-10-04 15:53:18 +03:00
iko1 bfc073b59d
feat(server): implement json.strappend & json.clear commands (#104) (#353)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-10-04 15:52:19 +03:00
Roman Gershman b616b1e1fd
feat(server): support epoll linux api (#351) 2022-10-04 11:11:09 +03:00
Boaz Sade af690668ca
fix(server): hscan command better support #347 (#350)
fix(server): hscan command better support - core review changes #347

Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-10-04 11:10:47 +03:00
Roman Gershman 881e8a1b0c
chore(server): Add more logging in case of check fail 2022-10-04 07:20:45 +03:00
Boaz Sade a1b800e23b
fix(server): better support for the --help option (#349)
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-10-03 15:15:15 +03:00
Roman Gershman d2b216077d
feat(setfamily): Add SADDEX command (#348)
Add SADDEX <key> <seconds> member member ....
Provides expiry semantics for set members with seconds resolution.

Important things to note:
1. The expiry is passive only, so if nobody touches that set then its members are being kept.
2. SCARD provides an upper bound estimation of set size for sets holding the expiring members.
   The reason for this is because SCARD returns a cached size and does not go over all members
   to check whether they expired. For regular sets it's exact, of course.

Fixes #335
2022-10-03 13:15:27 +03:00
Roman Gershman cf779c08a4
chore(server): reduce iouring exposure across the codebase (#342)
Refactor RdbSnapshot and consolidate iouring linux file handling in a single place.
2022-10-02 22:53:52 +03:00
Roman Gershman 21feebe47b
chore(hset): Get rid of helper functions in header file. (#341)
Move them to cc.
2022-10-02 14:33:11 +03:00
Roman Gershman 08779b154b
Add expiry semantics to the set. (#340) 2022-10-01 19:45:19 +03:00
Roman Gershman 44f98de128
chore(denseset): Simplify DenseSet, remove empty links. (#339) 2022-10-01 06:58:37 +03:00
Boaz Sade 7715ee06c6
feat(server): zrange family command support INF as case insensitive #326 (#336)
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-09-29 19:40:33 +03:00
Roman Gershman 2686b9316a
chore(DenseSet): Some code reorder without change in functionality. (#338) 2022-09-29 16:13:55 +03:00
Roman Gershman b74d65dee4
feat(server): Enforce decomitting memory pages upon flushdb call (#337)
This fixes #322 though some memory is still kept by memory allocator.
Tested by running:
1. debug populate 20000000 key 256
2. info memory
3. flushdb
4. info memory

Verified that the rss memory usage decreased from 5.07GB to 1.71GB
2022-09-29 14:01:22 +03:00
Boaz Sade ac90aecde1
feat(server): support for bitop command - unit tests added #213 (#319)
feat(server): bitop command - more code refactor from code review #213

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-09-28 19:32:30 +03:00
iko1 ad46e5e854
feat(server): implement json.objkeys command (#104) (#327)
Signed-off-by: iko1 <me@remotecpp.dev>
2022-09-26 10:44:54 +03:00
Roman Gershman 40c24d2d25
fix(server): dfly_main.cc now compiled with march=core2. (#328)
Fixes #325. Apparently, before the fix AVX instructions were produced
right at the start, thus the server crashed before it
had a chance to install sigill handler.
2022-09-26 09:46:01 +03:00
Roman Gershman 0925829afb feat(server): Introduce transaction clock.
Partially implements #6.
Before, each shard lazily updated its clock used for the expiry evaluation.
Now, the clock value is set during the transaction scheduling phase and is assigned
to each transaction. From now on DbSlice methods use this value when checking whether
the entry is expired via passed DbContext argument.

Also, implemented transactionally consistent TIME command and
verify that time is the same during the transaction. See
https://ably.com/blog/redis-keys-do-not-expire-atomically for motivation.

Still have not implemented any lamport style updates for background processes
(not sure if it's the right way to proceed).
2022-09-25 23:53:08 +03:00
Roman Gershman 347a619b44 chore(redis): remove noisy warnings. 2022-09-25 07:17:35 +03:00
Vladislav ca776119f0
fix(server): Fix ContainerEntry union initialization (#323)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2022-09-23 19:14:48 +03:00
Vladislav 0377481e28
feat(server): SORT command prototype (#311)
feat(server): Implement SORT

Fixes #287

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2022-09-23 07:17:49 +03:00
Roman Gershman 082ac36ac1
fix(bug): dashtable split crashes when moving items from the old segment (#318)
fix(bug): dashtable split crashes when moving items from the old segment.

Segment's Insert function uses an opportunistic heuristic that chose a bucket with smaller items among two available.
This creates a problem with split that goes from a smaller bucket to the biggest one and moves items to the new segment.
In rare case, the heurstic fills up a the next bucket with items that could reside in earlier buckets and then that bucket
does not have space for its own items. Eventually, items that had enough space in the old segment do not find space in the new one

The fix is to adopt a conservative approach during split and try to use a home bucket first. Home bucket is usually a smaller one.
This approach should be optimal because Split starts with smaller buckets first (can be proven iteratively).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-09-20 20:57:41 +03:00
Acheev Bhagat 751bcb9aec
feat(packaging): create deb packaging #249 (#313) 2022-09-20 11:10:19 +03:00
Roman Gershman 0a1b5eb297
chore(server): rdb save can now save into tcp socket directly. (#317)
In more detail, RdbSaver uses AlignedBuffer that writes into io::Sink in chunks of 4KB.
It's great for the direct file I/O, but bad for sockets that receive blocks of 4KB with garbage
at the end. I improved the code around this and actually simplified the logic, so now AlignedBuffer
is just another Sink that is passed into serializer when writing into files. When sending to
sockets a socket sink is passed instead.

Also many other unrelated changes grouped into this pretty big cr.
1. dashtable readability improvements.
2. Move methods from facade::ConnectionContext - into facade::Service,
   make ConnectionContext a dumb object.
3. Optionally allow journal to be memory only (not backed up by a disk)
   by using a ring buffer to store last k entries in each journal slice. Also renamed
   journal_shard into journal_slice because journal has presence in each DF thread and not
   only in its shards.
4. Introduce journal::Entry that will consolidate any store change that happens in the thread.
5. Introduce GetRandomHex utility function.
6. Introduce two hooks: ServerFamily::OnClose that is called when a connection is closed,
   and ServerFamily::BreakOnShutdown that is called when process exits and any background fibers neet to
   break early.
7. Pull some noisy info logs out of rdb_load class.
8. Snapshot class now has the ability to subscribe to journal changes, thus it can include concurrent changes into the snapshot.
   Currently only journal::Op::VAL is supported (it's part of RDB format anyway).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2022-09-20 11:09:03 +03:00
Boaz Sade 1733af4cf6
feat(server): bitcount commands support and unit tests
feat(server): bitcount commands support and unit tests - 

update docs/api_status.md, update license 
Partially implements #213 

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-09-20 11:07:26 +03:00
Roman Gershman de9369f518
chore(server): update license text (#312) 2022-09-19 09:01:10 +03:00
odedponcz f1ee9b4b8d
feat(docs): Update broken link to grafana dashboard (#314)
Signed-off-by: Oded Poncz <oded@poncz.com>
2022-09-19 09:00:23 +03:00
Vladislav 16b6b11cf2
feat(server): Implement MOVE command (#298)
* feat(server): Implement MOVE command

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2022-09-18 20:57:12 +03:00
Boaz Sade 1aef3c1030
feat(server): adding support for EXAT PXAT option at set command (#306)
feat(server): EXAT PXAT support changes according to code review #284

Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
Co-authored-by: Boaz Sade <boaz@dragonflydb.io>
2022-09-18 19:51:54 +03:00