v1.14.0

[romange]

Dragonfly v1.14.0

This is our first release for this year. We've mainly focused on stability and performance improvements for this release.
Among other things it includes:

• Tighter memory tracking
• support for ACL keys
• json parser allows '-' in field names.
• Initial support for client tracking feature
• XREADBLOCK fix for blocking case.

What's Changed

• feat: DispatchTracker to replace everything by @dranikpg in feat: DispatchTracker to replace everything #2179
• fix(server): Call PostUpdate from LPUSH even for new keys by @chakaz in fix(server): Call PostUpdate from LPUSH even for new keys #2254
• refactor: conn_context and reply_builder refactoring by @BorysTheDev in refactor: conn_context and reply_builder refactoring #2251
• fix(connection): Add WeakRef to replace pubsub wait token by @dranikpg in fix(connection): Add WeakRef to replace pubsub wait token #2227
• bug(server): remove CO::READONLY for QUIT command. by @theyueli in bug(server): remove CO::READONLY for QUIT command. #2263
• chore: add a test demonstrating on how to pass a dash within json path by @romange in chore: add a test demonstrating on how to pass a dash within json path #2267
• bug(hset): make hrandfield reply match Redis by @theyueli in bug(hset): make hrandfield reply match Redis #2266
• feat(Server):support Verbatim strings resp type, using it for CLIENT LIST and INFO commands by @theyueli in feat(Server):support Verbatim strings resp type, using it for CLIENT LIST and INFO commands #2264
• feat(acl): add storage for acl keys by @kostasrim in feat(acl): add storage for acl keys #2257
• fix(tiering): fix crash when item was deleted before offloaded by @adiholden in fix(tiering): fix crash when item was deleted before offloaded #2225
• feat(server): Return per-type memory breakdown from INFO and /metrics by @chakaz in feat(server): Return per-type memory breakdown from INFO and /metrics #2262
• feat(acl): add acl keys to acl setuser command by @kostasrim in feat(acl): add acl keys to acl setuser command #2258
• feat(acl): add acl keys to acl get command by @kostasrim in feat(acl): add acl keys to acl get command #2260
• feat(acl): add acl keys to acl list command by @kostasrim in feat(acl): add acl keys to acl list command #2261
• feat(acl): add validation for acl keys by @kostasrim in feat(acl): add validation for acl keys #2272
• feat(acl): add acl keys to acl save/load by @kostasrim in feat(acl): add acl keys to acl save/load #2273
• feat(server): Support CLIENT TRACKING subcommand (1/2) by @theyueli in feat(server): Support CLIENT TRACKING subcommand (1/2) #2277
• fix: accept '-' character when parsing json fields by @romange in fix: accept '-' character when parsing json fields #2271
• chore: update helio by @romange in chore: update helio #2286
• WIP: Auto PostUpdate() by @chakaz in WIP: Auto PostUpdate() #2268
• fix(tiering): fix tiering crash on setting expire by @adiholden in fix(tiering): fix tiering crash on setting expire #2285
• chore: eliminate most of clang++ warnings by @romange in chore: eliminate most of clang++ warnings #2288
• feat(acl): add acl keys to acl log command by @kostasrim in feat(acl): add acl keys to acl log command #2274
• fix(server): Fix client pause and add test by @dranikpg in fix(server): Fix client pause and add test #2298
• fix(server): handle no-key-transactional commands in multi/exec by @dranikpg in fix(server): handle no-key-transactional commands in multi/exec #2279
• feat(server): Better reporting of per-type memory in /metrics by @chakaz in feat(server): Better reporting of per-type memory in /metrics #2303
• feat(server): Implement NUMSUB subcommand by @highpon in feat(server): Implement NUMSUB subcommand #2282
• fix(memory): Add missing fields in connection memory by @chakaz in fix(memory): Add missing fields in connection memory #2306
• refactor(server): Refactor AddOrFind() by @chakaz in refactor(server): Refactor AddOrFind() #2299
• chore: update helio library by @romange in chore: update helio library #2305
• feat(cluster): Cancel blocking commands on cluster update by @dranikpg in feat(cluster): Cancel blocking commands on cluster update #2255
• feat(server): Convert DbSlice's AddOr* to return AutoUpdater by @chakaz in feat(server): Convert DbSlice's AddOr* to return AutoUpdater #2290
• feat(server): Add dispatch queue bytes to /metrics by @chakaz in feat(server): Add dispatch queue bytes to /metrics #2310
• feat(replica): add master id on replica output for info replication by @adiholden in feat(replica): add master id on replica output for info replication #2314
• fix(server): Track all IoBuf capacity changes in Connection by @chakaz in fix(server): Track all IoBuf capacity changes in Connection #2309
• refactor(DbSlice): Replace FindExt() with FindMutable() and FindReadOnly by @chakaz in refactor(DbSlice): Replace FindExt() with FindMutable() and FindReadOnly #2308
• refactor(server): Make FindFirst() read-only by @chakaz in refactor(server): Make FindFirst() read-only #2317
• fix(bug): access invalid prime table iterator by @adiholden in fix(bug): access invalid prime table iterator #2300
• feat: add command flow for slot migration process by @BorysTheDev in feat: add command flow for slot migration process #2292
• feat(server): Support CLIENT TRACKING subcommand (2/2) by @theyueli in feat(server): Support CLIENT TRACKING subcommand (2/2) #2280
• feat(server): Better accounting of DenseSet memory by @chakaz in feat(server): Better accounting of DenseSet memory #2325
• feat(server): Account for RObj concrete objects by @chakaz in feat(server): Account for RObj concrete objects #2324
• chore: remove support for save_schedule flag by @romange in chore: remove support for save_schedule flag #2327
• fix(stream): make fix for XREADBLOCK function by @BorysTheDev in fix(stream): make fix for XREADBLOCK function #2323
• feat: introduce transaction statistics in the info output by @romange in feat: introduce transaction statistics in the info output #2328
• chore: refactor slowlog code, no functionality changes by @romange in chore: refactor slowlog code, no functionality changes #2331
• feat: add keyspace_mutations metric by @romange in feat: add keyspace_mutations metric #2329
• feat: expose tx_queue_len metric by @romange in feat: expose tx_queue_len metric #2330
• chore(search): Block list by @dranikpg in chore(search): Block list #2307
• chore: refactor VersionMonitor into a separate file by @romange in chore: refactor VersionMonitor into a separate file #2326
• refactor(server): Privatize PreUpdate() and PostUpdate() by @chakaz in refactor(server): Privatize PreUpdate() and PostUpdate() #2322
• feat: introduce 'debug tx' command and periodic overload logs by @romange in feat: introduce 'debug tx' command and periodic overload logs #2333
• feat: track differrent patterns of multi/exec transactions by @romange in feat: track differrent patterns of multi/exec transactions #2334
• feat(search): Aggregator, step 1 by @dranikpg in feat(search): Aggregator, step 1 #2332
• feat: expose transaction types via /metrics by @romange in feat: expose transaction types via /metrics #2336
• feat(server): Add reply count & latency metrics by @chakaz in feat(server): Add reply count & latency metrics #2340
• fix: Invalid key lock strings with squashing by @dranikpg in fix: Invalid key lock strings with squashing #2341
• bug(server): reject replicaof while loading from snapshot by @adiholden in bug(server): reject replicaof while loading from snapshot #2338
• chore: expose the multi length in slowlog by @romange in chore: expose the multi length in slowlog #2339
• fix(server): Fix reply seconds & better metrics names by @chakaz in fix(server): Fix reply seconds & better metrics names #2342
• feat(makefile): Build with full debug symbols and strip in make package by @chakaz in feat(makefile): Build with full debug symbols and strip in make package #2343
• fix(SimpleLruCounter): Correctly set bumped node's next by @chakaz in fix(SimpleLruCounter): Correctly set bumped node's next #2346
• fix: relax the requirement for parsing successfully container limits by @romange in fix: relax the requirement for parsing successfully container limits #2352
• chore: transaction simplification by @romange in chore: transaction simplification #2347
• feat(getslotsinfo): Add memory usage per slot by @chakaz in feat(getslotsinfo): Add memory usage per slot #2355
• refactor(generic_family): Don't allocate StringSink on the heap by @chakaz in refactor(generic_family): Don't allocate StringSink on the heap #2356
• chore: add "send" state to client list by @romange in chore: add "send" state to client list #2357
• fix "debug exec" command by @romange in fix "debug exec" command #2354
• feat(tiering): add max file size limit by @adiholden in feat(tiering): add max file size limit #2344
• feat: add SLOT-MIGRATION-STATUS cmd for source node by @BorysTheDev in feat: add SLOT-MIGRATION-STATUS cmd for source node #2349
• fix: unblock transactions only if requirements are correct by @BorysTheDev in fix: unblock transactions only if requirements are correct #2345
• refactor(rdb): Expose default compression mode without direct flag by @chakaz in refactor(rdb): Expose default compression mode without direct flag #2360
• refactor(serializer): Move DUMP logic into RdbSerializer by @chakaz in refactor(serializer): Move DUMP logic into RdbSerializer #2363
• fix(dash table): bucket count returns the number of buckets by @adiholden in fix(dash table): bucket count returns the number of buckets #2359
• test: Adding integration test using Relay benchmark by @theyueli in test: Adding integration test using Relay benchmark #2348
• feat: introduce user timeout by @romange in feat: introduce user timeout #2361
• chore: add comment for c_rehash in ca_cert_dir flag description by @kostasrim in chore: add comment for c_rehash in ca_cert_dir flag description #2365
• chore: add memory usage test for skiplist encoding by @romange in chore: add memory usage test for skiplist encoding #2367
• chore: consolidate facade stats under a single struct by @romange in chore: consolidate facade stats under a single struct #2368
• fix(test): Fix flaky test by @chakaz in fix(test): Fix flaky test #2372
• feat(server): Add RestoreSerializer by @chakaz in feat(server): Add RestoreSerializer #2366
• refactor(SerializerBase): Move some logic from RdbSerializer to SerializerBase by @chakaz in refactor(SerializerBase): Move some logic from RdbSerializer to SerializerBase #2373
• fix(server): crash on rename save command on background save by @adiholden in fix(server): crash on rename save command on background save #2375
• refactor(cluster): remove SYNC cmd and do data transferring during FLOW by @BorysTheDev in refactor(cluster): remove SYNC cmd and do data transferring during FLOW #2369
• feat(server): allow running memory commands under script by @adiholden in feat(server): allow running memory commands under script #2382
• feat(lru): add generic lru class by @adiholden in feat(lru): add generic lru class #2351
• feat: add flag to enable round robin sharding in non cluster mode by @kostasrim in feat: add flag to enable round robin sharding in non cluster mode #2364
• chore: improvements in dash code by @romange in chore: improvements in dash code #2387
• fix(regTests): assertion failure during load in cancel_replication_immediately by @kostasrim in fix(regTests): assertion failure during load in cancel_replication_immediately  #2371
• chore(transaction): Avoid COORD_SCHED_EXEC ambiguity with multi transactions by @dranikpg in chore(transaction): Avoid COORD_SCHED_EXEC ambiguity with multi transactions #2392
• fix: ignore unexpected contents for /sys/fs/cgroup/cpu.max by @romange in fix: ignore unexpected contents for /sys/fs/cgroup/cpu.max #2394
• chore: simple traffic logger by @dranikpg in chore: simple traffic logger #2378
• feat(cluster): Add RestoreStreamer. by @chakaz in feat(cluster): Add RestoreStreamer. #2390
• chore: reduce Makefile for release procedures only by @romange in chore: reduce Makefile for release procedures only #2397
• feat: Store and diststore for GeoRadiusByMember by @azuredream in feat: Store and diststore for GeoRadiusByMember #2350
• chore: server_family cleanups by @romange in chore: server_family cleanups #2398
• feat: add os string by @romange in feat: add os string #2401
• bug(server): log evicted keys in journal in PrimeEvictionPolicy. by @theyueli in bug(server): log evicted keys in journal in PrimeEvictionPolicy. #2302
• feat(info): add new persistence section fields by @BorysTheDev in feat(info): add new persistence section fields #2396
• feat(cluster): Add params to slot migration full sync cut by @chakaz in feat(cluster): Add params to slot migration full sync cut #2403
• feat: allow checking the compressability of the data by @romange in feat: allow checking the compressability of the data #2370
• fix: do not return an error in memory-usage command by @romange in fix: do not return an error in memory-usage command #2405
• fix(server): call OnCbFinish after debug populate by @adiholden in fix(server): call OnCbFinish after debug populate #2358
• Fix flush when migrating connection by @dranikpg in Fix flush when migrating connection #2407
• chore: remove atomic<> from ReplicaInfo::state by @romange in chore: remove atomic<> from ReplicaInfo::state #2409
• chore: fix our release pipeline by @romange in chore: fix our release pipeline #2408
• feat: introduce segment merge operation by @romange in feat: introduce segment merge operation #2412
• feat(server): Implement CLIENT KILL by @chakaz in feat(server): Implement CLIENT KILL #2404
• fix: missing error reply to client after AddOrFind throw std::bad_alloc by @kostasrim in fix: missing error reply to client after AddOrFind throw std::bad_alloc #2411
• fix(tx): guard parallel writes to local result by @dranikpg in fix(tx): guard parallel writes to local result #2417
• cleanup: Remove unused PerformDeletion() overloads by @chakaz in cleanup: Remove unused PerformDeletion() overloads #2418
• feat: allow throttling tiered writes by @romange in feat: allow throttling tiered writes #2414
• feat(transaction): Single hop blocking, callback flags by @dranikpg in feat(transaction): Single hop blocking, callback flags #2393
• fix(rdb): Remove transaction from pre/post load search index rebuild by @dranikpg in fix(rdb): Remove transaction from pre/post load search index rebuild #2419
• fix: access to wrong thread local after command migrates fiber to a different thread by @kostasrim in fix: access to wrong thread local after command migrates fiber to a different thread #2410
• fix: non reset fields in command config resetstat by @kostasrim in fix: non reset fields in command config resetstat #2425
• fix(info): Reply with correct redis_mode in `INFO by @chakaz in fix(info): Reply with correct redis_mode in `INFO #2429
• fix: release.sh by @romange in fix: release.sh #2432
• server(tiering): load data on read by @adiholden in server(tiering): load data on read #2415
• chore(transaction): Clean up scheduling code by @dranikpg in chore(transaction): Clean up scheduling code #2422
• chore: add both df tree impelementations to memory usage test by @romange in chore: add both df tree impelementations to memory usage test #2430
• feat(server): Tracking memory usage for client tracking table by @theyueli in feat(server): Tracking memory usage for client tracking table #2431
• fix: Invalid memory access by @chakaz in fix: Invalid memory access #2435
• fix(replication): Correctly replicate commands even when OOM by @chakaz in fix(replication): Correctly replicate commands even when OOM #2428
• fix(server): Use custom journal for multi-key PFMERGE cmd by @chakaz in fix(server): Use custom journal for multi-key PFMERGE cmd #2437
• fix: release pipeline by @romange in fix: release pipeline #2439
• feat: measure latency of pipelined commands by @romange in feat: measure latency of pipelined commands #2436
• fix(server): dont allow snapshoting in tiering mode by @adiholden in fix(server): dont allow snapshoting in tiering mode #2438
• fix: allow lock_on_hashtags with any cluster mode by @romange in fix: allow lock_on_hashtags with any cluster mode #2443
• fix(tiering): update iterator after await throttle by @adiholden in fix(tiering): update iterator after await throttle #2440

Huge thanks to all the contributors! ❤️

Full Changelog: v1.13.0...v1.14.0

---

This discussion was created from the release v1.14.0.