-
|
I encountered an issue when setting keys with similar values in my Redis environment (version 7.0). Specifically, I ran the following commands: 127.0.0.1:6379> set aaaaaa abcdef
OK
127.0.0.1:6379> set aaaaaaa abcdef
OK
127.0.0.1:6379> memory usage aaaaaa
(integer) 64
127.0.0.1:6379> memory usage aaaaaaa
(integer) 72This led me to the following questions:
I attempted to calculate it myself using the following logic:
However, even with these calculations, the total memory usage exceeds 64 bytes. Could you help me understand where I might be going wrong? Feel free to ask if you need further clarification or have any additional questions! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 9 replies
-
|
In jemalloc, aaaaaa falls into bin 8, whereas aaaaaaa falls into bin 16. ref |
Beta Was this translation helpful? Give feedback.
-
|
@sundb This is really helpful for my question 1.( Why does the memory usage increase to 72 bytes when I only add one additional character to the key?). Thanks for the answer. I remain confused about the memory calculation: The key will use sds5 and the value will use sds8. (https://cloud.tencent.com/developer/article/1837860)
|
Beta Was this translation helpful? Give feedback.
-
no, key was saved as sds, not robj, so it's 8B.
there are two cases here:
so the memory is (24B or 32B) + 8B + 24B(dictEntry) |
Beta Was this translation helpful? Give feedback.
-
Thanks for the answer. @sundb But I checked the source code. Looks like the value stored as redisObject. Am I missing something here? Thanks. The source code version is 7.0. /* Overwrite an existing key with a new value. Incrementing the reference
* count of the new value is up to the caller.
* This function does not modify the expire time of the existing key.
*
* The 'overwrite' flag is an indication whether this is done as part of a
* complete replacement of their key, which can be thought as a deletion and
* replacement (in which case we need to emit deletion signals), or just an
* update of a value of an existing key (when false).
*
* The dictEntry input is optional, can be used if we already have one.
*
* The program is aborted if the key was not already present. */
static void dbSetValue(redisDb *db, robj *key, robj *val, int overwrite, dictEntry *de) {
int slot = getKeySlot(key->ptr);
if (!de) de = kvstoreDictFind(db->keys, slot, key->ptr);
serverAssertWithInfo(NULL,key,de != NULL);
robj *old = dictGetVal(de);
val->lru = old->lru;
if (overwrite) {
/* RM_StringDMA may call dbUnshareStringValue which may free val, so we
* need to incr to retain old */
incrRefCount(old);
/* Although the key is not really deleted from the database, we regard
* overwrite as two steps of unlink+add, so we still need to call the unlink
* callback of the module. */
moduleNotifyKeyUnlink(key,old,db->id,DB_FLAG_KEY_OVERWRITE);
/* We want to try to unblock any module clients or clients using a blocking XREADGROUP */
signalDeletedKeyAsReady(db,key,old->type);
decrRefCount(old);
/* Because of RM_StringDMA, old may be changed, so we need get old again */
old = dictGetVal(de);
}
kvstoreDictSetVal(db->keys, slot, de, val);
/* if hash with HFEs, take care to remove from global HFE DS */
if (old->type == OBJ_HASH)
hashTypeRemoveFromExpires(&db->hexpires, old);
if (server.lazyfree_lazy_server_del) {
freeObjAsync(key,old,db->id);
} else {
decrRefCount(old);
}
}
// dict.c
void dictSetVal(dict *d, dictEntry *de, void *val) {
assert(entryHasValue(de));
de->v.val = d->type->valDup ? d->type->valDup(d, val) : val;
} |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the answer. This makes sense for me. Thanks again. I pasted the related code to record. /* Add the key to the DB. It's up to the caller to increment the reference
* counter of the value if needed.
*
* If the update_if_existing argument is false, the program is aborted
* if the key already exists, otherwise, it can fall back to dbOverwrite. */
static dictEntry *dbAddInternal(redisDb *db, robj *key, robj *val, int update_if_existing) {
dictEntry *existing;
int slot = getKeySlot(key->ptr);
dictEntry *de = kvstoreDictAddRaw(db->keys, slot, key->ptr, &existing);
if (update_if_existing && existing) {
dbSetValue(db, key, val, 1, existing);
return existing;
}
serverAssertWithInfo(NULL, key, de != NULL);
kvstoreDictSetKey(db->keys, slot, de, sdsdup(key->ptr)); // Here set the key to sds.
initObjectLRUOrLFU(val);
kvstoreDictSetVal(db->keys, slot, de, val);
signalKeyAsReady(db, key, val->type);
notifyKeyspaceEvent(NOTIFY_NEW,"new",key,db->id);
return de;
} |
Beta Was this translation helpful? Give feedback.
-
I took a look at the source code of Redis. The data structure sdshdr5 is not used, so what you said that the key is sds5 (1B + 6B + 1B = 8B) is incorrect. |
Beta Was this translation helpful? Give feedback.
- 760C
no, key was saved as sds, not robj, so it's 8B.
there are two cases here:
OBJ_ENCODING_RAWencoding, consume 24B.OBJ_ENCODING_EMBSTRencoding, consume 32B, becauserobjandsdsare in contiguous memory, occupying a total of 24 bytes, so they are allocated to bin 32so the memory is (24B or 32B) + 8B + 24B(dictEntry)