Redis fundamentals and working with redis through python
Modify /etc/redis/redis.conf file
redis-cli: CLI utility to run redis commandskeys *: To get all key-valuesSET key value: To store key valueGET key: To get value of a keyredis-benchmark -q -n 1000: to benchmark requestsdbsize: gives number of recordsflushall: removes all recordsSET course_name:1:chetan "Python, 9999, 4.7" ex 10: key= course_name:1:chetan value="Python, 9999, 4.7" expires in 10 secondsDEL key_name: Deletes a keyincr key_name: increments a key value by onemset key value [key value ...]: to set multiple key,values at oncemget key1 key2 ...: to get multiple values for key at onceexists key1 key2 ...: checks if key exists or notttl key: gives time to live for a key until expiresexpire key seconds: to set expire for existing keypersist key: to remove expirelpush list_name value [value ....]: creates a list of values if not exists else adds values at toprpush list_name value [value ...]: creates a list of values if not exists else adds values at bottomlrange list_name 0 -1: gives all values of list from 0 to -1 both included.llen list_name: gives lenght of list_name.lpop list_name [count]: pops "count" item from top of a list.rpop list_name [count]: pops "count" item from bottm of a list.ltrim list_name start stop: keeps only values form start to stop in a listhset key field1 value1 [field2 value2 ...]: creates key if not exist else adds in existing key pairs of key-value.hget key field: gives value of a field in a key/hashhmget key field1 field2 ...: gives values of fields in a key/hashhmset key field1 value1 [field2 value2 ...]: similar to hset, deprecated.sadd users_ip ip1 ip2 ...: adds values into setsmembers users_ip: shows all members of setscard users_ip: length of a setsismember users_ip ip1: checks if member is in setsdiff set1 set2: members of set1-set2spop setname [count]: pops one or more random members from a set.srem setname memeber: removes a member from a set.watch key: watches a key during multi transactionmulti: to start multiple transactionexec: to run all commands in a transactiondiscard: to discard a transactionsentinel master master_name: gives master node details.slaveof ip_address port: makes a redis slave slave of node having given ip and port.save: saves current data in dump.rdb in redis-server.
-
RDB: Performs point in time snapshot of data at specified interval of time.
save 60 1000 30 500(if every 60 seconds 1000 keys, or 30 seconds 500 keys)
dir path/to/dir/to_save/redis_dump.rdb
add above 2 line in redis.conf
Advantages:
- Compact, single file of redis data
- Perfect for backups
- Performance is good
- Allows faster restarts
Disadvantages:
- Chances of data loss
- Fork process, may impact write operations for few milliseconds. -
AOF: Append Only File - Keeps log of every write operations in redis.
Define fsycn policies, always, everysec
appendonly yes
Add above line in redis.conf
Advantages:
- High Durability
- Allows different fsync policies
- Append only logs, no chances of corruption
- Corrupted files can be fixed by redis-check-aof tool
- AOF files can be parsed, lines can be removed.
Disadvantages:
- Size is bigger than RDB
- Slower than RDB -
Hybrid: (RDB+AOF) - Both can work together.:
- Add
replicaof master_ip master_portin slave's redis.conf file. - change port, logfilename, dump.rdb file name for slaves in redis.conf file.
- Run redis server for master and slaves.
- slaves can only be used to read data.
- Install sentinel on minimum 3 servers.
- Quorum (No. of sentinel which needs agree on master down and elect a new master)
- down-after milliseconds
- Redis port = 6379, Sentinel port = 26379
- redis.conf, sentinel.conf
-
Sentinel Deployment 01:
- Never do this deployment
- 2 sentinels, 1 master 1 slave
- quorum =2
- Only works if redis is down but sentinels are working on both servers.
- if complete box is down,redis will become unavailable.
-
Sentinel Deployment 02:
- 3 sentinels, 1 master, 2 slaves
- Quorum = 2
- Good approach if we have min 3 servers.
- issues:
- 1. If nodes are seperated by network partition
- Mitigation - min-replica-to-write 1 (stops writes if can't write to replica), min-replicas-max-lag 10
- 2. If two replicas are down, master stops accepting writes.
-
Sentinel Deployment 03:
- 3 sentinels(on application server), 1 master 1 slave
- Quorum 2
- Good approach if we have only 2 redis servers.
- Issue:
- Network disconnect issue between redis server and application server.
-
Sentinel Deployment 04:
- 4 sentinels (2 on application server, 2 on redis server), 1 master, 1 redis
- Quorum = 3
- Good approach if we have min 3 servers.
- minimum 3 masters and 3 slaves needed.
- Multi master architecture
- Data is partitioned into 16k buckets.
- Each bucket are assigned a master.
- Pros:
- Scalable
- Can scale upto 1000 nodes cluster
- Performance is good
- Allows multiple masters
- Cons:
- Need atleast 6 servers to setup a cluster
- Upgrading or downgrading to other topology is difficult.
-
create n1_redis.conf file with details as below in each node:
port 7001/7002/7003/7004/7005/7006
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes -
Start redis-server on each node
redis-server n1_redis.conf & -
Create a cluster:
redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1 -
Connect to server:
redis-cli -c -p 7001 -
Check cluster health:
redis-cli --cluster check 127.0.0.1:7001
redis-cli -p 7001 cluster nodes