Reference: http://openmymind.net/redis.pdf

Databases

Databases are simply identified by a number. The default is 0. Change to another database using select 1.

Data structures

  • Redis has 5 data structures: string (scalar), hashset, list, set, sorted set
  • Each of these has at least a key and a value
  • Organise keys using : e.g. a typical key would be users:leto
  • Values can be anything: strings, numbers, json, xml etc (redis stores as a byte array and doesn't care)
  • Objects cannot by retrieved by their value. This requires the data to be modelled in a certain way and can limit the usefulness of Redis.

String (scalar)

  • Simple key value pairs
  • As always, the values can be anything (not just strings)

Useful for:

  • Storing instances of objects by key e.g. with the object represented by JSON
  • Counters
  • Caching data (since key look up is so fast)

Common operations include:

  • set <key> <value> to set a value e.g. set users:leto '{"name": "leto", "planet": "dune", "likes": ["spice"]}'
  • get <key> to get a value e.g. get users:leto
  • strlen <key> to retrieve a value's length'
  • getrange <key> <start> <end> to get a substring of the value
  • append <key> <value> to append to an existing value (or create if not already exists)
  • incr <key> and decr <key> to increment / decrement an existing integer value
  • incrby <key> <step> and decrby <key> <step> to increment / decrement by a specified amount
  • setbit and getbit to perform bitwise operations on values

Hashes

  • Similar to strings, but provide an extra level of indirection: a field

Useful for:

  • Better control storing objects in terms of being able to get and set individual fields
  • More flexibility in querying data

Common operations:

  • hset <key> <field> <value> e.g. hset users:goku powerlevel 9000
  • hget <key> <field> e.g. hget users:goku powerlevel
  • hmset <key> [<field> <value>] to set multiple fields at once e.g. hmset users:goku race saiyan age 737
  • hmget <key> [<field>] to get multiple fields at once e.g. hmget users:goku race powerlevel
  • hgetall <key> to list all fields and values
  • hkeys <key> to list all fields
  • hdel <key> <field> to delete a specific field

Lists

  • Store and manipulate an array of values for a given key

Useful for:

  • Storing keys to other objects which have a certain attribute in common e.g. a list of newusers
  • Storing other list type data structures e.g. logs, the path a user takes through a site, queued user actions in a game

Common operations:

  • lpush <list> <value> to push a user onto the front of a list
  • ltrim <list> <start> <end> to trim a list
  • lrange <list> <start> <end>

Sets

  • Store a set of unique values
  • Provide efficient value based operations

Useful for:

  • Tagging or tracking properties of a value for which duplicates don't make sense
  • Storing items of e.g. friends where operations will include unions and intersections on values

Common operations:

  • sadd <key> [<value>]
  • sismember <key> <value> in O(1) time
  • sinter <key1> <key2> to given the intersection of values
  • sinterstore <key_to_store> <key1> <key2> to store the intersections of values in a new key

Sorted sets

  • Like sets but with a score
  • The score provides sorting and ranking capability

Common operations:

  • zadd <key> [<value> <score>] to add items to a sorted set in O(log(N)) time (where N is the number of items already in the set)
  • zcount <key> <start_score> <end_score> to return the count of values with scores in a certain range
  • zrank <key> <value> and zrevrank <key> <value> to get a value's rank within a sorted set

Memory and persistence

Redis runs in memory and periodically persists to disk. By default Redis saves the whole database every 60 seconds if 1000 or more keys have changed up to 15 minutes if less than 9 keys have changed.

In addition to snapshotting Redis can run in append mode whereby an append-only file is updated every time a key changes.

Leveraging data sturctures

Lookup by value

By default redis only allows lookups by key. In order to perform queries using values, an seperate structure needs to be defined which makes these values in one object, keys in another. Hashs are perfect these indexes.

set users:9001 '{"id": 9001, "email": "leto@dune.gov", ...}' hset users:lookup:email leto@dune.gov 9001

get users:9001 to look up someone by key hget users:leto@dune.gov to get someone's key by email and then use this to lookup via key

Remember, such indexes need to be created / managed / updated / deleted manually.

Maintaining references and indexes

sadd friends:leto ghanima paul chani jessica

In the set above, the values are likely also keys to the details of that person. If chani changes her name, or deletes her account there is no easy way to work out which sets we need to update in order to ensure the reference set is still consistent. Using a numeric id would be preferable for the set. But this doesn't resolve managing account deletions. Here a reverse index would need to also be maintained.

sadd friends_of:chani leto paul

Round trips and pipelining

Making round trips to the database are a common pattern in Redis. There are certain features in Redis which can be leveraged to get the most out of this pattern.

  • mget takes multiple keys and returns values
  • sadd can add one or members to a set
  • Pipelining allows sending multiple requests at the same time in a batch. In Ruby use the redis.pipelined block.

Transactions

  • Every redis command is atomic.
  • Groups sets of commands into one atomic transaction using the multi and then exec (or discard) keywords.

*** BOTTOM OF PAGE 18 - DON'T UNDERSTAND***

Keys anti-pattern

The keys <pattern> command usefully takes a pattern and returns all matching keys. However, it does this by scanning through all existing keys and is therefore very slow. Avoid using this in production code and instead use a hash etc to provide an index for data retrieval on specific criteria.

Beyond the data structures

Expiration

  • expire <key> makes a key for expiration (useful in caching scenarios)
  • ttl <key> returns the ttl
  • persist <key> removes any ttl
  • setex <key> <ttl> <value> allows setting a value and expiration time in one command

Pub and sub

  • A simple queue can be powered using a list and the blpop and brpop commands which remove the first / last elements respectively or block until one is available.

  • subscribe [<channel>] to subscribe to a channel(s)

  • publish <channel> <message> to send a message to a specific channel, will return the number of clients that received the message

  • psubscribe <pattern> to subscribe to channels based on a pattern

  • unsubscribe and punsubscribe to unsubscribe

Monitor and slow log

  • monitor at the command prompt will begin showing all commands being exected against the redis database
  • config set slowlog-log-slower-than 5 will log details of all commands which take longer than 5 microseconds to run
  • slowlog get <count> gets the history of slow commands (optional tail count)

Sort

  • Allows you to sort the values within a list, set or sorted set (sorted sets are only sorted by score)
  • sort <key> returns the values in ascending order
  • sort limit <offset> <count> desc alpha for paged results starting at page <offset> of <count> pages in descending order sorted lexicographically rather than numerically
  • Also allows sorting on a referenced object either by strings or by hashes using a pattern which is substituted for the actual sort value in order to look up the key and hence the actual value to sort by
  • sadd watch:leto 12339 1382 338, set severity:12339 3, set severity:1382 2, set severity:338 5