Basic Redis Data Types

Redis supports various basic data types. Although you don’t need to use all the data types, it is important to understand how they work so that you can choose the right ones.

Strings

Strings are the most versatile data types in Redis because they have many commands and multiple purposes. A String can behave as an integer, float, text string, or bitmap based on its value and the commands used. It can store any kind of data: text(XML), JSON, HTML, or raw text), integers, floats, or binary data(videos, images, or audio files). A String value cannot exceed 512MB of text or binary data.

Cache Mechanisms

It is possible to cache text or binary data in Redis, which could be anything from HTML pages and API responses to images and videos. A simple cache system could be implemented with commands SET, GET, MSET, MGET.

Store string to Redis.

1jedis.set("string", "this is a string");

Store byte[] to Redis.

1jedis.set("string".getBytes(StandardCharsets.UTF_8), "this is a string".getBytes(StandardCharsets.UTF_8));

Store multiple key-value pairs to Redis.

1jedis.mset("key1", "value1", "key2", "values2", "key3", "values3");

Get values stored on Redis.

1List<String> values = jedis.mget("string", "byte", "key1", "key2", "key3"); // ["this is a string", "this is a byte", "value1", "values2", "values3"]

Cache with Automatic Expiration

Strings combined with automatic key expiration can make a robust cache system using the commands SETEX, EXPIRE, and EXPIREAT. This is very useful when database queries take a long time to run and can be cached for a given period of time. Consequently, this avoids running those queries too frequently and can give a performance boost to applications.

Expire key after set

The TTL(Time To Live) command returns one of the following:

A positive integer: This is the number of seconds a given key has left to live
-2: If the key is expired or does not exist
-1: If the key exists but has no expiration time set

Store key without expiry

1jedis.set("keyNoExpiry", "valueNoExpiry");

Check TTL of the key

1long ttl = jedis.ttl(keyNoExpiry); // -1

Check TTL after set expiry

1jedis.expire("keyNoExpiry", 3);
2ttl = jedis.ttl("keyNoExpiry"); // 3

Sleep for 1s
```
1Thread.sleep(1000);
```
Check TTL after sleep for 1s
```
1ttl = jedis.ttl("keyNoExpiry"); // 2
```
Sleep for another 2s
```
1Thread.sleep(2000);
```

Check TTL after sleep for another 2s

1ttl = jedis.ttl("keyNoExpiry"); // -2
2String valueExpired = jedis.get("keyNoExpiry"); // null

Specify expiry while set

Store key that will expire in 3s.

1jedis.set("keyExpiryIn3s", "valueExpiryIn3s", SetParams.setParams().ex(3));

Get value within 3s.

1String valueBeforeExpiry = jedis.get("keyExpiryIn3s"); // "valueExpiryIn3s"

Sleep for 3s
```
1Thread.sleep(3000);
```

Get value after 3s

1String valueWithinExpiry = jedis.get("keyExpiryIn3s"); // null

Counting

A counter can easily be implemented with Strings and the commands INCR and INCRBY. Good examples of counters are page views, video views, and likes. Strings also provide other counting commands, such as DECR, DECRBY, and INCRFLOATBY.

Set counter to 100
```
1jedis.set("counter", "100");
```

Increase counter by 1

1long afterIncrease = jedis.incr("counter");
2// afterIncrease = 101

Increase counter by 5

1long afterIncreaseBy5 = jedis.incrBy("counter", 5);
2// afterIncreaseBy5 = 106

Decrease counter by 1

1long afterDecrease = jedis.decr("counter");
2// afterDecrease = 105

Increase counter by 100

1long afterDecreaseBy100 = jedis.incrBy("counter", 100);
2// afterDecreaseBy100 = 205

Increase counter by 1.8

1double increaseByFloat = jedis.incrByFloat("counter", 1.8);
2// increaseByFloat = 206.8

Lists

Lists are a very flexible data type in Redis because they can act like simple collection, stack, or queue. Many event systems use Redis’s Lists as their queue because Lists’ operation ensure that concurrent systems will not overlap popping items from a queue - List commands are atomic.

There are blocking commands in Redis’s Lists, which means that when a client executes a blocking command in an empty List, the client will wait for a new item to be added to the List.

Redis’s Lists are linked lists. The maximum number of elements a List can hold is 2^32 - 1.

A List can be encoded and memory optimized if it has less elements than the list-max-ziplist-entries configuration and if each element is smaller than the configuration list-max-ziplist-value(by bytes).

Basics

Since Lists in Redis are linked lists, there are commands used to insert data into the head and tail of a List. The command LPUSH inserts data at the beginning of a list(left push) and RPUSH insert data at the end of a List(right push)

1jedis.lpush("books", "Clean Code");
2jedis.rpush("books", "Code Complete");
3jedis.lpush("books", "Peopleware");

The command LLEN returns the length of a list.

1long length = jedis.llen(books); // 3

The command LINDEX returns the element in a given index(indices are zero-based). It is possible to use negative indices to access the tail of the list.

1String firstItem = jedis.lindex("books", 0); // "Peopleware"
2String lastItem = jedis.lindex("books", -1); // "Code Complete"
3String nonExistItem = jedis.lindex("books", 999); // null

The command LRANGE returns an array with all elements from a given index range, including the elements in both the start and end indices.

1List<String> range1 = jedis.lrange("books", 0, 1); // "Peopleware", "Clean Code"
2List<String> range2 = jedis.lrange("books", 0, -1); // "Peopleware", "Clean Code", "Code Complete"

The command LPOP removes and returns the first element of a list. The command RPOP removes and returns the last element of a list.

1String lpop = jedis.lpop("books"); // "Peopleware"
2String rpop = jedis.rpop("books"); // "Code Complete"
3List<String> remaining = jedis.lrange("books", 0 , 1); // "Clean Code"

Event Queue

Lists are used in many tools, including Resque, Celery, and Logstash, as the queueing system.

In following section, we are going to implement a Queue prototype using Redis Lists.

Define `Queue` class

 1@Getter
 2@Setter
 3public static class Queue {
 4    private final String queueName;
 5    private final Jedis redisClient;
 6    private final String queueKey;
 7    private int timeout = 0;
 8                   public Queue(String queueName, Jedis redisClient) {
 9        this.queueName = queueName;
10        this.redisClient = redisClient;
11        this.queueKey = "queues:" + queueName;
12        this.timeout = 3; // 3s
13    }
14
15    /**
16     * @return size of the queue
17     */
18    public long size() {
19        return this.redisClient.llen(this.queueKey);
20    }
21
22    /**
23     * Push {@code data} into the queue
24     */
25    public void push(String data) {
26        this.redisClient.lpush(this.queueKey, data);
27    }
28
29    /**
30     * Pop data from the queue
31     * <p>
32     * if queue is empty, block until data is added to the queue.
33     * <p>
34     * With this blocking api, we don't need to implement polling and no need to worry about empty list.
35     */
36    public String pop() {
37        List<String> results = this.redisClient.brpop(this.timeout, this.queueKey);
38        if (results == null) {
39            return null;
40        }
41        return results.get(1);
42    }
43}

Create a Producer Worker

 1public static class ProducerWorker extends Thread {
 2    private final static int MSG_NUM = 5;
 3    private final Jedis redisClient;
 4    private final String queueName;
 5
 6    public ProducerWorker(Jedis redisClient, String queueName) {
 7        super("Producer");
 8        this.redisClient = redisClient;
 9        this.queueName = queueName;
10    }
11
12    @Override
13    public void run() {
14        Queue queue = new Queue(queueName, redisClient);
15        for (var i = 0; i < MSG_NUM; i++) {
16            queue.push("Hello world #" + i);
17        }
18        logger.info("[Producer] Created " + MSG_NUM + " messages");
19    }
20}

Create a Consumer Worker

 1public static class ConsumerWorker extends Thread {
 2    private final Jedis redisClient;
 3    private final String queueName;
 4
 5    public ConsumerWorker(Jedis redisClient, String queueName) {
 6        super("Consumer");
 7        this.redisClient = redisClient;
 8        this.queueName = queueName;
 9    }
10
11    @Override
12    public void run() {
13        execute();
14    }
15
16    private void execute() {
17        Queue queue = new Queue(queueName, redisClient);
18        String message = queue.pop();
19        if (message != null) {
20            logger.info("[Consumer] Got message: " + message);
21            long size = queue.size();
22            logger.info(size + " messages left");
23            execute();
24        }
25    }
26}

Start Producer and Consumer Workers

 1ProducerWorker producer = new ProducerWorker(jedis, "logs");
 2producer.start();
 3producer.join(2000);
 4
 5long size = jedis.llen("queues:logs"); // 5
 6
 7ConsumerWorker consumer = new ConsumerWorker(jedis, "logs");
 8consumer.start();
 9consumer.join(1000);
10
11boolean isAlive = consumer.isAlive(); // true
12consumer.join(5000);
13isAlive = consumer.isAlive(); // false

Hashes

Hashes are a great data structure for storing objects because you can map fields to values. In a Hash both the field name and the value are Strings.

A big advantage of Hashes is that they are memory-optimized. The optimization is based on the hash-max-ziplist-entries and hash-max-ziplist-value configurations.

Internally, a Hash can be a ziplist or a hash table. A ziplist is a dually linked list designed to be memory efficient. In a ziplist, integers are stored as real integers rather than a sequence of characters. Although a ziplist has memory optimizations, lookups are not performed in constant time. On the other hand, a hash table has constant-time lookup but is not memory-optimized;

Basics

The command HSET sets a value to a field of a given key. The command HMSET sets multiple field values to a key, separated by spaces. Both HSET and HMSET create a field if it does not exist or overwrite its value if it already exists.

The command HINCRBY increments a field by a given integer Both HINCRBY and HINCRBYFLOAT are similar to INCRBY and INCRBYFLOAT.

Set key-value pairs for Hashes.

1jedis.hset("movie", "title", "The Godfather");
2jedis.hmset("movie", Map.of("years", "1972", "ratings", "9.2", "watchers", "1000000"));
3jedis.hincrBy("movie", "watchers", 3);

Get title field value for given key

1String title = jedis.hget("movie", "title"); // "The Godfather"

Get multiple values for given key

1List<String> multipleValues = jedis.hmget("movie", "title", "watchers"); // [ "The Godfather", "1000003" ]

Removed field for given key

1long removed = jedis.hdel("MOVIE", "watchers"); // 1 field removed

List all fields for a given key

1Map<String, String> allValues = jedis.hgetAll("movie"); // { "title" : "The Godfather", "ratings" : "9.2", "years" : "1972" }

List all field keys

1Set<String> keys = jedis.hkeys("movie"); // [ "ratings", "title", "years" ]

List all field values

1List<String> values = jedis.hvals("movie"); // [ "The Godfather", "9.2", "1972" ]

Voting System

This section creates a set of functions to save alink and then upvote and downvote it. First, we create function saveLink to store value in Redis then add upVote and downVote to update the score. Function showDetails shows all the fields in a Hash, based on the link Id.

 1public void saveLink(Jedis client, String key, String author, String title, String link) {
 2    client.hmset(key, Map.of("author", author, "title", title, "link", link, "score", "0"));
 3}
 4
 5public void upVote(Jedis client, String key) {
 6    client.hincrBy(key, "score", 1);
 7}
 8
 9public void downVote(Jedis client, String key) {
10    client.hincrBy(key, "score", -1);
11}
12
13public Map<String, String> showDetails(Jedis client, String key) {
14    return client.hgetAll(key);
15}

Use the previously defined functions to save two links, upvote and downvote them, and then display their details.

 1saveLink(jedis, "link:123", "dayvson", "Maxwell Dayvson's Github page", "https://github.com/dayson");
 2upVote(jedis, "link:123");
 3upVote(jedis, "link:123");
 4
 5saveLink(jedis, "link:456", "hltbra", "Hugo Tavares's Github page", "https://github.com/hltbra");
 6upVote(jedis, "link:456");
 7upVote(jedis, "link:456");
 8downVote(jedis, "link:456");
 9
10Map<String, String> details123 = showDetails(jedis, "link:123"); // { "title" : "Maxwell Dayvson's Github page", "link" : "https://github.com/dayson", "score" : "2", "author" : "dayvson" }
11Map<String, String> details456 = showDetails(jedis, "link:456"); // { "title" : "Hugo Tavares's Github page", "link" : "https://github.com/hltbra", "score" : "1", "author" : "hltbra" }

The command HGETALL may be a problem if a Hash has many fields and uses a lot of memory. It may slow down Redis because it needs to transfer all of that data through the network. A good alternative in such a scenario is the command HSCAN.
HSCAN does not return all the fields at once. It returns a cursor and the Hash fields with their values in chunks. HSCAN needs to be executed until the returned cursor is 0 in order to retrieve all the fields in a HASH. {: .prompt-warning }

1Map<String, String> fields = new HashMap<>();
2for (var i = 0; i < 1000; i++) {
3    fields.put("field" + i, "value" + i);
4}
5jedis.hset("manyFields", fields);
6ScanResult<Map.Entry<String, String>> scanResult = jedis.hscan("manyFields", ScanParams.SCAN_POINTER_START, new ScanParams().count(10));
7int size = scanResult.getResult().size(); // 10
8String cursor = scanResult.getCursor(); // "192"
9boolean isCompleted = scanResult.isCompleteIteration(); // false

Basic Redis Data Types

Strings

Cache Mechanisms

Cache with Automatic Expiration

Expire key after set

Specify expiry while set

Counting

Lists

Basics

Event Queue

Define Queue class

Create a Producer Worker

Create a Consumer Worker

Start Producer and Consumer Workers

Hashes

Basics

Voting System

Related

Define `Queue` class