Migrating Elasticache from Redis to Valkey
1 — Introduction
Valkey is a fork from Redis just before it transitioned from the open-source licensing to the Redis license.
It is a data structure server (or you can call a database as well) that serves key/value workloads, similar to Redis, supporting a wide range of native structures and additional features, such as extensible plugins for adding new data structures and access patterns, allowing to extend the solution beyond it’s original capabilities.
Being fully open-source also adds credibility to the project. Adding to this the fact that other big companies, including AWS, are backing up and actively contributing to the project helps in making the decision to transition from a “proprietary” license to Valkey’s open-source model.
AWS started supporting Valkey in its Elasticache managed service early on October 2024, as can be seen in this announcement: https://aws.amazon.com/about-aws/whats-new/2024/10/amazon-elasticache-valkey/
2 — Pricing
Elasticache for Valkey also provides better pricing, when compared with Elasticache for Redis.
Using AWS Pricing calculator, for a 3-node cache.r6g.8xlarge cluster, pricing for both Redis and Valkey can be seem below (pricing was based in Ireland region):
#############
### Redis ###
#############
3 instance(s)
x
3.664 USD hourly
x
100% utilization in a month (24/7)
x
730 hours in a month
----------------------------------------
8024.1600 USD
##############
### Valkey ###
##############
3 instance(s)
x
2.9312 USD hourly
x
100% utilization in a month (24/7)
x
730 hours in a month
----------------------------------------
6419.3280 USDAs can be seem above, the hourly price of Valkey is around 20% cheaper compared to Redis. This can result in massive savings if you are a heavy user or Elasticache for Redis.
3— Upgrading Elasticache from Redis to Valkey
AWS allows to upgrade from an existing Elasticache for Redis cluster, to use the Valkey engine.
Here is a complete scenario, where I move from a new Elasticache for Redis to a Valkey cluster:
3.1 — Create the Redis infrastructure
###
### As a best security practice, disable the "default" user
###
aws elasticache create-user --user-id "default-do-not-use" \
--user-name "default" \
--engine "REDIS" \
--passwords "Strong@Password123" \
--access-string "off +get ~keys*" \
--region eu-west-1
aws elasticache modify-user-group \
--user-group-id "grp-admin" \
--user-ids-to-add "default-do-not-use" \
--user-ids-to-remove "default" \
--region eu-west-1
###
### create a new admin user
###
aws elasticache create-user --user-id "fabricio" \
--user-name "fabricio" \
--engine "REDIS" \
--passwords "Strong@Password123" \
--access-string "on ~* +@all" \
--region eu-west-1
###
### associate the new admin user to the user group
### this user group will then be associated with the new cluster
###
aws elasticache modify-user-group --user-group-id "grp-admin" \
--user-ids-to-add "fabricio" \
--region eu-west-1
###
### create a new Elasticache for Redis cluster, with cluster mode enabled
### this allows Redis to have shards
###
aws elasticache create-replication-group \
--replication-group-id "valkey-cluster-tests" \
--replication-group-description "valkey-cluster-tests" \
--automatic-failover-enabled --num-node-groups 2 \
--replicas-per-node-group 0 \
--cache-node-type "cache.t3.small" \
--cache-parameter-group-name "redis-parameter-group-tests" \
--engine "redis" \
--engine-version "7.0" \
--transit-encryption-enabled \
--security-group-ids "<your security group id>" \
--cache-subnet-group-name "subnet-grp-valkey-tests" \
--user-group-ids "grp-admin" \
--at-rest-encryption-enabled --kms-key-id "<your KMS key ID>" \
--region eu-west-1
###
### checking the cluster
### it should have 2 shards
###
aws elasticache describe-replication-groups \
--replication-group-id "valkey-cluster-tests" \
--query "ReplicationGroups[*].NodeGroups[*].{NodeGroupId:NodeGroupId,Status:Status,NodeGroupMembers:NodeGroupMembers}" \
--region eu-west-1
---------------------------------------------------------------------------------
| DescribeReplicationGroups |
+-----------------------------------------+-------------------------------------+
| NodeGroupId | Status |
+-----------------------------------------+-------------------------------------+
| 0001 | available |
+-----------------------------------------+-------------------------------------+
|| NodeGroupMembers ||
|+--------------------------------+--------------+-----------------------------+|
|| CacheClusterId | CacheNodeId | PreferredAvailabilityZone ||
|+--------------------------------+--------------+-----------------------------+|
|| valkey-cluster-tests-0001-001 | 0001 | eu-west-1a ||
|+--------------------------------+--------------+-----------------------------+|
+-----------------------------------------+-------------------------------------+
| NodeGroupId | Status |
+-----------------------------------------+-------------------------------------+
| 0002 | available |
+-----------------------------------------+-------------------------------------+
|| NodeGroupMembers ||
|+--------------------------------+--------------+-----------------------------+|
|| CacheClusterId | CacheNodeId | PreferredAvailabilityZone ||
|+--------------------------------+--------------+-----------------------------+|
|| valkey-cluster-tests-0002-001 | 0001 | eu-west-1b ||
|+--------------------------------+--------------+-----------------------------+|3.2 — Connect and add some data to the cluster
###
### connect using "redis-cli" using the cluster endpoint
###
redis-cli -c -h <cluster-endpoint> --tls -p 6379 --user fabricio -a "Strong@Password123"
###
### checking engine and version
### to see ALL information, run an "info all" instead
###
redis> info server
# Server
redis_version:7.0.7
redis_git_sha1:0
redis_git_dirty:0
redis_build_id:0
redis_mode:cluster
os:Amazon ElastiCache
arch_bits:64
monotonic_clock:X86 TSC @ 2500 ticks/us
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:0.0.0
process_id:1
run_id:959a009cc0ace397588f7fba7c5955ee2142bef6
tcp_port:6379
server_time_usec:1729354977899395
uptime_in_seconds:892
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:1301729
executable:-From the above “info server”output, it’s possible to see that the engine is “redis” running version 7.0.7.
Now adding some data to the cluster:
###
### checking if there is any data already
###
redis> keys *
(empty array)
###
### creating some hashes
###
redis> hset "person1" name "name1" address "address1"
redis> hset "person2" name "name2" address "address2"
3.3 — Modify the engine from Redis to Valkey
Modify the current cluster’s engine and version, as Elasticache for Valkey is only available in version 7.2. Also provide a parameter group of the Valkey family:
###
### Specify the new engine in the "--engine" flag
### As of now, the only available version for Valkey is 7.2
### flag "--cache-parameter-group-name" specifies the Valkey parameter group
###
aws elasticache modify-replication-group \
--replication-group-id "valkey-cluster-tests" \
--engine valkey \
--engine-version 7.2 \
--cache-parameter-group-name "valkey-parameter-group-tests" \
--region eu-west-1 NOTE1: Valkey’s family custom parameter group must have the same parameters as of the Redis family custom parameter group.
After the modification has been completed (it will take some time), running “info server”, it’s possible to see that the engine has been modified, along with the version:
redis> info server
# Server
redis_version:7.2.4
server_name:valkey
valkey_version:7.2.6
redis_git_sha1:0
redis_git_dirty:0
redis_build_id:0
redis_mode:cluster
os:Amazon ElastiCache
arch_bits:64
monotonic_clock:X86 TSC @ 2500 ticks/us
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:0.0.0
process_id:1
run_id:bfb998f613ca184bda2ac6f3167127806330b7b1
tcp_port:6379
server_time_usec:1729358198629161
uptime_in_seconds:1061
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:1304950
executable:-
config_file:-
availability_zone:eu-west-1bOne important thing during this modification is that the data is persisted between engine changes. After the modification, I still could fetch the hashes created prior to the change:
> hgetall "person1"
1) "name"
2) "name1"
3) "address"
4) "address1"
> hgetall "person2"
1) "name"
2) "name2"
3) "address"
4) "address2"Another useful way to perform this change is to simply restore from a snapshot. You can take a snapshot of your Elasticache for Redis cluster, and restore directly as a Valkey cluster.
The only difference from the create command previously mentioned is the flag “--snapshot-name”, which points to a Redis snapshot previously taken:
aws elasticache create-replication-group --replication-group-id "valkey-cluster-tests-restored" \
--replication-group-description "valkey-cluster-tests-restored" \
--automatic-failover-enabled --num-node-groups 2 \
--replicas-per-node-group 0 \
--cache-node-type "cache.t3.small" \
--cache-parameter-group-name "valkey-parameter-group-tests" \
--engine "valkey" \
--engine-version "7.2" \
--transit-encryption-enabled \
--security-group-ids "<your security group id>" \
--cache-subnet-group-name "subnet-grp-valkey-tests" \
--user-group-ids "grp-admin" \
--at-rest-encryption-enabled --kms-key-id "<your KMS key ID>" \
--snapshot-name "snapshot-valkey-cluster-tests" \
--region eu-west-14 — Should I really upgrade?
This is a fair question, specially when Redis is a well established and consolidated caching solution and Valkey is fairly new and may still have some questions on the air.
I can see 2 main reasons for making the decision of migrating:
- Elasticache for Valkey is less expensive when compared to Elasticache for Redis, and this can make a huge difference, depending on the Elasticache footprint.
- Valkey is backed up by AWS (One of its main developers and maintainer is https://www.linkedin.com/in/madelyn-olson-valkey/ which is a principal AWS engineer), so Elasticache for Valkey can get different features, when compared to Redis, as generally AWS listens to its customers, deploying features that are required by its enterprise customers.
Other advantages that Valkey has:
- Better support for multi-threading.
- Community support and permissive license
- Enhanced observability features, with per-slot metrics
- Big company supporters, such as AWS (mentioned above), Google and Oracle. This may or may be not an advantage, but given the fact that Valkey license is open source, we can expect to see new implementations of services based on Valkey
I hope this was helpful.
Thanks.
