This document defines common Vitess concepts and terminology.
A keyspace is a logical database. In the unsharded case, it maps directly to a MySQL database name, but it can also map to multiple MySQL databases.
Reading data from a keyspace is like reading from a MySQL database. However, depending on the consistency requirements of the read operation, Vitess might fetch the data from a master database or from a replica. By routing each query to the appropriate database, Vitess allows your code to be structured as if it were reading from a single MySQL database.
When a database is sharded, a keyspace maps to multiple MySQL databases. In that case, a single query sent to Vitess will be routed to one or more shards, depending on where the requested data resides.
The keyspace ID is the value that is used to decide on which shard a given record lives. Range-based Sharding refers to creating shards that each cover a particular range of keyspace IDs.
Often, the keyspace ID is computed as the hash of some column in your data, such as the user ID. This would result in randomly spreading users across the range-based shards. Using this technique means you can split a given shard by replacing it with two or more new shards that combine to cover the original range of keyspace IDs, without having to move any records in other shards.
Previously, our resharding process required each table to store this value as a
keyspace_id column because it was computed by the application. However, this
column is no longer necessary when you allow VTGate to compute the keyspace ID
for you, for example by using a hash vindex.
A shard is a division within a keyspace. A shard typically contains one MySQL master and many MySQL slaves.
Each MySQL instance within a shard has the same data (excepting some replication lag). The slaves can serve read-only traffic (with eventual consistency guarantees), execute long-running data analysis tools, or perform administrative tasks (backup, restore, diff, etc.).
A keyspace that does not use sharding effectively has one shard.
Vitess names the shard 0 by convention. When sharded, a keyspace has N
shards with non-overlapping data.
Vitess supports dynamic resharding, in which the number of shards is changed on a live cluster. This can be either splitting one or more shards into smaller pieces, or merging neighboring shards into bigger pieces.
During dynamic resharding, the data in the source shards is copied into the destination shards, allowed to catch up on replication, and then compared against the original to ensure data integrity. Then the live serving infrastructure is shifted to the destination shards, and the source shards are deleted.
A tablet is a combination of a mysqld process and a corresponding vttablet
process, usually running on the same machine.
Each tablet is assigned a tablet type, which specifies what role it currently performs.
The keyspace graph allows Vitess to decide which set of shards to use for a given keyspace, cell, and tablet type.
During horizontal resharding (splitting or merging shards), there can be shards
with overlapping key ranges. For example, the source shard of a split may serve
c0-d0 while its destination shards serve c0-c8 and c8-d0 respectively.
Since these shards need to exist simultaneously during the migration, the keyspace graph maintains a list (called a partitioning or just a partition) of shards whose ranges cover all possible keyspace ID values, while being non-overlapping and contiguous. Shards can be moved in and out of this list to determine whether they are active.
The keyspace graph stores a separate partitioning for each (cell, tablet type) pair.
This allows migrations to proceed in phases: first migrate rdonly and
replica requests, one cell at a time, and finally migrate master requests.
During vertical resharding (moving tables out from one keyspace to form a new keyspace), there can be multiple keyspaces that contain the same table.
Since these multiple copies of the table need to exist simultaneously during
the migration, the keyspace graph supports keyspace redirects, called
ServedFrom records. That enables a migration flow like this:
new_keyspace and set its ServedFrom to point to old_keyspace.new_keyspace.
Vitess will automatically redirect these requests to old_keyspace.ServedFrom redirect to begin actually serving from new_keyspace.old_keyspace.There can be a different ServedFrom record for each (cell, tablet type) pair.
This allows migrations to proceed in phases: first migrate rdonly and
replica requests, one cell at a time, and finally migrate master requests.
The replication graph identifies the relationships between master databases and their respective replicas. During a master failover, the replication graph enables Vitess to point all existing replicas to a newly designated master database so that replication can continue.
The Topology Service is a set of backend processes running on different servers. Those servers store topology data and provide a distributed locking service.
Vitess uses a plug-in system to support various backends for storing topology data, which are assumed to provide a distributed, consistent key-value store. By default, our local example uses the ZooKeeper plugin, and the Kubernetes example uses etcd.
The topology service exists for several reasons:
A Vitess cluster has one global topology service, and a local topology service in each cell. Since cluster is an overloaded term, and one Vitess cluster is distinguished from another by the fact that each has its own global topology service, we refer to each Vitess cluster as a toposphere.
The global topology stores Vitess-wide data that does not change frequently. Specifically, it contains data about keyspaces and shards as well as the master tablet alias for each shard.
The global topology is used for some operations, including reparenting and resharding. By design, the global topology server is not used a lot.
In order to survive any single cell going down, the global topology service should have nodes in multiple cells, with enough to maintain quorum in the event of a cell failure.
Each local topology contains information related to its own cell. Specifically, it contains data about tablets in the cell, the keyspace graph for that cell, and the replication graph for that cell.
The local topology service must be available for Vitess to discover tablets and adjust routing as tablets come and go. However, no calls to the topology service are made in the critical path of serving a query at steady state. That means queries are still served during temporary unavailability of topology.
A cell is a group of servers and network infrastructure collocated in an area, and isolated from failures in other cells. It is typically either a full data center or a subset of a data center, sometimes called a zone or availability zone. Vitess gracefully handles cell-level failures, such as when a cell is cut off the network.
Each cell in a Vitess implementation has a local topology service, which is hosted in that cell. The topology service contains most of the information about the Vitess tablets in its cell. This enables a cell to be taken down and rebuilt as a unit.
Vitess limits cross-cell traffic for both data and metadata. While it may be useful to also have the ability to route read traffic to individual cells, Vitess currently serves reads only from the local cell. Writes will go cross-cell when necessary, to wherever the master for that shard resides.