Architecture of SingleStore / MemSQL

 

SingleStore / MemSQL Cluster

Most of the SingleStore DBAs will be fully aware of this. Just for starters, an information of what a typical SingleStore / MemSQL Cluster looks like.

Master Aggregator - Heart of the cluster as the name suggests. Clients can connect to any aggregator including master aggregator. Master Aggregator contains the metadata about the leaf nodes,  version, end user security, metadata on which leaf node to look into when a client requests data, cluster realtime information etc.

Master Aggregator is the master control of the cluster. User management, upgrades, create database  and few important cluster operations can only be performed on the master aggregator. In the case of a failure,  cluster can be operational via child aggregator. But important functions cannot be performed. In that case, a child can be promoted to master.

Child Aggregator - High availability for the clients can be provided using child aggregators. Also, having multiple child aggregators can handle more traffic, than a single  aggregator cab. Although, child aggregator is not necessary, in production clusters, it is always recommended to use atleast one child aggregator.

Child aggregator, aggregates data from the leaf nodes and provides it to the clients which are requesting data from it. Typically a load balancer is installed connecting master and the child aggregators and the load balancer DNS/IP is provided to the client.

Leaf Nodes - Leaf Nodes are the data store of SingleStore cluster. It can store data In Memory in RowStore format, On Disk in ColumnStore format. Leaf Data gets a request from the Master/Child aggregators to provide data whenever a user requests for data. 

Data in SingleStore is always partitioned at the database level. All the tables are therefore partitioned. Number of partitions can be defined while creating the database. By default, the number of partitions will be decided by the engine variable 'default_partitions_per_leaf' which will be discusses in the later part of this blog. It is decided by the number of cores in the CPU socket which will also be discussed  later. 

For instance, in the above examples, we have 4 leaf nodes. If we create a database with 16 partitions, 4 partitions will be stored in each leaf node. Each partition includes both RowStore and ColumnStore.

Please note that, data in memory is always rowstore and on disk is always columnstore. It cannot be changed in SingleStore.

There are some special tables called reference tables, which will be fully stored (and not partitioned) in all the leaf nodes. This is done for performance requirements which will be discussed in the later sections.