Scriptable State Snapshot Transfer

0.8 series MySQL/Galera introduces interface to customize state snapshot transfer via external script (or program). The script is to work under assumption that storage engine initialization on the receiving node happens after state transfer is complete. In short, the idea of this transfer is to copy the contents of source data directory to destination data directory (with possible variations).

Beginning with patch level 23.7 SST parameters are named. There is wsrep_sst_common.sh file to be sourced by individual scripts. It contains common functions for parsing argument list, logging errors and the like. There is no constraint on on the order and number of parameters. New parameters can be added. Any parameter may be ignored by the script.

Parameters that are always passed to any SST script:

• –role
• –address
• –auth
• –datadir
• –defaults-file
• –parent

Parameters that are passed to SST script by SST process.

• –socket - local server (donor) socket for communication, if that is required.
• –gtid - global transaction ID in the form of <uuid>:<seqno>
• –bypass - whether the actual data transfer should be skipped and only GTID passed to remote (receiver) server (to go straight to incremental state transfer)

Parameters that are passed only to wsrep_sst_mysqldump script as it is special.

• –user - MySQL user to use to connect to both remote and local servers (must be the same).
• –password - MySQL user password.
• –host - remote server (receiver) host address.
• –port - remote server (receiver) port.
• –local-port - local server (donor) port.

Before patch level 23.7 SST scripts were called with positional parameters as shown below. This complicated upgrading interface.

The script should be named wsrep_sst_<name>, like

wsrep_sst_myway

where <name> is just about any meaningful string, and placed in the PATH of mysqld server.

To use this script for state transfer, wsrep_sst_method variable on the receiving node must be set to <name>, like

wsrep_sst_method=myway

The same value will be put into state transfer request and sent to state snapshot donor node, which will also invoke a script called wsrep_sst_<name>.

Technically the scripts on sender and receiver sides can be different, only the name should be the same, but for administrative convenience it is recommended that the same script would handle both sender and receiver roles.

On receiving side the script should accept the following positional arguments:

1. Role. This will be 'joiner'.
2. Address to receive snapshot at. It will be either the value of wsrep_sst_receive_address or some sensible default in the <ip>:<port> shape if the former is not set.
3. Authentication information as set in wsrep_sst_auth.
4. The value of mysql_real_data_home (MySQL data directory path).
5. Path to my.cnf as used by mysqld.

After script prepares the node for accepting state snapshot (e.g. wsrep_sst_rsync starts rsync in server mode), the script should print the following string to standard output:

ready <address>

where <address> is the real address at which node is waiting for state transfer. This address, the value of wsrep_sst_auth, and the name of the script will be passed in a state transfer request to sender and will be part of input to sender script.

When the state transfer is over the script should print to standard output the global transaction ID of the received state:

e2c9a15e-5485-11e0-0800-6bbb637e7211:8823450456

and exit with a 0 exit status.

On the sending side the script should accept the following positional arguments:

1. Role. This will be 'donor'.
2. Address to send snapshot to. It will be the value received in state transfer request.
3. Authentication information as received in state transfer request.
4. The value of mysql_real_data_home (MySQL data directory path) on this node.
5. Path to my.cnf as used by mysqld.
6. state UUID.
7. seqno of the last committed transaction.

The script is run in a total order isolation which guarantees that no more commits will happen until the script exits or prints “continue\n” to the standard output.

The following signals from the script are accepted:

1. “flush tables\n” — ask the server to flush tables. When done the server will create tables_flushed file in the data directory. Optional.
2. “continue\n” — tell the server that it can continue committing. Optional.
3. “done\n” — tell the server that state transfer has completed successfully. Mandatory. The script then should exit with a 0 code.

In case of failure the script is expected to return a code that most closely corresponds to the error encountered. This will be returned to receiver through group communication and receiver will leave the cluster and abort (since its data directory is now in inconsistent state).