Incremental State Transfer (IST)

Galera 2.x introduces a long awaited functionality: incremental state transfer. It means that if

  1. the joining node state UUID is the same as that of the group and
  2. all of the writesets that it missed can be found in the donor's Gcache

then instead of whole state snapshot it will receive the missing writesets and catch up with the group by replaying them.

For example:

  • local node state is

5a76ef62-30ec-11e1-0800-dba504cf2aab:197222

  • group state is

5a76ef62-30ec-11e1-0800-dba504cf2aab:201913

  • if writeset number 197223 is still in the donor's writeset cache (GCache), it will send writests 197223-201913 to joiner instead of the whole state.

IST can dramatically speed up remerging node into cluster. It is also non-blocking on the donor.

The most important parameter for IST (besides ist.recv_addr) is GCache size on donor. The bigger it is, the more writesets can be stored in it and the bigger seqno gaps can be closed with IST. On the other hand, if GCache is much bigger than the state size, serving IST may be less efficient than sending state snapshot.

Trace: • ist

Login