How Hard Could That Be? Take II.

When we started to develop Galera replication about 5 years ago, we faced a question of how to test it under load: we needed a fast, flexible and easy to use load balancer. Thus GLB project was born, explained in my "How Hard Could That Be?" old blog post (lost in site upgrade, but miraculously preserved by Wayback Machine).

It turned out that it was not that hard at all — it was unsurmountable. Linux splice() function turned out to be a joke and you still had to do at least 3 system calls per packet: (e)poll(), read(), write() - 6 per SQL request. And that's a lot of CPU. And a lot of latency.

And while it is not that much faster than pen or haproxy (insignificantly so for production purposes), glbd IS the fastest user-space load balancer for SQL traffic. Yet it takes roughly the same amount of CPU as a sysbench client, easily becoming a bottleneck in benchmarking.

It took 5 years to realize that Linux has another wonderful feature: libc call overloading. Enter libglb.so — a shared library that overloads libc connect() call when it is preloaded with LD_PRELOAD. Now any Linux application can be made capable of balancing its connections without any modifications or functionality loss. And since only connect() call is overloaded, all communication happens directly between client and server absolutely unhindered, without any proxies or any additional code in between. Now this can't be beat by ANY load balancer! It turned out to be WAY easier than it looked 5 years ago.

Try it out here. Example command line:

$ LD_PRELOAD=/usr/lib/libglb.so GLB_BIND=3306 GLB_TARGETS=192.168.0.1,192.168.0.2,192.168.0.3 mysql -h127.0.0.1 -P3306 ...


P.S. This is so trivial, that I would not be surprised to learn that there already is a project like that. I might have googled for wrong terms. Anyway we had GLB already, so turning it into a library was like a 1 day job.