Wating For The Miracle

A short discussion with Baron at Henrik's blog has stirred my eloquence.

Baron points to a great post by Josh Berkus where Josh contemplates the database clustering issues from a novel viewpoint. The post is really insightful. But I'm going to top that (albeit not so skilfully).

In his post Josh maintains that existing PostgreSQL clustering solutions do a poor job satisfying user needs because developers concentrate too much on technological choices and too little on use cases, which he identifies three: Transactional User, Analytic User, Online User. And all developers need to do is just make three (only three) clustering solutions that would satisfy each of those use cases well. And all will be nice and pleasant!

And there he goes into one conceptual fallacy which may better be illustrated by a lousy metaphor.

Let's take the Online User of PostgreSQL:

"I'm driving this comfy and safe car. And now you should make 1000 of them into a car train. To transport 10000 people. At a supersonic speed. And it should have the same controls and the same ride quality I'm used to. And yeah, it should do its own maintenance and refueling. While driving at full speed, yeah. Ok, I will ease (!) the task for you - sometimes people may fall off. But sometmes they may not."

And my first (and last) reaction to that would be:

"Whoa, dude! Who told you that you can do it with a car? Perhaps you should buy an airline company instead?"

"But you can't drive-through McDonalds in a jet!"

I mean, notice that we're talking about clustering of PostgreSQL here, specifically a relational database server conforming to SQL standard, not an arbitrary piece of software. We're concerned with Postgres clustering solutions that don't satisfy an Online User. Something else won't do. But why should one expect that SQL database server cluster can in principle satisfy the needs of an Online User as they were stated? Who said that clustering can do that at all?

Look, the guy has developed an application on top of a transactional SQL server (apparently because that server had necessary qualities). And now he wants this server to be not really a transactional SQL server. But still run his application. And keep the necessary qualities. Plus more. And all this must be achieved by specifically clustering of the original transactional SQL server, not by creating an entirely new software product from scratch.

I don't even care if his wants make any sense (as we know, human mind has no bounds), but perhaps multiple failures in this direction are a good indication, that he should probably start looking for something different.

And, surprisingly, that's what many of them do! There are plenty of companies we all know who successfully run large-scale online operations by creatively adapting their architecture, their requirements and their expectations to the database clustering realities. So, nice try, Josh, but while you may have beautifully formulated the problem, you seem to have misplaced the responsibility.

Database clustering developers are doing what they can. They are working very-very hard. They have implemented just about any solution possible within the limits of logic, so that users could have a choice of what is possible. It is not their fault when users fail to choose. (Ok, some of these solutions are poorly implemented, but that's beyond original Josh's point.)

So what was my point here? (uh, almost forgot my point fighting for database clustering developers) Well, the point is that many people like to align their thinking along Josh's lines and keep on waiting for the miracle. Well, get real (or something!) Database clustering can solve some problems (in return of giving you more), but it won't lay diamonds. Deal with it.