[openamq-dev] round-robin queues

[Martin Sustrik]

Esteve,
> Following the library use case, the idea is that each box has a database that 
> contains the complete set of a specific type, and there's an application in 
> some other machine that requests elements that are either magazines, books or 
> dvds. Some types are requested more often than others, so they're replicated 
> across several boxes (that's why I would like to do round-robin at this 
> level). A box could hold more than one type, but given two boxes that contain 
> the same type, they might not share the same database with the same types, 
> e.g. A and B boxes could contain the book type, but B contains the dvd type 
> as well.
>   
Your use case looks like standard SOA architecture, i.e. client sends a 
request with routing_key=service type=DB table name ('book', 'dvd', etc.)
The request is then routed to the appropriate queue (1 queue per 
service) and afterwards the queue content is load balanced between 
service instances (individual boxes where the table clones reside). When 
the request reaches the service, SQL select is done and content of the 
appropriate database row is sent back to the requester.

So far, so good.

The strange thing is that you have 1,000,000 service types. We haven't 
seen more than several hundred services and even that results in quite 
an unmanageable beast.

How can you manage that amount of DB tables? I just cannot make myself 
believe that they are really "independent". How big is the team that 
administers the distributed database of that size? It seems strange. 
Maybe the categories form a tree-like hierarchy? Maybe they are 
cartesian product of two criteria with 1,000 possible values for each of 
them? Etc.

My feeling is that there is some piece of the design I am still missing.

Martin