Front page | perl.kiokudb |
Postings from May 2009
From: Yuval Kogman [mailto:nothingmuch@woobling.org] > Hi, > > > 2009/5/24 atomicstack <m@qx.net.au>: > > I notice BerkeleyDB::Manager has attributes for replication, but no > mention > > in the documentation. Has anybody had luck using it in a replicated > > environment? I'm tinkering with MemcacheDB but the tests seem to fail > when > > BDB::Manager calls BerkeleyDB::Hash->new. My XS knowledge isn't great > so I'm > > not sure where/how to start digging into BerkeleyDB to see if it's > honouring > > the replication flags properly... > > That flag is probably a leftover from my experiments, it's not > supported yet. And it isn't supported as yet in my BerkeleyDB module. > Adding replication to the BerkeleyDB module is possible, but requires > quite a bit of XS work, unfortunately. > > There are several steps required to support replication in BerkeleyDB > (the module): > > 1. Berkeley DB (the library) support 4 modes of memory management. > BerkeleyDB.xs uses only one of these, I assume you are referring to ENV->set_alloc and DB_DBT_MALLOC, DB_DBT_REALLOC,... ? > where the library allocates > everything (instead of using a callback for instance). Unfortunately > this mode is incompatible with the threaded environment, My BerkeleyDB module is *NOT* thread safe at the moment, especially where transactions are concerned. Major surgery to BerkeleyDB.xs will be required to make it work. > which is > necessary for replication (bdb spawns a background thread for this). > > 2. The XS api for replication is not written yet. This is > comparatively easy though. > > 3. the replication API doesn't support automatic forwarding to the > current master, only finding out who the master is. This means that > the RPC layer also has to be supported. To allow transparent writes on > the slaves all writes must be forwarded to the current master and then > replicated to the slave. The RPC api is also not too hard to expose. > > 4. After RPC is available, writing a layer to automatically forward > all write operations to the master is still a big undertaking. For > instance if there is an open transaction with write operations that > probably means all reads need to be done off the master too, so to > benefit from replication this might not be a good approach, and > something a little more explicit is necessary. When I get round to adding replication, I intend to start with the replication manager interface only. Whether I get to any of the lower level replication api remains to be seen. PaulThread Previous | Thread Next