CouchDB - Filtered Replication
October 28, 2012
CouchDB is an amazing database project that runs pretty much everywhere you might need it to. It also has an awesome replication engine that is doing the majority of the grunt work moving tons of data on my current project.
One great feature of the replication engine is the ability to selectively filter documents.
HOWEVER its documentation is a moving and incomplete target since its being actively developed. Something i hope to contribute to in the future!
There is a new set of features in CouchDB as of
v1.20 that gives you much better control over your replication jobs. You can find the docs on this new DB on the wiki.
By posting the typical replication json documents to the
replicator database you can get detailed stats on that replication process.
Recently i was running into issues using a filter and noticed that the documents i was deleting on my source database were not getting deleted in the target.
Something the documentation
DOES NOT tell you is that when you use a filter you have to be mindful of the
_deleted attribute. If you dont check this variable and return
true your deleted documents will not get deleted from the target database when you trigger replication. Below is a basic template i now use when making filters for replication.