NZB over XMPP PubSub

NZB over XMPP PubSub

Recently I’ve been on an XMPP kick, studying the technology, and identifying some of the areas I think it could be applied. Although this is similar to the concept of “When all you have is a hammer, everything looks like a nail”, I think RSS is one area that XMPP’s PubSub (XEP-0060) could improve in some scenarios.

One that I recently ran across was the being notified of new NZBs becoming available. I currently use an NZB client that includes a RSS client and a set of regular expression filters. At regular intervals (every 15 minutes in my case) it checks the RSS feeds I’ve specified and applies the regular expressions to each of the latest entries. For any entry that matches, that entry is sent on to the main NZB client.

The same content that the RSS client accesses is also available in a human readable format as a simple webpage with a table of NZBs and meta data (size, time, etc). Any NZB that I’m interested in, I can either download and have the NZB client pick up through its monitoring of my default downloads folder, or use a link on the page inserted by a plugin to my web browser and the NZB will be sent to the NZB client through an API.

In either case, I either have to manually check for regular NZBs I’m interested in, or wait for up to 15 minutes for the RSS client. This is where I started to think about the application of XMPP and PubSub. Instead of polling the server via RSS, why not have it actively notify me about all NZBs(name, url, etc; the same contents available via RSS) in a category, then I can apply my filters as usual.

The major benefit to me is the downloading of content starting sooner. I think this could also benefit NZB indexers by reducing their bandwidth. If a client checks an RSS feed 24 hours a day, but contents is most often published in a smaller window of time, then the client will repeatedly request an RSS feed that is mostly the same, wasting bandwidth on redundant information. When new NZBs are published, only a single copy of the information needs to be sent to the XMPP server, then the XMPP server manages contacting the clients.

Update 4-16-2012

Put my code up at GitLab: https://gitlab.com/bettse/xmppnzb and I’ve rewritten it to use the SleekXMPP pubsub examples as a base.

Update 10-26-2012

I’ve continued to develop xmppnzb and now use it exclusively for pulling down recurring content. I maintain a server with the xmpp, newznab, and ‘scandb’ portions and anyone interested can contact me about access for beta testing purposes.

Update 1-17-2015

The server part of xmppnzb was recently merged into the development-postgres branch of pynab!