QuoraGrabber is Dead!

Long live RSS Backup!
Really, a separate script / project just to back-up Quora is overkill. Now I have a more general script for backing up from any RSS feed. (Which I'll be able to use to ensure I have copies of what I write here and on Composing.)
I also made it a bit saner at keeping the useful HTML markup (ie. links etc.)
It's on GitHub.

Quora Scraping As A Service

If you missed it, I launched a Fiverr Gig.

Quick explanation.

GrabQuora on GitHub

A couple of tweaks to the Quora grabbing script. Makes it worth upgrading to a full GitHub project.

Quora Scraper

I love Quora. It's a great site and community. But I started getting a bit concerned how much writing I was doing there which was (potentially) disappearing inside their garden and not part of the body of thinking I'm building up on ThoughtStorms (or even my blogs).

Fortunately, I discovered Quora has an RSS feed of my answers, so I can save them to my local machine. (At some point I'll think about how to integrate them into ThoughtStorms; should I just make a page for each one?)

Anyway here's the script (powered by a lot of Python batteries.)




And this turns the files back into a handy HTML page.

Tags in RSS?

A quick question. What's the right way to add "tag" information to an RSS feed? (So that a story can have a number of tags associated with it? Eg. my last story here was tagged "wiki", "bill seitz" etc.)

Looking at the spec there's a "category" sub-element. Is it this? Does each category need to have a different "category-domain" value? Or can I have multiple categories for an item with the same domain? (This is what Blogger itself seems to do.)

What's Like RSS?

Dave Winer asked a great question back in December. What standards are like (his ideal for) RSS?

That is, basically fixed forever by convention, large userbase and multiple suppliers?

My suggestions :

In practice, a few Unix classics : SSH, the diff / patch formats, rsync, finger. All used on a grand scale by many parties. Multiple implementations. Multiple pieces of software committed to them. No one really trying to change them.

Email protocols are pretty widely supported and fixed.
Git. It's notionally owned by Linus Torvalds, but he doesn't seem to have any commercial interest in extending or breaking it. GitHub showed you can build a great commercial site around it without trying to make proprietary extensions. And I can use the same clients to push and pull from my server running the default Git daemon, from Github, or from rival offerings (I'm pretty sure BitBucket / SourceForge / Google Code now offer Git as an option)

Possibly Jabber / XMPP

Social Media 2012

My comment on Alex's blog :

Well, you already know but I still think wiki has a future, as pointed to by Smallest Federated Wiki. There are some flaws / issues with SFW, mainly I think because not enough people are working on it, but it’s still the signpost for how wiki could evolve. 

Would still love to see you and other UseMod / OddMuse people look at ways to engage, even if you don’t switch over. 

2012 is the year when it just became more and more clear that we need our own space and shouldn’t be dependent on Fb / Tw / G+ etc. 

Fb / Tw / G+ offer two compelling things : 1) an aggregate river of stuff from people we care about, 2) really easy transclusion from various rich media sites. 

We could have a distributed river architecture if we took RSS and some kind of pubsub architecture (eg. RssCloud) seriously. SFW has made transclusion protocols central to its philosophy. If we pick up on both, figure out how to get the most important things we get from the mainstream working smoothly, we can create a compelling alternative on our terms. And one of the interesting, overlooked, facts about G+ is that it showed that significant numbers of people are still willing to experiment with alternatives. As long as you can get a critical mass of around 20 people you care about to use it, G+ is as valuable as anything else. You don’t need 1 billion users. You aren’t trying to take over the world at this point, just to have a syndication / discussion architecture which isn’t owned by THEM.

JSON / RSS

Dave Winer is considering an official(?) JSON flavour of RSS.

I just want to say here that I like RSS, for what it is, and what it does, and I like JSON to actually work with, because parsing XML is still a faff. So it gets my vote.

Walled River

Apple join the war against RSS.

We need to defend the principle of a platform independent / open feed of news items from all the companies like Facebook, Twitter, Google and Apple who have seen the future as feeds insided their own proprietory walled "gardens".

Not sure if a garden is the right metaphor for a feed routing system, maybe "walled river"?

Something like this? :-(
 

 
Hat-tip Scribe.

Open rivers of news are wonderful things. Recently I've started using the venerable Planet feed agregator to make some public planets (rivers) such as this mind-boggling "Future Manufacturing" one. Glance at that and see exactly how awesome open RSS is. And how it can be way more compelling than the constrained Twitter or your riddiculously cramped Facebook wall. Look at a torrent of exciting information that can actually "breath", where text can be as long as it needs and where pictures are wide-screen rather than crammed into a cage designed to make you look at adverts.

Show Your RSS

Dave Winer reminds us to help people find our RSS feeds, as it seems that browser-makers are increasingly trying to obscure them from us.[1].

My approach is non-standard, but hopefully conveys the message :-)

[1] Rather like Steve Jobs trying to hide the file-system, some people love to take away anything that it might actually empower you to learn about.

A while ago on Platform Wars I wrote :

99% of the world's "semi-structured" data is not in Microformats but in tables in spreadsheets.

Wildly inaccurate estimate I'm sure. But I'll bet it dwarfs XML formats including RSS. So where's the Yahoo Pipes for CSV and spreadsheet data? The mashing, pivot-tabling, cartesian joining of live grids?

Yahoo Pipes does have a CSV reader ... but I'd like to see more. Particularly pivoting and SQL-like selects, projects and joins.