OWL Fix

There's a big fix for OWL today. There were some mysterious times when pages that I thought I was changing were getting reverted. I thought originally that this was a glitch from me btsync-ing between my laptop and tablet. Or maybe my attempts at doing background synchronization between the browser localStorage and the server were failing.

Nothing seemed to completely eliminate this intermittent problem. But today I realized it was much simpler. I was basically using web.py's "static" file serving to pull the OPML files off the server into Concord. But "static" is meant for static files (doh!). The browser was caching them. (Maybe because of some header web.py was putting out.) Anyway, I just changed the server to reading the files into memory and spitting their contents out, just like any other dynamic web-page, and the problem looks like it's gone away.

I'll keep an eye out, but I think that was it.

Dave open-sources his Outliner

Concord is Fargo open-sourced.

Dave is, of course, awesome.

Cthulhu

My software is more or less like Cthulhu. Normally dead and at the bottom of the sea, but occasionally stirring and throwing out a languid tentacle to drive men's minds insane. (Or at least perturb a couple of more recklessly adventurous users.)

However there's been a bit more bubbling agitation down in R'lyeh recently. The latest weird dream returning to trouble the world is GeekWeaver, the outline based templating language I wrote several years ago.

GeekWeaver was basically driven by two things : my interest in the OPML Editor outliner, and a need I had to create flat HTML file documentation. While the idea was strong, after the basic draft was released, it languished. 

Partly because I shifted from Windows to Linux where the OPML Editor just wasn't such a pleasurable experience. Partly because GW's strength is really in having a templating language when you don't have a web server; but I moved on to doing a lot of web-server based projects where that wasn't an issue. And partly, it got led astray - spiralling way out of control - by my desire to recreate the more sophisticated aspects of Lisp, with all kinds of closures, macros, recursion etc.

I ended up assuming that the whole enterprise had got horribly crufty and complicated and was an evolutionary dead end.

But suddenly it's 2013, I went to have quick look at GeekWeaver, and I really think it's worth taking seriously again.

Here are the three reasons why GeekWeaver is very much back in 2013 :

Fargo

Most obviously, Dave Winer has also been doing a refresh of his whole outlining vision with the excellent browser-based Fargo editor. Fargo is an up-to-date, no-comprise, easy to use online OPML Editor. But particularly important, it uses Dropbox to sync. outlines with your local file-system. That makes it practical to install GeekWeaver on your machine and compile outlines that you work on in Fargo.

I typically create a working directory on my machine with a symbolic link to the OPML file which is in the Fargo subdirectory in Dropbox and the fact that the editor is remote is hardly noticable (maybe a couple of seconds lag between finishing an edit and being able to compile it).

GitHub

What did we do before GitHub? Faffed, that's what. I tried to put GeekWeaver into a Python Egg or something, but it was complicated and full of confusing layers of directory.  And you need a certain understanding of Python arcana to handle it right. In contrast, everyone uses Git and GitHub these days. Installing and playing on your machine is easier. Updates are more visible.

GeekWeaver is now on GitHub. And as you can see from the quickstart guide on that page, you can be up and running by copying and pasting 4 instructions to your Linux terminal. (Should work on Mac too.) Getting into editing outlines with Fargo (or the OPML Editor still works fine) is a bit more complicated, but not that hard. (See above.)

Markdown

Originally GeekWeaver was conceived as using the same UseMod derived wiki-markup that I used in SdiDesk (and now Project ThoughtStorms for Smallest Federated Wiki). Then part of the Lisp purism got to me and I decided that such things should be implementable in the language, not hardwired, and so started removing them. 

The result was, while GeekWeaver was always better than hand-crafting HTML, it was still, basically hand-crafting HTML, and maybe a lot less convenient that using your favourite editor with built-in snippets or auto-complete.

In 2013 I accepted the inevitable. Markdown is one of the dominant wiki-like markup languages. There's a handy Python library for it which is a single, install away. And Winer's Fargo / Trex ecosystem already uses it. 

So in the last couple of days I managed to incorporate a &&markdown mode into GeekWeaver pretty easily. There are a couple of issues to resolve, mainly because of collisions between Markdown and other bits of GeekWeaver markup, but I'm now willing to change GeekWeaver to make Markdown work. It's obvious that even in its half-working state, Markdown is a big win that makes it a lot easier to write a bigger chunks of text in GeekWeaver. And, given that generating static documentation was GeekWeaver's original and most-common use-case, that's crucial.

Where Next?

Simplification. I'm cleaning out the cruft, throwing out the convoluted and buggy attempts to make higer-order blocks and lexical closures. (For the meantime.) 
  
Throwing out some of my own idiosyncratic markup to simplify HTML forms, PHP and javascript. Instead GW is going to refocus on being a great tool for adding user-defined re-usable abstractions to a) Markdown and b) any other text file.

In recent years I've done other libraries for code-generation. For example, Gates of Dawn is Python for generating synthesizers as PureData files. (BTW : I cleaned up that code-base a bit, recently, too.)

Could you generate synths from GeekWeaver? Sure you could. It doesn't really help though, but I've learned some interesting patterns from Gates of Dawn, that may find their way into GW.

Code Generation has an ambiguous reputation. It can be useful and can be more trouble than it's worth. But if you're inclined to think using outlining AND you believe in code-gen then GeekWeaver is aiming to become the perfect tool for you.

GeekWeaver

OK ... not shouting much yet, but what with the relaunch of OPML and Fargo, there's a bit of a GeekWeaver refresh going on.

I put the code on GitHub and starting to clean it up, making it suitable for use ...

Watch this space ...

Fargo For LinkBlogging

I'm a couple of days into LinkBlogging using Fargo, (at Yelling At Strangers From The Sky) and I have to say, I'm getting into the swing and it's great.

If you keep the outline open in a tab, it's about as fast and convenient to post to Fargo as posting a link to Plus or Twitter. (Which is where traditional blogs like WordPress / Blogger often fall short). In fact, G+ is now getting bloated that it can take 10 seconds just to open the "paste a new message" box. It's a lot faster than that.

It would be nice if it could automatically include a picture or chunk of text from the original page the way FB / G+ do, that's turned out to be a compelling experience for me, but it's a nice not must-have.

A question, is there any kind of API for the outline inside the page which a bookmarklet could engage with? (Is that even possible given the browser security model?)

Winer's Back!

This is really good news.

Dave Winer finally comes out with a decent outliner in the browser.

I've been looking for one for a long time. (Thought of trying to write it too, but it's not my speciality. Now you get one from the world's biggest Outlining evangelist.)

This is also great news for Winer himself, I think. As always, he has a lot of crucial ideas for where the web should be going. But for a while it's seemed like the main thing holding him back has been a code-base that's a Windows desktop application. (Which is NOT where either users or developers want to party these days.) The few times I've thought I'd like to look into the open-sourced Frontier / OPML Editor I've been put off by that.

A new browser-based UI (and Javascript-based server?) hopefully means that he'll be able to get more people involved in his code, interacting with his services, and start to have an impact via technology as well as evangelism.

And me, I'm holding on for the OPML export / import ... ahem ... cough ...  GeekWeaver ... cough. ;-)

Worth reading "Where's your data?"

Remember, your Mind Traffic Control data is easily exportable. Just go to : http://mindtrafficcontrol.appspot.com/exports (Via the "Export Data" menu item) and choose whether you want your data exported in CSV format (which you can import into Excel or EditGrid etc.) or OPML (which can be read in the OPML Editor or (less conveniently) in any XML editor).

Dave Winer back on Instant Outlining.

Would certainly be interesting if there was some movement on this. And some development of the OPML outliner in this direction.

Dave Winer's back on the Windows OPML Editor ... which is great for me and for GeekWeaver ('cos I don't have a Mac and there's no Linux port yet.)

Meanwhile ... anyone know other decent OPML editors? I'd be particularly interested in ones that run in the browser.

What do people think about the Grazr widget along the side here? I introduced it using an OPML outline of my online life to explore the evolving OPML ecology.

But that's turned out be pretty inflexible compared to the gutter-tools that blogger is starting to offer. The OPML went out of date, (although I like the collapsible aspect). And the fact that the widget takes up a lot of room even when showing only the highest level, makes it kind of clunky.

So maybe I'll remove and replace by blogger list. Or should I give Grazr another chance?

Update : Cool! Mike from Grazr solved one of my issues. Read comments. Grazr stays for the moment.

Update 2: And I refreshed my OPML file too.

Day 2 of Mind Traffic Control in public.

Development continues with MTC exporting OPML

Mind Traffic Control roll-out continues.

Nothing spectacular in terms of numbers, but friends are making interesting comments and suggestions; and some strangers are discovering it via the Twitter announcement.

Eikeon pointed to twemes as a place to search for the #tag #MTC on Twitter.

Good chat with Folknology. He's encouraging me to open up the data. Hence I did CSV export of your tasks. Al, likes RSS/Atom but I still need to find out whether feed-readers can do the required Google login to get at a non-public feed.

Rup3rt weighed in with a number of comments starting :

Wow an invisible list machine almost like a gumball/toy dispenser ...

and continuing

I like to see what is ahead but it is good with the delegation - I can line up things for the estagiarios. This is great for non-negotiables like driving instructions, recipes.

* The font for "This task was delegated to you .. may report the sender for blacklisting and delete this task" is a bit big

* I would like a difficult autoaddress (1gg23ff3d3reedd@mindtrafficcontrol.appspot.com) to delegate tasks by email and maybe I could get other tasklists to feed into MTC using this (to just top it up with priority1 or todotoday tasks)

* Maybe paste a list of (line break delimited) tasks that are automatically added

* Maybe show in a widget

* Maybe show total number of tasks

* How would time estimates of task duration fit in? It could be a diversion to see "You have 13 hours of traffic to negotiate")

Good suggestions. I'll be interested to know what everyone else thinks.

But here are some initial thoughts :

Font issue, yep. Mailing tasks in to Mind Traffic Control would be great; a very useful addition. But I'm not yet sure I can do it on the Google Application Engine.

I might be able to do it using another hosting service but that's introducing a lot of new complexity into the system : running two different hosting services, writing scripts on the old-skool one to pass emails it receives into MTC in some way, being responsible for security (how do I stop spammers mailing tasks to you via the system?) etc. Going to take a while to figure that one out.

Importing other formats and / or entering multi-line, multi-task lists is easier and more likely to appear soon. I'll definitely go for the multi-line input box in the short-term future. I'd be interested to hear what formats readers currently use for to-do lists so that I can import them. I immediately think OPML, of course. But are there others?

Widgets ... actually I've been playing with OpenSocial and Orkut Widgets for my other GAE project. It's cute ... but I'm not yet sure what I'd put in a MTC widget.

Task counts and time estimates I'm ambivalent on. I don't want to make MTC a paranoia inducing space where you arrive on the front page to be presented with something saying "Welcome to Mind Traffic Control. You have 280 outstanding tasks". On the other hand, I see that people would like some sort of overall measure of what they have to do. More thinking on this.

Finally, there's a couple of issues of "philosophy", particularly why I'm resistant to letting you flag tasks with "priorities" but leaving that for the next post when I'll be more coherent.

Good weekend for GeekWeaver development ... you can now pass arguments to functions that are more or less table-shaped - like this :

:f Fruits
  apples,, oranges,, pears
  grapefruit,, passion-fruit,, grapes
  bananas,, lychees,, mangos  

It's not public yet, but it will be available in the next installer.

Also, although GeekWeaver is designed to be written in an outliner, I'm close to a plain-text mode which will look something like the above.

But I have a problem which I'm still not sure of how to solve. GW has to preserve literal text. So using spaces for indentation like Python is not so easy.

abc
  xyz

could mean either a node "abc" with a child "xyz" or two nodes at the same depth : "abc" and " xyz". How can we tell the two apart?

Currently my plain text parser works with code that looks like this :

* abc
** xyz

Which keeps the continuity with wiki-markup and disambiguates from

* abc
*   xyz

But it's pretty ugly if you really were going to do any serious programming with it.

Suggestions welcome.

Listen everyone, I gotta come out to you all ...

I am now officially an outliner.

For a long time I thought that outlines, like all hierarchical documents, were limited and inferior to graph-shaped wikis.

Now I get it.

The point of the outliner is not the hierarachical structure as a navigation aid - free-form hypertext is still superior.

No, the point of the outliner is the collapse which lets you manage and manipulate bundles of items at the same time. That's something I never managed to get right in SdiDesk. Although I perceived the need for a "PageSet" to create a bundle that could be used for, say, exporting etc. I a) never got that working technically, which was partly because b) I never really made sense of it "conceptually" to myself.

What's great about the outliner is its "scale-free" / "fractal" / "recursive" / "self-similar" nature - which means the same operations (collapse, copy, move, publish) can work on anything from a short list, to a chapter to a volume composed of multiple chapters. I've really started to realize this over the last few months as I've used the OPML editor for more things that I'd have once used SdiDesk for.

Now, don't get me wrong. I still love wiki. It's still my favouritest type of software in the world, ever. And I still use SdiDesk every day. But now, I'm starting to appreciate that there's a need to manage a hierarchy of scales, and until I find out how to combine that with wiki-nature (and into SdiDesk), I'll probably be outlining most days too.

(In my day-job I also spend a whole lot of time with spreadsheets, but that's another story. SdiDesk was always meant to handle grid-shaped data, it just wasn't developed enough to be really usable.)

Q : Phil! Why have you put a Grazr widget in your gutter?

A : Something's happening. And it's about OPML, Grazr, and widgets ...

;-)

Q : Oh, and I noticed you changed the template.

A : Er ... yeah. The look didn't go with the widget. I wanted something that was more, I dunno, evolvable.

Q : What are you up to, exactly?

A : You know, general house-keeping, wanting to learn about new things. The truth is I've been using the OPML Editor a lot recently. Even tried to write a several thousand word essay in it.

And it's good. Perhaps I'm getting the outlining religion ...