OWL Server

OWL now has a simple Python server that saves OPML files to your local machine.

More here.

GeekWeaver : Fixed Variable Substitution in Markdown

GeekWeaver : Fixed a bug that prevented variables being evaluated in Markdown mode.

Cthulhu

My software is more or less like Cthulhu. Normally dead and at the bottom of the sea, but occasionally stirring and throwing out a languid tentacle to drive men's minds insane. (Or at least perturb a couple of more recklessly adventurous users.)

However there's been a bit more bubbling agitation down in R'lyeh recently. The latest weird dream returning to trouble the world is GeekWeaver, the outline based templating language I wrote several years ago.

GeekWeaver was basically driven by two things : my interest in the OPML Editor outliner, and a need I had to create flat HTML file documentation. While the idea was strong, after the basic draft was released, it languished. 

Partly because I shifted from Windows to Linux where the OPML Editor just wasn't such a pleasurable experience. Partly because GW's strength is really in having a templating language when you don't have a web server; but I moved on to doing a lot of web-server based projects where that wasn't an issue. And partly, it got led astray - spiralling way out of control - by my desire to recreate the more sophisticated aspects of Lisp, with all kinds of closures, macros, recursion etc.

I ended up assuming that the whole enterprise had got horribly crufty and complicated and was an evolutionary dead end.

But suddenly it's 2013, I went to have quick look at GeekWeaver, and I really think it's worth taking seriously again.

Here are the three reasons why GeekWeaver is very much back in 2013 :

Fargo

Most obviously, Dave Winer has also been doing a refresh of his whole outlining vision with the excellent browser-based Fargo editor. Fargo is an up-to-date, no-comprise, easy to use online OPML Editor. But particularly important, it uses Dropbox to sync. outlines with your local file-system. That makes it practical to install GeekWeaver on your machine and compile outlines that you work on in Fargo.

I typically create a working directory on my machine with a symbolic link to the OPML file which is in the Fargo subdirectory in Dropbox and the fact that the editor is remote is hardly noticable (maybe a couple of seconds lag between finishing an edit and being able to compile it).

GitHub

What did we do before GitHub? Faffed, that's what. I tried to put GeekWeaver into a Python Egg or something, but it was complicated and full of confusing layers of directory.  And you need a certain understanding of Python arcana to handle it right. In contrast, everyone uses Git and GitHub these days. Installing and playing on your machine is easier. Updates are more visible.

GeekWeaver is now on GitHub. And as you can see from the quickstart guide on that page, you can be up and running by copying and pasting 4 instructions to your Linux terminal. (Should work on Mac too.) Getting into editing outlines with Fargo (or the OPML Editor still works fine) is a bit more complicated, but not that hard. (See above.)

Markdown

Originally GeekWeaver was conceived as using the same UseMod derived wiki-markup that I used in SdiDesk (and now Project ThoughtStorms for Smallest Federated Wiki). Then part of the Lisp purism got to me and I decided that such things should be implementable in the language, not hardwired, and so started removing them. 

The result was, while GeekWeaver was always better than hand-crafting HTML, it was still, basically hand-crafting HTML, and maybe a lot less convenient that using your favourite editor with built-in snippets or auto-complete.

In 2013 I accepted the inevitable. Markdown is one of the dominant wiki-like markup languages. There's a handy Python library for it which is a single, install away. And Winer's Fargo / Trex ecosystem already uses it. 

So in the last couple of days I managed to incorporate a &&markdown mode into GeekWeaver pretty easily. There are a couple of issues to resolve, mainly because of collisions between Markdown and other bits of GeekWeaver markup, but I'm now willing to change GeekWeaver to make Markdown work. It's obvious that even in its half-working state, Markdown is a big win that makes it a lot easier to write a bigger chunks of text in GeekWeaver. And, given that generating static documentation was GeekWeaver's original and most-common use-case, that's crucial.

Where Next?

Simplification. I'm cleaning out the cruft, throwing out the convoluted and buggy attempts to make higer-order blocks and lexical closures. (For the meantime.) 
  
Throwing out some of my own idiosyncratic markup to simplify HTML forms, PHP and javascript. Instead GW is going to refocus on being a great tool for adding user-defined re-usable abstractions to a) Markdown and b) any other text file.

In recent years I've done other libraries for code-generation. For example, Gates of Dawn is Python for generating synthesizers as PureData files. (BTW : I cleaned up that code-base a bit, recently, too.)

Could you generate synths from GeekWeaver? Sure you could. It doesn't really help though, but I've learned some interesting patterns from Gates of Dawn, that may find their way into GW.

Code Generation has an ambiguous reputation. It can be useful and can be more trouble than it's worth. But if you're inclined to think using outlining AND you believe in code-gen then GeekWeaver is aiming to become the perfect tool for you.

GeekWeaver

OK ... not shouting much yet, but what with the relaunch of OPML and Fargo, there's a bit of a GeekWeaver refresh going on.

I put the code on GitHub and starting to clean it up, making it suitable for use ...

Watch this space ...

Winer's Back!

This is really good news.

Dave Winer finally comes out with a decent outliner in the browser.

I've been looking for one for a long time. (Thought of trying to write it too, but it's not my speciality. Now you get one from the world's biggest Outlining evangelist.)

This is also great news for Winer himself, I think. As always, he has a lot of crucial ideas for where the web should be going. But for a while it's seemed like the main thing holding him back has been a code-base that's a Windows desktop application. (Which is NOT where either users or developers want to party these days.) The few times I've thought I'd like to look into the open-sourced Frontier / OPML Editor I've been put off by that.

A new browser-based UI (and Javascript-based server?) hopefully means that he'll be able to get more people involved in his code, interacting with his services, and start to have an impact via technology as well as evangelism.

And me, I'm holding on for the OPML export / import ... ahem ... cough ...  GeekWeaver ... cough. ;-)

World Outline Screencast

Nice screencast from Dave Winer showing where the World Outliner (the successor to the OPML Editor) is at.

Reminds me of GeekWeaver of course, though obviously slicker (and more specialised).

Lunascript looks very interesting.

Looks like Staticmatic is Rubyland's solution to the problem that inspired GeekWeaver.

ThinkLinkr, another pretty slick web-based outliner.

The Smart Disorganized reboot is still in progress ....

Today, a sad story about GeekWeaver.

A couple of weeks ago I needed to revamp Gbloink!'s web presence. Quickly. And, I was in no position to do it in GW. So I fired up a copy of WordPress and made : Gbloink!

I also needed a new OPTIMAES site. And one for Gisel.

You spot the trend? The answer to "how do I knock up a decent-looking site? fast?" is to use WordPress. I'm seriously thinking of doing it for my own homepage too.

Obviously, these are the kinds of scenarios for which I envisaged GeekWeaver. So what's gone wrong?

Several things :

- I got over-ambitious. The basic GeekWeaver as templating language, worked great. But I wanted to make it into a full sophisticated Lispish sort of a functional programming language. That side-tracked me into several attempted rewrites before I shelved it with other unfinished projects.

- The OPML Editor worked great in Windows XP. But was horrible in Vista. And now I'm using Linux most of the time. I can run it under Wine, but it feels clunky to do so. So I don't have a decent OPML editor. There are still, surprisingly few outliners in Linux, and still no convenient outline editing widget for the browser.

- Great templates are the real win. An earlier version of GeekWeaver shipped with decent free template I got from somewhere or other. But I'm not a good HTML / CSS designer and I could neither adapt it to my changing requirements nor really design another. I guess the answer is to work with a designer. But as one of the proposals for GW was to make web-design more "programmer-friendly" (by introducing the programmer's favourite tool, abstraction, to HTML) that's rather a contradictory point. Chalk one up to web-designers and one against GeekWeaver.[1]

- Moreover, it's hard to compete against a rich ecosystem like WordPress. Among thousands of templates and plug-ins from dozens of contributors, are some pretty damned good ones.

- Also, while GW had a couple of sprinkles of syntactic sugar to make authoring XHTML / XML in the outliner a rather pleasant experience, the outliner is merely OK for CSS and not really all that good for javascript. (For a real programming language, it's useful to have the standard syntax checking, bracket counting, line numbering etc.) As web-pages are increasingly made of CSS and javascript over and above the HTML, GW is decreasingly useful. To do GW properly, the editor needs to be both a good code-editor AND a good outliner.

So is GeekWeaver a failure? Am I abandoning it?

Well, it's not yet a success, I'll agree. :-)

I still *believe* that there's a niche for a GeekWeaver-like language : something with the quick and dirty characteristics of early Perl or PHP; that let's you get a lot done quickly; and who's philosophy is "templating" at a granularity above the individual page or file. There's no reason that, if I had time and another burst of interest, I might not make further progress taking GeekWeaver in that direction.

But I now have a (more than) full-time day-job writing social software in Python[2] which leaves little time (or inclination) for a lot more experiments in python for web-site making. So GW is definitely on hiatus while I pursue other projects.

Nevertheless, I'm always coming back and thinking what I should do with it. You never know when inspiration might strike again.

[1] This raises another sceptical doubt. Allowing you to define abstractions is meant to make things easier. If it doesn't make "web-design" easier, am I barking up the wrong tree?

[2] In fact, I'm working with Django. Which brings a lot of other concerns and ideas. It was easy to see how GW could compile down to PHP. But would it make sense to try to compile it down to Django?

XHP actually looks pretty cool. On the surface, it's just a cleaned up PHP. But the cleaning up (putting XML into the language) actually gives it some of the character I was hoping for in GeekWeaver.

A quick thought ... working with a lot of Django and Pinax at the moment, I'm seeing a lot of use of Python's varargs. Functions that just take *argv, **kwargs

I find I don't like this. I want to see argument lists as explicit as possible. I feel uncomfortable and a bit lost. I feel unprotected when the compiler can't even check the number of arguments I'm sending.

So I wonder if this is legitimate. Or isn't it analogous to the case of static typing? Varargs give more flexibility than fixed arguments. Much as dynamic typing is more flexible than static. So why shouldn't I prefer the power and flexibility over restriction and security in this case?

Update : Of course, this is the way that GeekWeaver works, where there's no explicit definition of the list of arguments passed to a reusable block. I've been unhappy with that there too. Thinking that I should add positional explicit named arguement lists. But if this kwargs thing is a trend, then GW should probably be left as it is.

Phil Windley on creating your own Domain Specific Language.

Dave Winer's back on the Windows OPML Editor ... which is great for me and for GeekWeaver ('cos I don't have a Mac and there's no Linux port yet.)

Meanwhile ... anyone know other decent OPML editors? I'd be particularly interested in ones that run in the browser.

Smart Disorganized Philosophy #1

I've been on a Smart Disorganized Individuals tip for several years, now. Always writing software compatible with that philosophy.

But what is the philosophy? What does this software mean?

In this series I'll start to make some specific notes towards that. Here's the first.

SdiDesk is wiki : a network of texts.

GeekWeaver is a programming language written in an outliner : a hierarchy.

Mind Traffic Control is a multi-user, dynamic queue, a "flow" of tasks.

Each is SDI. Each is completely different. Each is for specific purpose.

SdiDesk excels at capturing ideas and the relationships between them that are static.

Outlines excel at authoring or creating structure which is ultimately intended for a reader.

Flows excel at capturing change and movement.

OTOH, each is bad for something. These are true, even if you might imagine them not to be.

Wiki is surprisingly bad for authoring. Outliners are surprisingly bad for managing todo-lists. Flows are a surprisingly bad place to put ideas that you want to keep forever.

Wiki is great for writing, but awkward for the kind of reworking and structuring needed to polish a document for an external audience.

Outliners fail to match the dynamism of shifting tasks and priorities in the real world. They focus on making a structure of something which needs little structure.

MTC will lose your ideas when they are no longer in the future.

Sometimes you need to take a step back from a problem, to see the forest for the trees.

GeekWeaver was being held up by something that seemed a rather complicated knot; one that I've wrestled with a number of times but never really untangled to my satisfaction.

So last night I decided to restart with a clean slate. I didn't look at the existing code or existing unit-tests. Didn't even open my IDE or project file.

I just started up IDLE and in a single file, redid the whole thing from scratch, in a fast test-driven stylee.

Amazingly I think I've come out with a cleaner solution, messing around with fewer classes and no attempt to over-re-use with inheritance. The whole thing took about two hours total. Of course, it's not quite finished, and still needs to be integrated back into the main codebase. But I like it a lot more. It's shorter and easier to understand.

Sometimes (at a certain granularity) old code *is* more of a burden than an asset. Understanding what it does and what you can do it is a cognitive cost that outweighs its value. Don't be afraid to throw code away when you need to.

Caveat for small granularities. Obviously rewriting an entire application from scratch is a different matter.

I'm way too busy now ... but I'm having some kicking ideas about my own grandiose bid to improve programming ...

... think GeekWeaver in an SdiDesk-alike editor (obviously).

Then imagine that *everything* is a template.

That's the way that GeekWeaver is already going. Everything is a template (ie. has named slots that can be filled). Function calls are just the injection of a data-block into that template. Another way of putting it, all objects know how to handle the "call" message with a data-block as argument, even if they don't do anything very useful with it.

But now imagine that all the types of things you can get in SdiDesk ... text pages, grids, network diagrams, are also templates. And you can plug and pipe them together any way you like. One page can hold a table, another a network diagram-shaped template, and a third can be specified as the result of injecting the first into the second.

Hmmm ... this definitely looks like it's going in the right direction.

And I thought I was ambitious, trying to write a programming language!

My friend Oli has decided to reinvent programming as we know it. Details are still trickling out via his web-site : Semantic Programming. And I'm in frenzied skype conversation with him, trying to figure out what it's all about.

In outline, it starts from some intuitions behind the Semantic Web : that there should be a massively parallel, distributed graph-shaped database of facts (relational assertions) represented on different machines across the world. But it then layers programming on top of that. Instead of a passive data-structure crawled by scutters, SemProg agents (roughly, the servers which manage different data-nodes) are active. There is message passing between the facts themselves, and agents may have hardwired interpretation to act on some facts (what Oli is calling "axiomatic" understanding), or a "deductive" understanding (I guess rather like Prolog inference), and even a "behavioural" understanding via (I guess again) learning from observing other agents.

I'll keep following this here on Smart Disorganized. Very interesting if it works out.

Would GeekWeaver support Semantic Programming? It seems like Oli is thinking of multiple editors for different types of information, all of which compile down to the same underlying graph-structured format so that the data can be combined. (Rather like Language Oriented Programming.) It seems quite possible that GeekWeaver could output something like his graph-format. I'll certainly be experimenting.

I'm also trying to persuade Oli to look at Erlang as a potential implementation language for the distributed virtual machine. I'm increasingly impressed by Erlang. Finding it very powerful and concise.

@adrianh suggests looking more at Parrot Compiler Toolkit ... which is very cool.

I'm finding that language design is hard. Not just the implementation part (although that's hard too) ... but just figuring out how to make a syntax that allows all the things you'd expect to be done in an elegant way.

I'm starting to appreciate how clever certain common design patterns are in language design, and how hard it is to deviate from them. More on this soon ...

Update : for people wondering how the whole new-SdiDesk-in-Adobe-Flex? thing is going. I solved something I thought was a problem yesterday.

I now have a (very fragile) Flex front end which can pass plain-text GeekWeaver programs to a web-server with GeekWeaver embedded, and get a compiled chunk of GeekWeaver out.

That's very cool ... unfortunately it also revealed yet more problems with the GeekWeaver compiler which need fixing before I can release the next version.

Still, it looks promising. The signs are better and better that something interesting is coming out in the next 3 months. :-)