Ruby on Javascript VM?

Sometimes you need to take a step back from a problem, to see the forest for the trees.

GeekWeaver was being held up by something that seemed a rather complicated knot; one that I've wrestled with a number of times but never really untangled to my satisfaction.

So last night I decided to restart with a clean slate. I didn't look at the existing code or existing unit-tests. Didn't even open my IDE or project file.

I just started up IDLE and in a single file, redid the whole thing from scratch, in a fast test-driven stylee.

Amazingly I think I've come out with a cleaner solution, messing around with fewer classes and no attempt to over-re-use with inheritance. The whole thing took about two hours total. Of course, it's not quite finished, and still needs to be integrated back into the main codebase. But I like it a lot more. It's shorter and easier to understand.

Sometimes (at a certain granularity) old code *is* more of a burden than an asset. Understanding what it does and what you can do it is a cognitive cost that outweighs its value. Don't be afraid to throw code away when you need to.

Caveat for small granularities. Obviously rewriting an entire application from scratch is a different matter.

Sweet Expressions ... great idea for making S-Expressions more like other, more readable programming languages

without losing their power (such as generality, macros, quasiquoting, homoiconicity, and easily-manipulated program fragments).

Hat-tip Folknology

Wow! I need to get into Buildout.

Update : Good blog-post accompaniment

Very interesting. Recently my "ProgrammingInWiki" page on WardsWiki seems to have come back to life. Along with a companion WikiIDE.

A WikiDE is, of course, something I've been working on in the background for some time. But I'm far from having anything to show yet. So if these guys get there first, good luck to them.

I just want to be able to start working this way.

Damien Katz has more interesting comments about Erlang's "ugliness".

(hat-tip @Folknology)

I've been wondering what Ward Cunningham's been up to at Eclipse. Jon Udell is on the case : Interesting tools for software developers.

Apparently's Ward's now at a wiki-startup. (Mark Dilley seems to be involved too.)

Cobra seems to be like Python with optional compile-time type-checking and other quirks removed.

It also has contracts, which ensure certain conditions when calling a function. However, wearing my new Erlangish hat, I wonder what contracts buy you over pattern-matching arguments?

Unit testing built in to functions is interesting. Not sure what I think of that .... yes, it's useful and convenient. But a) it may clutter up clean code (I tend to let a certain amount of messiness leak into my unit-tests, that I don't let into my code); b) it's not so easy to see how it can be used for test-driven development - how can I write my test before my class? And, worst of all, c) I tend to use unit-tests when refactoring and moving functionality from one class to another.

It's very simple to change a :


x = MyClass()
self.assertEquals(x.f(4),10)


into


x = MyClass2()
self.assertEquals(x.f(4),10)


and have my unit-tests already defining the new target I'm aiming for.

But if the test is is actually *part* of MyClass, then that's going to require more premeditated moving stuff around during my experimental refactoring.

I'm way too busy now ... but I'm having some kicking ideas about my own grandiose bid to improve programming ...

... think GeekWeaver in an SdiDesk-alike editor (obviously).

Then imagine that *everything* is a template.

That's the way that GeekWeaver is already going. Everything is a template (ie. has named slots that can be filled). Function calls are just the injection of a data-block into that template. Another way of putting it, all objects know how to handle the "call" message with a data-block as argument, even if they don't do anything very useful with it.

But now imagine that all the types of things you can get in SdiDesk ... text pages, grids, network diagrams, are also templates. And you can plug and pipe them together any way you like. One page can hold a table, another a network diagram-shaped template, and a third can be specified as the result of injecting the first into the second.

Hmmm ... this definitely looks like it's going in the right direction.

Somehow I feel I should be exploring WikidBase

Looks clever ... and in Python.

Here's a question ... why is Erlang so ugly?

I don't mean that in a pejorative way (not much, anyway). I mean, I really love what it does. I'm totally impressed with Erlang's power and simplicity. I'm writing simulations which are about a quarter of the size of the Python equivalent. So this is not to be taken as a criticism of Erlang which I'm definitely committing myself to, this year. Rather this is some random speculation about programming language aesthetics.

Erlang is wonderfully concise. And yet, somehow, unlike Python, unlike Haskell, it just doesn't come across as beautiful. It's confusing. It looks cluttered.

A couple of lines look fabulous. But the simplicity doesn't scale.

At first guess, there seem to be three issues.

a) As people have noted, the record type is ugly. It is. And counter-intuitive to use in patterns (although I may just be stupid).

b) In general I think it's good and brave thing to take a stance *against* objects. But I haven't figured out how to do encapsulation (data-hiding, abstract types)

Sure, I can define polymorphic functions (one clause at a time per input type) which is a lot shorter and more powerful than overloaded methods in Java. But it has the effect of jumbling all my data-types together. Which just feels *wrong* to me. (Of course, maybe that's some residual OOness in my thinking.)

But I think that may be part of the bigger issue :

c) erlang doesn't seem to have resources for "programming in the large". And, ironically, because erlang is so powerful, that problem becomes visible at a smaller scale - precisely because in erlang "large" programs are actually "small".

Or rather, the only resource is "modules" (which means breaking up into multiple files - always an extra overhead.)

But if you avoid breaking things up into files, the opposite problem becomes apparent. I can do the equivalent of a small Java class (let's say something around 50-80 lines) in about 6 to 10 lines of Erlang. But 10 lines of erlang is too small for a separate file. So I'll add the next 10 line packet of functionality to the same file. By the time I'm up to 4 or 5 packets that would be handled as different classes in Java or Python I may have written only about 50 lines of code ... but it's all running together!

There's no higher level of organization to distinguish and separate the code. In Python I often put 5 or 6 small-medium classes in a single file or around 300-500 lines. But the indentation and editor make these reasonably distinct and identifiable. In contrast, my equivalent 200 lines of Erlang have no visual cues to break them up. I can't use functions as a visual element because pretty much every line is a new function (except when I'm doing I/O, which has its own "issues").

I'm left with using comments but my editor (Komodo), excellent in many ways, doesn't actually know Erlang and so doesn't colour them differently. And, in general, because functions are powerful, they *are* short : one or two lines. But those lines are typically much denser than Python. Even if a dedicated editor would colourize them, I'm not convinced that's such a big win at this density. On the other hand, I don't want to artificially scatter them out into multiple lines. I'm not trying to recreate Python with a slightly less appropriate syntax. I want to take advantage of Erlang's power and conciseness.

But I wonder what the right aesthetics for a language as high level, dense and abstract as Erlang is. Haskell looks cleaner to me - maybe because it does abstract data-types right. But can it solve the problem of organizing your larger-scale things? Lisp is no role-model. ML and its offspring have always repelled me visually. (Nothing can be more ugly, dispiriting and patronizing in a programming language than an explicit "begin" statement.)

Suggestions, anyone?

And I thought I was ambitious, trying to write a programming language!

My friend Oli has decided to reinvent programming as we know it. Details are still trickling out via his web-site : Semantic Programming. And I'm in frenzied skype conversation with him, trying to figure out what it's all about.

In outline, it starts from some intuitions behind the Semantic Web : that there should be a massively parallel, distributed graph-shaped database of facts (relational assertions) represented on different machines across the world. But it then layers programming on top of that. Instead of a passive data-structure crawled by scutters, SemProg agents (roughly, the servers which manage different data-nodes) are active. There is message passing between the facts themselves, and agents may have hardwired interpretation to act on some facts (what Oli is calling "axiomatic" understanding), or a "deductive" understanding (I guess rather like Prolog inference), and even a "behavioural" understanding via (I guess again) learning from observing other agents.

I'll keep following this here on Smart Disorganized. Very interesting if it works out.

Would GeekWeaver support Semantic Programming? It seems like Oli is thinking of multiple editors for different types of information, all of which compile down to the same underlying graph-structured format so that the data can be combined. (Rather like Language Oriented Programming.) It seems quite possible that GeekWeaver could output something like his graph-format. I'll certainly be experimenting.

I'm also trying to persuade Oli to look at Erlang as a potential implementation language for the distributed virtual machine. I'm increasingly impressed by Erlang. Finding it very powerful and concise.

Towards End-User Programming with Wikis

Paul Graham's Arc Challenge . Examples in Lisp, Smalltalk Seaside, Ruby, Perl and Python (using generators instead of continuations)

And an Erlang response.

@adrianh suggests looking more at Parrot Compiler Toolkit ... which is very cool.

I'm finding that language design is hard. Not just the implementation part (although that's hard too) ... but just figuring out how to make a syntax that allows all the things you'd expect to be done in an elegant way.

I'm starting to appreciate how clever certain common design patterns are in language design, and how hard it is to deviate from them. More on this soon ...

Ian Bicking on what PHP deployment gets right.

I'm playing with pipelining in Python over on Composing.

Here's an interesting question (after reading this).

Why does anyone want to write "large" programs? Everyone knows large programs are difficult to build and maintain. Want we want is *small* programs that happen to be powerful (do a lot) and scale over large numbers of users. Is there any reason to think that "large" program correlates with either of these things?

Supplimentary question. Are 10 programs of 1000 lines each, which have to nevertheless interact (via pipes or XML-RPC calls etc.) actually any easier than one 10000 line program?

Update : Folknology has been making a pretty decent case for Erlang as a language optimized for writing software as a swarm of small programs.

Discipulus, patterns for programmers.

LOLCODE IZ IN UR COMPUTR RUNNG THINGZ

Python on Silverlight?

I'm intrigued despite myself.

E.E.Kim on tools

Looking at some slides about Perl 6(pdf) ... some cute features.

Quick note : Blahsploitation is no more ... long live "Composing"

The impotance of being earnest :-)

Update : which reminds me of a discussion I'm having with Sig

Update 2 : Seems like a lot of ongoing discussion around the place. Joel Spolsky explains.

Update : for people wondering how the whole new-SdiDesk-in-Adobe-Flex? thing is going. I solved something I thought was a problem yesterday.

I now have a (very fragile) Flex front end which can pass plain-text GeekWeaver programs to a web-server with GeekWeaver embedded, and get a compiled chunk of GeekWeaver out.

That's very cool ... unfortunately it also revealed yet more problems with the GeekWeaver compiler which need fixing before I can release the next version.

Still, it looks promising. The signs are better and better that something interesting is coming out in the next 3 months. :-)

Restless Mind Syndrome (propaganda for the ultra-cool looking LiveScribe)

He he! Perhaps NooRanch is NooVictorian ....

(I know, bad, bad pun)

What's this?

• 1
• 2
  • 21
    • 211
  • 22
• 3
• 4
  • 41
  • 42
  • 43
    • 431
      • 4311
• 5

An HTML list that came out of GeekWeaver, when I called this recursive function :

::rec
  .<ul>
    :for x,, #__
      .<li>
        ${x/=}
        :hasChild x
          :rec ++ #x__

on a chunk of OPML.

That's pretty much how it's going to look, folks.

::rec defines the recursive block (or function)

.<ul> creates the UL tag

:for x,, #__

is creating a loop through all the anonymous (unlabelled) children of the tree which is being passed to the call. (This list is represented by the symbol __)

Note that I'm trying to keep GeekWeaver as functional as possible so there's no real "assignment". x is scoped so that it only exists inside the block of the for-loop, it's bound once before the block is evaluated, but can't be updated after that.

${x/=}

This is the new way of accessing variables. x is actually bound to a sub-tree (Almost everything in GeekWeaver is a sub-tree, except a couple of weird cases like __ which is a list.)

Although a simple $x still works if you want the whole of a tree flattened into a string, ${} expressions give you a way to pull data from a single path in the tree. In this case we're just getting the text content of the root node of x.

But it's also possible to write stuff like this : ${company/employees/__3/name} which means from the symbol "company" get the labelled child "employees" from which get the non-labelled 3rd child, from which get the child labelled "name".

:hasChild, like :for, is a build-in function. If the tree in x has any children, it evaluates the body of the :hasChild tree. If not, it returns nothing. This tests for our "base-case" when we've hit a leaf of the tree we're recursing through.

The final line :

:rec ++ #x__

is the one which I've been struggling with for the last three months, ever since I started trying to figure out how to support recursion. I'm still not 100% happy with the solution I've come up with, but it's getting there.

:rec is calling the recursive block again. And obviously I need to pass the children of x as arguments. However the way the function is written, it is working on the anonymous children (inside the default variable __)

How am I going to get the children of x into the __ variable inside the next call of :rec? That's what ++ does. Not sure what I'm going to call this, I may call it a "pivot" although that may confuse as much as help. It captures something of the idea that I need to swing the anonymous children of x (#x__) around so that they can go into the next call of :rec as though they were children of the calling node

Like I say, don't know if I like this name or the ++ symbol being used for it. So consider both as provisional for now.

OK, I'm too tired to make the build with this stuff working tonight. I'll try to get a build together in the next couple of days, but meanwhile, if this looks interesting to you and you can't contain yourself, send me an email (interstarATgmail.com), say hello and I'll see what I can do.

Wasabi part 3

And some of Martin Fowler's book on DSLs is online.

Hey! I like NeoVictorianism.

There's a movement I can sign-up for :

Built for people

Built by people

Crafted in workshops

Embracing irregularity

Inspired

Just spent the last few hours downloading and playing with the beta of Flex 3, Adobe's IDE for Rich Internet Applications (ie. applications running on the Flash Virtual Machine) which is based on Eclipse and has an XML-based UI / form description language more or less like HTML.

I'm having two thoughts about it. One is a kind of sigh of relief. This is, after all, finally, The One. After years of fruitless searching I'm pretty sure here's a framework I can settle down with and commit to, and start making babies with. At least, it's more or less mature enough, handsome enough and well endowed enough to put these thoughts into a girl's head. A browser and a desktop? Holidays in Windows, Mac and Linux? Own grid and canvas. Tabbed notebook and some cute chunky buttons.

And it's all done in a way that's pretty self-evident when you look at a few examples. Forms and input widgets are described in XML. They layout nice; and you can start banging them in and prototyping the look of your interface in a couple of minutes. The round-trip from coding to running and testing is a bit slow on my poor 512MB Vista laptop, but it's going to be bearable. And the Eclipsyness of it all is comfortingly familiar if a trifle overblown.

As long as my next experiments turn out right (the one where I try to find a tutorial example of pulling data off a server over http, and the one where I try to compile with AIR into a stand-alone application) then I'm sold.

But there's another part of me going, "huh? Is this all there is? WTF?"

I mean, it's 2007 and I'm happy because I've finally found a way to make GUIs that's sufficiently lower than my pain threshold that I might actually get a piece of software released again. Zowie! But that's what I had with Visual Basic 6 - which came out in about 1997!

In fact, I was already a Pythonista before I started writing SdiDesk in VB. And I only pulled out VB (a language I thought I'd left behind for good) because I got impatient to see what the UI of an SdiDesk could look like and thought I'd prototype it. As often happens, the prototype spiraled out of control as I kept thinking, "maybe I can just also add ... " and within a month or so it had already started to grow into a real program. Another phase of development with some serious refactoring and cleaning up the internal architecture, and it was a quite respectable and powerful bit of software (If I say so myself; I'm talking about the time I made the tutorial screen-casts.)

(Then, of course, I hit the crisis of not wanting to be on the Microsoft treadmill and forced to upgrade to .NET; even though it was obvious that VB6 was as extinct as a very extinct thing from the Lower Devonian period. But also of not having any viable alternative. )

So there's a sense that Flex smells extraordinarily similar. I can see how you can knock out your prototype interface and start building backwards from it. That feels good. That's why it seems like this is plausible option to get development rolling again.

But, like I say, it *is* basically what I had 10 years ago. Except with javascript dressed up in Java's suit and tie to look more grown-up and respectable. And XML.

In fact, I was gchatting to Zbigniew earlier today, and realizd that all of this stuff is hardly a big advance on Hypercard back in 1985. Or perhaps Smalltalk 1972. Why the hell haven't we progressed further? Why am I struggling on each new platform to rediscover the level of comfort I had on the previous one? What's going wrong here?

I suppose it could just be that the idea of quick GUI builders is inherent in the idea of a GUI?

Or maybe we programmers of the noughties need to get our acts together and start coming up with serious new, cool shit. Stuff which couldn't have been thought up in the 60s and 70s. Stuff which is radically easier and more productive than something we had 10 years ago. Something like reinventing Lisp with a cleaner (non)syntax ... erm ... well anyway, I'm off to do more experiments with Flex and try to get it to talk to some kind of server.

If I succeed, then expect to see some interesting developments along the lines I mentioned earlier today ... steps towards a new SdiDesk, possibly a GeekWeaver development environment ... maybe even the long fabled, but never released SystemSketch. Or even the more outré things I've got buzzing around in my fevered imagination like "SexyCells" and "FlowerBrush".

Of course, it still sucks that Flex / Flash seems to have no musical ability whatsoever so Gbloink! doesn't look like an option. Which is a double pity because I think it would make a great Chumby widget and that would have justified me buying one.

:-(

37 Signals on why enterprise software sucks

Quick Note : I just had a revolution in my thinking, triggered by Enso but influenced by several other recent trends.

You write Enso "extensions" as XML-RPC servers sitting on your local machine, register them with Enso and it calls them using XML-RPC. I tried the example from the tutorial and it's very cute and simple. So I've decided this may be a way forward for the dilemma which has kept the "new SdiDesk" on hold for several years(!!!)

If you remember, the issue has been whether the new SdiDesk should be a "web"-application (accessed through the browser) or a desktop application? And how to implement the UI.

The problem with the "browser" answer has always been : "but how to do the network diagramming bit?" - which requires interactive vector graphics. SVG doesn't seem quite stable or cross-browser enough. Canvas isn't cross-browser. Flash is proprietory.

The problem with the "desktop application" is, well, the many rival Python GUI libs with different degrees of maturity. And the question of getting them to work cross-platform.

A meta-problem : I just don't have time and energy to go through learning lots of different GUI frameworks to decide which I want to use. And the ideal answer to the browser / desktop question is probably "both" which doubles the amount of work.

In fact, one of the motivations for GeekWeaver is to see if I can use it define a higher-level UI description that can be compiled down into both XHTML and a GUI widget-set.

Of course, with XUL, Open Laszlo, Adobe Flex, XUML etc. lots of people have been looking at something like HTML for defining desktop apps. too, but the free Python frameworks don't seem to have caught up with that yet. XUL sounds most promising especially with the open sourcing of Active State's Komodo ... but that's also a big complicated thing to even start looking into.

Anyway, inspired by the Enso extensions and Bruce Eckel's interesting example of hooking up a Python XML-RPC server behind an Adobe Flex application I've been wondering about blowing the project apart entirely into a number of services connectd only XML-RPC or something more ReSTful.)

So there'll be a WADS (SdiDesk PageStore) service, a GeekWeaver interpreter service, a "user navigation history service" etc. Even the glue which ties all this together will be another service that's neutral about the user-interface. Effectively the model, view and controller will be completely separate programs. (I know, I know ;-) )

Then there'll be a variety of different types of access with different UIs.

Now that Adobe's AIR has put Flash on the desktop and made it a serious rival to the Java virtual machine - I *am* tempted to put some time into learning it. It would give me a reasonable GUI widget set (including TreeGrid, yay!) and the vector graphics needed for networks. It will run both on the desktop and in the browser and on Linux, PC and Mac. The tools are free-as-in-beer (Flex beta) or free-as-in-speech (Open Laszlo). It's also possible to imagine generating the Flex XML or the Open Laszlo format directly from GeekWeaver. (Aside : I wish ActionScript hadn't borrowed quite so much from Java, but still, I'd only be using it for the front-of-the-front-end UI stuff.)

Then I want to experiment with Enso access - for example, Enso commands like "page HelloWorld" or "pagehistory ChocolateCake"

And there'll be web-browser access probably through a "gateway" that accepts http requests and spits out HTML for a browser. The only thing I don't know is whether I *really* should be using WSGI somewhere here.

Anyway ... that's a quick update of my thinking this week ... tune in soon for the next GeekWeaver release (really, it's coming together, and gonna be very powerful). Then it's back to work on this larger project.

Next installment of the must-read series on FogCreek's Wasabi.

Fascinating to see the problems they've come up against and the solutions. What Stefan calls "picture functions" sound close to GeekWeaver "blocks".

Cool RedMonk TV on open source startup.

Over at Platform Wars I'm tracking Google's "square circles" (hierarchical tags)

Very interesting ... seems that FogCreek's Wasabi is a language to compile into the various parts of a web-app in ASP or PHP.

So kind of a competitor to GeekWeaver. :-) Better take notes.

I wonder if it's gonna be released to the public.

DAn McWeeney has a great post on what he calls "synthesizers" : which seem like generalist / hacker / blogger / communicator type.

Personally I think the existence of the web is pushing us all in the direction of being synthesizers.

This story about rejecting Rails in favour of PHP is very interesting to me now I'm on a PHP-tip.

Some good links on Diseconomies of Scale ... via comments at Bruce Eckel's

Important statement (actually essay / braindump) on my thoughts on widgets and YASNS over on my personal blog.

Mark Bernstein has a great post (and sounds like a great talk) on NeoVictorian Computing.

Wants to return software development to the personal and to making meaning.

Via Bill Seitz

Cool!

Thanks to Dave Stein I now have "programming" on my Behance Profile

New meme : GeekWeaver is Language Oriented Programming for everyone else.

Good weekend for GeekWeaver development ... you can now pass arguments to functions that are more or less table-shaped - like this :

:f Fruits
  apples,, oranges,, pears
  grapefruit,, passion-fruit,, grapes
  bananas,, lychees,, mangos  

It's not public yet, but it will be available in the next installer.

Also, although GeekWeaver is designed to be written in an outliner, I'm close to a plain-text mode which will look something like the above.

But I have a problem which I'm still not sure of how to solve. GW has to preserve literal text. So using spaces for indentation like Python is not so easy.

abc
  xyz

could mean either a node "abc" with a child "xyz" or two nodes at the same depth : "abc" and " xyz". How can we tell the two apart?

Currently my plain text parser works with code that looks like this :

* abc
** xyz

Which keeps the continuity with wiki-markup and disambiguates from

* abc
*   xyz

But it's pretty ugly if you really were going to do any serious programming with it.

Suggestions welcome.

Thanks to Zby for turning me on to Behance which seems to be an interesting combination of social networking for creatives backed by a GTD-style how-to-organize methodology for disorganized creatives, and has special stationery too. Yay!

Could fall between a lot of stools, or could be the next-big-thing (more fun than Linkedin, more serious than Bebo, easier to use than degenerate art)

I do like the explicit emphasis on trying to solve the problems of disorganized people.

I don't like that "software architect" is a creative label you can attach to yourself but "programmer" isn't.

Amusing : job vacancy advertised in Twitter.

That's how evolved the coral reef is getting.

Just commented on a topic dear to the hearts of all smart, disorganized individuals over at Ward Cunningham's Vision Thing.

Resolver Systems seem to be a cool, python + spreadsheet company based in London - with the right attitude. Nice demo.

Hmmm ... and they're hiring ... (drifts off into daydreaming)

Joel Spolsky :

And your programmers are like, jeez louise, GMail is huge, we can’t port GMail to this stupid NewSDK. We’d have to change every line of code. Heck it’d be a complete rewrite; the whole programming model is upside down and recursive and the portable programming language has more parentheses than even Google can buy. The last line of almost every function consists of a string of 3,296 right parentheses. You have to buy a special editor to count them.

Spooky.

I mean, that wasn't exactly the whole GeekWeaver gameplan. But "high level" lisp-like language that compiles down into complex web-applications, is not so far off. I was thinking of Dojo as the Javascript library, PHP at the server, and Facebook itself (or Ning) as the layer at which applications can be glued together. So read with Marc Andreesen too.

It's the zeitgeist I tell ya'

I'd forgotten that Processing Processing by Paul Ford was one of the inspirations for GeekWeaver.

Reading more discussions on the business of Joel has got me thinking about the problem of marketing GeekWeaver.

So here's the pitch ... in the classic 5 points of selling ;-)

1) The pain you didn't know you had!



Oh, the horror, the horror, too many tags too many files, you have to reach for the mouse and click to find the file menu and open up a new file and then scroll around in that silly little common dialog where the files are all represented by little yellow blobs that are 2mm square and then you click one and open and your hand has to come off the mouse and down to the keyboard and you try to scroll around but there are tags and tags and tags then you change something and you forget to close the tag and everything's position gets screwed up and you think ... oh if only I'd used CSS, but there wasn't time and anyway you have twenty pages the graphic designer made by copying and pasting in Dreamweaver (oh BAD Dreamweaver) and now the client wants them all changed and if you were working with a content management system you'd have put some common things into includable subtemplates but you don't have that and you think why am I watching the grains of sand draining through the hourglass of my life doing the same mechanical transformations on dozens of static html and xml files and you know the customer will only want them changed again next week .... aaaaiiiighhh why can't html be like a proper programming language instead of this ???? WHY GOD? WHY??



2) The generic solution.


Yes, a programming language. That would have the requisite controls over abstraction that I need. And a decent IDE. If only writing HTML files could be more like a programming lanaguage.



3) Our solution is your solution.


Fear not because GeekWeaver will solve all your problems.

All your pages are kept in one file, which you can navigate around with the keyboard in the comfort of your favourite OPML editor. You never have to close an HTML or XML tag again with the .<tag> notation that takes advantage of the outline structure to know what blocks are meant to be inside what blocks. Never, ever copy and paste when you can define your own re-usable, parameterizable blocks at any level you like including outside the level of a single page ... that's right folks, you can define a reusable component which actually evaluates to a number of pages or other output files.



4) Accept the price


Think ... just think ... of the hours of your time that could be saved if your static web-pages (and everyone has a few tucked away somewhere - remember that old departmental intranet-site no-one's been updating in years? Remember that customer who needed to access the documentation off-line?) were all safely in GeekWeaver format? How much would that be worth? (And at 50 quid an hour you're underselling yourself, but hours of tranquility can be bought for a mere ... well, let's see download and install Python 10 mins on Windows, probably already have it on Linux and Mac. OPML editor? 5 mins on broadband. GeekWeaver ... 2 mins?



5) Act now!


Get it today! because it's Saturday and you know you'll be too busy wasting time manually editing HTML once Monday comes around. And if you're an early adopter, and send me emails, then I'm more likely to listen to your requests and stuff ...

An idea I had, over at Joel's Business of Software :

Here's a thought.

I was re-reading this a couple of days ago :

http://www.joelonsoftware.com/articles/fog0000000348.html

as I'm trying to persuade the overworked developers in the place I work to take some time out for this kind of cleaning.

And it occurred to me that there could be a market for specialist code-cleaners. After all, you have specialist office cleaners instead of making the clerical staff take the Dyson around after work. Why not somewhere you can outsource legacy code to simply be carefully and lovingly refactored and polished without any claim that this company are doing new development?

Normally companies from rich countries imagine they can bang out a few UML diagrams, ship them off where labour is cheaper and have it coded up. We all know that this is a fallacy (coding and designing are too tightly entwined).

But could "take my OK but ugly code, with these existing unit tests, and return it with identical behaviour but all the HTML tags closed and the variable names consistent" be a more viable way of dividing labour?

Anyone know of anyone currently offering this? Could "code cleaning" be a small service business? Maybe even part-time for people who have a few hours free each day to work from home, know how to code, but can't commit to a full development job?

I've done a slightly improved GeekWeaver screencast.

Still very sketchy, and without audio, but the SWF is smaller, and shouldn't crash.

It just concentrates on the basics :

* making several pages
* using the .tag notation with outline indenting to save on closing HTML / XML tags
* defining a reusable and parameterizable block (in this case, page template) and calling it for each page.

I'll try to get the next one done in a few days, with some more surprising capabilities.

I've been struggling with Python's distutils over the weekend, and I'm finally making some progress. There's a new GeekWeaver page and a new release.

The page is part of a revamp of my personal site, which is now being built with GeekWeaver. And the release is the first built with distutils.

I was hoping that it would put the code somewhere on the PATH so that you can include the GeekWeaver modules from anywhere, but that doesn't seem to be happening. So although the code goes into "site-packages" I'm not sure what that buys you at present. Any Python experts able to shed some light on this?

What's definitely happened is I tidied up the whole directory structure. Unit tests and library modules are in sub-directories, and you can run the program from wherever you like as long as you give the right path to the gwMain program.

If you go into the "examples" directory in this new distribution, you'll see some examples of .bat and .opml files. You should be able to run the .bat directly there. You can put your own projects in their own separate directories using this as a model.

As always, I'm keen to have feedback, questions and suggestions for GeekWeaver. It's starting to come together although I also need some proper docs and screencasts to really show what it's capable of. The example files show the most basic (but most useful) features of GeekWeaver, the fact you're using an outliner for your whole site, the fact you can manage multiple pages within the same file, the way you can use .< notation to save yourself having to close tags, and the definition of re-usable, parameterisable blocks.

There's a more complex example of GeekWeaver's power in the tests sub-folder. It's not that well explained yet, but it shows how you can produce tables, how you can map a block-call to a sub-list of items, define blocks which are bigger than a single page, and pass block-names as parameters to other blocks.

I see Zbigniew beat me to blogging about wiki-wednesday. That's one of the perils of laying around in bed all morning.

He's dead right about Alan Wood's presentation being very exciting. I'm probably even more impressed by Wood's "Rel3" than Zby is.

What most impressed me was Alan's apparent good taste that seems to be formed by real experience. Rel3 isn't an attempt to take a particular genre of social software as it exists and force it into the enterprise, but a crafted, pragmatic response to particular needs.

Yes, it looks very like a non-threaded discussion forum. In fact it's also a bit like a page of "River of News" for a single issue being handled within a group. There are echoes of TopicExchange and explicit inspiration from Twitter.

Behind the scenes things seem just as interesting. Written originally in Java, then ported to Rails and now to Erlang. It's designed to use Amazon's S3 and EC2. (The fact that it keeps data in files on S3 prompted leaving Rails.) Each page (or conversation) allows files to be attached so you can manage Word, Excel, PDF docs, etc. When asked if he's thinking of creating some kind of Ajaxy online editor for collaborative documents, Alan sensibly says he won't re-invent the wheel but will provide hooks to other providers of this kind of stuff.

In other words, this is a very nice, minimal "glue" to tie together the people and resources that temporarily need to be brought together to solve a particular problem. After which the page doesn't hang around cluttering up your ontology when you have lost interest in it. (I mean, it probably *does* hang around. But Alan Wood, interestingly, contrasted Rel3 with wikis which allow a small group to work producing a more permanent document that must stake its claim to some part of the name-space.)

In this, it has some of the best features of other social software, in particular, I think it has potential to rival email's special power of spontaneous group forming and dissolving. Let's see if actually creating, subscribing and unsubscribing to pages can be quick enough.

Also, Alan has a very smart blog at Folknology

Know what QEDWiki reminds me of?

Visual Basic.

And I mean that in a good way ... ;-)

IBM's QEDWiki looks pretty nice.

This is really taking wiki in the direction of application development tool that I've been dreaming about for ages.

Fairly crummy screen-cast of GeekWeaver basics (no sound, sorry)

Need to find a decent screen-recording software. Anyone got any suggestions?

More soon ...

OK ... everyone who's wondering why I've been so quiet and apparently unproductive lately ... very soft (in little more than a whisper) announcement ...


GeekWeaver is a mutant cross-breed of outlining, wiki-attitude, templating and some duct-tape that would like to be a programming language ...

tell you more soon ...

Listen everyone, I gotta come out to you all ...

I am now officially an outliner.

For a long time I thought that outlines, like all hierarchical documents, were limited and inferior to graph-shaped wikis.

Now I get it.

The point of the outliner is not the hierarachical structure as a navigation aid - free-form hypertext is still superior.

No, the point of the outliner is the collapse which lets you manage and manipulate bundles of items at the same time. That's something I never managed to get right in SdiDesk. Although I perceived the need for a "PageSet" to create a bundle that could be used for, say, exporting etc. I a) never got that working technically, which was partly because b) I never really made sense of it "conceptually" to myself.

What's great about the outliner is its "scale-free" / "fractal" / "recursive" / "self-similar" nature - which means the same operations (collapse, copy, move, publish) can work on anything from a short list, to a chapter to a volume composed of multiple chapters. I've really started to realize this over the last few months as I've used the OPML editor for more things that I'd have once used SdiDesk for.

Now, don't get me wrong. I still love wiki. It's still my favouritest type of software in the world, ever. And I still use SdiDesk every day. But now, I'm starting to appreciate that there's a need to manage a hierarchy of scales, and until I find out how to combine that with wiki-nature (and into SdiDesk), I'll probably be outlining most days too.

(In my day-job I also spend a whole lot of time with spreadsheets, but that's another story. SdiDesk was always meant to handle grid-shaped data, it just wasn't developed enough to be really usable.)

Sig :

You talk to one person. You lunch with one purchaser. You present for one or more, same Powerpoint. Why the difference? It's all about people and a person is singular, always.

David Brin has a nice post summarizing his vision for "what we need from science" :

Given the daunting range of problems and opportunities that we face, I'd have to say that our most urgent scientific and technological need is to develop better methods for problem-solving.

Some pieces to the puzzle are already getting attention. Governments and big institutions are developing ways to combine sensor meshes and data mining with powerful analytic and projection tools. But this emphasis on centralized or professional-level anticipation ignores the other half of the solution -- generating a resilient citizenry. A populace so knowing and capable that all problems get noticed and addressed, quickly, by a billion eyes

Great point!

What Brin understands is that the "collective" and the "individual" are not in opposition but that smart, disorganized individuals can come together in dynamic, ad-hoc networks to solve problems.

Traditionally, we've relied on certain types of static "organization" (typically hierarchical and procedural) to achieve the benefits of acting collectively. These work. But at a high cost. They're often dumb (information flows badly through hierarchies because no one passes bad news up, and the top becomes a bottleneck); inefficient (participants waste their energy politicking against each other); and unpleasant.

In contrast, we want (and can start to seriously imagine being able) to retain our capacity for individualistic action while working together in loosely co-ordinated, ever changing but highly effective groups. The internet has taken us a long way in this direction, but as Brin points out, there's still room for more scientific understanding and other tools to take us further.

Q : Phil! Why have you put a Grazr widget in your gutter?

A : Something's happening. And it's about OPML, Grazr, and widgets ...

;-)

Q : Oh, and I noticed you changed the template.

A : Er ... yeah. The look didn't go with the widget. I wanted something that was more, I dunno, evolvable.

Q : What are you up to, exactly?

A : You know, general house-keeping, wanting to learn about new things. The truth is I've been using the OPML Editor a lot recently. Even tried to write a several thousand word essay in it.

And it's good. Perhaps I'm getting the outlining religion ...

Cool video showing WikiCalc embedding YouTube. And showing several evolving lines of thought in the multiple texts being created.

Bleaahhhgh! Least inspiring Hughtrain cartoon evah!

There's a new crop of business wikis. And some interesting discussion.

But what's wrong with this picture?

SystemOne, an enterprise-knowledge-management system masquerading as an ordinary business wiki. What's cool about this product is that it automatically creates, at the bottom of each page, a list of relevant other wiki pages, feeds, and Web search results. The autocreation of the links removes some of the need to manually create links to connect wiki pages together. This is a key feature if the wiki is to be used by a lot of people who aren't hypertext-savvy.

Answer : Everything is wrong here.

Firstly, what's the point of automatically making hyperlinks for people who aren't web-savvy? Or rather, why are you trying to get people who don't understand hypertext to use wiki? (In fact, in 2007, why are you even employing people who aren't hypertext savvy? But that's another story.) At the very least you should ask how exactly they are going to use wiki if they can't understand what hypertext is, or have some intuition about how to use it?

Remember, this is wiki we're talking about : all the hard graft of making links (like the trivially fiddly writing of "anglebracket a href", and the genuinely tricky part about finding the right URL to point to) has been eliminated by the simple WikiWord or [[double square bracket]] conventions. What's left is the only other difficult question : deciding what links to make.

SystemOne "solves" that part for you. But remember, the links are part of the valuable decision-making and information that goes into your wiki. Wiki links are not meant to mean that "this page is vaguely similar to that web-page". If I want to know what pages are vaguely similar I can use Google. Wiki links mean "I, the author, want the reader to notice this kind of similarity between this page or this paragraph or this word, and that page because I think it's significant." Which is far more precise and subtle piece of information.

A wiki which automatically creates links is as useless as a word-processor which claims to write your letters for you.

The purpose of creating documents is not to kill trees or fill up as much disk space as possible. The purpose of creating documents is to capture the value added by human intelligence. Attempting to automate away that, in order to help consume more wood-pulp or fill the screen with more blue-underlines is counterproductive. Bad links devalue the good links. The reader of a page is overwhelmed and confused.

Joel Spolsky on a good hack :

I think what makes a good hack is the observation that you can do without something that everybody else thinks you need. To me, the most elegant hack is when somebody says, "These 2,000 lines of code end up doing the same thing as those 2 lines of code would do. I know it seems complicated, but arithmetically it's really the same." When someone cuts through a lot of crap and says, "You know, it doesn't really matter."

For example, Ruby on Rails is a framework that you can use with the Ruby programming language to access databases. It is the first framework that you can use from any programming language for accessing databases to realize that it's OK to require that the names of the columns in the database have a specific format. Everybody else thought, "You need to be allowed to use whatever name you want in the database and whatever name you want in the application." Therefore you have to create all this code to map between the name in the database and the name in the application. Ruby on Rails finally said, "It's no big deal if you're just forced to use the same name in both places. You know, it doesn't really matter." And suddenly it becomes much simpler and much cleaner. To me, that is an elegant hack—saying, "This particular distinction that we used to fret over, just throw it away."

When I wrote SdiDesk I explicitly decided that I wanted many small, hand-layed-out diagrams which are hyperlinked together, rather than automatically generating a large diagram. (Of course, it helped that that was the lazy thing to do, in terms of coding :-)

Manually drawing diagrams doesn't help with *discovery* of new connections or of other structure but, for me, it's the right way to capture some of the structure in *my* thinking. I tend to create diagrams with around 10 (15 max) nodes where I might be deliberately representing a hierarchy or a supply-chain or some other small chunk of the world. And I can arrange the nodes in a way that makes sense for that kind of thing. If I really need more nodes, I probably want to break it up into an overview page and sub-pages with details. Of course, SdiDesk nets are more primitive than even I want, visually (so yeah, I am thinking about how to improve on that in my next opus) But I prefer them to Buzan-style mind-maps where I find most software forces me to build one "butterfly" shaped hierarchy. Or to something like TouchGraph which is very clever but actually not much use when you want to get acquainted with the shape of your information.

3-D web of idea nodes : TiddlyWiki ... - Smart Disorganized Individuals - tribe.net

ZenPundit on Super-empowered individuals

Wiki-based software for the enterprise

SocialText 2.0 is out.

Check out the screencast. Some nice work on the UI.

Here's what looks good to me :

* the automatic backlinks

* deciding you can be "expert" (an explicit recognition that SmartAscii mode corresponds to expertise),

* watched pages,

* the sortable table views of RecentChanges, watched pages and other queries. (I like this very much.)

What I'm still ambivalent about :

* "tagging". Why are tags special? Why are they not simply a convention on top of full-text search (as in CategoryCategory)? Maybe it's simple optimization. Faster to search an "index" of explicit tags. Maybe the flickr / deli.cio.us folksonomic culture is too deeply ingrained to fight?

* they haven't quite got there yet with blog integration. As I'm always banging on about, the tough problem with wikilogs or blikis is that you have two incompatible addressing schemes : pagenames and dates. Deciding which addressing scheme you need for any particular item of information is difficult. As SocialText 2.0 allows everyone in a group to edit blog-posts as wiki pages, blog-items can blossom into full pages. What's not clear is how you can link to a particular blog-item, or how such an item could migrate out of the context of a specific conversation and become a more independent citizen of the wiki.

As always, the problem isn't technical but one of shared concepts. If you have two types of objects with distinct addressing schemes : temporal and permanent, how do you make sure that you and the rest of the community understand which information is of which type? Are people putting it in the right place? Are they looking for it in the right place?

I don't have any answers. But I'll only really get excited about a wiki that *also* provides blogging when someone finally does find a nice way to help the users navigate this conceptual gap. Otherwise, you might as easily use two different products. (Not trying to beat up on SocialText here. I'm just saying that this in one of the grand unsolved problems in the area, and they don't seem to have solved it yet.)

Why I'm still on the edge of my seat :

* WikiCalc? What's happening here? When will grids become first class citizens of the SocialText product?

Note here that conceptually, grids are much less trouble to integrate than blog-posts. Yes, they're technically very different from ordinary pages. But addressably they're the same kind of animal. Individual cells are perhaps more problematic, although the spreadsheet convention is very strong.

Given that Chris Dent's at SocialText, and EEK's involvement, do they (still / yet) have purple numbers in the SocialText wiki? Perhaps they do but that's too geeky for this screencast.

In fact we may see SocialText having to come to grips with questions about integrating four different addressing schemes : page-names, purple-numbers, cell-addresses and blog-items. I suspect proliferation of incompatible addressing schemes will be one of the major issues faced by wiki as it starts to take over the enterprise.

Making business beautiful

Very interesting.

Now HyperScope has been released, don't miss Brad Neuberg's great Screencast of Douglas Engelbart's Augment System

How much was SdiDesk influenced by Engelbart? Directly, not so much. But indirectly, obviously a lot. My direct influences were wiki, the Smalltalk environment, Emacs, HyperCard and the web-browser. Obviously, all, heavily indebted to Engelbart's work.

Great idea for a productivity hack.

Brudnopis: Mailing Lists and WWW

Dave Winer says he made 2.3 million with his blog, without advertising.

What I think he means (as I explain to Bill Seitz, here) is that he sold the weblogs.com server last year.

For Winer, the blog is a full media and communications strategy. It’s how he does his personal branding, hypes his projects, and most importantly makes his connections with important users and developers.

Without Scripting News Dave wouldn’t get invited to the right kinds of parties and conferences, wouldn’t have people working with OPML, or reading River of News, or his podcast directories etc. Without Scripting News how would he (or anyone else) understand that there was even an opportunity called “weblogs.com” or what form it should take?

Update : Somehow this reminds me of another Steve Pavlina article : the $10,000 hour.

Not in the letter, of course, - Pavlina's tactics for commercializing his blog are antithetical to Dave's - but in the spirit.

Both eschew mere optimization of the normal microchunk in favour of the hard-to-measure, riskier, higher potential stroke of genius. For Pavlina that means merely increasing your hourly rate by gradual, evolutionary steps is a bad deal if by doing so you miss out on the $10,000 flash of inspiration. Winer says the same for merely monitizing eyeballs rather than making the connections, and having the conversations, that lead to the next weblogs.com.

I like Steve Pavlina's list of 10 Stupid Mistakes Made by the Newly Self-Employed

Stowe Boyd interview : The individual is the new group

WTF!!! I've been Googling for references to SdiDesk, like, forever, but I've never seen August Agricola before today!

Wow! The guy has an SdiDesk mod that looks beautiful.



And sounds like he's done a tonne of cool stuff with it. Damn! I wanna see more!

Good point too, about SourceForge. The stupid thing is I even have an account ... just never got round to uploading it ... doh!

Here's more from August.

SdiServer screengrab
Originally uploaded by interstar.

Don't get too excited. This is just eye-candy at the moment. :-)

But it is what you think it is ... a page from an SdiDesk PageStore served by a web-server (I'm using web.py), and viewed in Firefox with some cute graphic tabs.

Hey, guess what ...

SdiDesk rocks!!!

Seriously.

I've been working, on and off, on the Python translation and making some progress, but the whole VB code-base had gone stale for me. And, although I've been using SdiDesk for the odd notes, I've been dabbling with other things like the OPML Editor and writing more on online wikis.

But this week I've started a new job in a fairly corporate environment. I've been expected to get up to speed on a lot of things fairly quickly. This is a local division of an international company, selling one component in a larger enterprise system. In the first three days I've had to get an idea of where our place is among various partners, suppliers, customers, systems integrators etc; I've had to get a picture of a range of different products and modules and libraries produced or consumed by different players; I went out yesterday and watched as someone upgraded the installation of our product on-site with one of our partners and will probably be expected to do it by myself the next time. SdiDesk has been absolutely, amazingly, invaluably useful as a place to dump all the information which is being thrown at me.

I've used most of the features : drawn a quick network diagram to sketch out our position in relation to our partners, clients, their clients etc. etc. And as each box in the diagram links to its own page, I can add other information about the parties : web-sites, contact info etc. I've drawn similar network diagrams to show the relations between the various tiers and modules in the software. My first drafts are often wrong but the diagrams are easily modified as new data comes in.

I've written out the steps to installing the software, using nested bullet lists. And, naturally, cross-linked to the modules when relevant. I can throw in login names and passwords. I've used footnotes, and tables, and hyperlinks to both online documentation and the actual web-based interfaces to the system. I'm confident that SdiDesk will become the dashboard that I will work through.

I've also seen its many flaws. SdiDesk has many rough edges, bugs, abandoned and unfinished roads, and is distinctly ugly. But, frankly, I haven't been this excited by, this in love with, it since the release of 0.2. I'm definitely back into it in a major way. (Of course, I don't have any time :-)

So if you don't know what I'm talking about because you haven't yet checked out SdiDesk; but you work in an environment where you need to cope with a lot of information including information that is list, table and graph-shaped; and you're a smart, if a little disorganized, individual; then you know what to do.

Yeah, there are still dozens of things that suck or are aesthetically challenged. Yep, I'm a slow and inattentive developer. But, to misquote Tati Quebra-Barraco, SdiDesk "is ugly but it's fashionable".

(And works off your pendrive.)

I notice Eastgate do stationary.

Is this the future for GTD / personal organizer / hyper-text companies? Give away software, sell paper based products?

Here's an interesting blog : Web 2.5

With some interesting ideas about pocket and pen-drive wiki-servers etc.

The Always-On-You Web: Web 2.5: Pocket Wikis in Sync

The Always-On-You Web: Privacy Promotes Productivity

Can you run SdiDeskOnAPenDrive?

Of course.

Hmmm. I wonder if there's a market for a generation of software aimed at pen-drives. As a complement for server-based web 2.0 stuff.

Another wiki-oriented blog : Wiki That!

Brilliant graph of user happines and feature creep. Well worth bearing in mind.

Recent discussion between Albert and me. Albert highlights what SdiDesk does, and what he wants it to do. I give my latest, still evolving plans, for future SdiDesk development. Yes, Python is the future.

(NB : it's a long page that needs refactoring. Scroll down.)

Here's a cool looking blog dedicated to wiki :

RecentChanges.info : Understanding Wikis

Hey! It wasn't me. But someone made an Sdidesk page on Wikipedia. !!! :-)

Paul Graham's Startup School and wiki

LOL! Dave Winer on DMOZ vs. Wikipedia

DMOZ has a bad rep for having editors with conflicts of interest. And it's exclusive, unlike Wikipedia which at least has battles (never thought I'd say that) of people with conflicts, DMOZ doesn't even have dissent among conflicted people, only one point of view exists, because there's only one editor for each category.

(My emphasis.)

Scripting News: 3/6/2006

Interesting to note how Dave is spinning against the DMOZ hierarchical catalogue (I never heard of any repution for conflict of interest that he claims, though it's plausible) just as he's about to launch a revolution in online hierarchical directories. :-)

Stowe Boyd : The Individual Is The New Group -- Part 1

which kind of reminds me of all those ideas about PersonalKnowledgeManagement that were floating around last year.

Mitch Kapor :

Wikipedia is the next big thing.

Ross Mayfield has more details

WikiMatrix / Wiki Feature Comparison - Compare them all

Quick comment I made about personal timeliness vs. personal individuality Blogger

100% right.

At some point we realize that some of our information consumption has to be a "minority game".

The winners will be those who know things that other people *don't*. That's what makes you valuable to other people. Being an expert in the top ten most known topics makes you worth what?

By definition, there's no algorithm which can give this to you. You need to cultivate your own tastes and ideosyncrasies. Your outlier blogs.

Don't worry about being the first to follow the fashions. Something which is number one on memeorandum or digg today (and is actually important) will probably turn up on one of your regular blogs within a week. So decide how you'll trade-off your own personal timeliness for personal individuality.

This isn't a problem. It's the solution.

Politicians who try to spin their wikipedia entry are getting named and shamed.

Latest news copied from the ShowStoppers page

I'm still thinking, more and more seriously, about rewriting at least the wiki-to-html engine in Python.

I'm also seriously thinking that the next release will actually be a complementary "web-server" edition, which allows you to edit pages via a browser (over the web) rather than through the VB program.

Right now I have a simple webserver application which let's you access the SdiDesk pagestore via the browser over HTTP.

Of course, this raises questions about security which will have to be answered before release. Will there have to be a login? Can you mark some pages as personal and private, while others are public, etc?

If I go 'browser only' I'll initially lose the network diagrams. (Would anyone miss them?) Although it may be possible to put them back for the more recent Firefox using SVG / Canvas tags.

None of this is ready yet. But it is looking promising. And the main reason is that the new code is in Python which will allow me to work on a proper Python based rendering engine. (Where I should fix all those bugs.)

If the Python rendering engine works out, I'll then try to retrofit it behind the current VB front-end.

Thoughts anyone?

The Fishbowl: In Defense of Wiki Markup

David Weinberger has an excellent piece on wikipedia.

JOHO - December 29, 2005