Tweaking / Refactoring OWL / OWLdroid

A bit of tweaking and refactoring OWL and OWLdroid over the last couple of days. If you’re trying them out, thinking of doing any development on OWL, it’s worth updating.
What’s up :
– the main purpose of this work was to bring the OWL and OWLdroid versions of the code-base back together. They’d drifted apart in my first attempt to figure out how to get this stuff working as an Android app. Now all the directory structures are back in the same place. The variation between versions is largely pulled out into specific local_setup.js files and a certain amount of cruft, debugging messages etc. have been removed.
This is obviously going to make it easier to do common development moving forward.
– there’s a new “Quick” menu which has buttons for moving the node up, down, left, right, and deleting it. This is mainly useful in OWLdroid where the menu was the only way to move nodes and there was no way to remove them. Nodes still have to be selected into the white-on-black non edit mode before they can be deleted (a useful precaution). This is awkward on OWLdroid, because there’s still the problem that the font-awesome arrows that show open / closed state of the items aren’t appearing. But it is possible to select this mode and, therefore, now possible to delete nodes on Android. (which previously wasn’t the case.)
– In OWLdroid I’ve experimentally moved the nav-bar up to the main menu bar. I’m concerned that this will screw up the layout on small screens. (Please tell me if it does.) Though it’s more or less OK on my Nexus 7 and I’d assume any tablet or in landscape mode.
There is a BIG advantage of doing this. The nav-bar controls no longer scroll off the top of long pages. So you can easily go back / forward or create links however far down the page you are. It’s definitely more usable for me.
I haven’t made the change in normal OWL because I think the cost / benefit calculation re: scrolling / layout problems probably works out a little differently. But if I get requests, then I’ll do it.
As a side-effect, it’s also removed something that was a catastrophic trap for the unwary in OWLdroid. The default bootstrap layout has a “brand” at the top left, which is a link back to the main page. When this was left in the Android version, if you clicked on it, it would cause weird behaviour. (On my tablet it would try to open the file system) That was confusing and hard to get back from. The branding link is now gone so this is no longer a danger.
– I’m still chasing down the Safari bug. It’s basically something to do with the way I handle requests for non-existent pages. The code which picks that up and deals with it on Firefox throws an error in Safari. The bug is appearing in jquery code, but it’s obviously because a null value is getting in somewhere. Just have to figure out what I’m relying on and make it an explicit test.

Questions for 2014 : #2 Project ThoughtStorms

Second in a series of questions occupying my mind at the beginning of 2014. Which may (or may not) inform what I’ll be working on.
2) What About Project ThoughtStorms?
Unlike the first question in the series, this one is relatively light.
It’s obvious that OWL is the new SdiDesk. And where my private personal organizer / wiki-notebook etc. work should now be focussed. It’s equally obvious that it’s not, in itself, a publication medium. Smallest Federated Wiki is still looking good for that. SFW also has some great features : federation being one. The embedded media types being another. It’s actively improving. And looks like it’s become the “official” wiki in the node.js library. ThoughtStorms isn’t moving off SFW in the foreseeable future.
So Project ThoughtStorms is still, largely, about tools that support SFW. At the same time, this dichotomy between one tool for writing and one tool for presenting is a little bit … disconcerting. It’s natural in the outlining world, of course, a separation between outliner as authoring tool and blog / slide-show / book as rendering. But in the wiki world, where writing / editing / reading are all blurred together in a kind of closed loop, it feels wrong.
Nevertheless, there’s little real question there. In 2014 OWL is my writing tool. SFW is my public thinking space. The open question is about what bridges to build between the worlds.
Should OWL export complete SFW pages? (Easyish, I think). Should it import them? (To be consistent with my promise not to abandon SdiDesk users?) Should I try to get it to speak the federation protocol of SFW? Where should I be capturing quick notes and draft thoughts that aren’t (yet) ready for publication?
No great soul-searching here. But a bit of quizzical head-scratching required.
(Next question coming soon.)

Questions for 2014 : #1 Mind Traffic Control

I’m not going to be making predictions for 2014. Or promises that I may or may not keep. Or even resolutions. At least, not public ones : who knows what crazy trajectories my life will take this year?
But I do have a couple of questions. These are the things occupying my mind at the beginning of 2014. And which may plausibly inform what I do.
1) What’s the relation between OWL and Mind Traffic Control?
There’s little doubt in my mind now. OWL is great. It’s the future. At least, as far as my private notebook is concerned. And that means it’s now my de facto to-do list manager as well.
Which is kind of embarrassing for Mind Traffic Control.
MTC does have some virtues :

  • firstly, a kick-ass name that I love. “Mind Traffic Control” is the best name ever! For personal organizer stuff.
  • it’s solid. It has remembered everything I put into it from the moment it was written. It’s all still hanging around to haunt me. (Even if I kind of wish some of it wouldn’t)
  • the task delegation is pretty sweet. I don’t know anything else as simple as MTC for delegating tasks to other people and tracking what you delegated. To be honest, I don’t do a lot of delegating tasks to people. But if I did, I’d want to use MTC.
  • the deferral part works as advertised and it’s a nice feature for a task-tracking app.
  • MTC is slicker than you remember, if you haven’t looked at it for a while. In 2013 I did a bunch of Ajaxing it up so it’s now sending more little json fragments around and doing a lot less recreating the page.

But it also has big problems.

  • It’s not OWL. (See above. Doh!)
  • There is a major impedance mismatch between queues and the ORM-style storage system that Google App Engine uses. Moving items around on the queues seems to be really inefficient. To such an extent that whenever I do a burst of development / testing on the live server, Google kick me off for going over my quota of data queries. Querying also seems to take longer than it needs.
  • It’s on the GAE cloud, at a time when I and, I think, many others are becoming wary of storing their personal / private data on servers belonging to large corporations under the jurisdiction of the NSA. And at a time when I, myself, am seeking to become less entwined with Google and its technologies.
  • While the UI has improved, it’s still not as slick as it ought to be. It still needs a lot of work to look as good or work as well as modern web-apps.
  • It never really caught on. People try it, some have used it off and on over a couple of years, but I don’t think I have any long-term users except me.

So if I look deep into the abyss and am honest, MTC is a failed experiment. Whatever its theoretical virtues.
And in 2014 I think there’s a straight choice :
Either admit to myself that MTC is no longer a going concern. Perhaps kill it entirely. (Like I did with Platform Wars last year.)
Or, to use a fashionable term, pivot the site to become something else. Probably something with OWL at its core. I don’t quite know what that would mean. Maybe doing the hard work of remodelling the queue data-model (use localStorage? synchronized to large BLOBs on the server?) and then trying to put OWL / Concord on the front-end of that? Maybe just a one-time export of MTC data? (Remember, MTC has exported to OPML since its inception.) Maybe becoming a GAE hosted farm of OWL wikis? If so, what, if anything, of the original queue and delegation model could / should be preserved / recreated?
Today, I only have these questions rather than answers. But this year, I’ll be trying to resolve them one way or another.
(I’ll post the next of my 2014 questions tomorrow. Maybe.)

Project ThoughtStorms Wikish Plugin Update

There’s a quick update to the wikish plugin for Project ThoughtStorms (that’s the plugin type that offers UseMode / SdiDesk style wiki markup for the Smallest Federated Wiki). I’ve now added the double-comma defined tables.
oranges,, apples,, pears
43,, 55,, 22

Now becomes

oranges apples pears
43 55 22

Although not nearly as elegant. The simple CSS styling I was trying to use seems to clash with SFW’s style sheet in a way I haven’t debugged yet. So I’ve stuck with good old-fashioned table borders. (Ouch!)
Given the SFW’s narrow columns, I wasn’t originally going to support tables. But I have a couple of things for which it makes sense. So if you’re using Project ThoughtStorms, the repo is already updated. ThoughtStorms itself I haven’t updated to the most recent SFW codebase. That’s coming ASAP.


My software is more or less like Cthulhu. Normally dead and at the bottom of the sea, but occasionally stirring and throwing out a languid tentacle to drive men’s minds insane. (Or at least perturb a couple of more recklessly adventurous users.)

However there’s been a bit more bubbling agitation down in R’lyeh recently. The latest weird dream returning to trouble the world is GeekWeaver, the outline based templating language I wrote several years ago.

GeekWeaver was basically driven by two things : my interest in the OPML Editor outliner, and a need I had to create flat HTML file documentation. While the idea was strong, after the basic draft was released, it languished. 

Partly because I shifted from Windows to Linux where the OPML Editor just wasn’t such a pleasurable experience. Partly because GW’s strength is really in having a templating language when you don’t have a web server; but I moved on to doing a lot of web-server based projects where that wasn’t an issue. And partly, it got led astray – spiralling way out of control – by my desire to recreate the more sophisticated aspects of Lisp, with all kinds of closures, macros, recursion etc.

I ended up assuming that the whole enterprise had got horribly crufty and complicated and was an evolutionary dead end.

But suddenly it’s 2013, I went to have quick look at GeekWeaver, and I really think it’s worth taking seriously again.

Here are the three reasons why GeekWeaver is very much back in 2013 :


Most obviously, Dave Winer has also been doing a refresh of his whole outlining vision with the excellent browser-based Fargo editor. Fargo is an up-to-date, no-comprise, easy to use online OPML Editor. But particularly important, it uses Dropbox to sync. outlines with your local file-system. That makes it practical to install GeekWeaver on your machine and compile outlines that you work on in Fargo.

I typically create a working directory on my machine with a symbolic link to the OPML file which is in the Fargo subdirectory in Dropbox and the fact that the editor is remote is hardly noticable (maybe a couple of seconds lag between finishing an edit and being able to compile it).


What did we do before GitHub? Faffed, that’s what. I tried to put GeekWeaver into a Python Egg or something, but it was complicated and full of confusing layers of directory.  And you need a certain understanding of Python arcana to handle it right. In contrast, everyone uses Git and GitHub these days. Installing and playing on your machine is easier. Updates are more visible.

GeekWeaver is now on GitHub
. And as you can see from the quickstart guide on that page, you can be up and running by copying and pasting 4 instructions to your Linux terminal. (Should work on Mac too.) Getting into editing outlines with Fargo (or the OPML Editor still works fine) is a bit more complicated, but not that hard. (See above.)


Originally GeekWeaver was conceived as using the same UseMod derived wiki-markup that I used in SdiDesk (and now Project ThoughtStorms for Smallest Federated Wiki). Then part of the Lisp purism got to me and I decided that such things should be implementable in the language, not hardwired, and so started removing them. 

The result was, while GeekWeaver was always better than hand-crafting HTML, it was still, basically hand-crafting HTML, and maybe a lot less convenient that using your favourite editor with built-in snippets or auto-complete.

In 2013 I accepted the inevitable. Markdown is one of the dominant wiki-like markup languages. There’s a handy Python library for it which is a single, install away. And Winer’s Fargo / Trex ecosystem already uses it. 

So in the last couple of days I managed to incorporate a &&markdown mode into GeekWeaver pretty easily. There are a couple of issues to resolve, mainly because of collisions between Markdown and other bits of GeekWeaver markup, but I’m now willing to change GeekWeaver to make Markdown work. It’s obvious that even in its half-working state, Markdown is a big win that makes it a lot easier to write a bigger chunks of text in GeekWeaver. And, given that generating static documentation was GeekWeaver’s original and most-common use-case, that’s crucial.

Where Next?

Simplification. I’m cleaning out the cruft, throwing out the convoluted and buggy attempts to make higer-order blocks and lexical closures. (For the meantime.) 
Throwing out some of my own idiosyncratic markup to simplify HTML forms, PHP and javascript. Instead GW is going to refocus on being a great tool for adding user-defined re-usable abstractions to a) Markdown and b) any other text file.

In recent years I’ve done other libraries for code-generation. For example, Gates of Dawn is Python for generating synthesizers as PureData files. (BTW : I cleaned up that code-base a bit, recently, too.)

Could you generate synths from GeekWeaver? Sure you could. It doesn’t really help though, but I’ve learned some interesting patterns from Gates of Dawn, that may find their way into GW.

Code Generation has an ambiguous reputation. It can be useful and can be more trouble than it’s worth. But if you’re inclined to think using outlining AND you believe in code-gen then GeekWeaver is aiming to become the perfect tool for you.

Personal Question

Question : Hey Phil, do you actually do any programming these days?

Answer : Yes. Quite a lot at the moment. Though it’s a bit all over the shop.

I’m dipping a toe into Android programming. (And, hmmm … Java …. I thought I’d got over my Java hangups by doing a lot of Processing, but it turns out that Processing just hides the crap and Android doesn’t. Why hasn’t Google picked up on Processing to turn it into a first-class Android art / game app. development environment?)

I’m mainly writing CoffeeScript. Some stuff related to my ongoing 3D modelling / desktop manufacturing projects. (Did I forget to mention those? I’m sure there’s a half-written blogpost somewhere.) Some work towards an SdiDesk-derived network diagramming plugin for Smallest Federated Wiki (held up by silly problems). Some other bits and pieces. I’ve recently been playing with Jison, which rocks. And I’m about to investigate angular.js which looks pretty good.

There’s a project for small stand-alone web-servers that I’ll talk about more if / when it takes off.

I’ve been trying to compile example VST instruments  (C++) for some of my work with the Brasilia Laptop Orchestra, but it’s driving me crazy. (I may go back to Pure Data which can be embedded in a VST.)

A bit of PHP, just simple small web-services.

I’m going to be teaching an Arduino course soon. So I’ll be writing a bit of C and I want to try Occam-.

I’m still writing Python too. Mainly for short file transformation scripts or to prototype algorithms that later get translated into CoffeeScript.

Some of this stuff is headed for GitHub soon.