Why, yes. This is, indeed, OWL running on the PocketCHIP.

Here’s the story.

I spent 12 days recently, travelling in the south of Brazil and Uruguay border. And I decided, as an experiment, to see if I could live without my laptop. I wasn’t meant to be working, but I always like to keep some kind of SDI / MTC style software on me to make notes and generally think about my insanely long queue of tasks and all my other half-baked ideas. I feel lost without having access to these. And would usually take a laptop, just to keep them near to me.

This time, though, I decided to see how well I could cope with a “device swarm” of very small (and cheapish) tech.

Here’s what I took :

  • The PocketCHIP
  • My Android phone
  • A small, portable USB mouse
  • The rubber-tipped stylus.
  • A 5V mini USB charger that plugs into the wall.
  • An A5 sized paper notebook and pen.

On my trip, I also acquired a small bluetooth keyboard / mousepad that’s actually just a capacitative touch surface with letters marked on it. This impressed me a lot when I bought it at a Duty Free shopping outlet just on the Uruguay side of the Yaguarón River in Rio Branco for under $8. It charges via USB and feels pleasantly heavy and solid, while still pretty small.

Here are some things I found out.

I should note, first, that I’m also pretty much against keeping things on public clouds and other people’s servers. I increasingly want my data to be private, on systems that I control, and mainly synced between my own machines rather than using, say, my hosting provider.

Although I’m far from a satisfactory solution to that, it turned out that the discipline was useful in the sense that I wasn’t as persistently connected to the internet as I’d hoped. And so if I had depended on the cloud I would have been without access to my data more often than I had it.

I also, of course, want to use my own software. I currently have three distinct trajectories of development in this area.

  • Mind Traffic Control, racket-version.
  • The new engine behind ThoughtStorms wiki which is written in Python and uses the light-weight Bottle web-framework. I use this for both ThoughtStorms and a second personal wiki-notebook which is descended from my old SdiDesk notebook.
  • OWL

All these projects are still alive. And my notes and todos (and some more significant chunks of writing) are somewhat haphazardly scattered between them. So I wanted to see how well I could take them on the road with me with such minimal hardware.

Before leaving I updated the Debian on the PocketCHIP and installed the relevant libraries / environments. I already had MTC-racket running on it. And Emacs. Python was there too, but I needed to pip install a couple of standard libraries. Then I could install both Project ThoughtStorms and the Python-served version of OWL. (Spoiler alert : 2017  is going to be a year of consolidation between all these projects, particularly I’m aiming to unify the Python-OWL and Project ThoughtStorms servers into a single code-base.)

I didn’t try installing the Android version of OWL on my mobile, because of … er … reasons … which currently prevent me having a working dev / deployment environment for Android on my laptop. I hoped, though, that I’d be able to access the OWL server on the PocketCHIP from the phone.

Results

The PocketCHIP is a wonderful machine. (And, seriously, a Debian box for $9 just insanely amazing.) It seemed to cope just fine with simultaneously running three Python servers (2 copies of the Project ThoughtStorms wiki, and the OWL server) + MTC. (I just ran unix “screen” on the terminal and ran each in a separate screen.)

Obviously these were not being heavily accessed. (I was the only user). But I’m still impressed.

The weakness of the CHIP is its WiFi. It is very weak. My original thought was to run the servers on the CHIP and then access them from either my phone, or other computers in the places I was staying. But even where WiFi was available, the CHIP typically failed to establish a usable connection with the router.

The only time I could get anything else talking to the CHIP was by turning the phone into a hotspot and then placing that within 10 to 20 centimetres of the PocketCHIP.

This way I was able to access both wikis and OWL from the phone. I didn’t get to remotely ssh-ing into the CHIP to see MTC in a terminal from the phone, but this would have been a particular fiddle and it’s not clear that it would have been worth it. In fact, nothing worked particularly well. OWL’s web-interface is pretty much impossible to work with on a small phone screen. It’s OK on a 7″ tablet, but the phone is way smaller and too difficult to manipulate. And the HTML UI doesn’t zoom in any effective way.

Reading the wiki pages was slightly better. I was surprised, though, how badly the phone handled the fairly simple, static html. I accept that there’s very little fancy “responsiveness” in the TS wiki at the moment. I hadn’t realized how little Chrome would help. On my laptop, ctrl + and ctrl – work beautifully to scale text up and down, reflowing and refitting the text. I have no hard settings for font-size or spacing. The page ought to be easy to automatically resize to convenience.

But pinch zooming a TS page on Android Chrome is diabolical. Not only does it not scale and reflow the text in a useful way. It also seems to remember (or guess) arbitrary different zoom levels for different pages, so you jump from a readable page to another page with illegibly tiny letters to another with enormous text, most of which is off screen. And zooming the menu seems to be independent of zooming the main page text. The whole thing is horrible.

Firefox on the phone is a bit better. You can go into “accessibility” and turn up the text size to full. And then the defaults are reasonably readable and consistent on all pages.

Editing is trickier. And this is somewhat my fault, I have set a fixed number of columns in the text-box in TS wiki which is too large for a phone with either browser.

Overall, I’m disappointed with the phone experience. TS wiki is just about readable on Firefox. But it’s useless for doing any kind of work. And horrible on Chrome. OWL looks OK on both browsers, but is too fiddly to actually edit.

I need to radically rework the UI for both these projects.

So I went back to see if I could actually look at these web-served applications on the PocketCHIP itself.

It turns out that OWL works surprisingly well with the surf browser which comes pre-installed … as long as you use a mouse!

The touch screen even with a rubber-tipped stylus isn’t viable for navigating and editing an outline. But attach an external mouse to the USB and it’s surprisingly usable. The PocketCHIP keyboard isn’t great for a lot of writing, but for short items in an outline it’s viable.

TS Wiki is also fine to read. But for some reason the text-area is coming out black, with black text. (I’m pretty sure I’m not setting this explicitly, so I assume it’s a bug in surf.) You, therefore, can’t edit the wiki with it. But reading is an acceptable experience. Once again, a mouse helps, but you can just about get away with the stylus.

The bluetooth keyboard I bought paired fine with the phone. But with the PocketCHIP there were some issues with the mapping between some symbols. I couldn’t find any combination of keys to make a backslash for instance. And sometimes the shift wouldn’t work. Also the bluetooth connection kept dropping. I found myself continually swapping between the PocketCHIP’s own keyboard for typing short commands with a lot of non-alphabetic symbols, and the bluetooth keyboard to type paragraphs of text. It was just enough to get some work done, and the bluetooth keyboard was just better enough to make it worthwhile, but it wasn’t really a viable solution.

Aside

This trip I also got interested in Logic Programming and tried out the Python Minikanren library, logpy, on the PocketCHIP. Unsurprisingly, it worked as expected. But, again, that is kind of startling when you think of it.

Conclusion

Well, this stuff works. If the PocketCHIP just had better wifi / bluetooth connections, then it would be a serious possibility to do some work on. It might be that a software update fixes the power-saving that may be weakening the wifi.

But it’s not something that even I can use yet.

Some of that is in my hands, of course, one task is to tweak the TS Wiki UI to ensure that the text-area is readable in surf. That would at least mean that PocketCHIP could be used (if uncomfortably) for working on my notebooks.

It’s not all working yet … but it’s getting closer.

Posted in Me.

qzhuyan responds to me tweeting Emacs on PocketCHIP.

Which is, er, very true. And wonderful.

But this haunts me continuously, as I explore the Mind Traffic Geometry of tools that support me tracking tasks and outlining ideas.

Will I one day end up simply falling into Emacs Org Mode? Isn’t that basically everything I really want?

Am I wasting my time with quixotic effort of writing my own software for this stuff when I could be writing something newer and more important?

Another thing that’s pushing me to think about this : this week I’ve been playing with Faust. A wonderful language for writing signal processing networks (ie. synthesizers, audio FX etc.) that compiles to multiple back-ends … including PureData, Supercollider, VST plugins and stand-alone programs.

It’s basically where I imagined Gates of Dawn eventually going.

But rather than a Python library, its a very nice “little-language”, with great operators for describing composition of data-flow blocks. It’s well developed and supported. I’m trying it out for writing small synths / FX units I can run on small boards like CHIP and Raspberry Pi.

I can see myself doing a lot with this. But it’s basically going to kill Gates of Dawn. Maybe there’s room for a Python library for those who don’t want to learn Faust. But for me, Faust is looking extremely viable.

So … another wasted project?

Perhaps I need to look at this positively. I’m not old. But I’m not as young as I used to be. I don’t have so many projects left ahead that I can afford to squander them. Perhaps its time to pivot. Time for a cull. A “spring-clean”. To remove some more cruft projects that occupy too much of my mind, but are actually just weak “me-too” versions of existing things that I would use perfectly happily if I made the effort to learn them. Enough with the Not Invented Here syndrome.

I’m not saying that OWL or MTC are going anywhere yet. I use them, and they work for me. And they are DIFFERENT from OrgMode, or todo.txt or any similar thing out there. They are what I want.

But I need to embrace this change. There are so many exciting NEW opportunities, there’s no point getting hung up on the old stuff.

Dawn is over. For me it’s 2PM. And there’s plenty of work to be done.

Posted in Me.

Today I killed the old Mind Traffic Control on Google App Engine. And replaced it with a new, fairly basic, site. (Though one that’s quite pretty, in a Packt Publishing kind of way.)

This is the beginning of a whole new MTC ecosystem.

1) I’m no longer interested in hosting your todos on Google App Engine. The only functionality that’s left on the site is the “export”. You can get the tasks you put into the old MTC in one of .txt, .csv or .opml formats.

.txt means you can use either an ordinary text editor, todo.txt or the new command-line based mtc program on your own machine.

.csv means you can use a spreadsheet if you prefer

.opml can be read into an outliner such as OWL.

2) The site has a new mission. Right now, it’s simply pointing visitors to the two pieces of software that I use to manage my todos and information : MTC command-line (in Racket) and OWL (in its three different versions : desktop, Android and web-served).

These are currently all simply source-code, hosted on GitHub. (Both MTC-racket and OWL are free-software, under the GPL).

It’s a geek view. But going forward, I’m going to be preparing, packaging and documenting these for actual users.

3) Right now, I still don’t know the relationship between MTC and OWL. They’re different programs, with different “mind traffic geometry”, in different languages. But I use both and I’m constantly musing about how they can and should interact.

The only thing which is clear to me at the moment, is that they should be brought together under the common Mind Traffic Control “brand”. As a statement of intent.

The question of 2014 is getting resolved.

4) “What about Project ThoughtStorms?”, you ask.

Well, I tend to think of “Project ThoughtStorms” as the developers’ view on my knowledge management / personal productivity software. While “Mind Traffic Control” is the user perspective.

There’s more to it than that. Project ThoughtStorms covers my experiments and add-ons to the Smallest Federated Wiki and thinking about wiki in general. MTC heavily emphasizes the dynamic flow of tasks. But that’s the broadest overview.

In practice I’ll continue to refer to MTC-racket and OWL within the contexts of both Mind Traffic Control and Project ThoughtStorms.

5) I am VERY happy to be deleting code. And collapsing several different overlapping ideas and codebases into … er … fewer. It’s a therapeutic decluttering that’s clearing space in my mind, and helping me focus and drive the surviving projects forward faster. It’s good.

But there is just a slight twinge of sadness. As I realize that this is a mile-stone in letting go of Python, a language I had a long and passionate engagement with. Both the old MTC code-base AND the server-side PageStore of OWL are Python.

But since porting OWL to Electron I can see that its future is very much in the Javascript (node / CoffeeScript) camp. Meanwhile the new MTC codebase is in Racket. And my latest quick conversion script persuades me that Racket is a good language for other small-scale tools. I have about a dozen of them in Python. But I’m pretty sure that new development in these areas will almost certainly be Lisp. (Racket or Clojure).

(Actually there is a project that’s still in Python where you’ll see some further development soon … but I’ll leave that to another post.)

Posted in Me.

A burst of development energy in a number of directions recently.

And things are starting to self-organize towards the new ecosystem.

Today’s exciting news : OWL makes a very nice desktop app, thanks to Electron.

Here’s the github repo.

I’ll be doing more testing, compiling, packaging this shortly. So that even non-geeks can play with it. But it seems to work fine.

To be honest, the few times I’ve installed OWL on non-geek friends’ machines, the “run a server and look at OWL in the browser” part has confused / put people off. Now things are VERY much simpler.

This is also going to give me some momentum to add a couple of extra features / ideas I’ve been thinking about over the last couple of years.

In the short term … this desktop repo is likely to be where I develop the next round of functionality. Though I’ll be porting these new features back to the Android and web-based versions.

Main issues in my mind at the moment : I’ve dropped the Python based server for this. We’re now purely Coffee / Javascript. That feels cleaner and more convenient. Will I now try do the same for the web version. It makes sense. But straight node? Express? Meteor?

What’s simplest and leads to least repetition of code?

Posted in Me.

A few years ago I looked at some of the programs I was writing, both for public release and libraries / scripts for internal consumption, and realized that even I was confused.

I needed a new map to understand what I was doing. Hence I came up with Project ThoughtStorms. A quick and dirty overview of what I was up to in this space of wikis, personal information and knowledge management.

I think that helps. I know what I’m up to, with the legacy of my ThoughtStorms UseMod wiki and SdiDesk, my experiments with Smallest Federated Wiki, the new OWL stuff, etc. It’s still a mess, but it’s conceptually “encapsulated” in a single place.

Now I find myself with a similar situation with a number of different tries at some tools to make it easier for me to produce static web-sites. Originally there was GeekWeaver, a way for me to generate static sites quickly and easily from an OPML outline. That was a great idea … except … it got mixed up with my n00bie enthusiasm for Lisp and desire to create a powerful Lisp-like programming language.

GeekWeaver also foundered on two other issues : soon after I wrote it, there were no popular OPML outliner tools. Although Dave Winer has resolved that issue and I have, indeed, used Fargo to author GeekWeaver programs.

The other, more subtle and pernicious issue is that few people want to develop a site from scratch these days. Most people use huge frameworks with not just Javascript, but CSS, LESS etc. It really doesn’t make sense to try to use a tool which denies the file-system or the fact that you’ll want to be working with a large number of external files.

So the next time I needed to produce some quick static pages I went back to the drawing-board and came up with BootDown : a very simple Python script that lets you write text in Markdown and wraps it in a BootStrap / BootSwatch template. It’s the opposite of GeekWeaver, it isn’t trying to be a rich and clever language that lets you define your own templates, it’s just a quick way to take advantage of a lot of templates that already exist.

BootDown adds two things to conventional Markdown. A shorthand for declaring div tags. And a page separator. So you can still make a multi-page site inside a single file. And you can explicitly reference the div classes and ids you need.

More recently still, I’ve been getting into a real Lisp : Clojure. Today, that’s the language I really want to be working with. And I have a current project that again involves generating a multi-page, fairly static site. But the inputs are a little bit more complicated : some more structured data, not just text. And it makes sense to store it in an outline. So I’ve started writing Clojure code to read OPML and spit out a BootDown Markdown file, to wrap in HTML.

I can see this project starting to grow too.

And so here I am … three Python code-bases : GeekWeaver, BootDown and the OWL back-end server. Two Clojure projects : the beginnings of a library to work on OPML outlines (OPMLKit) and my unreleased code I’m writing for this current project. And some clunky pipelines to use them together. It’s starting to be another mess. And that’s why I need Project GeekWeaver : an umbrella to pull together these different strands together, at least conceptually.

There are some thoughts … would it make sense to pull at least some of the original GeekWeaver code INTO the OWL back-end server? So that “run transformations on this outline” is a standard OWL feature, and OWL becomes GeekWeaver’s built-in IDE? But what about the OWLdroid version that doesn’t have Python? Should I rewrite that code in CoffeeScript. (Or ClojureScript … a working but still fairly slow option at the moment.) BootDown “just works”. Am I better off scrapping the GeekWeaver idea altogether and just focusing on such practical solutions?

Some of this will become clearer as I work through this current site I’m making. meanwhile Project GeekWeaver is the category and (shortly) the site for the high-level overview.

Posted in Me.

Good news. The latest version of BTSync actually lets me specify which folder on my Android tablet I want to sync.

That means I can explicitly tell it to sync. the OWL directory that OWLdroid uses by default. (Somewhere between updates of Android and BTSync last year, this facility got lost.)

So it’s looking like BTSyncing between my tablet and laptop are working again. So OWLdroid is a viable piece of software again. (The syncing with the laptop makes a huge difference in day-to-day usefulness.)

Yes, I know this isn’t great. I need to figure out how to ensure that OWLdroid works. The longer term plan is to move to having remoteStorage.js as an option. But that’s a little way down my todo-queue.

Posted in Uncategorised.

So, Dave has a nice template for showing essay-length writing.

And naturally, the way things are evolving in 2015, it’s a “single-page web-app” which can pull in the actual content from another URL.

Of course, I’d like to write essays using an outliner. And I’m using OWL which is really just the Concord editor that saves files in OPML. But myword.io has a new custom JSON format for its input (that also seems to allow Markdown).

So here’s the question : what’s the story for converting an outline to an essay? Is myword going to evolve to import OPML directly? Is the myword JSON format going to become a standard? How should OPML turn into myword documents?

In one sense it’s easy. You can always just flatten OPML and dump it to a raw text file. Perhaps with some indentation based on outline depth. (Although that might fight the Markdown.)

Or are there plans to represent any structure in the essay (sections, section headings etc.) with structure in the OPML? Is there a project to work on this? Or thoughts on a standard that I could start to support?

Obviously I have a couple of my own solutions in this area : GeekWeaver and bootdown. But if there are any emerging standards / conventions planned, then I’m up for supporting them. Both GeekWeaver and BootDown are useful, but they’re both rather old-skool command-line Python scripts which don’t necessarily fit into the workflow of many people in 2015.

Update : Dave replied :

It is what it is.

I have no plans at this time to do anything with it.

I might swing back around to this, or not.

MyWord is MIT Licensed, so if you really want it, you can do it.

OK. So right now, I’ll probably just do a flat export of text written in OPML and a quick JSON wrapper. And then think further. As this ties into questions about GeekWeaver, the future of some of my Python code base (as opposed to moving more into the browser)

Dave also points to his code :

Here’s the code I added to Fargo to generate the myword.io JSON.

https://gist.github.com/scripting/2a612b0e2cbb7077482a

As you can see, there ain’t much there! 🙂

Still. Very useful.

Posted in Uncategorised.

Bloody hell! I just wrote a blog post and WordPress lost it.

Grrrr …

The short resume was :

– yes, I’m late (Feb) continuing my “Welcome to 2015” blog posts.

– last year I got the functional programming bug. Haskell is pretty damned good, but I’ve fallen for Clojure because a) dynamic types, b) Lisp syntax(lessness), c) Paredit.

– last year I did more Python, but not much OWL. This year, OWL is calling again. And as well as making me want to rethink Mind Traffic Control in light of it, is also seeming to demand some kind of pivot on GeekWeaver.

I’ll be coding and blogging more on this ongoing development in coming weeks / months.

I’ve also been playing a bit with remoteStorage, but nothing to show yet.

Posted in Uncategorised.

So, 2015, here already?

Time to take stock. I haven’t been blogging as much as I should, but I’ve been pretty busy, so here’s the first of a couple of posts that give a quick run-down of the last year and thoughts on the next.

I started 2014 with a number of questions that I thought would guide my development work / play over the year. And then promptly went off and did something completely different. But it wasn’t an entirely wasted exercise. So let’s review those questions :

1) Given how satisfied I was with OWL, what was the future of Mind Traffic Control?
2) Given how satisfied I was with OWL, what was the future of Project ThoughtStorms?
3) How do I program on my tablet?
4) Can I learn visual design?
5) How do I support Mozilla and the new (open) web-stack?

So what happened?

1) Due to my other interests (see the next post) Mind Traffic Control got a stay of execution in 2014. I didn’t kill it and didn’t try to update it. What I did find myself doing (fairly recently) was throwing some new tasks into it. One reason MTC was a bit stale for me was that I wasn’t actually using it. But half-way through the year, my tablet broke, and when I got a new one, the rather convenient btsync connection between it and my laptop was impossible to get running again. This seriously degraded the value of OWL / OWLdroid for me, as btsyncing was my only way of keeping them automatically synced. (We’ll return to this theme in a minute, but just note here that MTC picked up some of the slack.)

2) I barely glanced at this question again after blogging it. I’ve been updating Thoughtstorms wiki sporadically this year. And it’s been perfectly viable without an OWL connection. Consider this a fairly non-urgent question as OWL and SFW seem pretty orthogonal in practice.

3) Right now, the best programming experiences I’ve had on the tablet are with Quoda and Clojure REPL. Both are pretty good as far as they go … which isn’t really far enough. I’ll come back to Clojure in a minute. But I’ll note that Quoda is the best code editor I’ve found so far (mainly for the very nice extra keys it provides). And Clojure REPL is, indeed, a REPL for Clojure. Which is good for trying out ideas. But far from a full blown Clojure dev. environment.

4) I’ve certainly been playing with visual designs this year (in Patterning), but I haven’t really touched on what I was getting at in this question at all. Maybe 2015?

5) I didn’t do much about this question in 2014, either. But that’s about to change.

I’ve just discovered the Unhosted movement and remoteStorage. And I believe this is going to very fundamentally shape what I’m thinking about and doing in 2015. I’m increasingly concerned with the risks to privacy and the concentration of power that comes from storing all our data in the clouds of a few large corporations. And remoteStorage seems a good solution to decouple the storage provision from app. provision and put control back in the hands of the user. More importantly, it seems to have both the right attitude and some momentum behind it.

So, in 2015, unlike 2014, I am going to make new year resolutions / commitments. In particular, I’m publicly committing, right now, that I’ll be adding remoteStorage to OWL and OWLdroid. This is not only good for you in the sense that it respects your right to privacy and control over your data, but it’s good for me, in that it should be a workaround for my Android file-system / btsync problems.

In fact, this year I’ve been playing with the javascript Dropbox API. And if I hadn’t discovered remoteStorage I’d have added Dropbox as an alternative file-system option, but it seems remoteStorage should nicely abstract away from that, allowing users to choose between Dropbox, Google Docs and any other storage provider.

OK. That’s the first big announcement for the year. In the next post I’ll come back to what I actually DID get obsessed with this year, and what I think that will mean for 2015.

Posted in Uncategorised.