Unification : BootDown

I’m working on a series of small “unifications”. Attempts to bring several of my different projects closer together. This is to share more code between them, or allow them to work more closely together.
Today, I’ve been updating BootDown, my quick and dirty static site generator so that it now uses the same library (txlib.py) as TSWiki to render markdown and embedded media like YouTube and SoundCloud.
This means that from now on, BootDown will use the same markup for embedding videos etc. as TSWiki : block defined with [] containing metadata in YAML format. Also means that TSWiki now inherits an include CSV file block which renders as a reasonable looking table from BootDown.
This is a small change to a couple of files in a couple of projects, but it’s part of a general campaign of forward motion that will hopefully result in a bunch of projects becoming more powerful, more flexible and faster evolving this year.

What's wrong with C++?

Another Quora answer : Phil Jones’s answer to Why is C++ considered a bad language?
This is one of those rare occasions I disagree with Simon Kinahan; although his answer sets the scene for this one.
[Simon says that C++ isn’t a bad language. It’s the right choice if you need low-level memory control and the ability to build powerful higher-level abstractions.]
C++ is a bad language because it’s built on a flawed philosophy : which is that you should add power to a language by kludging it in “horizontally” in the form of libraries rather than “vertically” by building new Domain Specific Languages to express it.
Stroustrup is very explicit about this, rhetorically asking “why go to other languages for new features when you can add them as libraries in C++?”
Well, the answer is, adding new higher level conceptual thinking in the form of a library doesn’t really hide the old thinking from you. Or allow you to abandon it.
C++’s abstractions leak, more or less on purpose. Because you can never escape the underlying low-level thinking when you’re just using this stuff via libraries. You are stuck with the syntax and mindset of the low-level, even as you add more and more frameworks on top.
Yes, you can explicitly add garbage collection to a C++ program. But you can’t relax and stop thinking about memory management. It can’t disappear completely beneath the horizon of things you need to be aware of the way it does in Java.
Yes, you can have higher-level strings, that can abstract Unicode-ness etc. via a library. But you can never be sure that you won’t confront strings that are simple byte-arrays coming from another part of your large system.
Etc.
C++’s ability to build high-level abstractions uncomfortably falls between two stools.
It encourage you to think you can and should be building large applications full of application logic. But doesn’t give you the real abstracting power to focus only at that application level.
This explains the otherwise mysterious paradox that C is a good language, so how could something that is “C plus more stuff” possibly be a bad one? Well, it’s exactly the “more plussing” that’s the problem.
With C, you KNOW you should only use it to build relatively small things. It doesn’t pretend to offer you mechanisms to build big things. And so you turn to the obvious tools for scaling up : lex and yacc to build small scripting languages, the Unix pipe to orchestrate multiple small tools . C++ gives you just enough rope to hang yourself. Just enough powerful abstraction building capacity in the language itself that you (or that guy who used to work in your company 15 years ago ) thought it might be possible to reinvent half of Common Lisp’s data-manipulation capability and half an operating systems’ worth of concurrent process management inside your sprawling monolithic application.
I don’t see why it shouldn’t be possible to combine low-level memory management and efficiency with high level abstraction building. But proper abstraction building requires something more like macros or similar capabilities to make a level of expression which really transcends the low level. That’s exactly what C++ lacks.

The Bug is Your Fault

My Quora answer : Phil Jones’s answer to What are some great truths of computer programming?

  1. The bug is your fault

  2. The bug is your fault

  3. The bug is your fault

  4. The bug is your fault

  5. The bug is your fault.

  6. No. The bug really IS your fault.

  7. The bug is your fault. Because you made an assumption that wasn’t true.

  8. The bug is your fault. And it won’t go away until you check your assumptions, find the one that isn’t true, and change it.

What do you think computers will be like in 10 years?

My Quora answer that’s pretty popular : (1) Phil Jones’s answer to What do you think computers will be like in 10 years?
Related to my previous story of trying to use the CHIP for work.

This is a $9 CHIP (Get C.H.I.P. and C.H.I.P. Prto)

It runs Debian. A couple of weeks ago, I took one travelling with me instead of my laptop to see what it would be like to use for real work.
I successfully ran my personal “productivity” software (three Python based wiki servers and some code written in Racket) from it. It also has a browser, Emacs and I was doing logic programming with minikanren library in Python. It runs Sunvox synth and CSOUND too, though I wasn’t working on those on this trip.
In 10 years time, that computing power will be under a dollar. And if anyone can be bothered to make it in this format, the equivalent of this Debian machine will be tantamount to free.
Of course most of that spectacular power will be wasted on useless stuff. But, to re-emphasize, viable computing power that you can do real work with, will be “free”.
The pain is the UI. How would we attach real keyboards, decent screens etc when we need them?
I HOPE that people will understand this well enough that our current conception of a “personal computer” will explode into a “device swarm” of components that can be dynamically assembled into whatever configuration is convenient.
I, personally, would LIKE the main processor / storage to live somewhere that’s strongly attached to my body and hard to lose (eg. watch or lanyard). I’d like my “phone” to become a cheap disposable touch-screen for this personal server rather than the current repository of my data.
I bought a cheap bluetooth keypad for about $8. It was surprisingly OK to type on, but connections with the CHIP were unreliable. In 10 years time, that ought to be fixed.
So, in 10 years time, I personally want a computer on my wrist that’s powerful to do all my work with (that means programming and creating music). That can be hooked up to some kind of dumb external keyboard / mouse / screen interface (today the Motorola Lapdock is the gold-standard) that costs something like $20. Sure, I’ll probably want cloud resources for sharing, publishing, storage and even high-performance processing, AI and “knowledge” etc.
And, of course, I want it to run 100% free-software that puts me in control.
This is all absolutely do-able.

Travelling without a laptop

Why, yes. This is, indeed, OWL running on the PocketCHIP.

Here’s the story.
I spent 12 days recently, travelling in the south of Brazil and Uruguay border. And I decided, as an experiment, to see if I could live without my laptop. I wasn’t meant to be working, but I always like to keep some kind of SDI / MTC style software on me to make notes and generally think about my insanely long queue of tasks and all my other half-baked ideas. I feel lost without having access to these. And would usually take a laptop, just to keep them near to me.
This time, though, I decided to see how well I could cope with a “device swarm” of very small (and cheapish) tech.
Here’s what I took :

  • The PocketCHIP
  • My Android phone
  • A small, portable USB mouse
  • The rubber-tipped stylus.
  • A 5V mini USB charger that plugs into the wall.
  • An A5 sized paper notebook and pen.

On my trip, I also acquired a small bluetooth keyboard / mousepad that’s actually just a capacitative touch surface with letters marked on it. This impressed me a lot when I bought it at a Duty Free shopping outlet just on the Uruguay side of the Yaguarón River in Rio Branco for under $8. It charges via USB and feels pleasantly heavy and solid, while still pretty small.
Here are some things I found out.
I should note, first, that I’m also pretty much against keeping things on public clouds and other people’s servers. I increasingly want my data to be private, on systems that I control, and mainly synced between my own machines rather than using, say, my hosting provider.
Although I’m far from a satisfactory solution to that, it turned out that the discipline was useful in the sense that I wasn’t as persistently connected to the internet as I’d hoped. And so if I had depended on the cloud I would have been without access to my data more often than I had it.
I also, of course, want to use my own software. I currently have three distinct trajectories of development in this area.

  • Mind Traffic Control, racket-version.
  • The new engine behind ThoughtStorms wiki which is written in Python and uses the light-weight Bottle web-framework. I use this for both ThoughtStorms and a second personal wiki-notebook which is descended from my old SdiDesk notebook.
  • OWL

All these projects are still alive. And my notes and todos (and some more significant chunks of writing) are somewhat haphazardly scattered between them. So I wanted to see how well I could take them on the road with me with such minimal hardware.
Before leaving I updated the Debian on the PocketCHIP and installed the relevant libraries / environments. I already had MTC-racket running on it. And Emacs. Python was there too, but I needed to pip install a couple of standard libraries. Then I could install both Project ThoughtStorms and the Python-served version of OWL. (Spoiler alert : 2017  is going to be a year of consolidation between all these projects, particularly I’m aiming to unify the Python-OWL and Project ThoughtStorms servers into a single code-base.)
I didn’t try installing the Android version of OWL on my mobile, because of … er … reasons … which currently prevent me having a working dev / deployment environment for Android on my laptop. I hoped, though, that I’d be able to access the OWL server on the PocketCHIP from the phone.
Results
The PocketCHIP is a wonderful machine. (And, seriously, a Debian box for $9 just insanely amazing.) It seemed to cope just fine with simultaneously running three Python servers (2 copies of the Project ThoughtStorms wiki, and the OWL server) + MTC. (I just ran unix “screen” on the terminal and ran each in a separate screen.)
Obviously these were not being heavily accessed. (I was the only user). But I’m still impressed.
The weakness of the CHIP is its WiFi. It is very weak. My original thought was to run the servers on the CHIP and then access them from either my phone, or other computers in the places I was staying. But even where WiFi was available, the CHIP typically failed to establish a usable connection with the router.
The only time I could get anything else talking to the CHIP was by turning the phone into a hotspot and then placing that within 10 to 20 centimetres of the PocketCHIP.
This way I was able to access both wikis and OWL from the phone. I didn’t get to remotely ssh-ing into the CHIP to see MTC in a terminal from the phone, but this would have been a particular fiddle and it’s not clear that it would have been worth it. In fact, nothing worked particularly well. OWL’s web-interface is pretty much impossible to work with on a small phone screen. It’s OK on a 7″ tablet, but the phone is way smaller and too difficult to manipulate. And the HTML UI doesn’t zoom in any effective way.
Reading the wiki pages was slightly better. I was surprised, though, how badly the phone handled the fairly simple, static html. I accept that there’s very little fancy “responsiveness” in the TS wiki at the moment. I hadn’t realized how little Chrome would help. On my laptop, ctrl + and ctrl – work beautifully to scale text up and down, reflowing and refitting the text. I have no hard settings for font-size or spacing. The page ought to be easy to automatically resize to convenience.
But pinch zooming a TS page on Android Chrome is diabolical. Not only does it not scale and reflow the text in a useful way. It also seems to remember (or guess) arbitrary different zoom levels for different pages, so you jump from a readable page to another page with illegibly tiny letters to another with enormous text, most of which is off screen. And zooming the menu seems to be independent of zooming the main page text. The whole thing is horrible.
Firefox on the phone is a bit better. You can go into “accessibility” and turn up the text size to full. And then the defaults are reasonably readable and consistent on all pages.
Editing is trickier. And this is somewhat my fault, I have set a fixed number of columns in the text-box in TS wiki which is too large for a phone with either browser.
Overall, I’m disappointed with the phone experience. TS wiki is just about readable on Firefox. But it’s useless for doing any kind of work. And horrible on Chrome. OWL looks OK on both browsers, but is too fiddly to actually edit.
I need to radically rework the UI for both these projects.
So I went back to see if I could actually look at these web-served applications on the PocketCHIP itself.
It turns out that OWL works surprisingly well with the surf browser which comes pre-installed … as long as you use a mouse!
The touch screen even with a rubber-tipped stylus isn’t viable for navigating and editing an outline. But attach an external mouse to the USB and it’s surprisingly usable. The PocketCHIP keyboard isn’t great for a lot of writing, but for short items in an outline it’s viable.
TS Wiki is also fine to read. But for some reason the text-area is coming out black, with black text. (I’m pretty sure I’m not setting this explicitly, so I assume it’s a bug in surf.) You, therefore, can’t edit the wiki with it. But reading is an acceptable experience. Once again, a mouse helps, but you can just about get away with the stylus.
The bluetooth keyboard I bought paired fine with the phone. But with the PocketCHIP there were some issues with the mapping between some symbols. I couldn’t find any combination of keys to make a backslash for instance. And sometimes the shift wouldn’t work. Also the bluetooth connection kept dropping. I found myself continually swapping between the PocketCHIP’s own keyboard for typing short commands with a lot of non-alphabetic symbols, and the bluetooth keyboard to type paragraphs of text. It was just enough to get some work done, and the bluetooth keyboard was just better enough to make it worthwhile, but it wasn’t really a viable solution.
Aside
This trip I also got interested in Logic Programming and tried out the Python Minikanren library, logpy, on the PocketCHIP. Unsurprisingly, it worked as expected. But, again, that is kind of startling when you think of it.
Conclusion
Well, this stuff works. If the PocketCHIP just had better wifi / bluetooth connections, then it would be a serious possibility to do some work on. It might be that a software update fixes the power-saving that may be weakening the wifi.
But it’s not something that even I can use yet.
Some of that is in my hands, of course, one task is to tweak the TS Wiki UI to ensure that the text-area is readable in surf. That would at least mean that PocketCHIP could be used (if uncomfortably) for working on my notebooks.
It’s not all working yet … but it’s getting closer.

Project ThoughtStorms Update

Been working quite a lot on Project ThoughtStorms, the new Python / Bottle based wiki engine behind ThoughtStorms in the last few days.
I can’t believe that I’ve spent 15 years building a wiki without creating my own software.
That error is going to be rectified from now on.
Sure, most of what I’m doing is adding the kind of functionality that most wikis already have : RecentChanges, Search, Sister Sites etc. But it’s pretty quick to write. And as I bring these functions back, ThoughtStorms becomes more subtle and alive.
The truth is that ThoughtStorms’ content is woefully outdated. There are a tonne of broken links, and news stories that were current in 2005, but equivalent stories between about 2008 and 2016 are no-where.
It needs a jolly good clean up / clean out / refactoring. But I’m starting to have some ideas about how to write some more innovative custom code to help with that.
It’s very much work in progress. And there’s still plenty that’s half-baked and unfinished.
But I’m starting to feel that ThoughtStorms is coming back to life.

Where I’m At, March 2017

A quick update (for me, as much as anyone else) of where I’m at and what I’m currently thinking about … March 2017:

ClojureScript and Reagent

I want to get up to speed on this as my standard way of writing browser based software. React seems to have conquered the world as browser based, reactive framework. And with React Native it’s also come to mobiles. It seems that the ClojureScript / React Native story is becoming compelling too.
So, yes, this is mainly how I want to do GUIs moving forward. I’m doing some experiments. Expect, eventually, Patterning and some of my other projects to move to this.

WebAudio

A friend of mine has sold me on the idea of going back to audio in the web-browser. Which is obviously a great way to make little musical apps that are easy to share with people. He’s not so keen on ClojureScript but does like Javascript in a functional style with Redux and Immutable.js so we’ll probably be using them on a collaboration. I’m considering porting versions of Gbloink! and Zewp! to it. Along with more of my current Processing based “ArtToys” Library.

Lisps on Android

It may be that ClojureScript + ReactNative is the right way to go. But I’m also looking at Clojure on Android. And Racket on Android.
Reading online it seems that Clojure on Android has been improving, but is maybe not as efficient as ClojureScript with ReactNative. And, also obliges you to deal more directly with the Android Java frameworks. Maybe something that’s useful for larger, more framework intense, applications.
I was originally tempted by LambdaNative, a Scheme for Android app. development. But it now looks like Racket’s Android story has improved considerably and is quite similar. Especially sharing the focus on using OpenGL for visuals. Racket on Android basically compiles to C and uses the NDK, which might be a faff to get working, but might also be more efficient. One to have in the arsenal.
Also, of course, Racket on Android would let me keep a common Racket code-base for both the command-line and an Android version of Mind Traffic Control.

Rosette

I’m kind of intrigued by Rosette and the idea of “solver aided” programming. And being a Racket DSL is a bonus. I want to get my head around this, this year.

Music Languages

Last year. I did a lot on Sonic-Pi. This year I thought I’d go back and take another look at Overtone. Which is, in many ways, the predecessor of Sonic-Pi done as a Clojure library.
It’s not quite as simple as Sonic Pi with its integrated development environment but Clojure is
more powerful than Ruby.
While I still think that Faust looks cool. I’ve also been tempted by the lower-level simplicity of CSOUND … wondering if it might make sense to write a CSOUND generator in Clojure. I’m slightly pulled backwards and forwards on this question. Plus, Overtone provides its own synth-design functions.

Project ThoughtStorms

So, I haven’t trumpeted it here. But last year I moved away from the Smallest Federated Wiki for ThoughtStorms. See ThoughtStorms:LeavingTheSFW for the full story.
Here I’ll just note … I’m very happy with the decision, and I have lots of ideas for how to develop my currently very minimal wiki engine to do more of the things I want to do. In many ways, this is like the MTC reboot of last year.

Small Boards

I’ve become a bit of a collector of small hobbyist / maker boards, from Arduino to last year’s CHIP. I’ve just got a couple of BBC Micro:Bits which are great fun. These are a really smart and original idea in what’s becoming a crowded and competitive market. The computing power is pretty minimal. But what gives them an edge is that they’re packed with sensors. And come with their own primitive display.
I’m looking forward to playing with and giving these a good run for their money.
Interestingly they run MicroPython. Something I hadn’t really looked at before.

Python

All of which means, despite last year’s thinking, that I’m actually renewing my relationship with Python. I really thought I was deprecating it in favour of Javascript. This year, that seems fairly unlikely. Python just is nicer than Javascript in so many ways. It’s the back-end of my new wiki, which I’m planning to extend. It’s the language of the Microbit and potentially useful on other boards. It’s getting ported to the GoLang infrastructure and perhaps has a great future in front of it there. Meanwhile, even a tiny serverside JS program with node.js seems to install a million dependencies.
So Python is definitely here to stay in my near future.

QaSaC

A rather crazy art-project I dabbled with last year spawned an interesting toy / esolang : QaSaC. This is little experimental “concatenative” language like Forth and Joy cross-pollinated with a visual data-flow language like Pure Data.
I think there’s some potential here. And I’ve really enjoyed the concatenative / stack-based thinking and elegance of this kind of programming. So I am working (albeit slowly and interspersed with everything else) on getting an actual viable version of this out soon.
OK. I think that’s a pretty comprehensive list of my obsessions for 2017. Let’s see how much progress I make with them all. This year, I promise, I’ll be trying to release, and blog, more often about them.

Grumpy: Python on Go

Google are making a Python to Go cross compiler
The idea here is that Google have a lot of Python to run, but Python’s Virtual Machines (especially with the Global Interpreter Lock) aren’t all that good for high-performance parallelism.
OTOH built Go! and the Go target machine for this. So they want to make a drop-in replacement that lets them run existing Python code on this infrastructure.
This is good news for Python. I wonder how long it will take to become the GAE python-hosting infrastructure. Perhaps this is Python’s answer to Elixir.

The End of Dynamic Languages

A comment I made over on The End of Dynamic Languages

The problem with the static / dynamic debate is that the problems / costs appear in different places.
In static languages, the compiler is a gatekeeper. Code that gets past the gatekeeper is almost certainly less buggy than code that doesn’t get past the gatekeeper. So if you count bugs in production code, you’ll find fewer in statically typed languages.
But in static languages less code makes it past the compiler in the first place. Anecdotally, I’ve abandoned more “let’s just try this to see if its useful” experiments in Haskell where I fought the compiler and lost, than in Clojure, where the compiler is more lenient. (Which has the knock on effect of my trying fewer experiments in Haskell in the first place, and writing more Clojure overall.)
Static typing ensures that certain code runs correctly 100% of the time, or not at all.
But sometimes it’s acceptable for code to run 90% of the time, and to have a secondary system compensate for the 10% when it fails. There might even be cases where 90% failure and 10% success can still be useful. But only dynamic languages give you access to this space of “half-programs” that “work ok, most of the time”. Static languages lock you out of it entirely. They oblige you to deal correctly with all the edge cases.
Now that’s very good, you’ve dealt with the edge cases. But what if there’s an edge case that turns up extremely rarely, but costs three months of programmer time to fix. In a nuclear power station, that’s crucial. On a bog-standard commercial web-site, that’s something that can safely be put off until next year or the year after. But a static language won’t allow you that flexibility.
The costs of static and dynamic languages turn up in different places. Which is why empirical comparisons are still hard.