Fixing the Lower-Case Mistake

After writing The Lower-Case Mistake this morning, that basically convinced me that I had to move back to having CamelCase file-names for Cardigan Bay.
So … a burst of activity this afternoon, and the automatic lower-casing of file-names is gone.
If you create a page called HelloWorld, it will be in a file called HelloWorld.md
The GitHub / GitLab repos are now updated and this is now the official behaviour of Cardigan Bay.
Obviously it’s a breaking change. Because if you were using it yesterday and your HelloWorld page was in file called helloworld.md the new version of the code can’t see it.
The default Bedrock Wiki pages that come with Cardigan Bay have now been fixed. But I’m afraid you’ll have to change any other page names yourself.
I wrote a simple script to help me convert the ThoughtStorms page-base (of over 5000 pages)
It wasn’t perfect, but it did enough of them that I’m happy to commit to making the change. (I’ll clean up the problem cases by hand)
Here’s the code to fix the names. It basically searches all the links in pages to find CamelCase names. Then prints a list of “git mv” commands that would turn the lower-cased version of those named files back into CamelCased ones. You just have to pipe that output to a shell script file and run it and it will do the changes.
Like I say, not perfect, but took a couple of minutes to write, and does the bulk of the job.

You’ll have to fiddle with it a bit, and make sure you have https://github.com/interstar/FSQuery installed. But like I say, it does the bulk of the work.
If you have problems (eg. a big investment in Cardigan Bay data but can’t do this migration yourself), leave me a message and I’ll see what I can do to help.

The Lower-Case Mistake

I’m doing a lot of of work on Cardigan Bay at the moment.

And it’s getting pretty good.

I wish I’d done all this sooner. I spent years with wikis, not writing my own wiki-engine because I thought that would be pointless reinventing the wheel. Then when I did, I realized I could make wikis do all the things I wanted from them.

Then I loved Clojure but kept writing my engine in Python because I thought it would be more straight-forward, a known art, and just quicker and easier.

Now I’ve bitten the bullet and moved to writing my wiki in Clojure (something that seemed yet another round of self-indulgence when I started it at the very end of last year) I realize that it’s exactly right, and Clojure is a super-powerful and easy language to do it in.

I’m making great progress and loving Cardigan Bay … (and very soon there’ll be a new release with lots of nice new features … watch this space.)

BUT …

somewhere along the line I have made a huge, huge mistake.

This goes back to my earlier engine, Project ThoughtStorms, I think.

Or maybe I inherited it from somewhere else. I can’t even remember.

The mistake is that wiki page names are obviously CamelCased for readability. A convention I like very much and continue to use.

But I decided to make the file-names all lower-case. In other words, I throw away the capitalization of the name.

The only place that the capitalization exists, is in the link that leads to a page. So links look OK. And from following a link, we can deduce the capitalization we show the next page.

Which works reasonably. But you’ll notice that in RecentChanges and other queries for OrphanPages etc. the pagenames are all lower-case. They have to be, the file system has no knowledge of the capitalization.

That was a pain, but I could live with it.

Now, though, I’m working on the “export this wiki as flat files” functionality. (Which is going to be fundamental to my new thinking … more on which later)

And I have same problem. Running through the directory of pages, I can render them beautifully, but I have no information as to how the pagename should be capitalized. And so can’t give the static pages a reasonable looking title.

That is an absurd restriction which I certainly can’t live with. (And nor will anyone else)

So what is to be done?

I could decide to store the correct page name somewhere else. Either in the page files themselves, or another database. But that breaks the convention that pages are pure markdown files (or slightly extended markdown files). I don’t want to add my own special header convention and force people to deal with that when migrating into or out of Cardigan Bay. It should be possible to work on CB wikis in your own preferred text editor if you choose.

Similarly, I don’t want to depend on another database.

Or I can try to capture the capitalized version of the names when I scan the links into the temporary database at the beginning of each session. And use that when exporting. Which will work for pages which aren’t orphans.

Or I do an export the way I used to back in SdiDesk, not by grabbing everything in the page-store, but by crawling it via the links. Which would give me the capitalized version of the pagenames.

Or maybe I need to change the convention so we don’t lower-case the file-names. But keep the capitalization. I’m tending towards this. But I have 5000 pages of ThoughtStorms in files whose names aren’t capitalized. And another 4000 in my personal wikis. That’s a lot of work to fix.

Anyway, thanks for listening dear reader … just writing this has been useful for me.

Command line HTTP apps.

I just rediscovered a question I wrote on Stack Exchange. I can’t remember much about the context, but the idea is to have an ordinary “web server” application accessed through http, but simply sending plain-text backwards and forwards to a command-line based client.

command line – Is there a “terminal” style program that talks to a server over http? – Unix & Linux Stack Exchange

The question was :

I’m looking for something that acts like a terminal but lets me have a “dialogue” with a server over http. Something like this :


$ connect http://myserver.com
Welcome to myserver.com
Options
A - Fribble the obsticator
B - List Frogits
C - Show the log
Q - Quit
$ A
Obsticator fribbled
blah blah
blah
$ C
Log file
...
$ Q
Bye

Http is a useful protocol and it goes everywhere, it’s easy to write a server that handles it (thanks to libraries). But you aren’t always in a place where you have a browser or could run one. Sure there are browsers like lynx etc. that run in the terminal, but they aren’t particularly easy. Sometimes you just want something that’s more like the old Gopher etc. systems. But with the benefits of modern server technology.

No-one seems to have written one, so I mocked up a client in Python using the Requests (http://requests.readthedocs.io/en/master/) library :

[cc language=”python”]
import requests
import sys
url = sys.argv[1]
print “Connecting “, url
r = requests.get(url)
print r.text
flag = True
while flag :
s = raw_input()
data = {“opt”: s}
r = requests.get(url, params=data)
print r.text
[/cc]

A server now just has to respond to messages passed as “opt” on a particular URL, and to respond in plain-text.

Obviously, something like file-downloads etc. would also be useful.

Cardigan-Bay Early 0.0.3

Cardigan Bay development is slow, but continuing.
It’s starting to look a bit more respectable (I’m so NOT a CSS programmer, but I have to do something)

Get the latest : Release Cardigan-Bay Early 0.0.3 · interstar/cardigan-bay
This release is largely just bug fixes and cleaning up the UI etc.
But that makes a massive difference in usability on tablets and even my phone. It’s still not a mobile friendly UI, but it is at least readable and usable through the phone.

Carp: A statically typed Lisp, without a GC, for real-time applications.

This looks rather good.
Carp: A statically typed lisp, without a GC, for real-time applications.
It is what it says. It’s a Lisp, with some inspiration from Clojure in terms of syntax, that’s designed for writing fast native code like games and … (w00t!) audio applications. Etc.
It actually compiles into C.
It takes some syntactic inspiration from Clojure (so it looks like Clojure and Clojure editors should handle it fine)
But it also takes inspiration from ML.
It has an ML-style type-system with type-inference. And the best way of representing type information in a Lisp program that I’ve seen.
Using a form called “the“.
So you write something like
[cc lang=”clojure”]
(the Int sexp)
[/cc]
This declares that the sexp needs to evaluate to the type Int.
Because it has type-inference, the type of much of the rest of the code can be deduced just from a few of these explicit specifications.
Carp also (allegedly) has an ML style module system. I have no idea what that means because I’ve not used ML. But it is something I hear a lot of people raving about.
However, Carp is a low-level language designed as a C-level language. It has the usual Lisp-style dynamic lists, but only for use at compile-time. It also has macros executed at compile-time.
At run-time code, you only have more primitive types : Ints, Floats, etc. Plus data-structures including Arrays, some kind of map / dictionary, structs and Sum Types. So you get the power and type-safety of ML / Haskell algebraic data-types but at a C-level without depending on heap allocation.
It also has some kind of Rust-like “ownership semantics” for pointers. I haven’t quite understood this yet, but again it looks sensible.
In all, this looks a really tasteful and well thought out combination of good ideas from a number of places (much like Clojure itself) that will add up to a much nicer than C experience for writing the sorts of things that I usually write with C.
All I’ve done so far is installed it and run a couple of examples, but I can well see myself trying to use this to do the coding for some audio / VCVRack stuff.

Tablets for Content Creation

Source: Thread by @interstar: @msimoni @coreload I think the tablet COULD be a perfectly good device for content creation if people would just do the damned UI design wor…
Another part of the thread. Manuel Simoni (@msimoni) says :

But I don’t want a file system, I want Xanadu. But it should still be editable through a WIMP/desktop UI.

I reply :
Sure. We all want / work towards Xanadu 🙂 Though at the very least I want mine to run on a tablet, have swipe-able cards like Material Design, and allow me to write Prolog-like rules for making automatic inferences that generate new information / views.

Against Direct Manipulation

I’m debating on Twitter about Direct Manipulation.

Here’s the first part of my discussion unrolled. I’m asked why I think the Desktop Metaphor has held us back. My response in 6 tweets. (Note that there’s a lot more good discussion in further pushback I get)

 

Complicated for Twitter. Here’s what I think in summary.
Computing is about using language to tell computers to do things. Language enables grammatical composition and ever increasing levels of abstraction and expressivity \1
The great mistake and delusion in computer history, of which the desktop metaphor is just one major example, is “direct manipulation”.
People seem to love it and always fantasize about more of it … \2
But with DM you switch from finding more and more elegant ways to TELL the computer to do stuff, to just “doing it yourself”
And once there’s a DM metaphor for a task, rather than a linguistic instruction, it gets locked-in and evolution grinds to a halt. \3
Basically every time DM came to a paradigm of computing, the vocabulary froze at that point and at that level of abstraction.
Desktop metaphor for launching applications and WYSIWYG word-processors looks like the 70s. \4
File systems look like the Mac finder from the 80s.
Spreadsheets started as a promising mix of visual and linguistic, but have DEVOLVED into mere grid-drawing GUIs.
IDEs haven’t even changed their menu layout since the 90s \5
All progress in computer science depends on maintaining that gap between you telling the computer to do something, and what the computer does.
All the force multipliers live in that gap. Close it and progress stops. \end
—-

Browsers Accessing the Local Disk

After writing Why isn’t browser based programming or browser based IDEs more popular? – Smart Disorganized I want to emphasize and ask again.
Why aren’t the browser makers offering us a way for web-apps to read and write from the local disk?
I know that it’s a mega security issue. But it should be possible to wrap it up. So that only web-apps opened in a special mode. Which ask three times and need you to sign in blood etc. etc. (Or better, add the public key of the web-app to your browser) get to write to the local disk.
But if just one browser maker made that happen, it would be the biggest revolution in application software development since the invention of the mobile app.
Suddenly all web-delivered applications would compete on a level playing field with locally installed apps. Browsers would be more important than ever. Surely there is a huge incentive for browser makers to do this.
 
 

Why isn’t browser based programming or browser based IDEs more popular?

My Quora answer :

Basically because most programmers use a bunch of other tools that are local, on their hard disk. These include compilers, libraries, source control, unit-testing frameworks, CI/CD pipelines etc. etc.

And browsers, because of their security model, are really bad at talking to the local disk.

So, if you want to use a browser-based / web-IDE all your other tools and resources have to be in the cloud too.

And the problem with that is you can’t mix and match your own custom configuration of those resources. You are basically stuck with the ones that your cloud IDE offers. Obviously most cloud IDEs are for-profit companies, and few of them have the resources to replicate the best of breed other tools you are using. Or to successfully integrate all the possible third-party options within their cloud.

So … cloud development IDEs should be a really obvious and cool win.

But … they aren’t because of the problem of bridging the cloud and the local disk.

I often wonder why browser makers haven’t solved this problem. Eg. why not have a special “mode” – just like “incognito mode” is a special mode – that allows web-applications to actually read and write to the disk just as native apps do. Of course it would need some extra security etc. But if they had it, we could do wonderful applications in the browser to compete with everything on the local file system. Including fantastic IDEs.

But until we bridge the cloud-local gap, web IDEs are stuck with whatever resources the cloud offers but also the downsides of latency, limited tools, less control and flexibility for the users etc.

Source: Why isn’t browser based programming or browser based IDEs more popular? Why do you not like or use one? – Quora