Smart Disorganized

People seem to be very excited about Roam Research at the moment. I’m sure it has many qualities. But I’m slightly surprised to realize that one thing that fans seem to find very useful (and almost miraculous) is the automatic back-linking. Ie, the ability to see what pages link to the page you are currently on.

Bill Seitz has always been a fan of this feature, ensuring that it’s prominent on his WebSeitz engine. While, I confess, I’ve always thought that links that people bother to make explicitly are more interesting and should be valued more highly than links that are implicit.

So, my current Python-based ThoughtStorms engine, following on from UseMod etc. lets you search for “pages that reference this one” with the usual text search, simply by clicking the page name, but it doesn’t automatically give backlinks to you on each page.

But on ThoughtStorms you’ll see I do often explicitly put something like  “Context : BlahPage, AnotherPage” at the top of pages where I think context is important.

Nevertheless, automatic backlinks are clearly the fashion. And as I’m working on Cardigan-bay: A new wiki engine in Clojure I figured I might as well add this feature.

So, one of the main ideas in Cardigan Bay is that it keeps an internal database of meta-data about the wiki, including which pages link to which, using core.logic, Clojure’s standard miniKanren library.

So adding the backlinks feature is as simple as adding a core.logic query to this database.

(defn links-to [target]
  (pldb/with-db @facts
    (logic/run* [p q]
      (link p q)
      (page p)
      (page q)
      (logic/== target q)
   )))

That’s basically it. The logic query tests that there’s a link relation between p and q, that both p and q are actual  existing pages, and that q is equal to the target argument to the Clojure function.

This is pretty obvious and straightforward. So then I just create a card containing this data and attach it automatically to each page.

Took basically less than an hour to write the whole thing.

And if you get the source from GitHub, it’s now there.

However, right now this is fine for small wikis with a few pages. On the actual ThoughtStorms data (as you see in the screen shot above), which has over 4000 pages and an order of magnitude more links, it’s way too slow. So I’m going to have to figure out how (and more importantly, where) I’m going to cache the results. And when I have to recalculate them, etc.

Nevertheless, automatic backlinks are now added to Cardigan Bay.

Update :

I just did a quick experiment where I took out the (page p) and (page q) clauses from the query. And it’s now much faster.

Testing that the two pages exist doesn’t really matter much. There’s no real way we’re going to get a (link p q) clause if p doesn’t exist. And while there can be broken links ie. a (link p q) where q doesn’t exist, there’s no way we’ll actually be asking the backlink query about a q that doesn’t exist. So no need to join these extra clauses in. That makes this now a single match query on a single relation and much more tractable.