Category: Opinion

A Replacement for PHP

I asked Quora for ideas for how a replacement PHP might look.

On the whole people are not enthusiastic. Alexander Tchitchigin had an interesting answer, but which focused on the basic theme of “once we move away from PHP’s weaknesses, we might as well use any language.

Which prompted me to write this comment elaborating what I was interested in. Plus some ideas of my own.

I agree that there are downsides to “embedded in HTML”.

But I think we can also see various what I call “pendulums” in computer science. For example between centralization and decentralization. Centralization gives you economies of scale, eliminates redundancy and makes it easier to see the wood for the trees. Decentralization makes it easier to modularize (or divide and conquer), easier to test and find bugs, easier to scale, easier to improve individual modules etc.

The pendulum oscillates because whenever one of these principles becomes more dominant, everyone starts to feel the pain and see the attraction of the other pole. And then stories start proliferating of the virtues of shifting the other way. Once everyone does, of course, there’s a pull back to the first way again. And so the pendulum continues to swing.

Right now I’m seeing this centralized / decentralized tension in ClojureScript web-frameworks. Comparing using Reagent directly with devcards vs. using Re-frame. Trying to decide whether the convenience of the modularity of keeping state decentralized in individual reagent components and being able to use devcards, outweighs the extra transparency of centralizing state in the re-frame db.

Now I think this thing about “MVC vs. templates with code” is another pendulum rather than an absolute principle.

At the end of the day, HTML is the data-structure for the application’s GUI. And the GUI data-structure does need to be fairly tightly integrated with the code. It doesn’t make sense to try to decouple them too much. You need a button attached to a handler attached to some transformation in your business logic. There’s no point trying to keep these things apart. A button without functionality makes no sense. Nor does functionality that can’t be accessed through the UI.

I’m now using Hiccup to generate HTML. And the place to do it is obviously tightly integrated with the actual functionality of the app in the code itself.

Yes, there are still some MVCish intuitions at work. But I don’t need or want a language to try to hard enforce that separation between UI and functionality either in separate files or with separate languages, when a simple DSL in the main programming language is sufficient.

But when you look at a Reagent (ie. React) component it’s basically a Huccup template with some extra code in it.

Now the brilliance of PHP, the reason it’s so popular is that :

a) it’s just there, pretty much always.

b) it simplifies simple sites considerably by automatically mapping the routing onto the directory structure of the file-system. There are many cases when that is fine. Why should I have to hand code a whole layer of routing inside my code when the file system already provides me with a logical hierarchical layout?

I think there’s still value in the “map files to pages” part of PHP. And that’s what I’d expect a “new PHP” to keep. Along with the “available everywhere” bit.

Of course, how it might look, might be more like Hiccup, a light-weight DSL rather than the verbosity of HTML. With each file implicitly mapped to a React component. Perhaps something like I started describing here : Phil Jones’ answer to What’s the best programming language for applications and GUIs?

In that other answer, I talk about what I found interesting in Eve : the event-handling within the language through “when” clauses and an implicit underlying data-structure.

And I tried to sketch what a language that brought events to Hiccup might look like :

 <pre>
    (defcomponent click-counter
        {:localstate ['counter 0]
         :when (on-click "the-button") #(reset! counter (+ @counter 1))
         :view
          [:div
             [:p "Counter clicked " @counter " times"]
             [:button {:id "the-button"} "Click Me"]
          ]
        }
    )
</pre>

Many frameworks encourage you to put components etc. in different files and directories anyway. Why not make this “official” in the same way that Python makes indentation official. And use the directory structure to infer the program structure?

A powerful modern language with the easy accessibility defaults of PHP would be a powerful combination.

Expression in Programming Languages

Source: My Quora Answer :

If Clojure is so expressive, and is basically Lisp, hasn’t there been any progress in expressivity in the last 50 years?
Well, as Paul Graham put it quite well, Lisp started as a kind of maths. That’s why it doesn’t go out of date. Or not in the short term.
You should probably judge the evolution of expressivity in programming languages by comparing them to the inventions of new concepts / theories / notations in maths (or even science), not by comparison to other kinds of technological development like faster processors or bigger hard drives.
What I’d suggest is that it turns out that function application is an amazingly powerful and general way to express “tell the computer to calculate something”.
Once you have a notation for defining, composing and applying functions (which is what Lisp is), then you already have an extraordinarily powerful toolkit for expressing programs. Add in Lisp’s macros which let you deconstruct, manipulate and reconstruct the definitions of functions, programmatically, and you basically have so much expressive power that it’s hard to see how to improve on it.
You can argue in the case of macros that, in fact, while the idea has been in Lisp for a while, the notation and semantics is still being actively fiddled with. We know that macros in principle are a good idea. But we’re still working on the right language to express them.
Perhaps we can also argue that there’s still room for evolving some other bits of expressivity. Eg. how to best express Communicating Sequential Processes or similar structures for concurrency etc. In Lisp all these things look like functions (or forms) because that’s the nature of Lisp. But often within the particular form we are still working out how best to express these higher level ideas.
Now, the fact that functions are more or less great for expressing computation doesn’t mean that the search for expressivity in programming languages has stopped. But it’s moved its focus elsewhere.
So there are three places where we’ve tried to go beyond function application (which Lisp serves admirably) and improve expression :

  • expressing constraints on our programs through types.
  • expressing data-structures
  • expressing large-scale architecture

These are somewhat intertwined, but let’s separate them.
Types
Types are the big one. Especially in languages like Haskell and the more exotic derivatives (Idris, Agda etc.) Types don’t tell the computer to DO anything more that you can tell it to do in Lisp. But they tell it what can / can’t be done in general. Which sometimes lets the compiler infer other things. But largely stops the programmer shooting themselves in the foot. Many programmers find this a great boost to productivity as it prevents many unnecessary errors during development.
Clearly, the type declarations in languages like Haskell or Agda are powerfully expressive. But I, personally. have yet to see a notation for expressing types that I really like or find intuitive and readable. So I believe that there is scope for improving the expressivity of type declarations. Now, sure some of that is adding power to existing notations like Hindley-Milner type systems. But I wouldn’t rule out something dramatically different in this area.
One big challenge is this : by definition, types cut across particular bits of computation. They are general / global / operating “at a distance”. One question is where to write this kind of information. Gather it together in standard “header” files? Or spread across the code, where it’s closest to where its used? What are the scoping rules for types? Can you have local or “inner” types? Or are all types global? What happens when data which is typed locally leaks into a wider context?
Data Structures
Lisp’s lists are incredibly flexible, general purpose data-structures. But also very low-level / “primitive”.
I don’t really know other Lisps. But it seems to me that Clojure’s introduction of both a { } notation for dictionaries / maps. And a [ ] notation for data vectors has been useful. Complex data literals can now be constructed out of these by following the EDN format. And it’s given rise to things like Hiccup for representing HTML and equivalents for other user-interfaces or configuration data. EDN is pretty similar to the way you define data in other contemporary languages like Javascript’s JSON etc. So it’s not that “radical”. But it’s nice to have these data structures completely integrated with the Clojure code representation.
Can we improve expressivity for this kind of data representation language?
I’m inclined to say that things like Markdown or YAML, that bring in white-space, make complex data-structures even more human readable and writable and therefore “expressive” than even JSON / EDN.
In most Lisps, but not Clojure, you can define reader-macros to embed DSLs of this form within programs.
So Lisps have highish expressivity in this area of declaring data. In Clojure through extending S-expressions into EDN and in other Lisps through applying reader-macros to make data DSLs.
Can we go further?
By collapsing data-structure into algebraic types, Haskell also comes up with a neat way of expressing data. With the added power of recursion and or-ed alternatives.
This leads us to imagine another line of developments for expression of data structures that brings these features. Perhaps ending up like regular or context free grammars.
Of course, you can write parser combinators in any functional language. Which gives you a reasonable way to represent such grammars. But ideally you want your grammar definition language sufficiently integrated with your programming language that you can use this knowledge of data-structure everywhere, such as pattern-matching arguments to functions.
Haskell, Clojure’s map representation, and perhaps Spec are moves in this direction.
But for real expressivity about data-structures, we’d have a full declarative / pattern-matching grammar-defining sub-language integrated with our function application language, for things like pattern matching, searching and even transformations. Think somewhere between BNF and JQuery selectors.
Shen’s “Sequent Calculus” might be giving us that. If I understand it correctly.
A third direction to increase expressivity in defining data-structures is to go beyond custom languages, and go for custom interactive editors (think things like spreadsheet grids or drawing applications for graphics) which manipulate particular recognised data types. These increase expressivity even further, but are very domain / application specific.
Architecture
“Architecture” is everywhere. It describes how different modules relate. How different services on different machines can be tied together. It defines the components of a user-interface and how they’re wired up to call-back handlers or streams of event processors. “Config files” are architecture. Architecture is what we’re trying to capture in “dependency injection”. And class hierarchies.
We need ways to express architecture, but mainly we rely either on code (programmatically constructing UIs), or more general data-structures. (Dreaded XML files.)
Or specific language features for specific architectural concerns (eg. the explicit “extends” keyword to describe inheritance in Java.)
OO / message passing languages like Smalltalk and IO do push you into thinking more architecturally than many FP languages do. Even “class” is an architectural term. OO languages push you towards thinking about OO design, and ideas like roles / responsibilities of various components or actors within the system.
Types are also in this story. To Haskell programmers, type-declarations are like UML diagrams are to Java programmers. They express large scale architecture of how all the components fit together. People skilled in Haskell and in reading type declarations probably read a great deal of the architecture of a large system just by looking at type declarations.
However, the problem with types, and OO classes etc. is that they are basically about … er … “types”. They’re great at expressing “this kind of function handles that kind of data”. Or “this kind of thing is like that kind of thing except different in these ways”.
But they aren’t particularly good to express relations between “tokens” or “concrete individuals”. For example, if you want to say “this server sits at address W.X.Y.Z” and uses a database which lives at “A.B.C.D:e”, then you’re back to config. files, dependency injection problems and the standard resources of describing data-structures. Architecture is treated as just another kind of data-structure.
Yes, good expressivity in data-structure helps a lot. So EDN or JSON beats out XML.
But, really, it feels like there’s still scope for a lot of improvement in the way we talk about these concrete relationships in (or alongside) our programs.
OK, I’m rambling …
tl;dr : function definition / application is a great way to express computation. Lisp got that right in the 1960s, and combined with macros is about as good as it gets to express computation. All the other improvements in expressivity have been developed to express other things : types and constraints, data-structures and architecture. In these areas, we’ve already been able to improve on Lisp. And can probably do even better than we’ve done so far.

Could the internet disappear within 20 years?

From a Quora answer I just made :
The protocol will still exist.
But there is now a real danger that “the internet” will have been successfully enclosed by private corporations.
What will that look like?
1. the end of net neutrality means that the phone companies / network providers can choose to prioritise packets from certain companies / services. They start to demand a premium from popular services.
2. most people don’t notice or care because they only use a few popular services anyway : Facebook, Google, Microsoft, Twitter, Pinterest, Netflix etc. These large corporations complain but start to pay the premium to be first class citizens. Packets from other services / web-addresses get deprioritized.
3. the commercial Content Delivery Networks probably play a role here. Basically anyone who wants to provide content to the public will need to go via a CDN because they’ll be the ones cutting the deals between the phone companies and any company smaller than Facebook and Google. That’s probably how Quora and Pinterest etc. will keep going.
4. Meanwhile, governments will continue to try to reassert control over internet content. In the name of clamping down on piracy, pornography, fake news, terrorism, online harassment, illegal crypto-currencies etc. they’ll start putting more pressure on online services to police their content. Facebook and Google will be increasingly required to co-operate. Smaller services will find they need to be vetted to get onto the CDN.
5. Also networking “hardware” is increasingly moved into software. Routers become generic boxes whose protocols are just software configuration.
6. Finally, phone companies, CDNs and router companies figure out some “improved” protocols which make the shaping and inspection of traffic more efficient. Stripped of the need to route generic packets from anywhere to anywhere, this new infrastructure, brought through a software update, is sold to the consumers as an “upgrade”.
7. By now, consumer IP is dead. Email protocols stop working. (Google and other “web-mail” providers arrange an alternative that looks the same but works over the new protocol. Or perhaps integrates directly with other messaging providers.) Communication is via Facebook and Twitter which work on the new protocol. Other P2P protocols that rely on IP also stop working. No more bittorrent, btsync, syncthing, webrtc etc.
IP survives as a “legacy system” for corporate users of course. Too many large companies still depend on it. But it will now be expensive, like most “enterprise” kit.
Within 20 years, we’ll find that the remaining “free” internet, where any ordinary person can hire a web-server and put up their own content independent of any of the major corporations or systematic government oversight is reduced to a tiny minority of hobbyists like amateur radio. Most people never access it or even know of its existence.
If you don’t like this scenario, NOW is the time to do something about it.
Not just do what you can to defend net-neutrality, but start to use the free / peer-to-peer, open protocols that we still have. Run your own web-site or blog. Use RSS to share content. Use webrtc to do your video conferencing, and bittorrent to share legitimate files. Do what you can to make sure that there is just too much demand and usage of the open protocols and internet that the large corporations can’t afford to try to switch it off because they know they’ll be losing too many customers.
Once most people aren’t using the internet, taking it away becomes easy.

What's wrong with C++?

Another Quora answer : Phil Jones’s answer to Why is C++ considered a bad language?
This is one of those rare occasions I disagree with Simon Kinahan; although his answer sets the scene for this one.
[Simon says that C++ isn’t a bad language. It’s the right choice if you need low-level memory control and the ability to build powerful higher-level abstractions.]
C++ is a bad language because it’s built on a flawed philosophy : which is that you should add power to a language by kludging it in “horizontally” in the form of libraries rather than “vertically” by building new Domain Specific Languages to express it.
Stroustrup is very explicit about this, rhetorically asking “why go to other languages for new features when you can add them as libraries in C++?”
Well, the answer is, adding new higher level conceptual thinking in the form of a library doesn’t really hide the old thinking from you. Or allow you to abandon it.
C++’s abstractions leak, more or less on purpose. Because you can never escape the underlying low-level thinking when you’re just using this stuff via libraries. You are stuck with the syntax and mindset of the low-level, even as you add more and more frameworks on top.
Yes, you can explicitly add garbage collection to a C++ program. But you can’t relax and stop thinking about memory management. It can’t disappear completely beneath the horizon of things you need to be aware of the way it does in Java.
Yes, you can have higher-level strings, that can abstract Unicode-ness etc. via a library. But you can never be sure that you won’t confront strings that are simple byte-arrays coming from another part of your large system.
Etc.
C++’s ability to build high-level abstractions uncomfortably falls between two stools.
It encourage you to think you can and should be building large applications full of application logic. But doesn’t give you the real abstracting power to focus only at that application level.
This explains the otherwise mysterious paradox that C is a good language, so how could something that is “C plus more stuff” possibly be a bad one? Well, it’s exactly the “more plussing” that’s the problem.
With C, you KNOW you should only use it to build relatively small things. It doesn’t pretend to offer you mechanisms to build big things. And so you turn to the obvious tools for scaling up : lex and yacc to build small scripting languages, the Unix pipe to orchestrate multiple small tools . C++ gives you just enough rope to hang yourself. Just enough powerful abstraction building capacity in the language itself that you (or that guy who used to work in your company 15 years ago ) thought it might be possible to reinvent half of Common Lisp’s data-manipulation capability and half an operating systems’ worth of concurrent process management inside your sprawling monolithic application.
I don’t see why it shouldn’t be possible to combine low-level memory management and efficiency with high level abstraction building. But proper abstraction building requires something more like macros or similar capabilities to make a level of expression which really transcends the low level. That’s exactly what C++ lacks.

The Bug is Your Fault

My Quora answer : Phil Jones’s answer to What are some great truths of computer programming?

  1. The bug is your fault

  2. The bug is your fault

  3. The bug is your fault

  4. The bug is your fault

  5. The bug is your fault.

  6. No. The bug really IS your fault.

  7. The bug is your fault. Because you made an assumption that wasn’t true.

  8. The bug is your fault. And it won’t go away until you check your assumptions, find the one that isn’t true, and change it.

What do you think computers will be like in 10 years?

My Quora answer that’s pretty popular : (1) Phil Jones’s answer to What do you think computers will be like in 10 years?
Related to my previous story of trying to use the CHIP for work.

This is a $9 CHIP (Get C.H.I.P. and C.H.I.P. Prto)

It runs Debian. A couple of weeks ago, I took one travelling with me instead of my laptop to see what it would be like to use for real work.
I successfully ran my personal “productivity” software (three Python based wiki servers and some code written in Racket) from it. It also has a browser, Emacs and I was doing logic programming with minikanren library in Python. It runs Sunvox synth and CSOUND too, though I wasn’t working on those on this trip.
In 10 years time, that computing power will be under a dollar. And if anyone can be bothered to make it in this format, the equivalent of this Debian machine will be tantamount to free.
Of course most of that spectacular power will be wasted on useless stuff. But, to re-emphasize, viable computing power that you can do real work with, will be “free”.
The pain is the UI. How would we attach real keyboards, decent screens etc when we need them?
I HOPE that people will understand this well enough that our current conception of a “personal computer” will explode into a “device swarm” of components that can be dynamically assembled into whatever configuration is convenient.
I, personally, would LIKE the main processor / storage to live somewhere that’s strongly attached to my body and hard to lose (eg. watch or lanyard). I’d like my “phone” to become a cheap disposable touch-screen for this personal server rather than the current repository of my data.
I bought a cheap bluetooth keypad for about $8. It was surprisingly OK to type on, but connections with the CHIP were unreliable. In 10 years time, that ought to be fixed.
So, in 10 years time, I personally want a computer on my wrist that’s powerful to do all my work with (that means programming and creating music). That can be hooked up to some kind of dumb external keyboard / mouse / screen interface (today the Motorola Lapdock is the gold-standard) that costs something like $20. Sure, I’ll probably want cloud resources for sharing, publishing, storage and even high-performance processing, AI and “knowledge” etc.
And, of course, I want it to run 100% free-software that puts me in control.
This is all absolutely do-able.

The End of Dynamic Languages

A comment I made over on The End of Dynamic Languages

The problem with the static / dynamic debate is that the problems / costs appear in different places.
In static languages, the compiler is a gatekeeper. Code that gets past the gatekeeper is almost certainly less buggy than code that doesn’t get past the gatekeeper. So if you count bugs in production code, you’ll find fewer in statically typed languages.
But in static languages less code makes it past the compiler in the first place. Anecdotally, I’ve abandoned more “let’s just try this to see if its useful” experiments in Haskell where I fought the compiler and lost, than in Clojure, where the compiler is more lenient. (Which has the knock on effect of my trying fewer experiments in Haskell in the first place, and writing more Clojure overall.)
Static typing ensures that certain code runs correctly 100% of the time, or not at all.
But sometimes it’s acceptable for code to run 90% of the time, and to have a secondary system compensate for the 10% when it fails. There might even be cases where 90% failure and 10% success can still be useful. But only dynamic languages give you access to this space of “half-programs” that “work ok, most of the time”. Static languages lock you out of it entirely. They oblige you to deal correctly with all the edge cases.
Now that’s very good, you’ve dealt with the edge cases. But what if there’s an edge case that turns up extremely rarely, but costs three months of programmer time to fix. In a nuclear power station, that’s crucial. On a bog-standard commercial web-site, that’s something that can safely be put off until next year or the year after. But a static language won’t allow you that flexibility.
The costs of static and dynamic languages turn up in different places. Which is why empirical comparisons are still hard.

Conway’s Corollary

Ian Bicking’s post on Conway’s Corollary is a must-read thought on isomorphisms between the organization and product structures.

What, asks Bicking, if we don’t fight this, but embrace it. Organizational structures are allegedly for our benefit. Why not allow them to shape product? Or when this is inappropriate why not recognize that the two MUST be aligned and if product can’t follow organization, we should refactor organization to reflect and support product.

The Android Studio / Gradle Experience

Im sure my answer / comment on What is Gradle in Android Studio? will get downvoted into oblivion with short-shrift fairly soon. (Maybe deservedly).

But I’ll make it here :

[quote]
At the risk of being discursive I think behind this is the question of why the Android Studio / Gradle experience is so bad.
Typical Clojure experience :
* download project with dependencies listed in project.clj.
* Leiningen gets the dependencies thanks to Clojars and Maven.
* Project compiles.
Typical Android Studio / Gradle experience :
* “Import my Eclipse project”.
* OK project imported.
* Gradle is doing it’s thang … wait … wait … wait … Gradle has finished.
* Compile … can’t compile because I don’t know what an X is / can’t find Y library.
I’m not sure this is Gradle’s fault exactly. But the “import from Eclipse project” seems pretty flaky. For all of Gradle’s alleged sophistication and the virtues of a build-system, Android Studio just doesn’t seem to import the build dependencies or build-process from Eclipse very well.
It doesn’t tell you when it’s failed to import a complete dependency graph. The Android Studio gives no useful help or tips as to how to solve the problem. It doesn’t tell you where you can manually look in the Eclipse folders. It doesn’t tell you which library seems to be missing. Or help you search Maven etc. for them.
In 2016 things like Leiningen / Clojars, or node’s npm, or Python’s pip, or the Debian apkg (and I’m sure many similar package managers for other languages and systems) all work beautifully … missing dependencies are thing of the past.
Except with Android. Android Studio is now the only place where I still seem to experience missing-dependency hell.
I’m inclined to say this is Google’s fault. They broke the Android ecosystem (and thousands of existing Android projects / online tutorials) when they cavalierly decided to shift from Eclipse to Android Studio / Gradle without producing a robust conversion process. People whose projects work in Eclipse aren’t adapting them to AS (presumably because it’s a pain for them). And people trying to use those projects in AS are hitting the same issues.
And anyway, if Gradle is this super-powerful build system, why am I still managing a whole lot of other dependencies in the sdk manager? Why can’t a project that needs, say, the ndk specify this in its Gradle file so that it gets automatically installed and built-against when needed? Why is NDK special? Similarly for target platforms? Why am I installing them explicitly in the IDE rather than just checking my project against them and having this all sorted for me behind the scenes?
[/quote]