Fellow Journalistas!
Martin has asked me to help out with NextJournal in a variety of ways. I'm writing this notebook to make explicit some of my ideas around the project, but I hope it will ultimately serve as a starting point for a dialogue in which we incorporate everyone's perspective.
Current Product
When I first started talking with Martin about NextJournal, I mainly gave him advice about how to quickly find out if users want to pay for a more reproducible version of Jupyter, which -- for better or worse -- is how many will think of NextJournal when they see it. It now seems clear, a couple of years on, that what we're currently offering is not compelling enough to attract a large audience, and I'm not convinced that slight improvements to the current product can out-compete "free".
So how do we discover/invent the thing the world wants from us? We need to try many things until we find something that we ourselves are excited to use. And for that, we should try to build an architecture optimized for experimentation.
Architecture recommendations
Firstly, I'd like to separate NextJournal-the-product from something that I'll refer to in this document as the "NextJournal Toolkit". In doing so, I'd like to encourage us to think of the pieces you have built over the last few years -- browser-hosted editing, server side execution in a variety of languages, real-time collaboration with an immutable history, and so on -- as independent components that can be remixed to make many things rather than as parts of an single monolithic product.
If we take this approach to each architectural decision, it will make the code much easier to re-purpose for a wide variety of speculative product visions with a minimum of work. This property is especially useful at this stage given that it's still unclear whether NextJournal is anywhere close to having product-market fit.
Another thing we should strive for at the toolkit level is making components that are maximally flexible. If we have the choice between a mechanism that allows only one policy or a mechanism that gives us a spectrum of possibilities, we should err toward the latter (so long as we don't end up in the weeds of design paralysis).
Some example approaches of this type:
the command bar interface provides a general UX paradigm that can be used for almost anything that involves interacting with entities on screen using the keyboard and mouse
a simplified runner interface not only improves reliability and startup time, but allows us to play with different cell/runner cardinalities with less work
"graph nodes and links for everything" not only simplifies the code for the features we have now, but makes things like transclusion easier to implement at different granularities and also allows us to experiment with more radical structures
using dataflow between the graph of cells would give a better way to handle state and dependencies, which in turn might let us innovate past some of what's annoying about other notebook environments
The NJ Toolkit is not far from being able to provide the building blocks needed to prototype a fancy outliner in which an entire notebook can be transcluded under a single bullet point, a polyglot Hypercard interface, or a new kind of spreadsheet.
There are, of course, so many more things that we can do, but I hope that this short list will shed some light on the larger vision of the kinds of things we would be able to try (quickly and cheaply) if we choose wisely in terms of architecture.
Multiplayer
After using NJ for collaboration within our group, it seems clear that we need a better story on how things work in that context -- one that goes far beyond syncing edits. There's already thought going into comments, but in addition we should also take some ideas from services like GitHub. A quick wish list:
change notifications when someone edits a shared notebook
a diff-like mechanism to see changes since I last looked, maybe with visual replay slider to explore what happened
comments that don't just imitate, say, Google Docs
Because we have an immutable log of changes, it would be particularly exciting to combine these features into a time-travel-able interface that allows floating comments to be attached to ranges of text or code at specific versions, giving a lightweight way to handle some of the use cases that are currently spread out between version control and issue tracking systems. If everything is links and nodes, why can't a code comment link to a text paragraph or a paragraph to an s-expression?