The Node Solid Server

A dive into Solid, including a complete how-to for starting your own Solid server using Node.js.

Solid is a specification that lets people store their data in federated data stores called Pods. The information on each Pod is linked through a person's online identity. Identity is addressed with a WebID, which is just an HTTP URL that points to a profile.

As depicted in the image below, the personal data pod can include things like pictures, calendar information, and personal contacts. The left side of the image depicts how this information is stored in walled gardens on today's internet platforms. The right shows how the data is stored in a personal data pod. Applications can simply query a Pod to get the information they need to provide their service.

Figure: Applications can query a pod to get the data they need. CC-by-4.0 Ruben Verborgh

Individuals can therefore manage their personal information in whatever place they choose. The Solid standard makes it easy to grant and revoke access to personal information to specific individuals, organizations, and applications. This is a win for personal privacy.

Pods make the web more secure because data is not pooled in a single place. Personal information is spread across a federation of servers. While Solid does not prevent individual targeting, its adoption would curb the massive data breaches that have become commonplace.

Individual Pods can move between servers with very little friction. If a person does not like a server's security practices, terms of service, or notices other forms of abuse, they can simply move their Pod to another server or even host their Pod on their own server.

That last sentence is what this notebook is about. Running the notebook will stand up a Solid server that can be found on the public internet. Reading the notebook will take you through each step along the way, including how to access the server from an outside Solid application. Let's get started.

Ngrok

The Solid server runs locally; Ngrok will provide a public URL for the local Solid server.

I have configured Ngrok to make the notebook available at https://nextjournal.ngrok.io. The full ngrok.yml file can be found in the Appendix. See the Ngrok Configuration section.

A reserved subdomain (nextjournal in this case) is only available through the paid plan. If you don't have a paid plan, you'll have to provide Ngrok's generated subdomain to the Solid server before you start it up.

I'll add my authentication token to the ngrok.yml configuration file. The token is held in an environmental variable. For more information on using secrets and setting variables in Nextjournal, see the platform's documentation. 

echo 'authtoken:' $ngrok | cat - ngrok.yml > /tmp/out && mv /tmp/out ngrok.yml
1.4s

Now it's time to start the ngrok service in the background using nohup. 

nohup ./ngrok start --all --config="/ngrok.yml" --log="/tmp/ngrok.log" &> /tmp/ngrok-run.log & sleep 1
2.4s

The Ngrok tunnel is now live at https://nextjournal.ngrok.io/. The next step is standing up a Solid server at https://localhost:8443.

Solid

There are two Solid server implementations that run on Node.js. The Community Solid Server (documentation) is newer, but this notebook uses the Node Solid Server (documentation). Both are open-source Node.js implementations of the same Solid standard, independently maintained.

The Solid server provides the necessary tools to store and manage access to data held in Pods on the sever.

  • The server provides the tools to assert your identity (you are who you say you are, Authentication)

  • The server provides the tools to use your identity (sharing personal information, having conversations on the internet, etc..., Authorization)

Authentication

Once you prove you are in control of your Pod, the authentication is done. This authentication step is not application-specific, it's Pod-specific. Therefore a person does not need a unique name and password for each application they use, they only need a single name and password for the Pod.

Authorization

A Solid application will ask you to authorize certain access permissions. The ultimate authority on who has access to your information does not live within the application (which is generally true today), it lives within a person's Pod.

Install the server

Now that some of the Solid server's functions are clear, it's time to install the server using NPM.

npm install -g solid-server
79.2s

Configure the Server

The server's configuration is held in config.json below. A few specific notes.

  • The serverUri must be set to the reverse proxy's URI, https://nextjournal.ngrok.io.

  • The port is set to serve information over SSL at 8443. This number matches the port setting of Ngrok's addr (address), https://localhost:8443. See ngrok.yml in the Appendix for more context.

Setting webid to true enables authentication and access control by establishing a relation between the WebID URI and the person's public keys

{ "root": "/solid/data",
  "port": "8443",
  "serverUri": "https://nextjournal.ngrok.io",
  "webid": true,
  "mount": "/",
  "configPath": "./config",
  "configFile": "./config.json",
  "dbPath": "./.db",
  "sslKey": "/keys/privkey.pem",
  "sslCert": "/keys/fullchain.pem",
  "multiuser": false,
  "server": {
    "name": "The Nextjournal Solid Server",
    "description": "This is a Solid server running in a Nextjournal notebook.",
    "logo": ""
  }
}
JSON

The sslKey and sslCert still need to be generated and added to the environment. This can be done using the openssl req command. OpenSSL is a general purpose cryptographic SSL/TLS (Secure Sockets Layer/Transport Layer Security) toolkit, req is the specific command to fulfill a X.509 Certificate Signing Request.

Here is the configuration file that provides the necessary cryptographic information and fulfills the Distinguished Name requirements.

[ req ]
 default_bits       = 2048
 default_md         = sha256
 prompt             = no
 encrypt_key        = no # do not encrypt the keypair, same as -nodes
 distinguished_name = nextjournal_server
[ nextjournal_server ]
 countryName            = "US"
 stateOrProvinceName    = "Illinois"
 localityName           = "Chicago"
 organizationName       = "Beyond the Frame"
 organizationalUnitName = "None"
 commonName             = "example.com"
 emailAddress           = "example@example.com"

Run the req command to generate a 2048-bit (default_bits) private key (-keyout /keys/privkey.pem) and then fulfill a certificate request (-out /keys/fullchain.pem).

mkdir /keys
openssl req -outform PEM -keyform PEM -new -x509 -keyout /keys/privkey.pem -config openssl.cnf -days 365 -out /keys/fullchain.pem
1.1s

Finally, the Solid server needs a folder to serve. The folder is set by root in the configuration file above. Make the corresponding directory:

mkdir -p /solid/data
0.6s

Run the Server

solid start
2979.4s

Open another window and load https//nextjournal.ngrok.io. This will be the result:

A Solid server

Some explanation on the magic happening behind the scenes is worthwhile.

Solid requires valid SSL certificates to function properly. They normally cannot be self-signed. The command solid-test start can be used to accept self-signed certificates. The command is found in solid-server/bin under the opt/nodejs folder.

However, this notebook gets away with self-signed certificates because Ngrok's servers use the registered ngrok.com certificates. Ngrok assumes that the notebook is hosted privately, so it doesn't need to check our self-signed certificates. If this was production instance, we would want to use our own certificates rather than Ngrok's.

Working With Pods

WebID

Click the blue Register button at https//nextjournal.ngrok.io to create a WebID. The profile associated with the WebID will be at this URL: https://nextjournal.ngrok.io/profile/card#me. Use this identifier to interact with Solid pods and apps.

Note: if multiuser was set to true in the config.yml, this WebID could be customized with a user's name to differentiate it from other Pods on this Solid server.

Schema

Relationships are the stuff of which information is made. Just about everything in the information system looks like a relationship.

~ Data and Reality (87)

Now that the personal datastore is addressable at https://nextjournal.ngrok.io/profile/card#me, web applications can request personal information using a standard schema.

The schema does more than simply address the data, it can describe relationships between bits of information.

For example, you may host a photograph on your Pod that you share through an application. If I sign into the same application using my WebID, I might be able to leave a comment on the photograph. That comment could exist on my Pod because I am the original author, but you could still view it through the application interface. The relationship between the photo and the comment can be depicted by a schema. It may look something like:

<https://mypod.solid/comments/36756>
    <http://www.w3.org/ns/oa#hasTarget>
        <https://yourpod.solid/photos/beach>

As Ruben Verborgh illustrates in Solid: Personal Data Management Through Linked Data, the comment on my Pod is linked to the picture on your Pod through a type defined in a schema. We know that my comment is a response to your picture instead of the other way around.

Like the markup standards that ensure a website is displayed correctly, schema standards for the world wide web promised to create semantic relationships between pieces of information on the web. This effort is at least 20 years old and their website at schema.org just turned 10.

Wikidata is one of the most prominent sites built on structured data. They use an open schema called RDF. RDF graphs are sets of subject-predicate-object triples (e.g. <Bob> subject <is a> predicate <human> object). The schema also provides a data-modelling vocabulary for RDF data (e.g. <human> is a subClassOf <mammal>).

These structures allow programmers to create vocabularies - which are essentially rules about what information can and cannot be asserted about a domain. For example, a programmer can build a social graph using the Friend of a Friend vocabulary.

Solid is built to use RDF. Common vocabularies can parallel the power of social networks, shared calendars, document collaboration, music libraries, etc.... These vocabularies describe what interactions are possible between Pods. Since vocabularies are built on open standards, independent data sources and applications don't have to build their own custom API. Interoperability is baked in, centered on web identity.

Apps

Solid supports a rich ecosystem of applications that compete on the merits of their service. The best-performing applications will naturally be complimentary with other services that process the data on your pod. As Jacob O'Bryant opines on Solid:

There's an interesting analogy to functional programming: it's better to have 100 apps operate on 1 database than 10 apps operate on 10 databases.

Any one of Solid's official list of apps can be used to read and write data to this notebook's Pod. For example, Notepod is a simple Solid-compatible note-taking app. Visit the app, click Connect, and add this Pod's URL.

Locate the Solid server

When asked to authenticate, use the username and password previously created on the Solid server.

Solid authentication

Grant the requested permissions and the app is connected. Anything created in the app is stored in the Pod. It's worth re-emphasizing that the app doesn't have its own username and password. Solid not only solves the problem of siloed data, it solves the proliferation of usernames and passwords that plague today's internet users.

Conclusion

I hope this notebook helped convey the power of Solid and how easy it is to get started with your own Solid server. The technology will become more powerful and useful as the federated network of servers grow. Getting started is as easy as remixing this notebook and configuring Ngrok to your own subdomain (paid Ngrok plan) or dynamically adding the generated subdomain to your Solid server configuration (free Ngrok plan).

Find me at my blog where I discuss information science, art, and narrative: Beyond the Frame. I can also be reached at Mastodon and Twitter.

Appendix

Federated Servers

If you don't like the server that hosts your Pod, simply move it to another server. Here is a list of current Solid servers:

You can choose a server based for a variety of ethical or technical reasons. Perhaps you disagree with a change in the terms of service, want more storage capacity, or high-speed transfer. This is illustrated in the data market section below:

Figure: Data and app markets. CC-by-4.0 Ruben Verborgh

Ngrok Configuration

If you remix this notebook, you can use the same Ngrok service, but your subdomain will be random, rather than nextjournal. All other configurations are the same.

When you change the subdomin, remember that the full domain, subdomain, domain (.ngrok), and TLD (.io) must match in three places:

  1. This Ngrok configuration file at subdomain.

  2. The serverUri found in Solid's config.json above.

  3. The URL in your web browser's address bar.

Here is the Ngrok configuration used in this notebook:

region: us
console_ui: true
tunnels:
  btf:
    proto: http
    addr: https://localhost:8443
    subdomain: nextjournal
    bind_tls: true
YAML

Setting the https prefix of https://localhost:8443 forces Ngrok to speak HTTPS to the local server. The 8443 port suffix directs it to the Solid server's port.

  • proto: http - the protocol. The http protocol covers both HTTP and HTTPS

  • addr: https://localhost:8443 use the full URL to force HTTPS (equivalent to ngrok http https://localhost:8443 on the command line)

  • bind_tls - without this setting both HTTP and HTTPS will be available from ngrok

    • bind_tls: true - ngrok only listens on an HTTPS tunnel endpoint

    • bind_tls: false - ngrok only listens on an HTTP tunnel endpoint

  • host_header: localhost - unused. But some APIs will want to see the host header of your app rather than 'ngrok'

References

This work is licensed under a Creative Commons Attribution 4.0 International License.

Runtimes (1)