The Node Solid Server
A dive into Solid, including a complete how-to for starting your own Solid server using Node.js.
Solid is a specification that lets people store their data in federated data stores called Pods. The information on each Pod is linked through a person's online identity. Identity is addressed with a WebID, which is just an HTTP URL that points to a profile.
As depicted in the image below, the personal data pod can include things like pictures, calendar information, and personal contacts. The left side of the image depicts how this information is stored in walled gardens on today's internet platforms. The right shows how the data is stored in a personal data pod. Applications can simply query a Pod to get the information they need to provide their service.
Figure: Applications can query a pod to get the data they need. CC-by-4.0 Ruben Verborgh
Individuals can therefore manage their personal information in whatever place they choose. The Solid standard makes it easy to grant and revoke access to personal information to specific individuals, organizations, and applications. This is a win for personal privacy.
Pods make the web more secure because data is not pooled in a single place. Personal information is spread across a federation of servers. While Solid does not prevent individual targeting, its adoption would curb the massive data breaches that have become commonplace.
Individual Pods can move between servers with very little friction. If a person does not like a server's security practices, terms of service, or notices other forms of abuse, they can simply move their Pod to another server or even host their Pod on their own server.
That last sentence is what this notebook is about. Running the notebook will stand up a Solid server that can be found on the public internet. Reading the notebook will take you through each step along the way, including how to access the server from an outside Solid application. Let's get started.
Ngrok
The Solid server runs locally; Ngrok will provide a public URL for the local Solid server.
I have configured Ngrok to make the notebook available at https://nextjournal.ngrok.io
. The full ngrok.yml
file can be found in the Appendix. See the Ngrok Configuration section.
A reserved subdomain (nextjournal
in this case) is only available through the paid plan. If you don't have a paid plan, you'll have to provide Ngrok's generated subdomain to the Solid server before you start it up.
I'll add my authentication token to the ngrok.yml
configuration file. The token is held in an environmental variable. For more information on using secrets and setting variables in Nextjournal, see the platform's documentation.
echo 'authtoken:' $ngrok | cat - ngrok.yml > /tmp/out && mv /tmp/out ngrok.yml
Now it's time to start
the ngrok
service in the background using nohup
.
nohup ./ngrok start --all --config="/ngrok.yml" --log="/tmp/ngrok.log" &> /tmp/ngrok-run.log & sleep 1
The Ngrok tunnel is now live at https://nextjournal.ngrok.io/. The next step is standing up a Solid server at https://localhost:8443.
Solid
There are two Solid server implementations that run on Node.js. The Community Solid Server (documentation) is newer, but this notebook uses the Node Solid Server (documentation). Both are open-source Node.js implementations of the same Solid standard, independently maintained.
The Solid server provides the necessary tools to store and manage access to data held in Pods on the sever.
The server provides the tools to assert your identity (you are who you say you are, Authentication)
The server provides the tools to use your identity (sharing personal information, having conversations on the internet, etc..., Authorization)
Authentication
Once you prove you are in control of your Pod, the authentication is done. This authentication step is not application-specific, it's Pod-specific. Therefore a person does not need a unique name and password for each application they use, they only need a single name and password for the Pod.
Authorization
A Solid application will ask you to authorize certain access permissions. The ultimate authority on who has access to your information does not live within the application (which is generally true today), it lives within a person's Pod.
Install the server
Now that some of the Solid server's functions are clear, it's time to install the server using NPM.
npm install -g solid-server
Configure the Server
The server's configuration is held in config.json
below. A few specific notes.
The
serverUri
must be set to the reverse proxy's URI,https://nextjournal.ngrok.io
.The
port
is set to serve information over SSL at8443
. This number matches the port setting of Ngrok'saddr
(address),https://localhost:8443
. Seengrok.yml
in the Appendix for more context.
Setting webid
to true
enables authentication and access control by establishing a relation between the WebID URI and the person's public keys
{ "root": "/solid/data",
"port": "8443",
"serverUri": "https://nextjournal.ngrok.io",
"webid": true,
"mount": "/",
"configPath": "./config",
"configFile": "./config.json",
"dbPath": "./.db",
"sslKey": "/keys/privkey.pem",
"sslCert": "/keys/fullchain.pem",
"multiuser": false,
"server": {
"name": "The Nextjournal Solid Server",
"description": "This is a Solid server running in a Nextjournal notebook.",
"logo": ""
}
}
The sslKey
and sslCert
still need to be generated and added to the environment. This can be done using the openssl req
command. OpenSSL is a general purpose cryptographic SSL/TLS (Secure Sockets Layer/Transport Layer Security) toolkit, req
is the specific command to fulfill a X.509 Certificate Signing Request.
Here is the configuration file that provides the necessary cryptographic information and fulfills the Distinguished Name requirements.
[ req ]
default_bits = 2048
default_md = sha256
prompt = no
encrypt_key = no # do not encrypt the keypair, same as -nodes
distinguished_name = nextjournal_server
[ nextjournal_server ]
countryName = "US"
stateOrProvinceName = "Illinois"
localityName = "Chicago"
organizationName = "Beyond the Frame"
organizationalUnitName = "None"
commonName = "example.com"
emailAddress = "example@example.com"
Run the req
command to generate a 2048-bit (default_bits
) private key (-keyout /keys/privkey.pem
) and then fulfill a certificate request (-out /keys/fullchain.pem
).
mkdir /keys
openssl req -outform PEM -keyform PEM -new -x509 -keyout /keys/privkey.pem -config openssl.cnf -days 365 -out /keys/fullchain.pem
Finally, the Solid server needs a folder to serve. The folder is set by root
in the configuration file above. Make the corresponding directory:
mkdir -p /solid/data
Run the Server
solid start
Open another window and load https//nextjournal.ngrok.io. This will be the result:
![](/data/QmcjTsi3rhm5r9TvZfBBccTzRZ6B65TwQz7WEuR6anbGQW?filename=solid-prototype.png&content-type=image/png)
A Solid server
Some explanation on the magic happening behind the scenes is worthwhile.
Solid requires valid SSL certificates to function properly. They normally cannot be self-signed. The command solid-test start
can be used to accept self-signed certificates. The command is found in solid-server/bin
under the opt/nodejs
folder.
However, this notebook gets away with self-signed certificates because Ngrok's servers use the registered ngrok.com
certificates. Ngrok assumes that the notebook is hosted privately, so it doesn't need to check our self-signed certificates. If this was production instance, we would want to use our own certificates rather than Ngrok's.
Working With Pods
WebID
Click the blue Register button at https//nextjournal.ngrok.io to create a WebID. The profile associated with the WebID will be at this URL: https://nextjournal.ngrok.io/profile/card#me. Use this identifier to interact with Solid pods and apps.
Note: if multiuser
was set to true
in the config.yml
, this WebID could be customized with a user's name to differentiate it from other Pods on this Solid server.
Schema
Relationships are the stuff of which information is made. Just about everything in the information system looks like a relationship.
~ Data and Reality (87)
Now that the personal datastore is addressable at https://nextjournal.ngrok.io/profile/card#me, web applications can request personal information using a standard schema.
The schema does more than simply address the data, it can describe relationships between bits of information.
For example, you may host a photograph on your Pod that you share through an application. If I sign into the same application using my WebID, I might be able to leave a comment on the photograph. That comment could exist on my Pod because I am the original author, but you could still view it through the application interface. The relationship between the photo and the comment can be depicted by a schema. It may look something like:
<https://mypod.solid/comments/36756>
<http://www.w3.org/ns/oa#hasTarget>
<https://yourpod.solid/photos/beach>
As Ruben Verborgh illustrates in Solid: Personal Data Management Through Linked Data, the comment on my Pod is linked to the picture on your Pod through a type defined in a schema. We know that my comment is a response to your picture instead of the other way around.
Like the markup standards that ensure a website is displayed correctly, schema standards for the world wide web promised to create semantic relationships between pieces of information on the web. This effort is at least 20 years old and their website at schema.org just turned 10.
Wikidata is one of the most prominent sites built on structured data. They use an open schema called RDF. RDF graphs are sets of subject-predicate-object triples (e.g. <Bob>
subject <is a>
predicate <human>
object). The schema also provides a data-modelling vocabulary for RDF data (e.g. <human>
is a subClassOf
<mammal>
).
These structures allow programmers to create vocabularies - which are essentially rules about what information can and cannot be asserted about a domain. For example, a programmer can build a social graph using the Friend of a Friend vocabulary.
Solid is built to use RDF. Common vocabularies can parallel the power of social networks, shared calendars, document collaboration, music libraries, etc.... These vocabularies describe what interactions are possible between Pods. Since vocabularies are built on open standards, independent data sources and applications don't have to build their own custom API. Interoperability is baked in, centered on web identity.
Apps
Solid supports a rich ecosystem of applications that compete on the merits of their service. The best-performing applications will naturally be complimentary with other services that process the data on your pod. As Jacob O'Bryant opines on Solid:
There's an interesting analogy to functional programming: it's better to have 100 apps operate on 1 database than 10 apps operate on 10 databases.
Any one of Solid's official list of apps can be used to read and write data to this notebook's Pod. For example, Notepod is a simple Solid-compatible note-taking app. Visit the app, click Connect, and add this Pod's URL.
![](/data/QmY1Hu2BuQsa6r1abrSpT3hWmJjMhQSVpywy9Z5PvVp3Pe?filename=notepod.png&content-type=image/png)
Locate the Solid server
When asked to authenticate, use the username and password previously created on the Solid server.
![](/data/QmYEjhAYt2oGpQG4dQfZQR57ZXB76Li5JQS6xf1YrXAQy2?filename=notepod-2.png&content-type=image/png)
Solid authentication
Grant the requested permissions and the app is connected. Anything created in the app is stored in the Pod. It's worth re-emphasizing that the app doesn't have its own username and password. Solid not only solves the problem of siloed data, it solves the proliferation of usernames and passwords that plague today's internet users.
Conclusion
I hope this notebook helped convey the power of Solid and how easy it is to get started with your own Solid server. The technology will become more powerful and useful as the federated network of servers grow. Getting started is as easy as remixing this notebook and configuring Ngrok to your own subdomain (paid Ngrok plan) or dynamically adding the generated subdomain to your Solid server configuration (free Ngrok plan).
Find me at my blog where I discuss information science, art, and narrative: Beyond the Frame. I can also be reached at Mastodon and Twitter.
Appendix
Federated Servers
If you don't like the server that hosts your Pod, simply move it to another server. Here is a list of current Solid servers:
Inrupt Pod Spaces by Inrupt, Inc. hosted at Amazon in the USA. I have a Pod and public profile on this server.
inrupt.net by Inrupt, Inc. hosted at Amazon in the USA
solidcommunity.net by Solid Project hosted at Digital Ocean in the UK. I have a Pod and public profile on this server.
solidweb.org by Solid Grassroots hosted at Hosteurope in Germany
trinpod.us by Graphmetrix, Inc. hosted at Amazon in the USA
When this notebook is running, it is also a Solid server at https://nextjournal.ngrok.io
You can choose a server based for a variety of ethical or technical reasons. Perhaps you disagree with a change in the terms of service, want more storage capacity, or high-speed transfer. This is illustrated in the data market section below:
Figure: Data and app markets. CC-by-4.0 Ruben Verborgh
Ngrok Configuration
If you remix this notebook, you can use the same Ngrok service, but your subdomain
will be random, rather than nextjournal
. All other configurations are the same.
When you change the subdomin
, remember that the full domain, subdomain, domain (.ngrok
), and TLD (.io
) must match in three places:
This Ngrok configuration file at
subdomain
.The
serverUri
found in Solid'sconfig.json
above.The URL in your web browser's address bar.
Here is the Ngrok configuration used in this notebook:
region us
console_uitrue
tunnels
btf
proto http
addr https //localhost8443
subdomain nextjournal
bind_tlstrue
Setting the https
prefix of https://localhost:8443
forces Ngrok to speak HTTPS to the local server. The 8443
port suffix directs it to the Solid server's port.
proto: http
- the protocol. Thehttp
protocol covers both HTTP and HTTPSaddr:
https://localhost:
8443
use the full URL to force HTTPS (equivalent tongrok http https://localhost:8443
on the command line)bind_tls
- without this setting both HTTP and HTTPS will be available from ngrokbind_tls: true
- ngrok only listens on an HTTPS tunnel endpointbind_tls: false
- ngrok only listens on an HTTP tunnel endpoint
host_header: localhost
- unused. But some APIs will want to see the host header of your app rather than 'ngrok'
References
Kent, William, and Steve Hoberman. 2012. Data And Reality. 3rd ed. Westfield, N.J.: Technics Publications.
‘Linked Data – Design Issues’. Accessed 19 August 2021. https://www.w3.org/DesignIssues/LinkedData.html.
Verborgh, Ruben. ‘Solid: Personal Data Management Through Linked Data’, 2018. https://doi.org/10.25815/50W4-HK79.
This work is licensed under a Creative Commons Attribution 4.0 International License.
![](/data/QmRrj9F16JzBL3jWwwYfEEi2vJrhqjZvr2A8R8qMMwgKsU?filename=cc.png&content-type=image/png)