Serverless Knowledge Sharing with Evernote, Markdown and Clojure
This is a website created using a somewhat unusual set of tools which let me:
- Be serverless! Nothing to run or maintain
- Easily create content with Markdown and Evernote
- Be ready to iterate to version 2 and version 3 quickly
Ultimately, it's an experiment with combining together existing best-of-breed tools to make something bigger than the sum of it's parts, a platform for easily sharing knowledge.
Brain --> Internet
I have always believed strongly in brain dumping.
Like the computers we spend so much time on, there is only so much space in our heads. All the cool stuff I research, learn and tinker on unfortunately cannot stay in my "brain RAM" longterm. I have relearned and perfected the same things too many times to count because I've not had a repository where I could reliably store, find and share these useful bits of fleeting knowledge.
For example, the immensely useful
jq is a great tool to have in your tool belt. It's the self proclaimed
sed of JSON and makes JSON wrangling in bash trivial, but I rarely recall the specifics of command's filter syntax or related command line flags when I need to use it the second or third time in a new project. Instead of having to spend the time rereading and reprocessing the tool's (admittedly really good!) documentation, I want to be able store a processed summary of the learnings and specific use-cases I care about in a place that is easily searchable.
But why stop there? If I have a useful repository of learnings I will want to share it with the world. We, the internet collective, are so much more powerful when we sum our knowledge together. I want to contribute to this corpus of knowledge that has informed and influenced my work.
This site is the latest incarnation of me trying to achieve this goal, and I think the specifics of how it's implemented are interesting and worth sharing.
So, you might be asking:
Why the complexity?
Complexity is the root of all evil and a static website is so simple!
Can't you just use Jekyll?
Perhaps! But with this architecture, I am able to:
- Leverage my existing knowledge storage workflow that uses Evernote Markdown
- No servers to run or maintain, CDN relatively for free
- Have a decomplected platform for future additions and customizations
Sidebar: The term decomplect is a gem from Rich Hickey, the creator of the Clojure programming language. I recommend watching the original Simple Made Easy presentation on InfoQ in its entirety.
Knowledge Repository: Markdown Evernote
Markdown is a simple, unceremonious way to structure knowledge and it is especially useful to programmers for documentation. It's plain text syntax for composing headings, lists, paragraphs and code blocks is intuitive and productive.
Evernote is my information repository of choice. For me, it's the clear winner for knowledge storage as:
- Everything is stored and managed by Evernote
- Great interfaces for desktop, mobile, and the web
- I can access my information anywhere
- You can search everything
- Find without having to pre-organize or categorize
- Search works within documents (PDF, etc) and images (via automatic OCR)
- I can programmatically access and use my data in ways not imagined by Evernote
- The ecosystem of tools built on-top Evernote add even more value.
- Scannable allows me to easily get paper documents and receipts into Evernote
- and others
Marxico combines these best part of these two systems together. It is a browser based Markdown editor and previewer that saves to Evernote.
I have slowly reprogrammed myself to diligently put documents, notes, and code bits into Evernote via these tools. Doing this consistently has simplified knowledge access and replaced a poorly managed mess of text files, Google Docs, and sticky notes.
Site Design: Webflow
Today, all websites need to look good and look good on all devices.
Maintenance Free Deployments: Amazon Web Services
Amazon Web Services (AWS) is the backbone of most modern web deployments and using it as the deployment platform for this site was obvious. It allows me to deploy an infinitely scalable website with DNS, SSL/TLS and a CDN for a few dollars a month that requires zero maintenance.
aws command line tool and we set a couple specific flags to specify content type and cache expiration HTTP headers. Once the files are in S3, they are served to user's browsers via CloudFront.
CloudFront is a content delivery network (CDN). It's not required in this architecture but it's easy to add and helps ensure the site is fast for everyone around the world regardless of how close they are to the S3 servers in Virginia.
Route53 is a hosted DNS service that makes it trivially easy to point a domain name to a CloudFront or S3 served site. It is especially helpful if you want to use a "naked" domain (i.e.
https://www.webb.codes) due to technical details related to DNS. For more advanced infrastructures, Route53 is scriptable which makes it way more useful that most other DNS service providers.
Amazon Certificate Manager
Amazon Certificate Manager allows us to serve the site via HTTPS using a free, trusted SSL/TLS certificate with minimal effort. Once the certificate is created via the AWS web console, you configure CloudFront to use it to securely serve the site content.
Obviously I am glossing a number of details of the AWS configuration. Related, there are other important but boring things that AWS handles for us via configuration and without needing external tools, including:
- Redirecting https://www.webb.codes -> https://webb.codes
- Similar but different, redirecting http://webb.codes -> https://webb.codes, and
- Serving custom pages on HTTP error status codes (like 404)
The Glue: Clojure
We use Clojure to glue the data, content and assets from the various systems described above into the final directory of files that represent the complete website. I would open source this code but admittedly it isn't very pretty or generalizable for others to use easily.
But, what the code lacks in polish is far outweighed by it's flexibility and power. Since we are tying together many loosely-coupled systems and using a functional language like Clojure, it is really easy to swap one data source out for another. The post Markdown content can just as easily come from the file system instead of Evernote. Markdown -> HTML conversion is done via Pegdown but can also be easily changed. If you didn't want to generate the site's HTML/CSS/JS in Webflow anymore you can replace it's assets with static files or other more traditional templating tools like Handlebars or Hiccup.
At the end of the day, the site generator is a tree of function calls that roughly look like this:
As this is an experiment, some things work better than others. All programming and design projects are iterative. This is how I currently feel about the decisions made.
Things that are working really well
It just works! You get the simplicity of Markdown with the infrastructure and tooling around Evernote
Hosting with S3 CloudFront
After a bit of a configuration learning curve, I can do everything I needed for a static website
The more I use Clojure the more I find the functional abstraction right for my programming challenges, and with its JVM compatibility I can leverage Java's massive set of libraries and APIs.
Decisions that the jury is still out on
Webflow for site design and generation
The power of the Webflow site design tool I do not question at all. It is approaching a sweet spot between being easy to use while still creating credible HTML and CSS markup that more technical people can appreciate and use downstream. The friction I am running into is the handoff between designing the site and pages graphically in the Webflow UI and later injecting the Evernote/Markdown content in code. I am not able to easily see actual site content during the Webflow workflow and annotating the HTML in Webflow for later replacement.
For the purely static content the Webflow workflow is great...but for the content that is effectively templates that I later process in code, it's a bit cumbersome. But, I admit, I am probably trying to use Webflow in a way not envisioned by it's team.
So, how did it turn out? So far so good. The site (as you can see) is up and running. :) Of course this took a bit longer than using a pre-canned static site generator like Jekyll, but because of the effort we have a lean and highly customizable platform to build on for the future.