All my notes, articles, bookmarks, photos and more are hosted on my personal domain rather than in someone else’s silo. I can choose how it looks and works while having fun building it for myself.
- Microformats2 (h-entry, h-event, h-cite, h-card)
- Webmention sending and receiving
- Micropub create, update and delete/undelete
- POSSE syndication to Twitter via Silo.pub and Pinboard
- Reply-/repost-/like- contexts fetched and displayed above posts
- Backfeed of replies, reposts and likes imported via Brid.gy
- Authorisation via IndieAuth and
- PubSub(HubBub) hub pinging
How it works
Transformative has several parts. The most obvious is a Sinatra web app that serves content from a database cache. It also exposes APIs so that compatible clients can create and edit content. Posts are first stored as flat JSON files in a git repository and then sucked down into the database and cached. Any external contexts, replies, likes and reposts are also imported.
All content is stored on GitHub in my content repo. Whenever a file is added or changed, either via a Micropub post to my endpoint or via a git push, GitHub notifies Transformative via a webhook.
The post is then pulled from GitHub and copied to a local Postgres database as a full Microformats2 JSON document. Note: this database is a cache that can be rebuilt from the content repo at any time; the canonical store for all content is the GitHub repo.
Images and other media files are also stored in the content repo but are copied to and served from an Amazon S3 bucket.
Rather than build an admin system for creating and modifying posts, I’ve attempted to do without. I’ve exposed an API endpoint that’s compliant with the Micropub specification for posting and editing using a compatible third-party app.
Using a tool like Quill or Micropublish I can log in and then post notes, bookmarks and likes to my site. A successful post is stored on GitHub and cached on my server which then fires off any webmentions, syndicates to silos like Twitter and fetches reply/repost/like contexts for display if appropriate.
Alternatively, I can write (or edit) a post as a file and simply push via git to my GitHub content repo and Transformative pulls it down and does the rest.
Instead of a comments form, my site supports replies, reposts, likes or mentions via Webmention from another IndieWeb site. If someone wants to respond to a post they can write a note on their own site using Microformats2 h-entry markup and send a webmention ping to my endpoint.
The commenter’s h-entry will then be parsed, stored and added underneath my post with an icon indicating its type. Further webmentions from the same permalink will update or remove it if the link (or the whole post) is gone.
And using the magic of Bridgy, responses to my tweets, Facebook posts and Instagram photos that have been syndicated from my site are pulled back in as webmentions.
- Ruby 2.3.1 or newer
- PostgreSQL 9.4 or newer – required for its JSONB support
- GitHub account – post canonical storage
- AWS S3 bucket – media file hosting
Transformative currently powers my personal site but should be considered experimental and likely to change at any time. You’re welcome to fork and hack on it but its primary purpose is to evolve based on my needs. Use at your own risk!
I recommend hosting Transformative with Heroku. I started building a new VPS with this setup and realised I could save myself the time by using a relatively cheap Heroku instance.
Create a fresh database instance with one table named
# CREATE TABLE posts (url VARCHAR(255) PRIMARY KEY, data JSONB);
- Create a public repo in GitHub (suggested name: “content”)
- Create a new webhook under this repo
- Payload URL: your Micropub endpoint, e.g. https://barryfrost.com/micropub
- Content type:
- Secret: Generate/decide on a secure password or token to use when setting
- Select just the
- Generate a Personal Access Token with repo permissions and keep a note of it to use when setting
You will need to define the following environment variables:
GITHUB_REPO– name of an empty repo to use as your content store
GITHUB_ACCESS_TOKEN– a personal access token generated from your GitHub account
GITHUB_SECRET– a (strong) random password/token you’ve generated for the webhook
SILOPUB_TWITTER_TOKEN– a token generated by SiloPub when syndicating to Twitter
DATABASE_URL– your Postgres connection (Heroku will create this for you on deploy)
CAMO_KEY– your Camo instance private key
PUSHOVER_TOKEN– account details for use with Pushover
PINBOARD_AUTH_TOKEN– Pinboard API key
You can view the source code for Transformative on GitHub.