I'm bullish on two technologies; Elixir and Bitcoin. Both are remarkable and beautiful technologies. Today, I'd like to combine both topics in a simple Elixir project to explore some of the concepts one will encounter when diving deeper into Elixir.

The Project

We're going to build a Bitcoin Block Clock. (A very basic one - not even closely as feature rich as https://blockclockmini.com/ - Check them out! It's really an awesome project!).

We're going to build a web application, which displays the current block time in a browser. The block time gets updated in real-time as soon as the block time is updated. All code is open sourced at Github.

What is this block time?

Bitcoin miners secure the network and process Bitcoin transactions. Miners achieve this by solving mathematical problems. If a miner solved a mathematical problem, a new block was found. This allows the miner to add the newly found block to the existing blocks (forming the blockchain).
A new block is found roughly every 10 minutes. An event (or tick) which can be represented as a clock.

Expectations

While it's a simple Elixir project, it's not your typical "Hello World" project. I assume you understand the basic syntax and already played a little bit with Phoenix Framework. If you need to freshen up your knowledge on this, I'd suggest to read Elixir's awesome official documentation, or use a service like Exercism.io.

We're going to explore some basic concepts of Elixir which we'll encounter from time to time when using this beautiful language to build software systems. We're going to use GenServers and Supervisors, PubSub and LiveView and pack it all up in a simple Phoenix Framework project.

Don't worry, it's not going to be difficult. We'll have fun learning together! :)

Overview

We have a web view, displaying the current block time. The current block time is simply represents the currently total number of blocks mined. The displayed number is updated as soon as we know there is a new block (soft real-time).

We store the current block time in a stateful process, our store. We have a worker process which periodically checks if there is a new block, and then updates our store with the currently available data. As soon as our store is updated, we notify the web view via PubSub.

Deep Dive

Let's dive in a little deeper. I'm not trying to explain everything in full detail. We're still going to be somewhat high level. But I hope I can get across some concepts of Elixir which at took me a while to get it myself, when I started learning Elixir.

First things first.

A Block

A Block looks as follows. It's a simple Struct. The module provides a helper function Block.new to simply create the new struct from data which the external API returns.

The DataWorker

A simple worker process is going to fetch data from an external API periodically.

The process is implemented as a GenServer. A GenServer at first feels a little complex and heavy. At least that's how it felt to me. Probably, because of "GenServer", which sounds big and heavy. But essentially it's a simple thing. It's a lightweight process which provides an API. We use this API (simple functions) to talk to and interact with the process. The process responds to our interactions with callbacks (which are simple functions as well). These interactions allow us to store state in a process. Some of those functions need to fulfil a certain interface (an interface is called a Behaviour in Elixir) so the whole process fits nicely into the rest of the Elixir ecosystems (e.g. can be used by a Supervisor).

Our only interaction with the DataWorker GenServer will be to start it under a supervisor. After that, the process will be its own little thing and tell the rest of the application when a new block was fetched.

A call to DataWorker.start_link will start the GenServer process. The server can be initialised with a state. In our case, that's a map containing interval_seconds and timer_ref. Interval seconds can be configured. If no config was provided, then 10 seconds is used as default interval. That's how often we call the external API to fetch new data (every interval_seconds seconds). The timer reference is useful for updating or cancel an interval. We're not going to use it in this project. It's only added to show that it exists.

DataWorker.init is the callback implementation. Once start_link is called, the process is initialised and the GenServer behaviour makes sure to call init, giving us the chance to do something in the initialisation phase. In our case, we schedule the worker (DataWorker.schedule_worker) to fetch the external API every interval_seconds. We also fetch the API for the very first time (via DataWorker.fetch_data). We do this so we'll immediately have some data in our store and do not have to wait until the first interval has passed.

schedule_worker is nothing more than telling our process to send "ourselves" a message again after x milliseconds (that's what Process.send_after(self(), :run, after_ms) does). Elixir (or the BEAM ErlangVM more precisely) will then take care of sending our GenServer a message after x milliseconds. So, we send our GenServer process a message with the content :run.

handle_info, another callback, "listens" to all incoming info messages. We've implemented the function as such, to only listen to messages containing the content :run (that's what def handle_info(:run, state) do does). In that function implementation, we then simply call fetch_data and schedule our worker again. This, again, will send the GenServer process a message after the configured interval. And it'll do so until the process or the application is stopped.

In fetch_data, we use a library to call the external API, prepare the data and create a new block via Blockchain.create_block.

Blockchain Context

The Blockchain context in this application is pretty simple. It only provides one function, create_block. This function accepts a map which we got from the external API, creates a new Block struct (see above) and sets it as "latest known block" in our Store via Store.set_latest_block.

Store - the 2nd GenServer

Our store is also a GenServer. Remember: a GenServer is simply a process running somewhere allowing us to interact with it and store state. That's the purpose of our Store. We want a process that holds our data fetched from the external API. start_link and init does the usual things and initialises the process with an initial state.

Next, we have additional functions to interact with the store; get, set_latest_block (we already heard about this one) and subscribe. This functions are simple abstractions to interact with our GenServer. All they essentially do is to accept parameters and send messages to the correct GenServer process for us.

subscribe is a little bit different. More on that later.

So, when the Store.get is called, the function sends a call(..., :get) message to our GenServer process. The GenServer process responds to this message through handle_call(:get, ...). In our case, we simply return the whole state of the process.

When Store.set_latest_block(block) is called, the function sends a cast(..., %{set_latest_block: block}) message to our GenServer. The GenServer then will handle the message through handle_cast(%{set_latest_block: latest_block}, state). Here, we update the state of the GenServer process.

Noticed the difference? In get we use call, in set_latest_block we use cast. This essentially tells the process if the calling process is waiting for a message to be returned synchronously (in get, we want to wait until the message is returned from the callback and receive the state) or if the calling process does not wait for a message to be returned (asynchronously - in set_latest_block we just send our data we want to store and move on. The calling process doesn't wait until a value is returned).

Supervisors

We have two GenServer processes, but so far, nobody is starting them. That's where our supervisors come in to help. The role of Supervisors is simple. Supervisor are processes as well. Their job is to watch other processes. That's basically it. If a supervised process dies (e.g. because of a failure), then the supervisor will restart the process from scratch and with its initial state.

(Note: We're not going to cover Supervisors in detail here. But there's a lot more to Supervisors and they play an important role in an Elixir/Erlang application. It allows developers to structure a "process tree" and define in what order a process tree should be started - or restarted in the case of process failures. It is also a tool which allows us to optimize an application for resiliency and data throughput - if done wrong, a process tree can turn into a bottleneck.)

Our application starts the registered supervisors. The supervisors then start our GenServer process; Store and DataWorker (see line 16 and 17).

The Web View - our Phoenix LiveView

Our LiveView is also pretty simple. It initially gets data from our store via Store.get, sets the received data to the LiveView Socket and renders the template.

The Realtime Features - PubSub + LiveView

Now to the fun part! We can now leverage the full potential of Elixir, LiveView and PubSub and add real-time features. When our store gets updated, we want to immediately update the view in our web browser as well. We can do this through PubSub and LiveView.

As soon as the LiveView is rendered in the Browser, the Browser connects to a LiveView Socket. One LiveView process is created for one LiveView socket connection. This allows us to send and receive updates in real time, hence the name LiveView. In our LiveView implementation, we check if a client is connected. As soon as one is connected, we tell the LiveView process to subscribe to our store (Store.subscribe see line 13 through 15 in BlockClockWeb.PageLive). This function is implemented in our Store. All it does is to subscribe to a PubSub topic called "data".

Every time data is updated in our store, we broadcast a message to our PubSub "data" topic, see line 65 in BlockClock.Store. And that's it! Since the LiveView is subscribed to that topic, it gets notified as soon as the store broadcasts new messages on that topic and then updates the displayed block time.

The Reason Why I am Bullish on Elixir

We now have a worker process which periodically fetches the latest block information from an external API. The latest block info is then stored in our store process. A LiveView component renders the current block time, which is derived from the data stored in our store. The LiveView also subscribes to our store and gets notified as soon as our store is updated. This allows us to display changes in our web view in real-time.

Consider all the moving parts (PubSub, real-time, Supervisor and processes) and then look again with how little code these parts can be connected and managed. With little effort, we can build resilient and decoupled systems which provide real-time features. Most complexities are abstracted away and already taken care of through Elixir, Erlang and the BEAM. Battle-tested, ready for web scale and almost perfect for what we do in 2021 web development.

Let's connect!

Learning new things is more fun together!

Feel free to share and discuss this, or send me a DM on Twitter!
If you like pieces like this in the future, feel free to sign up!