Skip to main content

Architecting Systems Like A Rock Band

In Software, orchestration often means control, synchronization, mediation and scheduling of decoupled application services in order to fulfill groups of tasks as part of a business processes. In highly distributed systems, orchestrating could be a nightmare to develop and maintain. If not designed carefully, it can lead to a scenario where the orchestrator becomes the bottle neck to future architecture changes. In traditional SOA, the orchestrator is often called the service bus or enterprise service bus. Even Micro-services architecture has an orchestrator. It hides behind the name "Gateway" and "Control Plane".

In an Orchestra, if the conductor leaves the orchestra in the middle of a symphony, chaos could occur eventually. The percussion section could become out-of-sync with the woodwinds and the brass section overtime. This is because each musician and each section rely heavily on a set of rules and patterns in the sheet music and of the tempo and the instructions of the conductor. Remove the music sheets and the orchestra will simply stop playing as the musician will not know what to play next. But not in a rock band.

Rock bands don't need a conductor and often times they don't need music sheets. This is because each musician do not rely on a single person to give them instructions. Instead, they listen to each other. They listen to the drum beat for the tempo. They listen to the singer to know which section they are in the song. They listen to the guitarist to know when to bring down their instruments volume to bring up the guitar solo, and so on and so forth. Each musician also knows what they need to do and when they need to do their thing without being told when and how. They decide when to do things based on what they hear. And because they don't follow music sheets, they can improvise and make the music sound better than expected. They can recover quickly from failure, i.e. a broken guitar string or a thrown away drum stick or improvise when this happens so, but they never stop playing just because of it. This is why each live performance is a bit unique as opposed to the musicians in an orchestra. In other words, musicians in a rock band react to events instead of being conducted.

Event-Driven Architecture

Broker Event-Driven Architecture

Event-Driven Architecture is recently catching the attention of software architects and engineers Today because of its simplicity and versatility. Components are loosely coupled. The interface contract definition does not have to be defined up front and can be easily changed. There are no more request-response (synchronous type or communication) and messages are pushed and not pulled. When pulling data a service needs to know upfront where to pull the data from. But when you push events/data, one does not need to know to which specific service you need to push the data to. There are many advantages of this architecture except if the system is too large, say 1000 services/actors, in which case the risk of loosing the enterprise-wide event broker becomes unacceptable.


The Rock Band Architecture

Similar to EDA, in a rock band architecture, the role of orchestration is delegated and distributed among each of the services. Independent services listen for events and react to those events whenever necessary. Services also broadcast events but they don't need to know how other services would react to it or would use those messages, or if other services are even listening. Much like a rock band, each musician is responsible for playing his own instrument but they also communicate by listening to each others instruments and looking at each other from time-to-time so that collectively, they can deliver the music as a one pleasing rock symphony. 

But there is a fundamental difference. Rock Bands are small. When you put 1000 rock musicians together and play the same song, it's fun for sure but it does not sound as good as a single 5-pieace rock band on a small stage. This is because it is so difficult to synchronize a huge number of musicians and that is the reason why orchestras need a conductor.


Sound travels at 342 m/s. So if you put a drummer and a bassist 100 meters apart, they will be de-synchronized by 1/3 of a second and that's what you are hearing in the video. The drums are not crisp and the bass guitar lost its punch.

Similar principles apply to Event-Driven Architecture. If two-closely relate services need to send messages to each other but there are 5 topics to go through before reaching the other end of the channel, you are loosing time and it's not efficient. 

Another important aspect to consider when deciding an architecture is how well it fits in the enterprise organization. Teams maintaining and building applications in a huge organization don't often speak to each other and the bureaucratic processes behind this are counter productive. No matter how you try to break the silos between teams, sections and departments, you can only do too much. So when different silos are sharing a middleware such as an Event Broker, what organizations usually do is to create another silo, such as a dedicated middleware team, to maintain that middleware. And from then on, teams and departments has to go through them when they want to establish a communication interface. That's a new layer of red tape.

The Rock Band Architecture is a Federated Event-Driven Architecture. Because bands sound better when they are the only ones playing the music in a small stage closer to the audience.

Rock Band Architecture - Federated Event-Driven Architecture

Instead of one single event broker, each silo has it's own event broker. This maximizes the freedom of the teams to choose how to implement their event broker. This also reduces the work on coordination between the silos. Overall, this reduces the effort required for dependency management.

The Event Broker

You don't need an orchestrator or a conductor in a Rock Band. What the system needs is a channel of communication where services can broadcast to and listen for events. The Event Broker. A rock band only needs a stage and their instrument monitors (the big speakers and guitar amps you see on stage). Events are not some generic free-form messages but are specific, small, real-time and contextual. The messaging channel for the event broker must have the following properties:
  1. Events must be broadcasted in real-time or near real-time, otherwise the band would be out of sync.
  2. The communication channel must also use a protocol that is platform agnostic so that services can be built in any platform and developed in any language
  3. It supports any kind of endpoints, i.e. HTTP, HTTP/2, TCP, UDP, MQTT, etc...
  4. It implements a publish-subscribe or consumer-producer messaging pattern
The Players

The services in RBA are the band members or the "Players". Each player plays a specific role that makes up the entire system or subsystem. Service can be in any form of software component (or even hardware) that is either subscribed to one or more topic or is publishing to topics or doing both. Services in RBA are similar to the "Actors" of the Actor-Model architecture pattern, they are completely isolated and are not aware of each others existence but players in RBA are much richer and performs more than just primitive calculations compared to Actors. They are also "stateful" and are always up and running. A player can be as big as an entire application or as small as a serverless function. Players are monitored so that in the event of a crash, there is something or someone that can do something about it like a player service automatic restart or an severity 1 incident raised to the helpdesk. One way to do this is to design the player service so that they are sending heartbeats regularly to a specific topic on the event broker.

The Band

A Band is a group of Players that are connected to a common Event Broker. These player services are grouped functionally into a Band. The grouping can be done in an organization context or by a more detailed context like a subsystem or even an application. The goal is to minimize dependencies between systems and/or organizational unit.

Edge Services

The services that are publishing  messages outside of it's own band are called Edge Services. I wanted to call it the "Groupie", but my wife doesn't approve. There can be one or more Edge Services in a band. Edge services must only publish messages to other band's event broker and must not subscribe to other band's event broker. This simplifies the service as they don't have to maintain a persistent connection with an event broker outside its perimeter.

Summary

Enterprises continue to implement SOA architectures and have also begun adapting event-driven architectures (SOA 2.0) as the standard for many of their applications. But organizations are also becoming more and more agile. Being able to decentralize event brokers in a federated manner can reduce or eliminate organizational dependencies between different silos which in turn would produce a more empowered, agile team who can make important architectural decisions on their own with minimal/isolated risk of impacts.


Comments

  1. Nice article. Easier to understand with example of bands.

    ReplyDelete

Post a Comment

Popular

DIY Airsoft Chrono [PC-based]

Chrono or Chronograph, the term used by Airsofter and Paintball players to refer to the device that measures the muzzle velocity of firearms. This is also called MVMD, short for Muzzle Velocity Measuring Device by the British Army. Muzzle Velocity is the velocity of the bullet as it exits the barrel of the firearm.

A year ago I was an active Airsofter in Iligan who's more interested in modifying and improvising AEGs (Automatic Electric Gun) rather than playing airsoft. I have made my AEG susceptible to high current burns by using locally available MOSFETs. I have improved velocity by replacing cylinder gaskets and spring. I have increased the rounds per minute by altering the armature windings of the drive motor. And most of all, I have made my own cheap alternative PC-based Chrono.

I have posted an artivle before that details the development of my chrono but unfortunately the online forum was hacked/deleted by webmaster of proboards.com due to some violation in the contents. So, …

Disable PrintScreen on C# without Keyboard Hooks

Yes, there is a simple solution to prevent grabbing information on your C# application screen with the keyboard's printscreen key without using keyboard hooks or calling COM interops. The solution makes use of Windows Forms Message Filter to trap keyboard events on your application window.

To trap keyboard events with Windows Message Filter, you need to implement the IMessageFilter interface and override the member PreFilterMessage(ref Message WM) method. This is the method called whenever a Form receives a keyboard or mouse event. You may want to read more about IMessageFilter.

The problem is that even though you have trapped the PrintScreen keypress event, the captured image will still persist to the clipboard. Therefore the simplest solution is to clear up the clipboard right after the PrintScreen is pressed on the keyboard.

This is how the overridden method will look like this:

publicbool PreFilterMessage(refMessage WM)
{
Keys kCode = (Keys)…

DIY Mic Shock Mount

For the past couple of weeks I was looking for a cheap microphone shock mount and the cheapest I saw that was available in my country was around US$160 so I decided to build a cheap one. I only spent less than $6 and a 2 hours of my not very precious time.Before I left office today, bought materials and tools to for my DIY shock mount which are a piece of 4”-diameter PVC pipe coupling ($0.50), a set of hair ties($0.40), set of stove bolts ($0.50), a set of coping saw frame and blades ($2.5).First, I decided to cut the PVC coupling so it would look like Rode SM3 shock mount’s frame.Then I used the section of the PVC coupling that was removed to be a attachment brace to the original mic holders base and bolted it to the base of the frame. I had to heat that part so could bend it to a desired angle.And then cut slots on the edges for the rubber suspensions (hair ties). Right after positioning the suspensions, i found a problem. The rubber bands slip every time i attempted to position the…