Technologies
Heimdall
A global event bus system
Introduction
Heimdall is a inter-organization event bus system.
It was conceived, designed and implemented by the Stubber team to solve a wide range of architectural problems arising in dispersed IT systems.
The full scope of Heimdall is not covered in this documentation.
This documentation will focus on the use of Heimdall within the Stubber platform.
Heimdall has many uses independent of Stubber.
It is the intention of Stubber to open source the Heimdall system for wider use.
Basic Operation
Heimdall allows you to :
- Register for a known event
- Emit an event
With these two basic operations, you can use a global event bus to communicate between different parts of your application or other applications.
The Heimdall Global Event bus is a separate system to Stubber and is intended to be a general global event bus for all systems.
Example
Scenario: Event Emission in a Financial System
A financial system emits an event when a transaction is completed. The event is emitted with a unique path structure, which includes details like the transaction type and reference number.
Example event path:
Listening for Specific Events
A reporting system may only be interested in credit transactions. It can subscribe to events with a path pattern that matches all credit transactions:
Subscription path:
With this pattern, the system will receive all events related to credit transactions, regardless of the reference number.
Listening for All Transaction Types
If the reporting system needs to capture events for all types of transactions (not just credits), it can subscribe to a more general path:
Subscription path:
This allows the system to receive all events for transactions, including different types like debit, credit, refund, etc.
Listening for a Specific Transaction Reference
In some cases, a system might only need to track a specific transaction by reference number. It can subscribe to a path that matches a particular reference, such as 1234:
Subscription path:
This path ensures that only events for the transaction with reference 1234 are received.
Conclusion:
Heimdall Paths allow systems to efficiently filter and process only the events that are relevant to them. By using flexible path structures, systems can subscribe to broad categories (e.g., all transactions) or specific events (e.g., a single transaction by reference). This makes Heimdall an effective tool for event-driven architectures.
Wildcard Characters Explained
We can see from the above example that you can use * and # as wildcards in the path to filter events.
*(asterisk): Matches any value at any level deeper in the path.
# (hash): Matches any value at the current level of the path.
Example:
/users/*/orders (matches any user's orders)
/users/#/orders (matches orders for any specific user)
Heimdall Path
A Heimdall path is a string that represents a unique location in the Heimdall event bus. It is used to register for events and emit events.
The path is a string that :
- contains only alphanumeric characters, hyphens, and underscores
- starts with an alphanumeric character
- is case-insensitive
- uses "." as a separator for different parts of the path
Path Hierarchy
It is very important to have a lot of hierarchy in your path.
Hierarchy is achieved by separating the path into different levels using the "." separator.
Most events need to start with 2 levels of hierarchy for the organization that owns the event. Example : stubbergroup.internal
The path should contain at least 5 levels of hierarchy. Example : stubbergroup.internal.subsystem.component.key.value.key2.value2
Good practices for path hierarchy :
- The first level should be the organization name
- Using at least 5 levels of hierarchy
- Alternating the path levels between keys and values eg.
stubbergroup.internal.system.financial.transactions.type.credit.reference.1234
Having a good hierarchy and added some important keys and values into the path allow systems that listen for events to filter on a large number of events and only listen for the ones that are relevant to them.
Heimdall Mechanisms
Heimdall mechanisms are used to control who receives an event.
When registering for an event you specify which mechanism you'd like to use.
Then all listeners using the same mechanism will receive the event according to the rules of the mechanism.
The default mechanism is evt which simply allows all listeners to receive the event.
Mechanisms are a powerful way to control who receives an event.
Here are the available mechanisms :
Registering For An Event
To register for an event inside Stubber you use the Heimdall Register task.
When registering for an event you typically specify the following :
- A UUID so that you can deregister later if you want to stop receiving events
- The path of the event you want to listen for
- The heimdall mechanism to use (
evtis the default) - If the event should be kept or only triggered once
- How to trigger the event (Stub, action, data to pass etc)
When providing the path to register on you can filter the events you want to receive by using * and # as wildcards in the path.
* is used to match any value deeper in the path
# is used to match any value at a specific level of the path.
eg. stubbergroup.internal.system.financial.transactions.type.#.reference.* will receive all events for any type of transaction with any reference.
it would be possible to receive events for stubbergroup.internal.system.financial.transactions.type.credit.reference.1234 and stubbergroup.internal.system.financial.transactions.type.debit.reference.5678 but would not match on events without a reference like stubbergroup.internal.system.financial.transactions.type.credit.
To match on all events for a type of transaction you would use stubbergroup.internal.system.financial.transactions.type.*.
Emitting an event
To emit an event inside Stubber you use the Heimdall Emit task.
When emitting an event you typically specify the following :
- The path of the event you want to emit
- The heimdall mechanisms to allow (
*is the default allowing all mechanisms to be used) - The data payload to send with the event
Advanced Emitting
You can emit on multiple paths at the same time.
This allows you to emit at different levels of the path hierarchy.
Example : stubbergroup.internal.system.financial.transactions.type.credit.reference.1234 and stubbergroup.internal.system.general.references.1234
This would allow systems to find general events across all systems that mention the reference 1234.
So the financial system would emit on stubbergroup.internal.system.general.references.1234 for each transaction reference and the support system might emit on stubbergroup.internal.system.general.references.1234 for each support reference, and then the reporting system could listen for events on stubbergroup.internal.system.general.references.1234 and receive both events.
Emitting on multiple paths is encouraged to allow systems to filter by different hierarchies of the path.
Triggering
When an event is received by the central Heimdall server it is then evaluated to see which listeners should be triggered.
Heimdall will first find all the listeners that match the path or partial path of the event. It will find the listeners according to the mechanism of the registration. For the most basic mechanism, evt, it will find all listeners that match the path or a wildcard path. For the cbh mechanism, it will only find the last listener that matches the most specific path.
For the cbh mechanism it will only find the last listener that matches the most specific path.
Once a list of listeners is found, it will then trigger the listeners according to the specifications in the registration.
An example is shown here : Heimdall Register task