The traders of an internationally active energy group relied on multiple sets of data to make the right decisions at the right time: some of these data sets gave them a real-time overview of current market prices, the other sets of data provided an overview of their current positions. The two types of data sets originated from various sources and differed in format and structure. Hence, they could not be presented in a consolidated and combined manner, which inconveniently slowed down the decision-making process. Therefore, the client wished to combine the two data streams in a real-time manner in the backend and create a unified visualization of the combined data through a dashboard. This idea was to be realized with the potential of streaming, more precisely with the data processing software Kafka.
Among a couple of suppliers having experience with the Kafka technology, Trivadis were asked to answer a Request for Proposal (RFP). In the end, the clear and structured delivery approach, the broader set-up of a highly experienced and versatile team (instead of a single person), the prompt answers during the RFP phase and the demonstrated expertise and high commitment led the company to select the Trivadis team as their preferred partner.
With the customer's confidence gained, the team set about tackling the challenges of the project: combining two different data streams (one of them representing the real-time market situation) and visualizing the result in a real-time fashion. The traders obtain the data on specific real-time product prices in separate markets from a market platform. The overview of open positions (exposed to market price volatility) is provided by the company’s internal trading and risk management system.
Our traders have always been looking for the possibility of calculating and visualizing the financial value of their open positions in a real-time fashion. After some years of discussions and workshops, we wanted to try out and to justify the potentials of a streaming technology. By leveraging the benefits of Kafka, we have managed to combine our internal market-exposed positions with real-life market prices in a real-time manner and visualizing the back-end results on a dashboard. The project has greatly benefited from both, the conceptual Kafka knowledge as well as the hands-on Kafka experience of the Trivadis team.
Project manager at the IT department of the client company
The solution for this project was developed by means of the Kafka data processing software. For the external market data platform, a mapping function was used to transform its product names into their corresponding product names of the internal system. Through this alignment, the concluded external market data stream could be merged with the data produced by the internal system.
The data from the internal trading system was reused in a new Kafka stream: the information of the external market data stream (representing the real-time market prices) was combined with the position data of the internal system in order to calculate the real-time financial value of exposed positions. Persisting these data streams in a database secures a certain fault-tolerance of the solution and provides the possibility to visualize the historical trends on the dashboard. Additionally, an advanced splitting of the Kafka topics ensures the solution’s scalability.
The real-time trading dashboard enables the client's traders to get a comprehensive overview of market movements as well as the impact of such changes on their financial positions at any point in time, which leads to fast decision-making processes and thus to a competitive advantage. Due to the great success of this project, multiple other use-cases emerged, and more departments were keen on taking advantage of the potentials of this technology.