Optimization techniques for FIX-based trading applications

Our team has about 20 years of experience with FIX API, with the last six years primarily in crypto (or digital assets, if you will).

During this period, we accumulated particular experience and knowledge of what is good and what is not when it comes to using FIX protocol. Some of our know-how I will share in this short article.

These techniques are general and can be applied to digital assets, FX, equities, etc. It also will work (with some exceptions) with REST/WebSockets APIs.

General optimization methods

Never run on operating system default settings

By default, your OS is pre-configured to run on average hardware profile, and it has no idea how much memory (and what type) you have, what sort of network cards you’ll use, what storage devices are installed, and how they are configured. We don’t know much about Windows or Mac OS, but Linux offers a great set of different configuration mechanisms for tuning. It’s like a free lunch — you can get additional performance by configuring your hardware correctly.

Tune your 10Gbps NICs for better performance

Increase TCP/IP buffers

For example, the default buffer size for Linux is 8kb. This value works perfectly for web browsing but might create problems for applications that work with market data intensively.

If possible, consider changing threads priorities

Again, Linux-specific — you can increase threads priority for specific processes. In addition to this — bind your application to specific CPU cores (threads affinity)

If possible, choose the proper disk scheduler

Again… Linux has multiple disk I/O schedulers, each with its benefits and drawbacks. If you run on SSD/NVME then consider using none. (https://access.redhat.com/solutions/109223)

Keep the number of threads as low as possible

This tip is more on the software development side rather than system configuration. Try to reduce the number of working threads to a bare minimum — context switching comes with a price, and with two or three threads it might be ignored. But with hundreds or thousands of threads competing for resources, the overall impact becomes meaningful.

Market Data sessions optimization

Increase receiving buffers

Pretty straightforward.

Properly plan your processing architecture

Make an informed decision on how and where to process market data — update books, run calculations, send orders, or do anything else.

if these operations take some time, performing them in a separate thread is a good idea. This way, you can ensure that a FIX engine is busy emptying, receiving buffers, and not being part of the Monte Carlo calculation. Otherwise, there is a chance that you will become what we call a “slow consumer” and will be disconnected. Do not try to subscribe to all Binance, Coinbase, OKX, and Deribit instruments, and keep complex processing in a single thread.

On the contrary, if the number of instruments is low, algo is pretty straightforward (e.g. price checks), properly debugged, and profiled, you can always save extra time on context switching by placing your business logic inside FIX messages handlers. For example, the latency arbitrage system we built some time ago works precisely like this.

Turn off logging for market data sessions unless it is required

If you are not in the business of reselling historical market data, then turn off logging provided by FIX engines. Yes, we optimize market data streams, and our FIX-based feeds contain fewer data(in bytes) than native OKX, Huobi, ByBit, or GATE.IO formats. But it still will keep your disk system busy. And it takes some space too. A lot of space.

Trading sessions optimization

Increase receiving and sending buffers

Unlike market data sessions, where you mainly receive data, trading sessions have usually utilized both ways. This means that both sending and receiving buffers should be adequately increased.

Never turn off logging for trading sessions

Based on our experience, we always keep logging on for trading sessions. To mitigate I/O penalty, use asynchronous logging adapters.

If your FIX engine doesn’t have such a thing from scratch, it’s usually a good idea to invest some time to build it by yourself.

Use pre-cached messages

Use pre-cached messages instead of creating e.g. NewOrderSingle or ExecutionReports (in our case) every time you need it.

If your FIX engine supports this, run warm-up procedures before hitting the gas with actual trading. Sometimes developers use lazy-load initialization, and creating particular objects might happen only when you send an actual message. Pity if this message is your NewOrderSingle when you are chasing a price.

Use pre-cached timestamps

If you profile your code, you will be surprised at how long it takes to transform timestamps from long (data format) into strings. It might be beneficial to pre-populate formatted strings into memory. This task is challenging (populate, update), but the performance gain is significant.