When Joyridez's trading desk decided to rebuild their core execution engine, the team faced a familiar but painful question: which build system should support a real-time C++ application where every microsecond matters? The old Makefile-based setup had grown brittle, with manual dependency tracking and inconsistent compiler flags across developers. After evaluating several options, they chose modern CMake — not because it was trendy, but because it offered the right balance of expressiveness, portability, and tooling integration for a latency-sensitive project. This article explains how they made that decision, what alternatives they considered, and the exact steps they followed to ship on time without sacrificing quality.
The Decision Frame: Why the Build System Matters for Real-Time C++
For a trading application, the build system is not just a developer convenience — it directly affects runtime behavior. A poorly configured build can introduce unnecessary overhead, obscure compiler optimizations, or create subtle bugs that only appear in production. Joyridez's team had learned this the hard way: their previous project used a hand-rolled Makefile system that required every developer to manually set environment variables and link flags. The result was a constant trickle of "works on my machine" issues, wasted debugging time, and at least one incident where a missing optimization flag caused a 50-microsecond latency spike during a high-volume trading session.
The new project had to meet several constraints: compile times under five minutes for incremental builds, reproducible builds across Linux and macOS, integration with a custom profiler, and support for both static and shared library combinations. The team also wanted a build system that could handle complex dependency graphs — the trading engine depended on a proprietary messaging library, several Boost components, and a custom allocator that required specific compilation flags. With these requirements in mind, the team set a two-week evaluation window to choose a build system before starting the core development.
They knew that the wrong choice could delay the launch by months. A build system that forced manual configuration would slow down iteration. One that lacked proper dependency management could cause link errors at the worst possible moment. And a system that didn't support incremental compilation would make the feedback loop too long for rapid experimentation. The clock was ticking, and the stakes were high.
The Options They Considered: Three Approaches to Building a Trading Engine
Joyridez's team evaluated three main approaches: sticking with a refined Makefile setup, adopting Bazel, and moving to modern CMake. Each had its strengths and weaknesses for a real-time C++ project.
Approach 1: Enhanced Makefiles with External Tools
The first option was to improve their existing Makefile system by adding a dependency generator like makedepend or using a meta-build tool like ccmake. This approach would minimize disruption — the team already knew Makefile syntax and could incrementally add features. However, the team quickly realized that Makefiles lack built-in support for modern C++ features like modules, and they would need to manually handle cross-platform differences. Worse, Make's reliance on timestamps for dependency tracking could produce false negatives during parallel builds, leading to stale object files. For a latency-critical application, this risk was unacceptable.
Approach 2: Bazel — Powerful but Heavy
Bazel offered hermetic builds, precise dependency analysis, and strong caching — features that appealed to the team's desire for reproducibility. A few team members had used Bazel at a previous job and praised its ability to enforce build correctness. However, Bazel required learning a new build language (Starlark), and its strict sandboxing made it difficult to integrate with existing profiling tools that relied on specific environment variables. The team also found that Bazel's build graph, while accurate, added overhead to the edit-compile-test cycle for small changes. In a fast-moving trading project, even a few extra seconds per rebuild could frustrate developers and reduce productivity. After a two-week trial, the team concluded that Bazel's complexity outweighed its benefits for their relatively small codebase (about 50,000 lines of C++).
Approach 3: Modern CMake with a Prescriptive Style
CMake had been around for years, but the team had dismissed it earlier because of its reputation for cryptic syntax and bloated output. However, the CMake 3.20+ releases introduced features that changed the game: target-based properties, proper support for C++20 modules, and improved dependency management via FetchContent. The team spent a week prototyping a CMake-based build with a strict style guide: every library and executable defined as a target, all dependencies declared explicitly, and no global variables. The result was a build system that was both readable and maintainable. CMake's target_compile_options and target_link_options allowed fine-grained control over optimization flags for latency-critical modules, while its CTest integration made it easy to run unit tests and benchmarks. The team also appreciated that CMake generated native build files (Makefiles or Ninja), so they could still use their existing tooling.
Comparison Criteria: What the Team Used to Decide
To make an objective choice, Joyridez defined five criteria and scored each option on a scale of 1 to 5.
1. Build Correctness and Reproducibility
The build had to produce identical binaries given the same source code and toolchain. CMake's target-based model, combined with explicit dependency declarations, scored high here. Bazel earned a perfect score for hermeticity, but the team noted that perfect reproducibility was overkill for their environment, where developers used the same Docker image. Makefiles scored low because of the risk of stale dependencies.
2. Incremental Build Speed
For a trading application, developers need fast iteration when tuning algorithms. CMake with Ninja generator provided near-instant incremental rebuilds, often under two seconds for a single file change. Bazel's caching helped for full rebuilds but added overhead for small edits. Makefiles, even with optimizations, were slower due to their serial dependency resolution.
3. Tooling Integration
The team needed to integrate with a custom profiler that required specific compiler flags and link-time instrumentation. CMake's target_compile_options and generator expressions allowed conditional flags based on build type or platform. Bazel's sandboxing made this integration painful. Makefiles required manual flag propagation, which was error-prone.
4. Learning Curve and Team Adoption
Three of the five team members had prior CMake experience, and the other two picked it up within a week. Bazel required a steeper learning curve, and the team worried about long-term maintenance if the Bazel expert left. Makefiles were familiar but had evolved into a messy tangle of ad-hoc rules that were hard to audit.
5. Long-Term Maintainability
CMake's active community and widespread adoption meant that the team could find solutions to most problems online. The project's CMakeLists.txt files were self-documenting, with clear target dependencies. Bazel's documentation was good but the ecosystem was smaller. Makefiles, as the team had experienced, tended to accumulate cruft over time.
After scoring, CMake won with 22 out of 25 points, compared to 19 for Bazel and 14 for Makefiles. The decision was unanimous.
Trade-offs: What the Team Gained and What They Sacrificed
Choosing CMake was not without compromises. The team documented three key trade-offs they accepted.
Loss of Hermeticity
Unlike Bazel, CMake does not enforce a hermetic build environment. The team relied on a Docker container to ensure consistent toolchain versions and system libraries. This added an extra step to the onboarding process but was acceptable given the small team size. They also added a CMakePresets.json file to standardize build configurations across machines.
More Boilerplate for Complex Dependencies
CMake's FetchContent works well for simple dependencies, but the team's proprietary messaging library required custom build steps and conditional compilation. They had to write a custom FindMessaging.cmake module, which took about two days to debug. In hindsight, they could have used a package manager like Conan, but they decided to keep the dependency management simple to avoid another tool.
Learning to Avoid Anti-Patterns
CMake's flexibility can lead to messy code if not disciplined. The team adopted a strict style: no global variables, no add_definitions(), and every target defined in its own directory. They also used target_sources() with FILE_SET to explicitly list source files, avoiding the glob() anti-pattern that could miss new files. This discipline paid off when they needed to add a new module six months later — the build system required only a few lines of changes.
Implementation Path: How Joyridez Built the CMake Project
With the decision made, the team followed a structured implementation plan over three weeks.
Week 1: Project Scaffolding and Core Targets
They created a top-level CMakeLists.txt that defined the project name, language standard (C++20), and minimum CMake version (3.22). Then they added three library targets: core (order book and matching engine), messaging (wrappers around the proprietary library), and utils (allocators and logging). Each library had its own directory with a CMakeLists.txt that declared sources and dependencies. They used target_compile_features(core PUBLIC cxx_std_20) to enforce the C++ standard across all consumers.
Week 2: Testing and Benchmark Integration
The team added enable_testing() and created a tests/ directory with unit tests for each library. They used add_test() to register tests with CTest and integrated Google Test via FetchContent. For performance benchmarks, they added a separate executable that linked against the core library with specific optimization flags (-O3 -march=native -flto). CMake's generator expressions allowed them to apply these flags only for the benchmark target: target_compile_options(benchmark PRIVATE $<$.
Week 3: Packaging and CI Integration
They set up CPack to generate a tarball of the trading engine binary and its dependencies. For CI, they created a ci/ directory with a build.sh script that invoked CMake with a preset: cmake --preset=ci. The preset configured a Release build with address sanitizer disabled (to avoid runtime overhead) and static analysis enabled via clang-tidy. They also added a custom target check-format that ran clang-format on all source files, enforced by a CI gate.
Risks of Getting the Build System Wrong
Joyridez's team had seen firsthand what happens when a build system is neglected. In their previous project, a missing include path caused a silent type mismatch that corrupted the order book during a live trading session. The bug took three days to diagnose because the build system did not enforce consistent compiler flags across translation units.
Latency Regressions from Incorrect Optimization Flags
If the build system does not apply the same optimization flags to all critical paths, some functions may be compiled with less aggressive optimizations. For a trading engine, this can add microseconds to the critical path. CMake's target-based flags prevent this by ensuring that every source file in a library receives the same options. The team also used target_link_options(core PRIVATE LINKER:-Map=output.map) to generate a linker map, which they reviewed to verify that no unintended symbols were pulled in.
Dependency Hell During Rapid Iteration
When developers add new source files or libraries, a poorly designed build system can require manual updates to Makefiles or build scripts. CMake's target_sources() with FILE_SET automatically picks up new files in a directory, reducing friction. However, the team still encountered issues when they added a new dependency on a third-party library that used a different build system (Bazel). They had to create a custom ExternalProject module to integrate it, which took a day of debugging.
Team Morale and Onboarding
A complex build system can frustrate new team members and slow down onboarding. Joyridez's CMake setup, with its clear structure and consistent patterns, allowed a new hire to build the entire project and run tests within 30 minutes of cloning the repository. In contrast, the old Makefile system required a 10-page document of setup steps. The team considered this risk mitigation a major win.
Mini-FAQ: Common Questions from Teams Adopting Modern CMake
Should we use CMake's FetchContent or a package manager like Conan?
For small projects with a few dependencies, FetchContent is simple and keeps the build self-contained. Joyridez used it for Google Test and a header-only logging library. For larger projects with many dependencies or complex versioning requirements, a package manager like Conan or vcpkg may be more appropriate. The team evaluated Conan but decided it added unnecessary complexity for their two external dependencies.
How do we handle different compiler flags for debug and release builds?
Use CMake's built-in configurations: CMAKE_BUILD_TYPE set to Debug, Release, RelWithDebInfo, or MinSizeRel. Then use generator expressions to apply flags conditionally. For example, target_compile_options(core PRIVATE $<$. This approach is cleaner than manually setting flags in the CMake command line.
What about C++20 modules? Does CMake support them?
Yes, CMake 3.28+ has experimental support for C++20 modules via the add_library() MODULE keyword and target_sources() with FILE_SET of type CXX_MODULES. Joyridez's project used modules for a small internal utility library, and it worked well with Clang 16. However, support is still evolving, and the team recommends thorough testing before using modules in production.
How do we ensure reproducible builds across different machines?
Use CMakePresets.json to define a standard set of build options, and run builds inside a Docker container with a pinned toolchain. CMake's CMAKE_CXX_COMPILER_LAUNCHER and CMAKE_C_COMPILER_LAUNCHER can be set to ccache to speed up repeated builds. Joyridez also set CMAKE_CXX_STANDARD_REQUIRED to ON and CMAKE_CXX_EXTENSIONS to OFF to avoid compiler-specific extensions.
What is the biggest mistake teams make when moving to modern CMake?
Using global variables and add_definitions() instead of target-based properties. This defeats the purpose of modern CMake and leads to the same kind of fragile builds that Makefiles produce. Another common mistake is using file(GLOB) to collect source files — it can miss new files if the glob is not re-run. Always list sources explicitly or use target_sources() with FILE_SET.
Recommendation: What Joyridez Would Do Differently Next Time
Looking back, the team is satisfied with their choice but identified a few improvements for future projects. First, they would invest more time upfront in writing a custom Find module for their proprietary library, as the ad-hoc solution caused a few build breaks. Second, they would adopt cmake-format from the start to enforce consistent style in CMakeLists.txt files. Third, they would set up a nightly build that compiled with multiple compilers (GCC and Clang) to catch portability issues earlier.
For teams considering a similar move, Joyridez recommends starting with a small proof-of-concept that mirrors your project's dependency structure. Spend a week prototyping the build system, and involve at least two team members to ensure knowledge transfer. Use CMake's --trace option to debug unexpected behavior, and don't be afraid to write a custom module if a library doesn't fit the standard patterns. Finally, remember that the build system is a tool, not a goal — it should serve the development workflow, not dominate it.
Comments (0)
Please sign in to post a comment.
Don't have an account? Create one
No comments yet. Be the first to comment!