How We Deploy a 108GB Game Without Losing Our Minds

The Challenge

In April 2022, I joined the Limitless Flight team as a DevOps Engineer. At the time, our team was quite small, consisting of four on-site developers and a Senior Unreal Engine Developer based in France. My primary task was to build a CI/CD pipeline capable of automating the deployment of three .NET services and a large Unreal Engine game client.

As the project grew, so did the complexity of our deployment process. Managing multiple services alongside a resource-intensive Unreal Engine build required a scalable and efficient solution. This blog post outlines how we tackled these challenges and optimized our deployment workflow.

Larger and Larger Builds—Oh My...

Building a next-generation virtual flight simulation demands cutting-edge visual fidelity and performance optimizations. Our project leverages Unreal Engine 5, taking advantage of its latest technologies, including:

  • Nanite for virtualized geometry, enabling high-resolution assets without excessive performance costs.
  • Lumen for global illumination, delivering realistic lighting in our simulation environments.

Even after extensive optimizations, the game’s binary size remained large, averaging around 108GB per package. Additionally, builds took several hours to complete on a single developer machine. Clearly, a robust automation solution was necessary to manage these resource-intensive builds.

Choosing Jenkins for Build Automation

Given our team's prior experience with Jenkins, it was a natural choice for handling our CI/CD pipeline.

One of the complexities in our setup is that we maintain a custom Unreal Engine source build, incorporating patches and optimizations tailored to our flight simulation requirements. As a result, every build must be compiled from our modified Unreal Engine repository, adding another layer of complexity.

To efficiently manage our builds, we set up a Windows Server Virtual Machine (VM) to handle:

  • Windows & Linux client builds for the game.
  • Linux server builds, which are then packaged into Docker containers.

Additionally, a dedicated Linux build VM is responsible for packaging the game’s server binary into a Docker container before pushing it to our private repository. This setup ensures that our builds are reproducible, automated, and optimized for deployment.

Deployment with Octopus Deploy

With our build process automated, the next challenge was deploying these massive artifacts efficiently.

We use Octopus Deploy to manage the deployment of:

  • Docker services for our microservices-based backend.
  • Custom application artifacts, including our 100GB+ game builds.

By integrating Octopus Deploy into our pipeline, we streamlined the process of deploying both backend services and game client updates, ensuring that updates are consistent, reliable, and easy to manage.

Conclusion

Building a modern virtual flight simulator presents unique challenges, particularly when working with large-scale Unreal Engine projects. By leveraging Jenkins for build automation and Octopus Deploy for distribution, we created a scalable, efficient, and automated deployment process that keeps our development pipeline running smoothly.

As our project continues to evolve, we are constantly refining our approach to build optimization, cloud deployment, and automation strategies. We look forward to further improvements that will allow us to deliver an even more immersive and high-performance flight simulation experience.

This version improves flow, readability, and engagement while maintaining your original structure and intent. Let me know if you need further refinements! 🚀