Pax Dei
Subscribe to the Alpha

Excited about the game? Enter your email to sign up for playtesting and to receive updates about Pax Dei.

Subscribe
Pax Dei

Tech Insights: our Building System

Behind the scenes

    Hello, Paxians!

    All the beautiful structures you have built since the first Alpha amaze us every day, and the building system is the players' most amazingly utilized feature so far. As such, we thought you might be interested in knowing a bit more about the tech behind the system - which is probably one of the most complex systems in the history of video games.

    Currently, the hottest home valley zones have around 400,000 building pieces. With a total of 24 home valleys per world, there are many millions of pieces in each world. The typical build rate is 10,000 new pieces per week in a single-home valley.

    Our replication system

    In a networked multiplayer game, the replication system is used to ensure that all the players have a consistent and up-to-date game state. In other words, It receives the input information of a player and distributes it to the other players.

    The building data size for a single home valley is currently around 30MB. It does not sound a lot, right? But let’s say we have 100 players; we are now talking about 3GB of data that needs to be managed by the server, the client (the player’s machine), and our replication system.

    This amount is, in fact, so massive that it needs very special handling in all stages of the game pipeline. If we can use Unreal Engine replication for gameplay, we can’t use it for the buildings. The backbone of our building replication is the repli backend server - smartly named ‘repli’ - which sends build data from the world database to the clients and the servers.

    When logging in, both the client and the server connect to repli, which first sends the current full state of the zone. As said, initial data is currently around 30MB in the largest zones. After the initial state replication, the client and the server listen to new build events from repli. We use gRPC streaming as a communication channel to build events from repli. We won’t go into details about gRPC. Simply put, it ensures all parties involved, the server, repli, and the clients, communicate correctly and understand each other.

    In the matrix

    Adding a building piece: under the hood

    Here is simply what happens after a player swings their construction hammer:

    1. The client sends a build request to the Unreal server;

    2. The server does some permission checks to verify if the build request is allowed;

    3. If the server allows it, it then sends the build request to the backend;

    4. The backend does its own checks to allow or not the build request;

    5. Once allowed, the building piece is added to the main world database;

    6. The build event is sent from the database to our custom repli backend server;

    7. Repli will send the build event to both the server and the client;

    8. The server and all the clients in the zone - here, the Home Valley - spawn the building piece;

    9. The server does the integrity check that the structure still holds;

    10. The client calculates integrity as well to display it in the UI.

    Making the building visible to all

    Massive data replication is one of many challenges. After the client has all 400,000 pieces of the home valley, the 30 MB package we discussed earlier, we need to make them visible. Your computer must draw them, and here comes the second big challenge.

    It would be impossible to use a system that draws 400,000 pieces one after the other. We needed something tailored to suit our needs. The two key elements we use are instanced static meshes (ISM) and area managers.

    The ISM is used to draw several, even thousands, pieces in one draw call, instead of drawing them one by one.

    The area managers, the parallelepipeds boxes in the video below, control all props and building pieces in their areas. They handle the creation and destruction of building pieces and props, integrity calculations, and other things, including the ISM.

    The manager looks at all the standard building pieces in its area - not the crafters nor the containers - and sorts them spatially. Then, it inserts them in custom instanced static meshes. And now the magic can happen.

    That being said, we are not done yet. In addition to the buildings, we have containers, crafters, and decoration pieces that need to be drawn, and most of them have their own gameplay functionality. All these props are standard Unreal actors, but contrary to the buildings you can see from pretty far away, they spawn only around the character inside the streaming distance.

    In the video below, you can see how the managers are working and how the props become visible or not. Each box you can see is a manager. As explained above, each manager controls its own area for ISM and integrity, but they also control the visibility of the prop actors - crafters, containers, and other decoration pieces. Depending on the position of a character and the streaming distance of its client, the manager will activate or deactivate the visibility of all its prop actors.

    In the video, red shows the managers that deactivate the props’ visibility and cyan the active ones. Purple shows the ones pending ongoing activation, and magenta the ones pending ongoing deactivation.

    There is another special type of props that needs special treatment: the lighting sources. When your character is close to them, the lights are running through the Unreal Lumen lighting - the dynamic global illumination system from Unreal Engine 5 - to improve immersion. However, the lights are switched to lightweight light cards at a certain distance to improve performance. Some valleys have more than 20,000 lights that you can observe simultaneously!

    Lights in the night

    Of course, we also have some constraints on the server side. One single Unreal Engine server can handle only a maximum of 150-200 players, and if more players want to be in the same village, then we need to create some server instances. That’s another area where having some custom systems helps a lot. Our replication system allows the buildings to be visible across all the server instances and in real time! If one player builds in one server instance, then all the players in the other server instances see the building changing at the same time.

    A few last words

    Spawning all this building stuff around the player without visible stuns is complicated and is done over several seconds, even minutes, sliced into many frames. Also, as you can have guessed from our step-by-step above, the integrity calculation is a very complex process. Our servers also stream world content, so integrity calculation needs careful setup so that all supporting landscapes are there before they even start. Creating only physics states for pieces needs to be done without stuns, and after that, querying overlaps from physics and actual integrity calculations need to be done iteratively.

    Now you know why when logging in, you often can see the landscape first, then the buildings, and finally the props.

    Pax Dei builders have been doing building much faster than we anticipated initially. The whole system has seen many optimization passes so far, and we still have many tricks in our sleeves. The current system should work fine until around 500.000 pieces. After that, we will start seeing new issues that must be solved. Good news, though! We already have plans to scale the system to 1.000.000 pieces per zone.

    But, that is another story for later.

    In the meantime, you can keep building and follow these tips from players who master the pieces and environment to create gorgeous structures!

    Pax vobiscum,

    The Mainframe Team