Weekly Update - Reducing FPS Drops, Optimizing, and Terrain Changes Last week I focused primarily on reducing FPS drops when crossing terrain page boundaries and making a few optimizations. I also started on some improvements to the terrain system that'll finally allow for collisions with the cave walls and ceiling, and placing clutter on it.

Reducing FPS Drops

As early testing of Antilia continues I've been switching between addressing items that are glaring issues that make Antilia less fun to explore and smaller bugs that are easy to address. This week I tried to tackle one of the more annoying issues - the frame stuttering that occurs when a player character crosses a terrain page. Crossing onto a new terrain page triggers a re-centering of the world as well as a lot of load and unload tasks (streaming). I spent some time gathering metrics on how long various functions took to execute and how long resources of different types took to load (terrain page data, trees, meshes, textures, etc.) I then went to work reducing the time spent on those things in the primary thread, either moving processes into background threads or splitting up functions with large lists to iterate through to only iterate a few items per frame.

If I'm honest the results were a bit disappointing in the end. By the numbers I had made notable improvements, cutting the "stall time" of several resources to a fraction of what they were. If I disable character compositing then the page transitions are unnoticeable on my dev machine. With character compositing on there was still a small noticeable fps drop.

But when testing on other PCs the frame drop was still quite noticeable and more often that I would have liked - pretty bad. It seems to be at its worst in Tasii Garden, where there are a lot of objects and NPCs. The frustrating part is that by the performance numbers I saw I really thought I'd solved it somewhere around 80%, and on my dev machine it seemed like I had. I think it's a case where any fps drop is noticeable, and on systems where the CPU is more taxed and has nothing left to spare even those small split up tasks are still stalling the frame loop.

The conclusion I came to is that Antilia just can't be putting so much of a CPU load on a single core. I'm going to have to optimize CPU usage more, think about the bottlenecks more, and continue to reduce scene complexity where I can. There is also the character compositor stuff that I need to take a deeper look at. In theory it should be dividing it's workload up across several frames, but something is taking too much CPU time in that process (Probably parsing the compositor scripts, they are some of the most complex JSON I parse, with a bonus "variable reference" system hacked on top.)

Optimizations

In trying to improve CPU usage, I ran Antilia in a special performance measuring mode. This mode measures the time it takes to run hundreds of functions while the game is running and spits out a large tree of data revealing where the most CPU time is spent. These kinds of performance measuring modes can at times be a little misleading as "what you measure you change" is a very real effect here. The timing and recording functions introduce overhead, so small functions that are called very frequently appear worse than they are in reality. Still, it is the tool I have so I made use of it.

A few bits of code caught my eye. First I noticed that after every single frame a function was walking through all the currently loaded resources and trying to unload those that aren't referenced. At first I tried doing this every few seconds, but that introduced a noticeable spike in the fps graph (a spike that was previously occurring every frame). I prefer to avoid those kinds of spikes as they tend to add chaos and could potentially align with some other intermittent process on some frames. Instead I redesigned the function to run every frame, but only run for 0.0005 seconds before saving it's place to continue at on the next frame. Much better.

I looked for optimizations I could make to the rendering loop but ended up reverting everything I tried there. It still feels like there is room for improvement but I'm just not spotting it yet.

Updating the game objects tends to be the big time sink. I've tried various schemes to reduce the number of objects that are updated every frame, and that has helped but also at times introduced bugs that were easiest to fix by opting-out of that optimization in places. I feel like this problem is architectural and one I'm still pondering.

In the end last week I found a few optimizations that reduced the time it takes to render a frame by about 2-5%. No big wins but if I can find a few more of those they could start to add up.

Terrain Changes

On Sunday I was feeling a little exhausted from optimizing and wanted to do something a little more creative. I really wanted to work in the World Editor on some scene improvements, but ended up doing more programming. The terrain engine is still missing a few things, namely that no collision data is being generated for the cavern ceiling or the "roof" - the layer of terrain above a cave. If you try to walk over these areas or into a cavern wall you'll just pass right through. Also because of the lack of collision data I can't place any clutter on those surfaces yet either.

For some time I'd been building the solution in my head, and on Sunday I decided to start working on it so that I can finish up the caves around Gajoi's Claw and finally be done with a rough pass across all of the Isle of Kasau. Adding the collision data required changing the terrain file format, and I really didn't want to have the testers need to download the entire 4GB of terrain data again. Preferably only the pages with caves would change - so I took a little extra care in ensuring the existing terrain pages would continue to load in addition to the new files. The new files include distinct index buffers for the floor, sea, ceiling, and floor - meaning that the new pages will eventually be rendered in a more optimized way as well. The tricky part is along the edges where I need to perform a "scissor" operation on some triangles. That is the part that I've been putting off and I think I've got the math right in my head to finally finish it up.

Unfortunately I ran out of time on Sunday to get all the changes fully working, so as much as I was hoping for a reveal screenshot of my character standing above a cave entrance surrounded by clutter I'm afraid that will have to wait for now. All the programming has left me without a good screenshot for the week - so enjoy some silly bugs instead!

Dramatic Information Markers!

I may have shared this one already. If I wanted to deliberately create this effect it would have taken several lines of code. Apparently it can also be accomplished with a small typo.

That's all for this week, thanks for reading!