Behind the BANG (mini)! - The thing with the white-screens
"Behind the BANG!" is a new irregular blog series that looks behind the scenes of the game in different aspects.
As most of you have noticed, the game still experiences occasional white screen issues where textures do not load and only a blank (white screen) texture is displayed instead. In this blog I would like to talk a bit about this - what I have found out about it and what the underlying issue is.
It’s old but not gold
I don't know if this happened in the original game, but for Celeste we got reports very early on. The first instance was actually in 2017. At the time, it wasn't that big of an issue, so we didn't give it too much priority. But it was still on our radar. Our general suggestion was: If you hit a white screen texture, restart the game, and for the most part that worked. But that was more of a workaround than a solution.
Investigating
Back in 2021 I wanted to tackle this. I wanted to find out what the issue was that was preventing the game from loading textures. Therefore I've added a logging system. Not too long after implementing it, I got the first results: Out of memory.
“But hey, I’ve got 64GB of RAM. How can it be out of memory?” … you might ask.
The thing is, Age of Empires Online is a 32-bit application. Unfortunately, it was developed at a time when 64-bit games were just starting to become popular, but not late enough to be adopted for Age Online. I think the first game in the AOE franchise to be upgraded to 64bit was Age1: DE. With 32-bit, we're limited to 4GB because the memory addresses are 32-bit and we can't address more than that. But it gets worse: Normally 32bit Windows applications are not built with the "largeaddressaware" flag, which means that the game can only access 2GB by default. This has historical reasons. We could try patching that flag in, but this is also not a magic fix.
So generally speaking, if you see a white screen texture, you know that you have already reached some sort of memory limit. This also means that it is only a matter of time before any memory request other than texture loading is refused. You will notice this with the dreaded "Visual C++ Runtime Error" message box. Most of the time (not always) this means that the game has tried to request some memory from the system, but the system has refused it.
The thing with the “limit cores”.
One of the short-term fixes suggested by the community was that restricting the game to certain CPU cores would reduce the problems, and indeed it did. (Thanks to Donar and CWO!) However, this is more of a workaround than a fix. I am not sure why it works, but I suspect it has something to do with memory management. I imagine that the system creates its own memory "arenas" for each core to be very efficient in terms of speed, but at the cost of wasting memory.
So what can we do?
The best thing to do would be to upgrade to 64 bit. However, this is not feasible as we don't have the source code. This is very unfortunate, because it means that we will eventually have a limit when it comes to adding new content, such as new UIs, civs, models, ect... One good thing is that the game has been written to use memory efficient strategies. On the other hand, the game uses texture caching so that it doesn't have to reload textures from disk all the time. (This is where the white screen bug comes from).
LargeAddressAware
I will eventually try our LargeAddressAware, but that needs a lot of testing. If we can get the extra 2GB out, we might be able to buy some time.
Optimizing XLive
That's something I've started to do. There are places in xlive where we're a bit more wasteful with memory. I'm trying to reduce that.
Checking for in-game-overlays
Unfortunately, in-game overlays such as steam, discord, etc. inject themselves into the process and consume extra memory. I am considering adding a warning about this.
Unifying allocators
One thing that might help is to unify the memory management between the game and xlive. Currently they are two separate systems.
Conclusion
With this blog post, I just wanted to reassure you that the issue of memory hasn't gone off my list. Unfortunately, things are not that simple and we have limited options. At the end of the day, we all want to have a smooth experience in the game and I know that crashes (network disconnects and hard crashes) are frustrating. With that in mind, I still hope that you all have a great time playing Celeste.
I wish you happy questing for the Winter Event!
========
Previous Behind the BANG! blogs:
========
All current and future content will stay 100% free and accessible to everyone.
========
Thank you so much for reading, and we will see you all next time!
Project Celeste Development Team
The Indians are here! Watch the Announcement Trailer.
The Romans are here! Watch the Overview Trailer.
Project Celeste is completely free and always will be. However, we gladly accept donations for our overhead costs, which are larger than we have budgeted. If you want to support us, you can do so HERE.
Read every Romans related blog HERE.
Read every Indians related blog HERE.
Tell your friends! Join our Discord HERE.
========
