Compiling DLLs for video games using common household tools

I’ve decided to mod my Fallout New Vegas and Fallout 3 installations and found some mods that replace Vorbis DLLs and restore surround sound. These projects are open-source and just because I can I’ve decided to compile the DLLs myself and while doing so I’ve also decided to write a guide. Note: this guide only explains how to compile programs written in C++ and C. Also while the title of this page says DLLs, this tutorial can be applied to almost any other program written in C or C++.

Compilation

Why even go through all this if you can get pre-built binaries? Well sometimes the provided binaries are outdated and compiling them from source is the only way to get the latest version, or if the latest version of some library doesn’t work with the game while previous does and developer does not provide binaries for older versions, compiling the source is often the only way to get the needed version. Also before compilation you can often configure the program to suit your particular system better.

Programs we are going to need

• Microsoft Visual Studio
  • C++ compiler for Visual Studio
• CMake (Some programs already have .sln file for Visual Studio, but most often programs require CMake)

Other useful tools and utilities that I recommend

• CFF Explorer (Makes applying the 4GB flag easier and shows some additional information about DLLs)
• Process Hacker (Has lots of useful features. For example it shows which DLLs are loaded by a program)
• If you’re still using notepad to edit config files, I would highly suggest getting Notepad++, Atom or VSCodium.

Configuring the project

Cmake

First of all you need to download the source code and unpack it somewhere.

Lots of open source projects use a build system called CMake, it makes it much easier to build the same program on different platforms, but some programs just come with Visual Studio project file. If that’s the case you can skip this step.

Then you need to open CMake and up in the top there are two things that need to be set: source and build directories. Source directory is obviously the directory with the source code, and build directory is where you want to build it. Usually there is already a directory called “build” in the source directory, and if there isn’t, you can always make one.

After choosing the directories click “Configure”. In the newly opened window you need to choose the generator (Visual Studio 16 2019 in my case). Below it you have to choose the platform. This will depend on the game, if you know that the game is 64-bit you should choose x64. In any other case you should choose Win32. If you’re unsure you can always check the game executable with CFF Explorer or check the Task Manager/Process Hacker when the game is running.

Visual Studio

After configuring the project click “Generate” after which click “Open Project”. Now you will be presented with the Visual Studio window. In the solution explorer on the right click “ALL_BUILD” or just the project name, either should work, but with only the project name it might compile a bit faster. In the upper part of the window look for the word “Debug” and next to it is going to be an architecture (x64 or Win32), click on “Debug” and change it to “Release”.

Now we can configure some project properties.

In solution explorer right-click on the project and click “Properties”.

Multicore compilation

Multicore compilation is going to speed up the compilation quite a lot on large projects. From the properties window on the left expand “C/C++” and click “General”. From there enable “Multi-processor compilation”. This might not work for some projects, but for the most part it works flawlessly.

Optimization

This will make the code run a bit faster. Go to C/C++ -> Optimization and enable “Whole Program Optimization”. Also in “Optimization” select “Maximum Optimization (Favor Speed) (/O2)” Then go to Linker -> Optimization, click “Link Time Code Generation” and select “Use Link Time Code Generation (/LTCG)”.

Enabling Enhanced Instruction Set

Enhanced instruction set flag says the compiler to use advanced floating point and vector instructions. This might make the code faster in some cases. Under “C/C++” go to “Code generation” and in “Enable Enhanced Instruction Set” select and choose the latest instruction set that your CPU supports. Most likely it’s going to be AVX2, if your CPU is older it’s going to be SSE2. If something goes wrong, you should try disabling this setting.

Building the project

Now in the upper menu click “Build” and from the menu click “Build <project name>” it should now compile.

In the build log you should see some clue where to look for the binaries. Navigate to that directory and copy the .dll file to the game directory. Sometimes libraries included with game are going to have different name so you will need to rename your files accordingly. In case anything goes wrong be sure to back up the original game files.

Build log with location:

2>Generating code
2>Finished generating code
2>someproject.vcxproj -> C:\some\folders\Release\library.dll
2>Done building project "someproject.vcxproj".
========== Rebuild All: 2 succeeded, 0 failed, 0 skipped ==========

Notes for specific libraries

When compiling Vorbis libraries you first need to compile the Ogg library as a static library (BUID_SHARED_LIBS unchecked in CMake), then when configuring the Vorbis library in cmake, check “BUID_SHARED_LIBS” and specify the path to the “include” folder in the Ogg folder and to the static library that you previously built.

sfall (Engine improvements for Fallout 2) and many other libraries that use DirectX 9 are going to require DirectX SDK. If you don’t have it, you’re going to get a compilation error that says something like “d3d9.h is missing”. If you encounter error S1023 when installing the SDK, uninstall Microsoft C++ Redistributable 2010 and then try to install DirectX SDK. Also when compiling the code you may encounter an error that dinput.lib is missing. In that case go to project properties -> Linker -> Input and there in “Additional Dependencies” rename dinput.lib to dinput8.lib. You also might need to retarget the project if it was made with older version of Visual Studio. (It’s safer to use the same version as developers of the project but you can always try to use a newer version.) For that you need to right-click the project in solution explorer and click “Retarget projects”.

Conclusion

Now you should have newer versions of libraries which usually have many bugs and vulnerabilities fixed. Replacing libraries in games probably isn’t a thing you’re going to do very often, but otherwise compiling programs from source is a very useful skill to have. Should you encounter any errors, search engine of your choice is your best friend.