Creating a Content Patch
ACR 1.87 supports the creation and deployment of content patches, which are server-side hotfixes that are automatically deployed to live servers on startup. The content patch system is intended to make it easier to deploy important bugfixes to an already released ACR version.
Content patches can be used to perform the following tasks:
- Place a file in the Override directory for the server. Note that module and hak resources supersede "Override" resources (but Override supersedes in-box .zip files and can be used to deploy a new script that could be called by a hak, etc.). Override-targetted content patch files go in "override\ACR_ContentPatches" on each live server.
- Place an updated hak in the Hak directory for the server (for example, alfa2_acr.hak).
- Recompile module scripts.
- Patch moduledownloaderresources.xml with new hak file download information for game clients (module specific downloader resources are left as-is).
Content patches cannot be used to make client-side content changes alone (those must go through the standard release process in addition). As of ACR 1.90, the content patch system can recompile all module scripts (if the RecompileModule database column is set for a content_patch_files row that required updating), if necessary. This facility can be used to update includes in the ACR hak and recompile scripts in a module that reference the include.
Content patches automatically update the downloader XML for a server (see #Content Patch Downloader Resources for how to configure new dowmloader XML information for new client-side release). Server admins should be careful to not restage haks that have been hotfixed by the content patch system (typically, alfa2_acr.hak), but if any of the downloader XML descriptors for noticed files were incorrect, the server will correct these on next startup (incurring an additional server process restart).
Content Patch Components
A content patch contains several distinct components which must be assembled:
- The content files to patch (e.g. alfa2_acr.hak, or a free-standing file to place in the Override directory). Any number of files can theoretically be content patched, but you're encouraged to keep the set of file small. The content files must be placed on the central server vault, in a special directory, for them to be downloadable by the content patch system.
- A set of database entries in the content_patch_files table that describe what ACR hak version the patch applies to (e.g. 1.87), and what file is associated with the patch. You need a database record for each file that is contained within the content patch.
- An optional initialization script, called "acr_patch_initialize". If present, this script is run during module startup, to provide opportunity for a content patch to hook script callbacks or set things up. The script is never located in the ACR hak or the module, but should only be distributed as a content_patch_files record.
Creating a Content Patch
The following is a checklist of the steps necessary to build and deploy a content patch. Once deployed, it will be automatically picked up by live servers on their next restart. (If necessary, you can remotely restart a server using the Restart Server tool.)
These steps assume that you have already verified that the new hak to install works correctly, and that it compiles correctly for a module. These steps are highly recommended to make sure that the patch process goes smoothly.
- First, create the content files to patch. For example, the fixed script or the patched ACR hak. Typically this would be done by checking out the ACR branch for the released ACR version to be patched, making the necessary changes and checking them in to the repository, and then rebuilding the hak from that branch.
- Record the existing content_patch_files contents from the database, in case a rollback is necessary.
- Next, copy the patched content files to the Azure updater using the UpdaterConnectionString (formerly, the files were placed on the central vault). They should go under ACR_ContentPatches/Global/<ACR_Version> on the updater Azure storage (for example ACR_ContentPatches/Global/1.87 for an ACR 1.87 patch). AzCopy or Storage Explorer can be used to upload content patch files. Optionally, and recommended, content patch file hosted on Azure storage can be gzip compressed with a tool like gzip.exe from msysgit, or 7-zip (append .gzip to the filename uploaded to Azure, e.g. alfa2_acr.hak.gzip after compressing with gzip.exe). Compression was not supported for updates distributed by the central vault, though that method is deprecated. Remember that Azure storage is case sensitive, so path cases must match exactly, and that the compression format must be gzip.
- Note! It is highly recommended that the previous content patch files be downloaded or saved, for any files that will be replaced, so that a rollback can be performed.
- The content patch files are stored in the alfa-nwn2-acr-updater storage container on Azure storage. By convention, the content patches are stored in the ACR_ContentPatches\Global\<hak version> directory; the latter path is drawn from the config table "ContentPatchPath" variable.
- By current convention, the ACR version is left at 1.9201 and has not been incremented for some time. The content patch system allows different patches to be selected based on the previous ACR version of the server, but it has proven more conventient in many cases to simply keep the same version number and produce a new patch content.
- Presently, the server vault and ACR updater are on the same Azure storage account, though this is not architecturally required.
- Generate a md5 checksum of each of the patched content files. The md5sum utility (included with msysgit - http://code.google.com/p/msysgit/ - or cygwin, can do this). You'll need these checksums to insert into the database in the next step. If compressed files were stored on Azure storage, the checksum must be that of the uncompressed file.
- Create (or, as need be, update) a database record in content_patch_files for each file present in the content patch.
- Set the FileName column to the name of the file to patch (with extension, no spaces, for example "alfa2_acr.hak"). This should not include the .gzip appended for compressed content patches, rather it should reflect the final on disk filename of the content to update.
- Set the HakVersion column based on the ACR version that is matched when deciding whether the content patch is applicable (for example, "1.87").
- Set the Location column to either "override" or "hak", depending on where the content file should be stored on live servers.
- Set the Checksum column to the md5sum value (32 hex characters) for the file that is being patched.
- Set the RecompileModule column to true if the module needs to have all scripts recompiled, else false if a recompile is not required. A recompile is generally only necessary if an include in the ACR HAK has been changed and the include is referenced by a script distributed with the module. This option is only supported in ACR 1.90 or higher (otherwise the module is never recompiled).
- Only one row for a specific combination of both a given FileName and HakVersion should exist in the table.
- Connect a test server to the database and central server vault, and restart it. Check the log to make sure that the content patch applied successfully; the server should restart a second time after applying the patches. There will be server log entries containing "ModuleContentPatcher" updating you with the status of content patches as they're applied.
- If a content patch needs to cause new client content to be downloaded, follow the steps for #Content Patch Downloader Resource Updates to configure the moduledownloaderresources.xml entries that will be patched on each server. Patches that only have server-side script changes do not need this, but 2DA or template or GUI updates will need client-side updates, and have to go through that process.
It is highly recommended that you test the content patch on a test server before copying the patch data to the live server. You should still verify that the patch applied correctly to a live server, however, once the patch has been deployed to production.
Remember to set the RecompileModule flag on the ACR hak if any script includes that are included by the module are modified. Most script files are included by the module, so this is often necessary. Without this step, for a script update, servers may not see the update take effect.
The IRC bot will send status messages in #alfa-players and #alfa-tech as an update is under way, so being present in these channels is recommended.
Detailed logging information is logged to the server log. If that is not present, stop the server, edit nwn2player.ini for the server, and make sure that "Scripts Print to Log=1" is present (the default might be "Scripts Print to Log=0"). This is required for WriteTimestampedLogEntry to log to the server log, and most diagnostics are written with that mechanism. Within the server log, search for the strings ModuleContentPatcher and PatchContentFiles to find the log messages relevant for the ACR updater.
In order for the module recompile to work, NWNScriptCompiler.exe has to be installed in the NWNX4 directory of the server. This is part of the standard server configuration and deployment process. A file not found error when launching the compiler, in the server log, may indicate that NWNScriptCompiler.exe is not properly installed. The recompile process only takes effect when the given hak is updated, so you may need to copy the old hak onto the server and start it again to cause a recompile to take, if the recompiler flag was not properly set the first time around.
If a server does not pick up an update at all, the database entries in content_patch_files are likely to be incorrect.
Client downloader resources need to be processed using the #Content Patch Downloader Resources step below, so if a client side change does not take effect, make sure that both processes have been followed.
Content Patch Downloader Resource Updates
The content patch system can automatically maintain moduledownloaderresources.xml if new haks need to be pushed to client machines. There is a separate process, outlined below, to configure updates to moduledownloaderresources.xml on each server. This checklist needs to be followed in addition to the steps for #Creating a Content Patch when any .hak updates need to be downloaded by client machines. Note that this process can only update existing entries in moduledownloaderresources.xml; if an entirely new hak is created from scratch, this must be manually added to moduledownloaderresources.xml for each server, or the content patch system would need to be extended to support adding new download resources.
The database table content_download_config contains a set of rows that describe override values that are installed into each server's moduledownloaderresources.xml on startup, replacing any previous values. This provides a mechanism to update the autodownloader manifest on all servers as they deploy a content patch, and avoids the downloader resources from having picking up invalid values for ACR-controlled resources when a per-module content update is performed.
Note that this process does not replace publishing new hak files to the download server for clients to download, and that is a separate step fron the Azure ACR updater.
- First, go through the normal process to stage the new hak contents for the autodownloader using the toolset using any module that references the ACR haks after installing the new hak that will be published to the autodownloader, and take note of the specific hak files that need client updates. These are typically all haks, except that the ACR hak can be stale on the client in the case of server-side script only updates. Note that it is important to use the same hak on the client and server, so it is best to perform the steps listed in #Creating a Content Patch followed by those in this section at the same time, unless an update is a server-side only script change.
- For each hak file that needs client-side updates, take note of the following elements from the new moduledownloaderresources.xml produced by the toolset. Each of these values will need to be updated in the corresponding row of content_download_config, which is keyed off of by hak filename. Below, the name of each moduledownloaderresources.xml element is followed, in parenthesis, by the corresponding database column name in content_download_config.
- name (Name)
- hash (Hash)
- downloadHash (DownloadHash)
- dlsize (DLSize)
- size (Size)
- Record the existing content_download_config contents from the database, in case a rollback is necessary.
- Each hak that needs a new client-side update must have its corresponding content_download_config rows updated to match the new moduledownloaderresources.xml elements on the computer that you have used to stage a new ACR-controlled hak. There should be only one entry in the table for each .hak Name value.
- As normal, the staged lzma files from the toolset need to be uploaded to the download server so that clients can find them.
- Note! It is highly recommended that the previous LZMA files be downloaded or saved, for any files that will be replaced, so that a rollback can be performed.
Content Patches and the Standard Release Process
The content patch system ensures that patches are tagged with a specific ACR version that they apply to. When a server upgrades to a new ACR version, using the standard release process, previous content patches will no longer be applicable and won't be applied anymore. In most cases, the ACR version is not incremented, and is kept at 1.9201, though previously it used to have been incremented. This was found to create more overhead than benefit and so the old version is
For content patch files located in the "hak" directory, no action is taken. It's expected that if the administrator did an ACR upgrade, that they have manually upgraded the haks anyway. (No new haks can be added with the content patch system, only hotfixes to existing haks.)
For content patch files located in the "override" directory, the content patch system automatically removes any patch files not applicable to the ACR version the server starts up with at startup time. Thus it's not necessary to manually clean the content patch override directory out.
Content Patch Server Log Messages
Messages in the NWN2 server log file track the progress of the content patch system. When a content patch has been successfully applied, a log message of the form "ModuleContentPatcher.ProcessContentPatches: Successfully updated content patch file <filename>." will be written to the server log. Log messages related to the content patcher can be easily identified by searching for ModuleContentPatcher in the log file. To verify that a content patch was deployed successfully, examine the server log after a server restart. Be sure to check the log message timestamp; it is usually best to start searching from the end of the server log file.
If a content patch requests a module script recompile, then the console output of the script compiler is printed to the server log file. An error message prefixed with the standard ModuleContentPatcher prefix is printed to the server log if any scripts failed to compile.