Splitting servers into multiple mods

[Mord]

Oh hi there

This is an idea that's been floating around my head for about 2 years now, since I've been on a year and a half hiatus from playing NWN2, I haven't really thought of mentioning it until today.

We all know there are limitations with how NWN2 handles hosting modules, the server software is hampered when the issue of module size is concerned, due to the 32 bit 2 gigabyte per process cutoff.

We have a workaround to this through the travel maps, eliminating the need for travel areas in favour of roleplaying and adventuring hotspots . This is however not a rant about the pros and cons of travel maps, but more a suggestion of how to improve both the maximum server size, and possibly server lag as well.

The NWN2 server software is single threaded, most if not all CPU's in use these days, including those used for hosting our Servers, are multicored. What I am proposing is to effectively split our Servers (BG, TSM, WHL etc) into several mods, one mod per core.

One mod would be reserved for roleplay areas, such as Cities and travel areas, where players can congregate and stage whatever adventure they wish. Another mod would house the adventuring areas, such as for instance: The cloakwood, the frost hills, Nether Mountains, and everything that goes with those areas.

This was tried in the past in NWN with the Arabel city / Northern Cormyr split, and the result was that Arabel City was mostly devoid of players. However, this proposal differs significantly in that you have all your most frequently used areas, areas where players meet, roleplay and stage adventures on a single module instead of effectively splitting the playerbase as was the case with Arabel/North Cormyr.

I've recently been informed that a system to streamline transitioning between modules has now been implemented in NWN2, Activate Portal. This will need some testing before its use can be validated, but as it stands now it appears very promising. This would bypass any need to log out of the module, enter gamespy lobby and log into the module you're transitioning to.

This proposal would also possibly go some way to limit lag incurred by players constantly logging in and out of the server, as they would most likely be doing so on the roleplaying module and not where latency plays a more important part, on the adventuring module. This will need some testing before it can be confirmed.

What you give up in the process is transparency, the ability for DM's to monitor and instantaneously port to a portion of the playerbase should they happen to find themselves on the opposing module. However, by dividing the server into 2 distinct, defined sections, one can assert right from gamespy lobby where players are located.

This is of course only a proposal at this point, meant to gauge interest and encourage discussion.

[maralten]

The servers are already pretty big. If anything, they would benefit from reduction / fleshing out existing areas, not outright expansion.

[paazin]

I could see this as potentially providing a number of benefits for our larger servers, though the delineation would prove rather difficult hurdle. The Northern Cormyr/Arabel City separation was uniquely successful because of the enormous scope of Arabel City itself; currently we have nothing even remotely reaching this magnitude. Baldur's Gate, if expanded to something vaguely resembling its canonical size, could serve well for this sort of model but likely there are better solutions.

Adapting it, as you suggest, to center one on bulk of the server, within those areas heavily trafficked might prove fruitful to some extent but there's a further question as to what other qualities/metrics one would wish to consider when deciding upon an optimal division. Obviously, the primary reason is to increase the number of playable areas per server (most likely exteriors, as they contribute by far the most to the memory footprint). As mentioned, this allows us some variables to play with like player size, numbers of exteriors vs interiors, and simple utility (e.g. having to port servers constantly would be more trouble than it's worth).

The proposal considering keeping heavily-used areas on one server and lesser-used areas on another looks solid enough on the face of it, but problems will arise in implementation. Where is the specific line to be drawn? What of difficulties in fetching other players or DMs from an 'adventuring' area to the 'civilized' areas, and so forth? In slight contrast to the proposal, consider a more natural geographic split that would better 'fit' the lay of the server, e.g. dividing all that west of Rauvinwatch to one server and all that is east to another. This would keep that inherent split with players that often arises between Rivermoot and Silverymoon (that often ends up lagging the server), while still leaving the surrounding adventuring areas within the towns available to explore without changing servers.

One thing to consider is what exactly are the largest limits to including unique and interesting areas within our mods. Pretty plainly, it's the gargantuan amount of time and effort required to get good-quality areas out there. Even if we were able to split a server up into two or more pieces there's no guarantee that the resources freed will be used for anything of interest. If there were dozens of potential builders chomping at the bit to include areas, that would be another matter, but it doesn't appear to be the case...

[ElCadaver]

Is there any statistical function to NWN2? Is it possible to take a months data from the servers and show (for example) the total number if IG hours, or PC's per hour, per area of the module? Would be really useful determing the application of the above portal process.

[danielmn]

One tech fix if ALFA goes this route, I belive, is with any static that goes between modules. I think something extra needs to be done with those in order for the statics to register in their steps.

[Swift]

The Northern Cormyr/Arabel City separation was uniquely successful because of the enormous scope of Arabel City itself; currently we have nothing even remotely reaching this magnitude

Successful tech wise, wildly unsuccessful in almost all other ways. It did allow the HDMs to create a very detailed version of Arabel, unfortunately, thanks to being seperate, it was mostly wasted effort since very few people ever went in there unless absolutely required.

Overall, the NC/AC split was a nice experiment, but not a successful one, which was one of the reasons why the DM team pulled it back into the main NC mod when they did the complete rebuild of NC.

[Veilan]

Servers tend to have a lot of only marginally used space. Why's that? Because players like to play together, so one, or several, meeting spots crystallise.

Therefore, the simplest solution would be that every area apart from one inn, one temple, one shop and one cave needs a solid justification to make it into a server .

This is of course not a dig at the beautiful craft we can enjoy and experience on our servers... simple isn't always best. It does highlight, though, that splitting up / expanding is not really a good way to increase enjoyment... neither for players who want to hook up and interact, nor for DMs who may want to have quick access to, and ability to contact, everyone who's around.

[stormsong]

Is there any statistical function to NWN2? Is it possible to take a months data from the servers and show (for example) the total number if IG hours, or PC's per hour, per area of the module? Would be really useful determing the application of the above portal process.

I had a tracking function that showed time in module and in area as well as treasure and experience.

It could be used by the DM wand to track who was on and where as well. It let us get a lot of information on trends and possible cheats.

ActivatePortal() has been in my mind the absolute hole in NWN2. The failure to release it with the inital package was an admission that they had no interest in multiuser.

Now that it is working it offers huge chances for doing things. The key is that by itself I would be nervous about using a shared vault across the internet. In the past, it caused some bad lag spikes and spawned some real cheating. We had to run all the servers off a local network or do some real ugly workarounds to make it work.

Now however, using NWNX4/ActivatePortal the world is wide open. There is some testing that needs doing, but I think that it will make multiserver rings and spliting servers work nicely. I would be happy to work with anyone in doing the testing.

[AcadiusLost]

Now that it is working it offers huge chances for doing things. The key is that by itself I would be nervous about using a shared vault across the internet. In the past, it caused some bad lag spikes and spawned some real cheating. We had to run all the servers off a local network or do some real ugly workarounds to make it work.

We have been using a remote vault (webdav) since BG went Live, more than a year ago (unless you count the month and a half when I was hosting both, while Zic was waiting on ISP issues). The lag overhead is significant, but it's something we're already used to dealing with.

How smoothly it will work for ActivatePortal() remains to be seen. To me it seems like a pretty trivial thing to relog into the next server when portalling, takes 10 seconds or so... so I'm not sure I see how it would change anything fundamental with regard to ALFA's server organization.

As a DM on NC/AC when they were split, I can say it caused a wide variety of frustrations from that side of the wand; it doesn't seem like a choice we would want to make for other servers unless necessary.

[ElCadaver]

Though.. a 4 node 8 core 8GB supercomputer for $1300US running 8 modules could be worth the headache. That could almost bring back the NWN1 style servers with all that lovely scenery!

Maybe the guy will lend it to you to see if it can be done? Anyone live near Grand Rapids, MI?

http://www.calvin.edu/~adams/research/microwulf/design/

[Sandermann]

Its a couple of months on, but im interested to see how or if community thoughts on this topic have progressed. The BG mod is currently at 1.2 GB, thats sounds like theres tons of space left but even if all remaining 800 MB were sued to hit the max 2GB limit, that only results in another 25 or so exteriors.

Considering the only adveturing location we really have is the cloakwood and that needs expanding 25 areas doesnt cut it. Fields of the Dead, Wood of Sharp teeth, Troll hills, Cloudpeaks, Greenfields - these all need to go in and that doesnt even count the extra roads and hill areas required to link them to the "spine" of the tradeway in a believable manner.

Needless to say we ahve abandoned the idea of a travel map for BG, the current system of roads and link areas has met with enough support for me to believe this is the preffered route for players.

[edit]: nt to mention the ongoing inclusion of extensive underdark. I would like to use more exterior areas down there, currently ive had to limit its allocation as one 16x16.

[Regas]

This sounds like a great deal of work but if you think it can be done I wouldn't mind dealing with multiple servers. How would porting between servers work?

Also.. does this mean we might someday see the city built to something like the traditional size we'd expect? That would be amazing.