The user count for Netware 3.xx is embedded in the SERVER.EXE file that is loaded by DOS when the server boots. Copying that to the C:\SERVER.312 folder is the proper method to upgrade the user count.
Although the error message make it sound like this could be a problem with getting Netware to see all the memory in the server, I would think you'd have noticed that problem earlier.
The problem probably lies in the fact that you repaced a newer version of the SERVER.EXE program (v3.2) withan older one (3.12).
Version 3.2 is, generally speaking, the same as version 3.12 with all patches applied. Many netware patches re-wrote the SERVER.EXE. So, it's entirely possible that there's something internal to the 3.12 executable that'snot compatible with the rest of the system.
Replace the SERVER.EXE program with the original 3.2 executable, then see if the server starts normally. If is does, you know where the problem lies. If not, it's something else.