Skip to content

Fix four LP64 defects and add Oracle 19 client discovery - #34

Merged
AconiteX merged 5 commits into
SWG-Source:64-bit-typesfrom
Galaxies-Reborn:x64-dx9-no-docker
Jul 26, 2026
Merged

Fix four LP64 defects and add Oracle 19 client discovery#34
AconiteX merged 5 commits into
SWG-Source:64-bit-typesfrom
Galaxies-Reborn:x64-dx9-no-docker

Conversation

@swgsais

@swgsais swgsais commented Jul 25, 2026

Copy link
Copy Markdown

Four LP64 defects found while bringing up a 64-bit server from 64-bit-types, plus Oracle 19 client discovery. Each is present on 64-bit-types today. All five compile and none change 32-bit behaviour.

1. Packed maps desynchronise their own stream

AutoDeltaPackedMap<NetworkId,int>, <int,NetworkId> and <int,Unicode::String> write the command count as int32 but write the baseline count, and read both counts, as size_t.

pack writes unpack reads
ILP32 4 + 4 = 8 4 + 4 = 8
LP64 4 + 8 = 12 8 + 8 = 16

Under LP64 the reader consumes 4 bytes more than the writer produced. Little-endian makes this worse than a clean failure: the first get reads [count:4][zero:4] and yields the correct count, so the loop starts normally; the second get then swallows the remaining baseline bytes plus the first 4 bytes of command #1, and every subsequent key and value is misaligned. These run through SwgSnapshot, so the symptom is silent object-persistence corruption rather than a crash.

Fixed by using int32_t for both counts, matching the generic internal_pack/internal_unpack path and the already-converted PlayerQuestData specialization. This also keeps the serialized form byte-identical to a 32-bit server, so data written by either architecture stays readable by the other.

2. getCallStack() overruns every buffer by 2x

DebugHelp::getCallStack() passes the caller's uint32 buffer straight to backtrace(), which writes void* entries:

void DebugHelp::getCallStack(uint32 *callStack, int sizeOfCallStack)
{
    IGNORE_RETURN(backtrace(reinterpret_cast<void **>(callStack), sizeOfCallStack));
}
Call site Buffer backtrace writes Overflow
Fatal.cpp uint32[68] = 272 B up to 544 B 272 B (stack)
CallStack uint32[32] = 128 B 208 B 80 B (heap)
CallStackCollector uint32[26] = 104 B 208 B 104 B (heap)

Fatal.cpp's formatMessage() backs every WARNING and FATAL, so on a 64-bit build any warning that captures a stack corrupts memory. We first saw this as a SIGSEGV inside cfree() with si_addr 0x8 shortly after an unrelated missing-asset warning — the warning was the corruption and the crash was the aftermath, which makes it very hard to attribute.

Fixed by capturing into a native void* array and widening into a uint64 buffer, which is correct under both ILP32 and LP64, and widening the storage, lookupAddress() and the printing to match. On win32 the same code truncated StackWalk's DWORD64 address to 32 bits, so symbols were unusable there; that is fixed too.

Also corrects CallStackCollector copying a single element instead of the whole call stack (sizeof(*newCallStack) without the element count), and CRCing only half the buffer.

3. QueryPerformanceCounter is not monotonic on linux

The linux shim samples gettimeofday(), i.e. CLOCK_REALTIME, which time synchronisation can slew or step backwards. Clock::update() subtracts successive samples to derive frame time, so a backwards step yields a negative elapsed time and AlterScheduler reports:

doAlterConclude() called with elapsed time [-9.2e-05] resulting in no forward time movement

Under a hypervisor that re-syncs guest time this fires continuously. Combined with defect 2, every one of those warnings was also smashing 272 bytes of stack.

Fixed by using clock_gettime(CLOCK_MONOTONIC), keeping microsecond units so the value still matches QueryPerformanceFrequency(), with a gettimeofday() fallback if the call fails. Every getFrameStartTimeMs() consumer compares values relatively, so nothing depends on the previous epoch-based origin. This one is architecture-independent and applies equally to a 32-bit build.

4. SIGSEGV handler dereferences a 32-bit frame offset

JavaLibrary::fatalHandler() reads the faulting address from a hardcoded 0x44 byte offset into its own frame:

// the address that generated the segfault is stored at an offset of 0x44 from
// the frame address
// @todo: find documentation on the offset to verify it is stable
void *crashAddress1 = *((void**)((static_cast<char *>(frameAddress)) + 0x44));

That constant describes the 32-bit x86 frame layout. On x86-64 it is both misaligned and meaningless, and dereferencing it from inside a SIGSEGV handler risks a recursive fault that loses the crash report entirely. The existing @todo already flagged the offset as unverified.

Restricted to __i386__, relying on the portable __builtin_return_address() probe already present immediately below it. Also zero-initialises the lib/file buffers, which the strstr() checks read whenever a lookup fails.

5. Oracle 19 Instant Client discovery

FindOracle.cmake knows nnz18 and nnz21 but not nnz19, and does not consider ORACLE_HOME when searching. Adds libnnz19/nnz19 and ${ORACLE_HOME} to the search paths. Purely additive.

Verification

Built from this branch against SWG-Source/swg-main:64-bit-types with its stock Ant/CMake path and bits = 64:

[echo] Architecture is x86_64
[echo] Compiling as 64-bit application.
BUILD SUCCESSFUL

20 binaries produced; LoginServer, TaskManager, CentralServer, ConnectionServer, SwgDatabaseServer and SwgGameServer all verified ELF 64-bit LSB shared object, x86-64.

Separately, these same fixes were exercised at runtime on a downstream integration branch: Oracle schema creation and template load completed with 0 ORA- errors and 60,077 object templates, the full server process set reached a healthy state, and client login and logout worked. That run carried additional downstream deltas, so it is supporting evidence for these fixes rather than a clean-room test of this branch alone.

Not exhaustively swept: format specifiers server-wide, time_t in persisted structures, alignment and packing assumptions, and third-party ABI.

swgsais added 5 commits July 25, 2026 03:52
These three AutoDeltaPackedMap specializations wrote their command count as
int32 but wrote the baseline count, and read both counts, as size_t. Under
ILP32 size_t was 4 bytes so pack and unpack agreed. Under LP64 size_t binds
to the 8-byte uint64 Archive overload, so pack writes 4+8 bytes while unpack
reads 8+8 and leaves the stream misaligned by 4 bytes for every subsequent
key and value.

Switch both counts to int32_t, matching the generic internal_pack/
internal_unpack path and the already-converted PlayerQuestData specialization.
This also keeps the serialized form identical to the 32-bit server, so data
written by either architecture stays readable by the other.
DebugHelp::getCallStack() handed the caller's uint32 buffer straight to
backtrace(), which writes void* entries. Under LP64 that wrote 8 bytes per
frame into a 4-byte-per-frame array, overrunning every buffer by 2x:

  Fatal.cpp        uint32[68] = 272 bytes, up to 544 written (stack)
  CallStack        uint32[32] = 128 bytes, 208 written (heap)
  CallStackCollector uint32[26] = 104 bytes, 208 written (heap)

Fatal.cpp's formatMessage() backs every WARNING and FATAL, so any warning
that captured a stack corrupted memory. Observed as SIGSEGV inside cfree()
with si_addr 0x8 after an unrelated warning fired.

Capture into a native void* array and widen into a uint64 buffer, which is
correct under both ILP32 and LP64, and widen the storage and printing to
match. lookupAddress() already took uint64 on linux; win32 now agrees and no
longer truncates StackWalk's DWORD64 address to 32 bits.

Also fixes CallStackCollector copying only one element instead of the whole
call stack, and CRCing only half the buffer.
The linux shim sampled gettimeofday(), which reports CLOCK_REALTIME and can
be slewed or stepped backwards by time synchronisation. Clock::update()
subtracts successive samples to derive frame time, so a backwards step
produced a negative elapsed time, which AlterScheduler then reported as
"no forward time movement" once per affected frame.

Use clock_gettime(CLOCK_MONOTONIC) and keep microsecond units so the value
still matches QueryPerformanceFrequency(). Falls back to gettimeofday() if
the call fails. All getFrameStartTimeMs() consumers compare values
relatively, so nothing depends on the previous epoch-based origin.
JavaLibrary::fatalHandler() read the faulting address from a hardcoded 0x44
byte offset into its own frame. That constant describes the 32-bit x86 frame
layout; on x86-64 it is misaligned and points at unrelated stack memory, and
dereferencing it from inside a SIGSEGV handler risks a recursive fault that
loses the crash report entirely. The original code carried a TODO noting the
offset was never verified.

Restrict that probe to __i386__ and rely on the portable
__builtin_return_address() probe already present below it. Zero initialise
the lib and file buffers, which the strstr() checks read whenever a lookup
fails.
@AconiteX

Copy link
Copy Markdown
Member

just throwing in copilot review to see how it does but otherwise LGTM

2 nits on code style but need to just write a md for it

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses several LP64 (64-bit) correctness issues in serialization and debugging utilities, and extends CMake-based Oracle client discovery to support Oracle 19 Instant Client layouts. The changes aim to preserve 32-bit behavior and serialized compatibility while fixing 64-bit-only corruption/overflow risks.

Changes:

  • Fix packed-map serialization/deserialization count type mismatches by using int32_t for command counts.
  • Correct call stack capture/storage to be pointer-width safe (uint64), including Linux backtrace() handling and Win32 address preservation, and fix CallStackCollector buffer copy/CRC sizing.
  • Make Linux QueryPerformanceCounter use CLOCK_MONOTONIC, harden the SIGSEGV handler to avoid invalid frame-offset dereference on x86-64, and add Oracle 19 (nnz19) + ${ORACLE_HOME} library discovery paths.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file
File Description
external/ours/library/unicodeArchive/src/shared/UnicodeAutoDeltaPackedMap.cpp Uses int32_t for packed-map command counts to keep LP64 streams aligned.
engine/shared/library/sharedUtility/src/shared/NetworkIdAutoDeltaPackedMap.cpp Same packed-map count fix for AutoDeltaPackedMap<int, NetworkId>.
engine/shared/library/sharedFoundation/src/shared/AutoDeltaNetworkIdPackedMap.h Same packed-map count fix for AutoDeltaPackedMap<NetworkId, int>.
engine/shared/library/sharedFoundation/src/shared/Fatal.cpp Widen call stack storage/printing to 64-bit addresses.
engine/shared/library/sharedFoundation/src/linux/PlatformGlue.cpp Make QueryPerformanceCounter monotonic via clock_gettime(CLOCK_MONOTONIC).
engine/shared/library/sharedDebug/src/win32/DebugHelp.h Widen call stack APIs to uint64 on Win32.
engine/shared/library/sharedDebug/src/win32/DebugHelp.cpp Preserve full instruction addresses and widen stack storage/log formatting.
engine/shared/library/sharedDebug/src/shared/CallStackCollector.cpp Fix callstack buffer type, memcpy sizing, and CRC sizing.
engine/shared/library/sharedDebug/src/shared/CallStack.h Store instruction addresses as uint64 to match pointer width.
engine/shared/library/sharedDebug/src/shared/CallStack.cpp Print and resolve widened call stack addresses.
engine/shared/library/sharedDebug/src/linux/DebugHelp.h Widen call stack API to uint64 and document pointer-width requirement.
engine/shared/library/sharedDebug/src/linux/DebugHelp.cpp Fix Linux backtrace() usage by capturing into void*[] then widening.
engine/server/library/serverScript/src/shared/JavaLibrary.cpp Restrict frame-offset crash-address probe to __i386__ and zero-init buffers.
cmake/linux/FindOracle.cmake Add nnz19 and ${ORACLE_HOME} to Oracle client library discovery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@swgsais

swgsais commented Jul 25, 2026

Copy link
Copy Markdown
Author

just throwing in copilot review to see how it does but otherwise LGTM

2 nits on code style but need to just write a md for it

Would love any feedback on the nits so I can improve style for future PRs - Thanks!

// 4-byte-per-entry array and overran the buffer by 2x. Capture into a
// native pointer array and widen instead, which is correct for both
// ILP32 and LP64.
enum { cs_maximumFrameCount = 256 };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is functioning as a magic number, so could instead be at top of file, even though it is scoped and only relevant to the single function so it's 50/50 (I also don't love use of an untyped enum here since the compiler then has to make an inference and god knows what it may do in the future); alternatively, e.g., at top of file:

namespace
{
	constexpr int cs_maximumFrameCount = 256;
}

enum { cs_maximumFrameCount = 256 };

if (sizeOfCallStack <= 0)
return;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source generally (but admittedly inconsistently) uses allman style bracing and doesn't tend to use indentation block syntax/off-side rule, so code like this would properly be (but again doesn't technically matter and would surely start a not so fun debate)

if (sizeOfCallStack <= 0)
{
    return;
}

@AconiteX
AconiteX merged commit 22dd13d into SWG-Source:64-bit-types Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants