Create your own
Lesson illustration

Building a Hookable x64 Windows Test Fixture

Hello again. Previously, you separated the Node.js host from the injected Frida agent and established the rule that native, target-local work belongs in the agent while lifecycle, commands, storage, and presentation belong in the host.

You now need a target process whose behavior is known before you instrument it. Rather than beginning with an arbitrary Windows application, you will create a small x64 fixture consisting of a DLL and a console runner. The DLL exports one stable, hookable function; invoking that function performs a predictable CreateFileW call and a predictable Registry call. The runner loads the DLL before it announces that it is ready, then waits for explicit run commands. That wait is what lets your future host attach and install hooks before the interesting work happens.

Use this fixture only on your own machine and only as a controlled test target.


What makes an instrumentation fixture useful?

A fixture is not merely a small executable. It is a test contract between the code you are about to write and the environment in which it runs.

For this course, the contract is:

  1. fixture-runner.exe is a 64-bit Windows process.
  2. fixture.dll is loaded before the runner prints READY.
  3. The DLL exports a stable C symbol named Fixture_DoWork.
  4. The runner performs no observed workload until it receives run on standard input.
  5. Each run invokes Fixture_DoWork once with a predictable wide-string path and a positive sequence number.
  6. A successful call performs these Windows API calls in this order:
    • CreateFileW
    • WriteFile
    • CloseHandle
    • RegOpenKeyExW
    • RegCloseKey
  7. The function returns the supplied sequence number on success, and 0 on a fixture-level failure.

This gives later integration tests three clear observations:

Later capabilityControlled observation
Function interceptionOne entry to fixture.dll!Fixture_DoWork
File API tracingA CreateFileW call with a known UTF-16 path
Registry API tracingA RegOpenKeyExW call for HKCU\Software
Return-value captureThe exported function returns the sequence number
Attach timingThe process waits until the test explicitly requests work

The fixture deliberately does not contain Frida code, message protocols, or CLI behavior. It is external test infrastructure. Keeping it in its own directory prevents native test mechanics from leaking into your TypeScript domain and application layers.


Prepare an x64 MSVC command environment

For a reliable architecture match on Windows 11 x64, compile both the DLL and the runner from an x64 Native Tools Command Prompt. A regular Command Prompt may not have cl.exe, link.exe, the Windows SDK headers, or the library paths configured.

Use the Microsoft C++ Build Tools from the command line | Microsoft Learn

Read Microsoft Learn’s setup guidance to confirm the required C++ workload and, crucially, the correct architecture-specific command prompt.

In “Download and install the tools”, read the full section. The sentence linked as installer step is near the guidance to choose the Desktop development with C++ workload. Then, under “Developer command prompt shortcuts”, find the x64 prompt entry. Use that prompt rather than the default Developer Command Prompt.

Open x64 Native Tools Command Prompt for Visual Studio and confirm that the compiler is available:

where cl
cl

The compiler banner should identify an x64 target. The exact Visual Studio version is unimportant; the target architecture is not.

Create this structure inside your repository:

fixtures/
  windows-x64/
    src/
      fixture_api.h
      fixture_dll.c
      fixture_runner.c
    build/

Add fixtures/windows-x64/build/ to .gitignore. Commit the source files, not the generated DLL, import library, executable, or object files.


Export a stable hook target

Frida can resolve an exported symbol by module name and export name. The future request will look conceptually like:

fixture.dll!Fixture_DoWork

That is a symbolic request, not a runtime address. The future agent will resolve it inside the target process after the DLL is loaded, which correctly accommodates ASLR.

The critical native-code decision is to expose a C ABI function. C++ names can be decorated or mangled by the compiler, whereas a C function produces a stable, readable export name for this controlled build. On x64 Windows, calling conventions no longer change the machine-level ABI in the way they did on 32-bit Windows, but declaring __cdecl still documents the intended native interface and keeps the fixture explicit.

Exporting from a DLL Using __declspec(dllexport) | Microsoft Learn

Read Microsoft Learn’s explanation of DLL exports. It explains why __declspec(dllexport) is sufficient for this small fixture and why export naming deserves deliberate attention.

In the article’s opening discussion, read the explanation of exporting functions and the note about recompilation and name decoration. Focus on the name stability rationale. Then continue through the paragraph beginning “To export functions” and its code example, noting that the export declaration precedes the calling-convention keyword.

Create src\fixture_api.h:

#pragma once

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#ifdef FIXTURE_DLL_BUILD
#define FIXTURE_API __declspec(dllexport)
#else
#define FIXTURE_API __declspec(dllimport)
#endif

#define FIXTURE_NOINLINE __declspec(noinline)

/*
 * Contract:
 * - output_path must name a writable file location.
 * - sequence must be non-zero.
 * - returns sequence on success, or 0 on fixture-level failure.
 */
FIXTURE_API FIXTURE_NOINLINE DWORD __cdecl Fixture_DoWork(
    LPCWSTR output_path,
    DWORD sequence);

There are two details worth retaining:

  • __declspec(dllexport) places Fixture_DoWork in the DLL export table.
  • __declspec(noinline) prevents optimization from folding this small function into some caller. An exported DLL function already has a durable entry point, but explicitly preventing inlining communicates that this is the intended interception boundary.

The runner sees this declaration as dllimport; the DLL sees it as dllexport. Both compile against the same function signature.


Implement the deterministic native workload

Create src\fixture_dll.c:

#define FIXTURE_DLL_BUILD
#include "fixture_api.h"

FIXTURE_API FIXTURE_NOINLINE DWORD __cdecl Fixture_DoWork(
    LPCWSTR output_path,
    DWORD sequence)
{
    static const char marker[] = "grasp-fixture\n";

    HANDLE file = CreateFileW(
        output_path,
        GENERIC_WRITE,
        0,
        NULL,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_NORMAL,
        NULL);

    if (file == INVALID_HANDLE_VALUE) {
        return 0;
    }

    DWORD bytes_written = 0;
    BOOL write_succeeded = WriteFile(
        file,
        marker,
        (DWORD)(sizeof(marker) - 1),
        &bytes_written,
        NULL);

    CloseHandle(file);

    if (!write_succeeded || bytes_written != sizeof(marker) - 1) {
        return 0;
    }

    HKEY key = NULL;
    LSTATUS registry_status = RegOpenKeyExW(
        HKEY_CURRENT_USER,
        L"Software",
        0,
        KEY_READ,
        &key);

    if (registry_status != ERROR_SUCCESS) {
        return 0;
    }

    RegCloseKey(key);
    return sequence;
}

This is intentionally unambitious native code. Its value is its observability.

Why these choices matter

CreateFileW uses a wide string. The W suffix means its path parameter is a UTF-16 string, represented in C as LPCWSTR. Your future Frida agent will receive a pointer, not a JavaScript string, and must safely decode it as UTF-16. This fixture provides an expected value: fixture-output.txt.

CREATE_ALWAYS removes prior-file state. Every run produces a known final file content instead of appending to whatever an earlier test left behind.

The Registry call has a stable subkey argument. RegOpenKeyExW(HKEY_CURRENT_USER, L"Software", ...) lets the Registry trace decode both a predefined root handle and a UTF-16 subkey. In a normal interactive Windows user profile, this key is available. The function still checks the returned status so an environment problem becomes visible rather than silently producing a misleading test result.

The return value is useful for interception. A later hook can capture the return value and compare it with the input sequence. Returning 0 on failure makes the fixture’s own success condition unambiguous.

The function itself will later be intercepted at its entry point. The API calls it makes will be traced separately. That distinction mirrors real analysis: one probe tells you a particular application operation began, while another probe reveals the operating-system interaction it caused.


Build a runner that waits for your test

A fixture that immediately executes and exits is fragile: by the time your host attaches, the function call may be over. The runner below loads the DLL as an ordinary executable dependency, announces its PID, and waits for you or a test harness to send run.

Create src\fixture_runner.c:

#include "fixture_api.h"

#include <stdio.h>
#include <wchar.h>

static void trim_newline(wchar_t *text)
{
    size_t length = wcslen(text);

    while (length > 0 &&
           (text[length - 1] == L'\n' || text[length - 1] == L'\r')) {
        text[length - 1] = L'\0';
        --length;
    }
}

int wmain(void)
{
    wchar_t command[64];
    DWORD sequence = 1;

    wprintf(L"READY pid=%lu\n", GetCurrentProcessId());
    wprintf(L"Commands: run | quit\n");
    fflush(stdout);

    for (;;) {
        wprintf(L"fixture> ");
        fflush(stdout);

        if (fgetws(command, (int)_countof(command), stdin) == NULL) {
            break;
        }

        trim_newline(command);

        if (wcscmp(command, L"quit") == 0) {
            break;
        }

        if (wcscmp(command, L"run") != 0) {
            wprintf(L"Unknown command. Use: run | quit\n");
            continue;
        }

        DWORD result = Fixture_DoWork(L"fixture-output.txt", sequence);

        if (result == sequence) {
            wprintf(L"DONE sequence=%lu result=%lu\n", sequence, result);
        } else {
            wprintf(L"FAILED sequence=%lu result=%lu\n", sequence, result);
        }

        ++sequence;
    }

    wprintf(L"STOPPED\n");
    return 0;
}

The DLL is imported when the runner starts. Therefore, when READY appears, fixture.dll is already present in the target’s loaded module list. This is exactly the condition your future agent requires before it resolves the export.

The run gate also avoids a common integration-test race:

  • Start the fixture and wait for READY.
  • Attach your host to the reported PID.
  • Load the agent and install a probe.
  • Send run.
  • Assert that the probe observed exactly one call.

For now, you will trigger run manually. Later, a Node.js integration test can spawn this runner with piped standard input and write run\n programmatically.


Compile both artifacts as x64

From x64 Native Tools Command Prompt, change into the fixture directory and build the DLL:

cd path\to\your-project\fixtures\windows-x64

cl /nologo /TC /W4 /WX /O2 /LD /Fo:build\fixture_dll.obj src\fixture_dll.c ^
  /link /OUT:build\fixture.dll /IMPLIB:build\fixture.lib /INCREMENTAL:NO Advapi32.lib

Advapi32.lib is required because RegOpenKeyExW and RegCloseKey are Registry APIs supplied through Advapi32.dll.

Then build the runner against the DLL’s import library:

cl /nologo /TC /W4 /WX /O2 /Fo:build\fixture_runner.obj src\fixture_runner.c build\fixture.lib ^
  /link /OUT:build\fixture-runner.exe /INCREMENTAL:NO

The build directory should now contain at least:

fixture.dll
fixture.lib
fixture-runner.exe

It may also contain .exp, .obj, and debugging artifacts depending on your MSVC installation and options.


Verify the binary contract before involving Frida

First, verify that the DLL exports the exact symbol that the future agent will request:

dumpbin /exports build\fixture.dll

Look for Fixture_DoWork in the export list. Do not build future probes around an ordinal number. Ordinals may be present in the export table, but the fixture contract is the readable symbol name.

A `dumpbin /exports` listing for a DLL. The output associates each exported function name with an ordinal; for this fixture, verify the named export `Fixture_DoWork` rather than relying on its ordinal.

Next, verify the runner’s architecture:

dumpbin /headers build\fixture-runner.exe | findstr /i "machine"

You should see output identifying an x64 machine. If it reports x86, stop here and reopen the x64 Native Tools Command Prompt before rebuilding. An x64 fixture is important because the next Windows lessons use x64 calling-convention assumptions and x64 Frida agents.

Finally, run the fixture from its build directory so Windows can locate the sibling DLL:

cd build
fixture-runner.exe

Expected initial output resembles:

READY pid=12345
Commands: run | quit
fixture>

At the prompt, enter:

run

You should see:

DONE sequence=1 result=1

Then inspect the predictable file output:

type fixture-output.txt

Expected content:

grasp-fixture

Enter run a second time. The result should now be 2, and the file content should remain the same because CREATE_ALWAYS recreates it. Finish with:

quit

A successful manual validation means the fixture’s observable contract is ready for Frida.


Keep fixture failures separate from instrumentation failures

When you later automate this, distinguish these categories clearly:

SymptomLikely sourceFirst check
cl is not recognizedBuild environmentOpen x64 Native Tools Command Prompt
unresolved external symbol RegOpenKeyExWLink configurationEnsure Advapi32.lib appears in the DLL build command
Runner cannot find fixture.dllLaunch locationStart fixture-runner.exe from build
FAILED sequence=... result=0Fixture workloadConfirm the current directory is writable and inspect local permissions
Export not found laterSymbol contract or module nameRun dumpbin /exports and use fixture.dll!Fixture_DoWork
Probe installs but sees no callTest orderingAttach and install the probe before entering run
API trace sees an unexpected file pathRunner working directoryLaunch consistently from fixtures/windows-x64/build

A dumpbin export listing verifies the symbol in the DLL file on disk. It does not provide an address that you should reuse in a target process. In the next lessons, the injected agent will resolve the named export inside the live process, where ASLR and loaded-module state are known.


Key takeaways

You now have a deterministic x64 Windows target for authorized Frida integration work:

  • fixture.dll exports the C ABI function Fixture_DoWork.
  • fixture-runner.exe loads that DLL before announcing its PID and waits for an explicit run.
  • Each successful run produces one exported-function invocation, one CreateFileW call with a known UTF-16 path, and one RegOpenKeyExW call with a known subkey.
  • dumpbin /exports verifies the named export, while dumpbin /headers verifies the x64 architecture.
  • The fixture stays outside your TypeScript host and agent architecture, serving as controlled integration-test infrastructure.

Next, you will attach to this authorized local runner from the Node.js host and load the minimal compiled Frida agent into it.

Can't find a good explanation? Sign up and we'll make it for you

Sign up