Rust FFI

Sometimes, we need to implement some libs using rust, and these libs should be loaded by other programs, which means we have to implement some interface for these libs based on the specific foreign interface standard. Usually, this specific standard is called FFI(Foreign Function Interface). It is an important mechanism that allows libraries to access functions from other programming language. In this way, we can use this mechanism call C functions in Rust language or call Rust functions in C language.

Externally Exported functions

In many cases, it is necessary to export certain functions when implementing libraries or DLLs in any programming language. Therefore, it is important to understand how to export functions from Rust language libraries."

It is an easiest demo to export a function. no_mangle attribute enables us to maintain a clear function name, and non_snake_case allows us to name functions using the Pascal Case method without throwing warnings.

1
2
3
4
5
6


// lib.rs
#[no_mangle]
#[allow(non_snake_case)]
extern "C" fn GetValue() -> usize {
	return 0xDEADBEEF;
}

And we can use this method to call GetValue(), which is same as call any exported functions from dynamic libraries.

1
2
3


HMODULE hDll = LoadLibraryA("lib.dll");
auto GetValue = GetProcAddress(hDll, "GetValue");
( (size_t (*)())(GetValue) )();

External Class Instance

Sometimes, we have to export some instance variables of certain classes to programs, allowing programs to invoke member functions using these instances. For this requirement, we can define classes which we want to export in a .h, and include this .h in both the library and the program, if we use C language. We can follow the same approach as with Externally Exported functions by returning the instance ptr from an external function, and the program can receive this ptr and cast it to the class type defined in the .h file.

Here is an example implemented from C++.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


// lib.cpp
class Demo {
public:
	virtual bool Init(
		IN unsigned int timestamp,
	) = 0;
	virtual bool UnInit() = 0;
};

class TestDemo : public Demo
{
private:
	unsigned int init_timestamp;
public:
	virtual bool Init(
		unsigned int timestamp,
	);
	virtual bool UnInit();
};
bool TestPlugin::Init(
	unsigned int timestamp,
)
{
	this->init_timestamp = timestamp;
	return true;
}

bool TestPlugin::UnInit() {

	return true;
}


TestPlugin g_Obj;

extern "C" __declspec(dllexport)  Demo * __stdcall GetObj() {
	return &g_Obj;
}

1
2
3
4
5
6
7
8
9


// program.exe
HMODULE hDll = LoadLibraryA("lib.dll");
auto GetObj = GetProcAddress(hDll, "GetObj");
Demo* instance =  ( (Demo* (*)())(GetObj) )(); 
if ( instance->Init(0xDEADBEEF) == true )
{
	printf("[+] Instance initialize success");
	instance->UnInit();
}

Obviously, we can invoke GetObj and cast return value to the same type as in lib.cpp. This allows us to call any functions using the instance through the vtable.
Now the question is how we can implement the lib using Rust language and achieve the same goal.

The crucial key to solving this question lies in how we implement a class and structure it with a vtable. This is because external programs rely solely on the vtable to access member functions through instances obtained from exported functions. However, the raw classes in Rust don’t inherently possess vtable structure. Therefore, we need to construct a struct resembling a vtable to enable external program to call member functions using these instances.
Initially, I came across a crate called “vtable” and attempted to wrap the class using it. However, I was unable to find a valid vtable, possibly due to issues with my usage. As a result, I construct vtable struct manually, there is my demo.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52


// lib.rs
#[repr(C)]
pub struct DemoVtable{

    Init: *const unsafe extern "C" fn(&mut Demo, u32) -> bool,
    UnInit: *const unsafe extern "C" fn(&mut Demo, ) -> bool,
    none: usize

}

#[repr(C)]
#[derive(Debug)]
pub struct Demo { 
    vptr: *mut c_void,
    vtable: *const DemoVtable,

	init_timestamp: u32,
}

impl Demo {
    #[no_mangle]
    extern "C" fn Init(self: &mut Demo, init_timestamp: u32) -> bool {

		self.init_timestamp = init_timestamp;
        true
    }
    #[no_mangle]
    extern "C" fn UnInit(self: &mut Demo,) -> bool { 
        
        true 
    }
}

const DEMO_VTABLE: DemoVtable = DemoVtable{
    Init: Demo::Init as *const _,
    UnInit: Demo::UnInit as *const _,
    none: 0
};
pub static mut GLOBAL_DEMO: Lazy<Demo> = Lazy::new(|| {
    Demo {
    vptr: Box::into_raw(Box::new(DEMO_VTABLE)) as *mut c_void,
    vtable: &DEMO_VTABLE,
    init_timestamp: 0,
    }
});


#[no_mangle]
#[allow(dead_code)]
extern "stdcall" fn GetObj() -> usize {
	return unsafe{std::mem::transmute(&GLOBAL_DEMO.vptr)};
}

Obviously, this approach only implements a specific structure to enable external program to call member functions using the instance. Exactly, I do not implement the ABI for C++ class, so that this implement may be encounter some problems in some scenarios, for example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


// program.exe

class DemoEx : public Demo
{
public:
    virtual bool register_callback(void* event_ptr) = 0;
};

void data_event(void* data_buff);

HMODULE hDll = LoadLibraryA("lib.dll");
auto GetObj = GetProcAddress(hDll, "GetObj");
Demo* instance =  ( (Demo* (*)())(GetObj) )(); 
if ( instance->Init(0xDEADBEEF) == true )
{
    printf("[+] Instance initialize success");

    auto demo_ex = dynamic_cast<DemoEx*>(instance);
    if( demo_ex )
        demo_ex.register_callback((void*)data_event);

	instance->UnInit();
}

There is no denying that an error will be raised when program runs in dynamic_cast, primarily due to the absence of an ABI implementation for C++. The code in dynamic_cast registers SEH to catch any exceptions and subsequently raises a C++ exception. Unfortunately, I have not come acorss an excellent optimal approach to construct abi for C++. Consequently, I must handle C++ exception outside the scope of dynamic_cast.