Dan East

Minor update: I now use swig for my C++ / Lua binding, and I like it much better.

I’ve been using the Lua programming language for several years now as an embedded scripting language in a game engine.  In order to bind C++ classes into the Lua environment I use a 3rd-party library called tolua.

Recently I kept running into a bug with a new class interfaced via tolua that would manifest differently between Release and Debug builds.  The symptom was apparent corruption of other unrelated objects within Lua.  The specific situation was accessing a member of a class that was also a class.  Here’s an example to make things clearer:

class B {
int n;
};
class A {
B b;
};
//create an object a of type A
A a;


Then in lua when accessing the nested class the bug would be triggered:

a.b.n=5

tolua passes instances of C++ classes into lua using simple user data, which is basically just a pointer.  To reduce memory overhead there is no metadata passed into lua – just the pointer.  So whenever an C++ object is accessed in lua, tolua has to look the pointer up in a table or list in order to determine what the object’s type is.  This is a design flaw, because memory pointers are not necessarily unique to a specific class.  Specifically, the first member variable in a C++ class has the same pointer as its parent class.  In the above example, &a and &a.b will have the exact same memory address, even though they are two different objects of different types! So the limitation is that, for two classes interfaced via tolua, one cannot be the very first member variable of the other.

The workaround is straightforward – either rearrange the order of member variables in your class so the interfaced nested class is no longer first, or place a dummy variable in front of it.  If you do not have the flexibility to modify the class you are exporting then you may be in trouble – this would require fixing the design flaw in tolua itself.

I’m posting this in the hopes that it will keep someone else from wasting their time.