View RSS Feed

Development Team Blog

How does External_Function really work?

Rate this Entry
Let's take a closer look at External_Function. But first, there are currently two generic and well established technologies you can use from VDF to interact with external components, DLL functions and COM objects. Actually, there are several other techniques available as well, the obsolete and defunct DDE technology, and the more modern SOAP technology for example, and various other IPC mechanisms. But we'll ignore those for now.

DLL and COM are the two fundamental technologies for in-process and cross-language use. These technologies are very important because they are very simple in essence, programming language agnostic, and so widely used that almost no Windows program can do without them. VDF is no exception, it provides support for both technologies.

DLLs are loadable modules with an exported programming interface. Most DLLs export functions, but it's technically possible to export other types of interfaces as well, C++ classes etc. Of course, you have to know how to use the exported interface, so the fewer expectations the DLL has of the client, the more universally accessible it will be from any program. Most DLLs export functions following a de-facto standard, which includes a calling convention, a minimal set of data types etc.

Even a simple construct such as an exported function is not really as simple as it seems. The address of the function is usually not enough to be able to use it, you must also know how to pass parameters and how to access the return value. This is generally referred to as the calling convention, which defines how parameters are pushed on the stack, which CPU registers are used, etc. In 32-bit Windows there's one de-facto standard calling convention for exported functions meant to be used generically, it's stdcall or __stdcall. Other calling conventions exists, cdecl, thiscall etc., but they're programming language and even compiler specific. The data types generally used with exported functions are the fundamental C types, int and so on.

The good news is that you don't really need to know exactly how parameters are pushed on the stack etc., the External_Function command takes care of that. But it's useful to know that the complexity is there behind the scenes, and that unless the function is using stdcall calling convention and only the basic C types, it might be more difficult to use.

Windows DLL functions can be exported by name and/or ordinal number, we'll ignore ordinal number for now. A name is just what you'd expect, the function name is listed in the export table. But there could be more to that, the function name could be decorated with additional cryptic type information, or a trailing @4 or @8 etc. to encode parameter information. If the function is meant to be used generically, the name is typically stripped from any such name decorations, which makes it easier to use.

But there is one naming convention commonly seen with a trailing A or W, which denotes an Ansi or Wide/Unicode string version of the function. This is just a convention though, not a requirement, so you will mostly see that only with standard Windows API functions, and only with functions that take String (LPTSTR) parameters. In which case you usually would pick the Ansi version.

The External_Function command is used to declare the function so it can be called from VDF code. The documentation explains how to use this command, so I'll skip that. But it's useful to remember that the real function name you have to specify might end with a trailing A.

The VDF data types you can use when specifying the parameter types are listed in the documentation for External_Function, but most likely you will use Integer, Handle, Pointer, DWORD and String. Note that Handle, Pointer and DWORD are all actually Integer. The alias types are just to make the code clearer. You typically use Pointer or Address for any pointer type, char*, int* alike. Note that many Windows API functions also use alias types of their own, such as LPSTR for char*, which can be a little challenging when trying to understand the parameter types at first.

The attentive reader will notice that I said you can use String for parameter type. It can often be used directly in place of LPCSTR, char*, i.e. any string-type parameter, or specifically any in-parameter that is of string type. Doing so can significantly simplify calling the function, as you can directly pass any VDF string value, without messing with AddressOf() and stuff.

The important thing to remember about using String parameter type is that the runtime will automatically create a temporary copy and pass the address of that to the actual function call. This means that if it's an in-out or out-parameter, then you cannot use String, you have to use Pointer or Address. String can only be used with in-parameters.

If it's a String out-parameter you also have to make sure the string/buffer is large enough to hold whatever value the DLL function will write to it. The good old ZeroString command is very useful for that purpose, it resizes the string accordingly and fills it with null characters. Then you simply use Pointer/Address for parameter type along with AddressOf() as mentioned before.

If it's a String in-out-parameter, or you have declared the parameter type as Pointer/Address for some other reason, you'd basically do the same as outlined in the previous paragraph, but you'd presumably pass in some string value as well. Typically you must make sure it's null terminated (assuming it's supposed to be a null terminated C-style string). The practice in VDF here is a little inconsistent. Sometimes you'll see code explicitly adding a null character to the end of the string. This is especially important if you're using Char[] or manually allocating the buffer, as opposed to using a String variable. Oddly enough, it turns out that if you're using a String variable, most of the time you don't really need to worry about adding a null terminator. VDF String variables always have a trailing null terminator anyway (it's just not part of the string value itself).

Now when you have some background information, we're ready to take the leap and see how you can create a DLL which we will then use from VDF. Next time I'll show how to do that and demonstrate the interoperability of VDF with other programming languages. Stay tuned.