Monday, July 5, 2010

So what happened to the variadic marshaller?

As you may recall, I had a variadic marshaller I had been using for the RPC layer for some time now, which recently had to be dropped since it was causing trouble with passing float parameters. I wanted to talk about here a little bit since it's not really a specific problem concerning DSP or marshallers, but rather how certain arguments are passed to variadic functions.

Let's start by talking about what a variadic function is, since you may or may not have heard the term. A variadic function is a function that can take a varying number of arguments. There usually are a few "required" arguments but the sky (or rather, the bottom of the stack :)) is the limit. Sounds familiar? Yes indeed, probably the two most famous functions in the C library are variadic: printf and scanf.

Variadics in C are easily identifiable by their declarations, which looks something like this:

int summation_function(int count, ...)

notice the ellipsis ( ... ) - this is the notation used to state that there will be an arbitrary number of arguments here.

And for quick reference - accessing the variadic arguments is done via stdarg.h macros, for example:

va_list arg_list;
va_start(arg_list, count);
for(int i=0; i < count; i++)
  sum += va_arg(int);

whose detailed usage descriptions you can find easily by Googling.

Moving back to the problem I had with the variadic marshaller: I was passing regular 4-byte floats as arguments to be marshalled, but somehow the 4-byte region corresponding in the marshalled buffer always showed up to be corrupted somehow.

The issue was eventually revealed to be about the default argument promotions that are applied to variadics, as described in:

therefore, any short int or char arguments passed are automatically promoted to int's, and all float's are casted into double's - thus having a 8 byte representation whose first 4 bytes don't really mean all that much :)

My initial idea was to simply cast the obtained double parameter back into a float before marshalling it into the buffer, but unfortunately this leads to loss of precision during double->float conversion. Therefore, I've decided to switch to using macros and doing the parameter marshalling directly inside the stubs. Comparing what the DSP-side stubs look like in the old vs. new methods of marshalling:

void rpc_mixedprint(int a, char b, float c, double d, short e, int f)
    rpc_marshal("rpc_mixedprint", "vicfdsi",a,b,c,d,e,f);


void rpc_mixedprint(int a, char b, float c, double d, short e, int f)
    RPC_INIT("rpc_mixedprint", "vicfdsi");
    RPC_PACK(int, &a);
    RPC_PACK(char, &b);
    RPC_PACK(float, &c);
    RPC_PACK(double, &d);
    RPC_PACK(short, &e);
    RPC_PACK(int, &f);

I'm aware it doesn't look quite as elegant as the variadic marshaller did... but in terms of ease of stub generation, it's not all that different, and there's no loss of data precision involved. And the GPP-side stub uses macros to extract the parameters as well, once the unmarshaller unpacks the buffer into the void** param_buffer:

void rpc_mixedprint(void **param_buffer, void *result_buffer)
    mixedprint(RPC_CAST_PARAM(param_buffer[0], int),
               RPC_CAST_PARAM(param_buffer[1], char),
               RPC_CAST_PARAM(param_buffer[2], float),
               RPC_CAST_PARAM(param_buffer[3], double),
               RPC_CAST_PARAM(param_buffer[4], short),
               RPC_CAST_PARAM(param_buffer[5], int)


  1. I'm considering handling variadic arguments in a similar way, and am curious what your view now is after having several years to use this.

  2. While I haven't really been using this for the past few years, I still think the macro-method I describe in this post is the safer way to go. It probably all depends on the use case though. libffcall ( is also definitely work checking out.