Have Your Efficiency, and Flexibility Too

The Last Remaining Elephant In The Room: Runtime Conversion

Don't worry, I'm not really going to introduce an elephant into the story...

We've already seen various ways of configuring a compile-time option at runtime. But that was limited to creating a Gizmo. Once created, a Gizmo was stuck with those settings. So what if the number of ports and spinnablity of an existing Gizmo needs to change? If only a few Gizmos need to be able to change, we can just make those few out of the DynamicGizmo from earlier with only trivial modifications. But suppose all the Gizmos need to be changeable: Now how do we change the settings of an existing Gizmo without giving up the metaprogramming benefits?

In our original example, it would have been trivial to modify the Gizmo code so you could change a Gizmo's number of ports and spinnablity after it was created. They were already runtime values. But our metaprogramming examples have taken advantage of the fact that such values were fixed when a Gizmo is created. In fact, that's the fundamental key of the metaprogramming versions: "Turn your runtime options into compile-time options."

If one of these settings needs to change frequently during a program's run, then naturally it must be a runtime option. Turning it into a compile-time value won't improve its efficiency. That's because the "runtime to compile-time" trick works by taking advantage of the fact that certain values don't really need to change. So unfortunately, if a setting needs to change frequently, it must remain a runtime value. The only way to optimize it out is to eliminate the feature entirely.

However, if the settings only need to change occasionally, then we're still in business.

"What? Have you completely lost it? Configuring a compile-time option at runtime was crazy enough. And now you intend to change a compile-time option at runtime?! They're immutable!"

It not so strange, really. In fact, functional programming experts probably already know where I'm going with this...

Listen carefully: We converted the runtime values to compile-time ones by encoding them as part of the type, right? So to change the compile-time value, all we have to do is convert the variable to a different type. Just like converting an integer to a string. Easy. Except this is even easier because the types we're converting are fundamentally very similar.

Naturally, this does take extra processing. Likely much more than just simply changing an ordinary runtime variable. That's why I made a big deal about how frequently you need the value to change. If it changes a lot, you'll just slow things down from all the converting. But as long as it doesn't change enough to use up the efficiency savings from metaprogramming, you'll still end up ahead.

I'll demonstrate this by shuffling around a few Gizmos on every iteration. You'll notice that I'm always keeping the same number of each type of Gizmo, but that's not a real limitation: That's only so we can still compare benchmarks. It would be trivial to actually change a Gizmo without keeping everything balanced. All you'd have to do is remove the old Gizmo from the array for the old type, and append the newly converted one to the array for the new type. Of course, if you were to do this, you'd probably want to use a linked list or a stack instead of an array, since arrays have poor performance with such a usage pattern.

For simplicity, I won't use any of the "Flexibility Enhancements" covered in the previous section. But the techniques can certainly still be combined.

Conversion Downsides And Possible Fixes

There are a couple downsides to the runtime conversions above. But, they can be dealt with.

Hold It, Gizmo! Quit Squirming Around!

One issue with converting a Gizmo is that doing so puts it in a different memory location. This means that if anything was keeping a reference to the Gizmo, after conversion the reference is no longer pointing to the newly converted Gizmo. It's still referring to the old location.

This may not always be an issue. But if it does matter, you can solve it by using a tagged union. Something like this:

Hey Data, Get Back Here!

The other main issue with converting Gizmos is that some conversions can loose data. Suppose we convert a spinny Gizmo to a non-spinny and back again. The non-spinny Gizmos don't have a spinCount, so our newly re-spinified Gizmo has lost its original spinCount data. It's back at zero again. Same thing with the ports: Convert a 5-port to a 2-port and back again and you've lost the numZaps from the last three ports.

Sometimes that might matter, sometimes it might not. Or it might matter for only some pieces of data. In any case, there's an easy fix: just make sure every Gizmo type has the data you don't want to loose. Even if a certain type of Gizmo doesn't actually use that data, let the Gizmo hold on to that data anyway.

If you need to hold on to all the data, then as with the AnyGizmo above, each Gizmo type will take up as much space as the largest Gizmo. In fact, in such a case, using the AnyGizmo above may be a good idea. Except this time, the performance cost of doing conversions can be almost completely eliminated.

How? Suppose every type of Gizmo does need to hold all the possible Gizmo data, and you use AnyGizmo. In that case, if you arrange all the data members of all the Gizmo types so everything is always laid out in memory the same way, then to convert the AnyGizmo from one type to another, all you need to do is change the currentType member. That's it. Everything else will already be in the right place.

At this point, it might sound like we've merely re-invented the DynamicGizmo. While it is similar, there are two important differences:

First, this method allows us to have completely different functions for each Gizmo type. Each type can have a different set of functions, and each function can be specially-tailored to the features that Gizmo is supposed to support. So the Gizmos don't have to spend any extra processing time dealing with being flexible (for instance, by checking their own isSpinnable variables to see whether or not to actually spin). Essentially, we have a form of polymorphism. In fact, if a DynamicGizmo isn't included in the AnyGizmo, then there can still be a space savings over DynamicGizmo because variables like isSpinnable still won't need to exist at runtime at all.

Second, if we want to change multiple settings on a Gizmo, we only need to actually modify one value: the value of currentType.