Have Your Efficiency, and Flexibility Too

Metaprogramming Techniques For No-Compromise Code

by Nick Sabalausky

Full source code for this article is available on GitHub, or can be downloaded here.

View this article on a Single Page or Multiple Pages

Table of Contents:

  1. Have Your Efficiency, and Flexibility Too
  2. First Attempt: Send Efficiency and Flexibility to Dr. Oop's Couples Therapy
  3. Respecting the Classics: Old-School Handcrafting
  4. Success at Dr. Metaprogramming's Clinic
  5. It Walks Like a Duck and Quacks Like a Duck...Kill It!
  6. Metaprogramming Plus: The Flexibility Enhancements
  7. The Last Remaining Elephant In The Room: Runtime Conversion
  8. Curtain Call

It Walks Like a Duck and Quacks Like a Duck...Kill It!

Duck typing is a topic that divides programmers almost as much as tabs-vs-spaces or vi-vs-emacs. While I admit I'm personally on the anti-duck side of the pond, I'm not going to preach it here. I only bring it up because there are other anti-duckers out there, and for them, there's something about the metaprogramming example they may not be happy with. But there is a solution. If you are a duck fan, please pardon this section's title and feel free to skip ahead. I promise not to say anything about you behind your back...

Remember in the last section I pointed out the useGizmo() function was templated so it could accept all the various Gizmo types? Well, what happens when we pass it something that isn't a Gizmo? For most types, the compiler will just complain that doStuff(), spin, isSpinnable, and numPorts don't exist for the type. But what if it's a type that just happens to look like a Gizmo?

From snippet_notAGizmo.d:

struct BoatDock_NotAGizmo { // Places to park your boat int numPorts; void doStuff() { manageBoatTraffic(); } // Due to past troubles with local salt-stealing porcupines // swimming around and clogging up the hydraulics, // some boat docks feature a special safety mechanism: // "Salty-Porcupines in the Intake are Nullified", // affectionately called "spin" by the locals. bool isSpinnable; void spin() { blastTheCrittersAway(); } }

The templated useGizmo() function will happily accept that. Hmm, accepting a type based on its members rather than its declared name, what does that remind you of...? Yup, duck typing.

Granted, it's not exactly the same as the usual duck typing popularized by dynamic languages. It's more like a compile-time variation of it. But it still has the same basic effect: If something looks like a Gizmo, it will be treated as a Gizmo whether it was intended to be or not. Whether or not that's acceptable is a matter for The Great Duck Debate, but for those who dislike duck typing, it's possible to kill the duck with metaprogramming and constraints. This is almost identical to ex4_metaprogramming.d, but with a few changes and additions that I've highlighted:

From ex5_meta_deadDuck1.d:

template isIGizmo(T)
{
immutable bool isIGizmo = __traits(compiles,
// This is just an anonymous function.
// We won't actually run it, though.
// We're just making sure all of this compiles for T.
(){
T t;
static assert(T._this_implements_interface_IGizmo_);
int n = t.numPorts;
static if(T.isSpinnable)
int s = t.spinCount;
t.doStuff();
t.spin();
}
);
}
// Almost identical to the original metaprogramming Gizmo
// in 'ex4_metaprogramming.d', but with two little things added:
struct Gizmo(int _numPorts, bool _isSpinnable) { // So other generic code can determine the // number of ports and spinnability: static immutable numPorts = _numPorts; static immutable isSpinnable = _isSpinnable;
// Announce that this is a Gizmo.
// An enum takes up no space.
static enum _this_implements_interface_IGizmo_ = true;
// Verify this actually does implement the interface
static assert(
isIGizmo!(Gizmo!(numPorts, isSpinnable)),
"This type fails to implement IGizmo"
);
static if(numPorts < 1) static assert(false, "A portless Gizmo is useless!"); private OutputPort[numPorts] ports; void doStuff() { static if(numPorts == 1) ports[0].zap(); else static if(numPorts == 2) { ports[0].zap(); ports[1].zap(); } else { foreach(port; ports) port.zap(); } } static if(isSpinnable) int spinCount; void spin() { static if(isSpinnable) spinCount++; // Spinning! Wheeee! } } struct OutputPort { int numZaps; void zap() { numZaps++; } } struct UltraGiz { template gizmos(int numPorts, bool isSpinnable) { Gizmo!(numPorts, isSpinnable)[] gizmos; } int numTimesUsedSpinny; int numTimesUsedTwoPort;
void useGizmo(T)(ref T gizmo) if(isIGizmo!T)
{ gizmo.doStuff(); gizmo.spin(); if(gizmo.isSpinnable) numTimesUsedSpinny++; if(gizmo.numPorts == 2) numTimesUsedTwoPort++; } void run() { StopWatch stopWatch; stopWatch.start(); // Create gizmos gizmos!(1, false).length = 10_000; gizmos!(1, true ).length = 10_000; gizmos!(2, false).length = 10_000; gizmos!(2, true ).length = 10_000; gizmos!(5, false).length = 5_000; gizmos!(5, true ).length = 5_000; // Use gizmos foreach(i; 0..10_000) { foreach(ref gizmo; gizmos!(1, false)) useGizmo(gizmo); foreach(ref gizmo; gizmos!(1, true )) useGizmo(gizmo); foreach(ref gizmo; gizmos!(2, false)) useGizmo(gizmo); foreach(ref gizmo; gizmos!(2, true )) useGizmo(gizmo); foreach(ref gizmo; gizmos!(5, false)) useGizmo(gizmo); foreach(ref gizmo; gizmos!(5, true )) useGizmo(gizmo); } writeln(stopWatch.peek.msecs, "ms"); assert(numTimesUsedSpinny == 25_000 * 10_000); assert(numTimesUsedTwoPort == 20_000 * 10_000); } }

Those experienced with the D programming language may notice this is very similar to the way D's ranges are created and used, but with the added twist that a type must actually declare itself to be compatible with a certain interface.

Now, if you try to pass a boat dock to useGizmo(), it won't work because the boat dock hasn't been declared to implement the IGizmo interface. Instead, you'll just get a compiler error saying there's no useGizmo() overload that can accept a boat dock. As an extra bonus, if you change Gizmo and accidentally break it's IGizmo interface (for instance, by deleting the doStuff() function), you'll get a better error message than before. Best of all, these changes have no impact on speed or memory since it all happens at compile-time.

Under the latest version of DMD at the time of this writing (DMD v2.053), if you break Gizmo's IGizmo interface, this is the error message you'll get:

ex5_meta_deadDuck1.d(44): Error: static assert "This type fails to implement IGizmo" ex5_meta_deadDuck1.d(92): instantiated from here: Gizmo!(numPorts,isSpinnable) ex5_meta_deadDuck1.d(116): instantiated from here: gizmos!(1,false)

So it plainly tells you what type failed to implement what interface. In a language like D that supports compile-time reflection, it's also possible to design the IGizmo interface so the error message will state which part of the interface wasn't implemented. But the specifics of that are beyond the scope of this article (That's author-speak for "I ain't gonna write it.")

This is great, but announcing and verifying these dead-duck interfaces can be better generalized. Changes from ex5_meta_deadDuck1.d are highlighted:

From ex5_meta_deadDuck2.d:

string declareInterface(string interfaceName, string thisType)
{
return `
// Announce what interface this implements.
// An enum takes up no space.
static enum _this_implements_interface_`~interfaceName~`_ = true;
// Verify this actually does implement the interface
static assert(
is`~interfaceName~`!(`~thisType~`),
"This type fails to implement `~interfaceName~`"
);
`;
}
// Almost identical to the original metaprogramming Gizmo
// in 'ex4_metaprogramming.d', but with *one* little thing added:
struct Gizmo(int _numPorts, bool _isSpinnable) { // So other generic code can determine the // number of ports and spinnability: static immutable numPorts = _numPorts; static immutable isSpinnable = _isSpinnable;
// Announce and Verify that this is a Gizmo.
mixin(declareInterface("IGizmo", "Gizmo!(numPorts, isSpinnable)"));
static if(numPorts < 1) static assert(false, "A portless Gizmo is useless!"); private OutputPort[numPorts] ports; void doStuff() { static if(numPorts == 1) ports[0].zap(); else static if(numPorts == 2) { ports[0].zap(); ports[1].zap(); } else { foreach(port; ports) port.zap(); } } static if(isSpinnable) int spinCount; void spin() { static if(isSpinnable) spinCount++; // Spinning! Wheeee! } }

If you ever want to create another type that also counts as an IGizmo, all you have to do is add the declaration line:

From snippet_anotherGizmo.d:

struct AnotherGizmo // Even a class would work, too! {
mixin(declareInterface("IGizmo", "AnotherGizmo"));
// Implement all the required members of IGizmo here... }

Now isIGizmo will accept any AnotherGizmo, too. And just like a real class-based interface, if you forget to implement part of IGizmo, the compiler will tell you.

There are many further improvements that can be made to declareInterface(). For instance, although it's currently using a string mixin, it could be improved by taking advantage of D's template mixin feature. It could also be made to detect the name of your type so you only have to specify "IGizmo", and not "AnotherGizmo". But this at least demonstrates the basic principle.

For the sake of simplicity, the rest of the examples in this article will forgo the anti-duck typing techniques covered in this section.

Of course, none of this is needed if you're only using classes, which can truly inherit from one another. In that case, you can just use real inheritance-based interfaces. But if you want to avoid the overhead of classes, you can use these metaprogramming tricks to achieve much of the same flexibility.

Next: Metaprogramming Plus: The Flexibility Enhancements

Table of Contents:

  1. Have Your Efficiency, and Flexibility Too
  2. First Attempt: Send Efficiency and Flexibility to Dr. Oop's Couples Therapy
  3. Respecting the Classics: Old-School Handcrafting
  4. Success at Dr. Metaprogramming's Clinic
  5. It Walks Like a Duck and Quacks Like a Duck...Kill It!
  6. Metaprogramming Plus: The Flexibility Enhancements
  7. The Last Remaining Elephant In The Room: Runtime Conversion
  8. Curtain Call