Latest articles

Bash: Multiline Eval

Maybe this is obvious to all the linux and bash-fu experts out there, but I just discovered it, thought it was pretty cool and worth sharing for anyone else who isn't quite a bash expert.

Background

Ok, so bash has a feature which takes the output of a command and runs it (no, this isn't the part that's new to me ;), I'll get to that part in a bit). Let's use the print working directory command as an example:

$ cd ~ $ pwd /home/my-user-name

Now let's get a command that just simply outputs the command we want:

$ echo pwd pwd

And then bash's basic feature to execute that result (two different ways to do it):

$ `echo pwd` /home/my-user-name $ eval $(echo pwd) /home/my-user-name

Alright, neat, but basic stuff.

The Problem

Unfortunately, that trick only works when you're evaluating a single line. Try to evaluate multiple lines and it won't work. (The newlines get converted to spaces and it all gets interpreted as one command.)

To demonstrate, let's make a basic script:

$ touch display-a-command $ my-favorite-editor display-a-command

Copy-paste, save and exit:

#!/bin/bash echo pwd echo pwd

Then:

$ chmod +x display-a-command $ ./display-a-command pwd pwd

Ok. So we have a basic command that outputs two lines of commands. Try to evaluate (You get the same result using either eval method):

$ `./display-a-command` /home/my-user-name

Fucknuggets. It only ran "pwd" once. The two lines got mashed together into one line and bash really executed the single command pwd pwd. The second "pwd" was ignored as an unused argument to the first "pwd". That's not at all what we wanted.

The Cool Part

Turns out there's an easy way to do a multi-line eval:

$ eval "`./display-a-command`" /home/my-user-name /home/my-user-name

Yup. That's it. The internal `./display-a-command` evaluates to our two-line command, and the double-quotes ensure the newline gets preserved when passed to eval.

It even works when display-a-command's output includes backticks and quotes:

$ cat display-a-command2 #!/bin/bash echo \`echo pwd\` echo eval \"pwd\" $ chmod +x display-a-command2 $ ./display-a-command2 `echo pwd` eval "pwd" $ eval "`./display-a-command2`" /home/my-user-name /home/my-user-name

Nice. Cool, huh? At least I think it is, anyway.

Read more


Tagged Scriptlike release: v0.7.0

Scriptlike is a utility library to aid in writing script-like programs in D (available through DUB):

Two things in this release:

  • Easy input-prompting module "scriptlike.interact", by Jesse Phillips. This has actually been in master for quite some time, but now it's part of a tagged release.
  • Fixed: The unittests failed to compile on DMD 2.067.0

Early heads-up warning: While scriptlike currently supports DMDs as early as 2.064.2, I will likely be increasing the minimum supported DMD up to 2.066.0 between now and the next release, due to increasing reliance on the newer parts of std.process (which prior to 2.066.0 had "paths with spaces" problems on Windows, which I was previously able to work around using the outdated system() function).

Read more


Windows Equivalent of sudo

Yea, yea, I know what day today is. And NO, this post isn't some idiotic prank (or any prank at all, for that matter). Irritating psuedo-holiday...

I'd been a Windows guy most of my life, but lately (for various reasons) I've gravitated more to Linux. (Not that I consider any OSes to be all that fantastic). One thing that's quite handy in Linux is the "Don't give me this 'access' shit, I bought and own your circuit board ass, so fucking do what I tell you to" command: sudo.

Last few versions of Windows have started getting uppity about access privileges, so I've occasionally run into a similar need. For example, logging in as an Administrator, firing up an Administrator-priveledge command line, and still getting "access denied" shit from one CLI tool or another regarding files and paths I know damn well I have full access to.

As can probably be expected, the Windows equivalent is all goofy, verbose and awkward (and launches the provided command in a separate auto-closing windows, gee thanks), but at least it's there:

runas /profile /user:YourUserName "C:\Full\Path\To\Program.exe Your Args Here"

Make sure to properly escape anything in those double-quotes (whee...I miss sudo already...). And...oh yea, your PATH won't work here, so you need to know the full path to your program. Luckily, there's an equivalent to "which". Side note: I don't know why that's calling it "where.bat", it behaves more like Linux "which" (shows the path to the one program that would actually get run) than the built-in Windows "where" (shows all programs on the PATH with the given name).

REM This is 'which.bat' @setlocal @set P2=.;%PATH% @for %%e in (%PATHEXT%) do @for %%i in (%~n1%%e) do @if NOT "%%~$P2:i"=="" echo %%~$P2:i

Toss that into a "which.bat", save to your windows directory (yea, not real clean, but hey, it's Windows, I don't give a crap), and get your work done:

> doshit now Fuck off, access denied > sudo doshit now I'm Windows, I don't understand that > which doshit C:\Some\Big\Path\doshit.bat > runas /profile /user:MyUserName "C:\Some\Big\Path\doshit.bat now" Oh, snap!

Read more


SDLang-D v0.9.0

SDLang-D update (v0.9.0) has now been tagged/released:

It's a D lib to parse and generate the SDL data language (similar to XML/JSON, but more human-friendly, while still being very simple). Additional examples are available at the links above, but the language looks like this:

// Sample, but useful, SDL: latest-version "v0.9.0" release-date 2015/03/16 how-cool "So cool!" links { original-sdl "http://sdl.ikayzo.org/display/SDL/Language+Guide" sdlang-d:home "https://github.com/Abscissa/SDLang-D" // Full changelog sdlang-d:changelog "https://github.com/Abscissa/SDLang-D/blob/master/CHANGELOG.md" // How-to-Use and API Reference sdlang-d:how-to-use "https://github.com/Abscissa/SDLang-D/blob/master/HOWTO.md" sdlang-d:api-reference "http://semitwist.com/sdlang-d-api/sdlang.html" }

Most of the big changes have been sitting in master for awhile, so this is long overdue.

Aside from various bugfixes, the big changes in this release are the addition of a StAX/Pull-style parser (see the changelog), and utilization of D's package.d feature.

  • Breaking change: Changed package structure to use package.d. Most users will be unaffected, but the internal package names have changed slightly, and users of DMD 2.063.2 and below will need to import sdlang.package; instead of import sdlang; until they upgrade their compiler. The built-in command line tool and unittests, however, do now require DMD 2.064 or newer because of this change.
  • New: Added StAX/Pull-style parser via pullParseFile and pullParseSource. (Warning: FileStartEvent and FileEndEvent might be removed later: #17)
  • Fixed: Work around a DMD 2.064/2.065 segfault bug in a unittest.
  • Fixed: #5 & #7: Building with Dub produces package format warnings (@ColdenCullen).
  • Fixed: #8: Consecutive escape sequences not getting correctly decoded.
  • Fixed: #11: Newline immediately after // is ignored.
  • Fixed: #12: Incorrectly accepts "anon tag without a value" when the tag has children.
  • Fixed: The build-docs script was broken for newer RDMDs.
  • Improved: Better error message for anonymous tags with no values.

As always, it's available in the DUB package repository.

The codename for this release is "version zero point nine point zero". Alternate codename is "4a44e9e88e573444cd6e7a43fb0e574ff5e03ac2".

Update 2015/03/27: Just tagged v0.9.1, a small but important update that fixes this issue:

  • Fixed: #16: Access Violation when using the pull parser

Read more


Nemerle's "when": Bad Idea, Easily Solved

It's no secret I'm a big fan of D, and very critical of most other languages. But there is another language I actually do like: Nemerle.

Some highlights of Nemerle

Like D, Nemerle is one of the few languages which truly "gets" metaprogramming and provides a decent implementation[1] (unlike, say, C# which only offers a half-baked generics system plus a dynamic type and simply leaves it at that). D and Nemerle's approaches to metaprogramming are entirely different and both have their pros and cons, but I like them both.

Another thing I love about Nemerle is its powerful, yet extremely accessible, algebraic types and pattern matching. This is actually one area where Nemerle is vastly ahead of D: While D does offer algebraics via its standard library, they're nowhere near as nice and clean as they are in Nemerle.

Nemerle (as does D) naturally has its downsides, though. For one, it lacks D's fantastic "ranges and algorithms" system. Also, like C#, Nemerle's reliance on CLR (ie, Mono/.NET) renders it medicore at best for low-level bare-metal work. And unlike Nemerle's designer's, I remain entirely unconvinced that merging the ideas of "statement" and "expression" leads to an cleaner, nicer language. I realize the common statement/expression separation isn't necessary and hasn't always existed, but I happen to like it, and I find it more useful than problematic. I don't like implicit returns, and as for the benefits of blurring the statement/expression line: Having lambdas, the ternary operator and an expression equivalent to switch blocks are all I've ever desired. The merging just seems unneccesary and overboard. But that said, I can certainly live with it just fine.

Overall, D is still my preferred language. But Nemerle is the only other language I can say I genuinely like.

Nemerle's one big blunder

For all its good, there is one big thing about Nemerle that's been bugging the hell out of me: The if statement requires an else clause. If you don't want else, you say "when", not "if".

That may seem minor, and the Nemerle designers do have a very deliberate reason for it: "to avoid stupid bugs with dangling-else". They provide the usual old dangling-else example that relies on combining bad indentation with omitting brackets on multi-line blocks.

I have a problem with that reasoning.

In language design, any feature or deviation from common expectation needs to pull its own weight. The benefits must outweigh the costs. Let's see:

Benefit: "Eliminate stupid dangling-else bugs": Mmmm hmmm. Ok. So....Who the fuck actually hits this issue?!? I've been coding for twenty-five years. As such, I firmly believe that stupid mistakes happen, to even the best of programmers, and that the true mark of an amateur is believing you can (or have) overcome that. I openly admit I've made, and will continue to make on a regular basis, every stupid freaking mistake in the book...except this one. I have never made this one. I have never even seen this one get made by anyone, anywhere. I admit, I really do have no doubt it does get made. Sure. Occasionally. On rare occasions. But heck, who the fuck ever omits the curly braces around a multi-line "if" body anyway (a requirement for this bug to occur in a whitespace-insensitive language)? If it does happen, who the fuck lets it pass code review, whether dangling-else bug or not?

Is the benefit of eliminating this rare bug worthwhile? Naturally, that depends on the cost:

Cost: Take one of the most basic, common, fundamental constructs in all of programming, modify it, rename it, and force every addition/removal of every "else" clause anywhere to always involve swapping between "if" and "when" on a different line of code.

So, a major disruption to one of the most fundamental constructs in order to eliminate a bug that's quite frankly more theoretical than real? No. No Nemerle, that is not a good tradeoff.

Solution

Luckily, Nemerle's biggest problem also demonstrates one if it's greatest strengths: This design flaw is easily changed, in mere user code alone, without hacking or modifying the language or compiler at all:

macro plainIf(cond, body) syntax("if", "(", cond, ")", body) { <[ when($cond) { $body } ]> }

That's it. That solves it. Those few lines bring back a normal "if". Just toss that into a "plainIf.n", compile to a DLL:

ncc -r Nemerle.Compiler.dll -t:dll plainIf.n -o plainIf.dll

Then toss that DLL into your project:

ncc -r plainIf.dll [...the rest of your args here...]

And magically, you can use "if"s with or without an else. Nice.

For convenience, I've put this code (along with Posix/Windows buildscripts and a small test program) up on GitHub, in a simple project named plainIf.


[1] Ok, I realize there is *ahem* a certain enthusiastic subset of programmers who will look at Nemerle's macros and bring up the ol' "Everything is LISP" idea. And y'know what? They may well be mostly right. But, if I want to drown in a torrent of barely-readable symbols, I'll just use brainfuck and get to the "shoot myself" stage that much quicker.

Read more


Unity and Unreal Compete: Indies Win!

Big news from game engine developers at GDC this past week:

First, Epic dropped the up-front cost of Unreal Engine 4 to "free for everyone", with only a 5% royalty beyond each project's first $3000 revenue (per quarter).

Then, the following day, Unity released their long-teased Unity3D 5, along with an indie bonus of their own: Nearly all previously Pro-only features are now available for free to non-Pro users (excluding the snazzy Pro-only editor skin, darn!).

Also exciting is Valve's unveiling of Source 2, also free for developers. Although, I admit I know very little about Source, so I can't really comment any further on it.

I've had an eye on both Unity3D and Unreal Engine for awhile now, I even dabble in Unity3D a bit, and I find it interesting the directions both have taken. The two engines come from opposite backgrounds: Unity3D from the indie scene, and UE from the AAA industry (Although initially, Epic itself originated from the old DOS shareware scene - arguably the original indie game scene). But lately, they've been encroaching on each other's territory. Unity has been touting one AAA-targeted feature after another. Meanwhile, Epic has reversed the "expensive, exclusive and difficult" image from their UE3 days, and drastically reduced both technical and financial barriers of entry - a move aimed, in large part, straight at indies.

The wall between indie and AAA development tools has collapsed, attacked from both sides, with Unity and Epic being two of the biggest demolitionists (among others as well). And indies are quite possibly the biggest beneficiaries.

Here's what the two engines now offer to indies:

  • Anyone can get started. (Both Unity and Unreal)
  • Easy to use game editing environment, with rapid prototyping and development turnarounds, and loads of useful tools. (Both Unity and Unreal)
  • Engine with AAA-quality and speed. (Both Unity and Unreal)
  • Target all the most popular PC, console and mobile platforms... (Both Unity and Unreal)
  • ...and then some (Unity)
  • Develop and release without investing one cent on tools or engine. (Both Unity and Unreal)
  • Official online store for buying/selling assets. (Both Unity and Unreal)
  • Ease and safety of [an older version of] C# and other CLR languages (Unity)
  • Full power of native C-linkage languages (implying a potential for possibly using D!) (Unreal)
  • Full, unrestricted access to the entire build system and engine source. (Unreal)
  • Partial ability to develop on Linux (Unity via Wine, albeit unsupported and occasionally problematic[1]. Unreal via an in-progress community-driven Epic-sanctioned effort[2].)

Even the few differences above aren't so different after all: Unity is perfectly capable of interop with native C-linkage languages (it just requires taking extra trips manually across the managed/unmanaged barrier), and there's no reason a managed scripting system can't be used on top of Unreal Engine (you're just "on your own" with that, AFAIK).

Of course, Unity and Unreal aren't even the only options in town. There's still Source 2 as mentioned earlier, plus CryEngine, MonoGame (a resurrection of XNA), and even others.

It's a good time to be an indie: Engines compete, we win.


[1] I'll have to save my adventures using Unity3D's editor on Linux for a separate post. In brief: It gets the job done, although not spectacularly well. Certainly not well enough to obviate the need for a native Linux editor.

[2] At the moment, I have no idea what the exact state of Unreal Engine's editor is on Linux. I know it exists, my understanding is that it's usable, but I know nothing about how usable. Could be flawless, could be comparable with Unity under Wine, could be less, I don't know. But I am definitely interested.

Read more