Latest articles

Various Software Updates and New Tools

I've been fairly busy over the last couple months or so updating several projects. Enough has accumulated that instead of making individual posts for each update to each tool and library, I'm combining them all into this one post.

Most of this has been part of a behind-the-scenes push to get the next incarnation of DAuth ready for release. It will still be the same library, but will be rebranded as "InstaUser" (partly to avoid confusion with OAuth, which DAuth isn't directly related to) and will have an expanded scope.

The current DAuth will become "InstaUser Basic", the core of the InstaUser project. But InstaUser will also offer brand-new purely optional components: "InstaUser Store" to instantly get up-and-running with database-backed user account storage, and "InstaUser Web" which builds on InstaUser Store to provide ready-to-use web-based login and registrations. Eventually, there may even be an "InstaUser OAuth", but that's only in the "wishlist" stage right now and definitely won't be in the first release of InstaUser.

Anyway, that's all still in-progress, but working on it has revealed various direct and indirect needs in other projects. Here's what's new since my last "Announcements" post:


DAuth v0.6.2

[Project Homepage]
[Full Changelog]

This will likely be the last DAuth release before being rebranded InstaUser.

DAuth is an open-source salted password hash authentication library for D. It provides a simple, yet flexible API. With it, your software can easily incorporate user accounts with reliable, upgradeable security.

Changes since v0.6.1:

v0.6.2 - 2015/03/25

  • Fixed: #1/#2: Compilation failure when using DUB and DMD 2.067 (@NCrashed)


gen-package-version v1.0.2

[Project Homepage]
[Full Changelog]

Automatically generate a D module with version and timestamp information (detected from git or Mercurial/hg) every time your program or library is built. You can also generate a DDOC macro file (using the --ddoc=dir switch.)

Even better, all your in-between builds will automatically have their own VCS-generated version number, including the VCS commit hash (for example: v1.2.0-1-g78f5cf9). So there's never any confusion as to which "version" of v1.2.0 you're running!

Just add this to your project's dub.json:

"dependencies": { "gen-package-version": "~>1.0.2" }, "preGenerateCommands": ["dub run gen-package-version -- your.package.name --root=$PACKAGE_DIR --src=path/to/src"]

Replace your.package.name and path/to/src with the appropriate values for your project (but keep $PACKAGE_DIR exactly as written above), and you're ready to go:

module your.package.name.main; import std.stdio; import your.package.name.packageVersion; void main() { writeln("My Cool Program ", packageVersion); writeln("Built on ", packageTimestamp); }

Changes since v0.9.0:

v0.9.1 - 2015/06/14

  • Fixed: helper/gen_version.sh isn't set as executable when checked out through dub.

v0.9.2 - 2015/06/14

  • Fixed: The old recommended "preGenerateCommands" led to problems (project dependencies that use gen-package-version would run it from the wrong directory).

v0.9.3 - 2015/06/15

  • Enhancement: If detecting the version number via git fails, attempt to detect it via the current directory name (ex, ~/.dub/packages/[project-name]-[version-tag]).
  • Enhancement: Don't bother running git if there's no .git directory.
  • Enhancement: Bootstraps itself, so gen-package-version itself enjoys the following fix:
  • Fixed: Fails to detect version number for packages fetched by dub (since they lack .git).

v0.9.4 - 2015/06/16

  • Enhancement: Support detecting the version number via Mercurial (hg).
  • Enhancement: Support .hgignore for Mercurial working directories.

v1.0.0 - 2015/06/27

  • Change: The generated packageTimestamp is changed from ISOExt format to human readable. The ISOExt formatted version is now called packageTimestampISO.
  • Change: Value for --module is no longer allowed to contain periods.
  • Enhancement: Basic ability to be used as a library. See the README for details.
  • Enhancement: Add -r|--root to support projects in any directory, not just the current directory.
  • Enhancement: Minor improvements to --verbose and --trace outputs.
  • Fixed: Don't update the version file (and thus trigger a project rebuild) if the version file doesn't need updated. Bypass this check with the new --force flag.
  • Fixed: Don't rebuild gen-package-version if not needed.
  • Fixed: Failure on Windows when target project is on a different drive letter from current working directory.

v1.0.1 - 2015/06/28

  • Fixed: Don't use a broken scriptlike release (v0.9.0), use v0.9.1 instead.

v1.0.2 - 2015/07/01

  • Enhancement: Now works on DMD 2.066.1 (previously required 2.067.0 or up).


New tool: safeArg v0.9.7

[Project Homepage]
[Full Changelog]

Think of this like xargs -0, but simplified and ready-to-use in any dub-based project, even on Windows.

See the homepage for more info and examples, but essentially, the purpose is this:

Using eval or command substitution to pass arguments to a program is error-prone, non-portable and a potential security risk. This command-line tool utilizes the commonly-recommended approach of null-delimited stream to provide safety and reliability when passing arbitrary data to a program as command-line arguments.


Scriptlike v0.9.2

[Project Homepage]
[Full Changelog]

Utility library to aid in writing script-like programs in D.

Changes since v0.8.0:

v0.9.0 - 2015/06/27

  • Change: Split scriptlike.file and scriptlike.path into the following:
    • scriptlike.core
    • scriptlike.file.extras
    • scriptlike.file.wrappers
    • scriptlike.path.extras
    • scriptlike.path.wrappers
    Utilizes package.d to retain ability to import scriptlike.file and scriptlike.path.
  • Change: Convert changelog from markdown to ddox so links are more readable.
  • Enhancement: Add (opt-in) command echoing to most functions in scriptlike.file.
  • Enhancement: Add yap and yapFunc as improved versions of to-be-deprecated echoCommand.
  • Fixed: Make escapeShellArg const-correct.
  • Fixed: Make Path.toRawString and Ext.toRawString both be pure @safe nothrow.

v0.9.1 - 2015/06/28

  • Fixed: Fails to compile unless the makedocs script has been run.

v0.9.2 - 2015/07/10

  • Fixed: Properly flush all command echoing output (ie, in yap and yapFunc).
  • Enhancement: Add a "no-build" configuration for projects that need to import/depend on Scriptlike through DUB, but use their own buildsystem.


SDLang-D v0.9.2

[Project Homepage]
[Full Changelog]

A library to parse/generate SDL (Simple Data Language) files. Offers both DOM and StAX/Pull APIs.

SDL is like XML/JSON/YAML, but is low-verbosity, simpler than YAML, and supports comments and basic datatypes. It looks like this:

// An example of SDL: folder "myFiles" color="yellow" protection=on { folder "my images" { file "myHouse.jpg" color=true date=2005/11/05 file "myCar.jpg" color=false date=2002/01/05 } // Another folder folder "my documents" { document "resume.pdf" } }

Changes since v0.9.0:

v0.9.1 - 2015/03/17

  • Fixed: #16: Access Violation when using the pull parser.

v0.9.2 - 2015-07-31

  • New: Uses travis-ci.org for continuous integration testing.
  • Change: Updated package.json to newer dub.json name.
  • Fixed: #16: Now fixed for DUB users, too: Access Violation when using the pull parser.
  • Fixed: #21: Remove unneeded "buildOptions" from DUB package config (fixes a DUB warning) (@schuetzm)
  • Fixed: #28/#29: Wrong line count for Windows style line breaks. (@s-ludwig)
  • Fixed: Fixed running unittests via DUB. (Part of #29) (@s-ludwig)
  • Fixed: Trailing line comments incorrectly treated as line continuation instead of newline (Related: #20, plus libsdl-d's e565f30 and c6dc722) (@Dicebot)
  • Improved: #22/#23: Internal improvements (@schuetzm)

Read more


Those who rely on Moore's Law...

Those who rely on Moore's Law inevitably ignore and fall prey to Wirth's Law.

(See also Jevons paradox.)

Read more


gen-package-version v0.9.0: New tool: Auto-generate version/timestamp info

Automatically generate a D module with version and timestamp information (detected from git) every time your program or library is built.

Even better, all your in-between builds will automatically have their own git-generated version number, including the git commit hash (for example: v1.2.0-1-g78f5cf9). So there's never any confusion as to which "version" of v1.2.0 you're running!

The gen-package-version Homepage

Full usage and instructions are at the link above, but here's a brief primer for those using it with dub:

To use:

Add this to your project's dub.json file:

"dependencies": { "gen-package-version": "~>0.9.0" }, "preGenerateCommands": ["dub run gen-package-version -- package.name --src=path/to/src"]

Now, when you build your project with dub, it'll automatically download, install and run gen-package-version, which will generate the file path/to/src/package/name/packageVersion.d

Then just import that and use it:

module package.name.main; import std.stdio; import package.name.packageVersion; void main() { writeln("My Cool Program ", packageVersion); writeln("Built on ", packageTimestamp); }

Read more


Scriptlike v0.8.0

A bunch of new updates to Scriptlike: A library to aid in writing script-like programs in D.

Scriptlike links: [Home] [Dub] [Full changelog]

Highlights (more info at full changelog):

  • Change: Minimum officially supported DMD increased from v2.064.2 to v2.066.0.
  • Change: API reference now built using ddox and uses much improved styling (actually uses a stylesheet now).
  • Change: Misc changes to reduce symbol conflict issues with std.file/std.path.
  • Change: Eliminate remnants of the "planned but never enabled" wstring/dstring versions of Path/Ext/Args. There turned out not to be much need for them.
  • Enhancement: Add module scriptlike.only to import all of scriptlike, but omit the helper Phobos imports in scriptlike.std.
  • Enhancement: fail now accepts an arbitrary list of args
  • Enhancement: Added failEnforce.
  • Enhancement: Added runCollect and tryRunCollect, to capture a command's output instead of displaying it.
  • Enhancement: Added pause to pause and prompt the user to press Enter.
  • Enhancement: Added Path-based wrappers for std.file's getcwd, thisExePath and tempDir.
  • Fixed: No longer uses the deprecated std.process.system().

Read more


Ahh, Irony

You'd think a sudden stop to the flood of spam in the comment moderation queue would be a nice relief. Instead, I find it makes me paranoid that "The comment system didn't just suddenly break, did it?!?".

Go figure.

Oh well, I'm sure the spam traffic will pick back up again!

Read more


Bash: Multiline Eval

Maybe this is obvious to all the linux and bash-fu experts out there, but I just discovered it, thought it was pretty cool and worth sharing for anyone else who isn't quite a bash expert.

Background

Ok, so bash has a feature which takes the output of a command and runs it (no, this isn't the part that's new to me ;), I'll get to that part in a bit). Let's use the print working directory command as an example:

$ cd ~ $ pwd /home/my-user-name

Now let's get a command that just simply outputs the command we want:

$ echo pwd pwd

And then bash's basic feature to execute that result (two different ways to do it):

$ `echo pwd` /home/my-user-name $ eval $(echo pwd) /home/my-user-name

Alright, neat, but basic stuff.

The Problem

Unfortunately, that trick only works when you're evaluating a single line. Try to evaluate multiple lines and it won't work. (The newlines get converted to spaces and it all gets interpreted as one command.)

To demonstrate, let's make a basic script:

$ touch display-a-command $ my-favorite-editor display-a-command

Copy-paste, save and exit:

#!/bin/bash echo pwd echo pwd

Then:

$ chmod +x display-a-command $ ./display-a-command pwd pwd

Ok. So we have a basic command that outputs two lines of commands. Try to evaluate (You get the same result using either eval method):

$ `./display-a-command` /home/my-user-name

Fucknuggets. It only ran "pwd" once. The two lines got mashed together into one line and bash really executed the single command pwd pwd. The second "pwd" was ignored as an unused argument to the first "pwd". That's not at all what we wanted.

The Cool Part

Turns out there's an easy way to do a multi-line eval:

$ eval "`./display-a-command`" /home/my-user-name /home/my-user-name

Yup. That's it. The internal `./display-a-command` evaluates to our two-line command, and the double-quotes ensure the newline gets preserved when passed to eval.

It even works when display-a-command's output includes backticks and quotes:

$ cat display-a-command2 #!/bin/bash echo \`echo pwd\` echo eval \"pwd\" $ chmod +x display-a-command2 $ ./display-a-command2 `echo pwd` eval "pwd" $ eval "`./display-a-command2`" /home/my-user-name /home/my-user-name

Nice. Cool, huh? At least I think it is, anyway.

Read more