Announcements

DAuth v0.6.1 - Salted Hashed Password Library

An update to DAuth is out.

Main Changes:

Coming on the horizon: DAuth will be rebranded as "InstaUser Basic". It will be one pillar of a broader multiple-lib project called InstaUser, which will also include optional higher-level tools "InstaUser Store" and "InstaUser Web".

Read more


DAuth v0.6 - SHA-2 and Hash_DRBG

DAuth is a low-level authentication library for D with no external dependencies other than D's standard library, Phobos. It provides a simple-yet-flexible API, so your software can easily incorporate secure, upgradable user authentication based on salted password hashes.

By default, DAuth uses known-good hashing and randomization algorithms (currently SHA-512 and Hash_DRBG), but it accepts any Phobos-compatible hash digest or random number generator. You can have as much or as little control as you need, making DAuth suitable for both new projects and interfacing with any existing hashed-password store.

Links:

New in v0.6:

  • Added SHA-2 (hash digest) and Hash_DRBG (cryptographically secure psuedo-random number generator), and set them as the new hash/RNG defaults. DAuth is only a temporary home for these - the SHA-2 implementation has already been merged into phobos master, and I intend to prepare a pull request for the Hash_DRBG. But they're ready-to-use in DAuth in the meantime.
  • Split DAuth into separate modules (using a "package.d" to preserve "import dauth;")
  • Changed callbacks from functions to delegates.
  • Fixed a few bugs, including an error in the dub package file.

Read more


DAuth - Authentication Utility Lib for D - Initial Release v0.5.1

I've put up an initial release of DAuth: A simple-yet-flexible salted password hash based authentication utility lib for D.

Before you get too excited, know that actual cryptographic algorithms are outside the scope of this lib. Instead, it uses any Phobos-compatible digests and random number generators.

The upside: This makes DAuth fully extensible with plug-in cryptographic algorithms. The downside: It's currently limited to what little exists in Phobos right now (or in any Phobos-compatible third-party digests/RNGs I may not know about).

Full overview, sample code, source and (ugly) API reference are at DAuth's GitHub homepage.

(DUB project name "dauth")

Announcement thread at the D newsgroups is here.

Read more


Scriptlike v0.6.0

An update to Scriptlike is out. Scriptlike is a utility library to aid writing script-like programs in D.

https://github.com/Abscissa/scriptlike

Changes from v0.5.0 to v0.6.0:

  • Change: Path and Ext are now aliases for the UTF-8 instantiations, and the template structs are now named PathT and ExtT.
  • Change: Removed path() and ext() helper functions to free up useful names from the namespace, since they are no longer needed. Use Path() and Ext() instead.
  • Change: Internally split into separate modules, but uses package.d to preserve import scriptlike;.
  • Change: Rename escapeShellPath -> escapeShellArg.
  • Change: Rename runShell -> tryRun. Temporarily keep runShell as an alias.
  • Change: Rename scriptlikeTraceCommands -> scriptlikeEcho. Temporarily keep scriptlikeTraceCommands as an alias.
  • Enhancement: Added scripts to run unittests and build API docs.
  • Enhancement: Added opCast!bool for Path and Ext.
  • Enhancement: fail() no longer requires any boilerplate in main(). NG link
  • Enhancement: Added run to run a shell command like tryRun, but automatically throw if the process returns a non-zero error level.
  • Enhancement: #2: Optional callback sink for command echoing: scriptlikeCustomEcho.
  • Enhancement: #8: Dry run support via bool scriptlikeDryRun.
  • Enhancement: #13: Added ArgsT (and Args helper alias) to safely build command strings from parts.
  • Enhancement: Added this changelog.
  • Fixed: Path(null) and Ext(null) were automatically changed to empty string.
  • Fixed: #10: Docs should include all OS-specific functions.

Read more


Scriptlike: Shell Scripting in D: Annoyances and a Library Solution

Due to its power and general-purpose intent, D has great potential as a faster, easier (in terms of flow-control), and cross-platform alternative to shell scripting. But after quite a bit of experience using it as such, I've noticed several annoyances that consistently slow me down and over-complicate my D scripts:

  • Managing paths as stings is messy and error-prone: There are a lot of fantastic tools in std.path to deal with paths, but there are still annoyances:

    • No static compile-time checks to keep non-path strings out of an API that requires a path.

    • Extra naming conventions are sometimes needed, to help distinguish paths from other stings. Not a major problem, but it's still one more thing to keep consistent, and one more thing making your variable names longer.

    • Extra care required to make sure directories are properly separated by slashes: The buildPath function can help with this, but it's easy to forget and often very tempting to omit.

    • Slashes vs backslashes: Posix uses forward slashes. Windows almost always accepts forward or backslashes interchangeably, but there are a few occasional exceptions. Plus, displaying forward-slashed paths in Windows is bad style.

      D's recommended style is to always use the platform-native slash in all your internal variables, but this can complicate algorithms and make it harder to guarantee your software properly accepts both versions on any OS (which is good style). It also leads to awkward, verbose, easy-to-forget-or-deliberately-stop-caring ugliness like writing "some/big/path" as buildPath("some", "big", "path") or as "some"~dirSeparator~"big"~dirSeparator~"path".

      Or you can use forward-slash-only internally (like I do) and only convert to backslashes as needed. But then you have to obsessively sanitize all path inputs, and many of std.path's outputs. Either way, it's not exactly trivial script-like coding.

    • Trailing slashes? You can ignore the matter of whether trailing slashes exist by relying on the error-prone convention of always using buildPath, even in trivial cases where it verbosifies code and tempts you to omit it (and accept buildPath's unwillingness you let you use "all paths are internally forward-slashed" convention to not complicate your internal logic with "forward and/or back" concerns). Or you can use an "always trailing slash" or "never trailing slash" convention, try to remember which one you chose, and obsessively sanitize all function inputs and external return values while maybe avoiding ugly double-slashes and definitely taking care not to accidentally convert null paths into root directories. Neither options are particularly appealing for shell-like scripts.

  • Invoking a tool that's in the current directory is platform-specific: On Posix, you must prefix `./`. On Windows you must not prefix `./` (but `.\` will work and so will omitting the path). So more buildPath and dirSeparator, just to invoke a tool in the current directory! Whee! Obviously more the fault of the system's command interpreters than D, but still an annoyance that could be improved.

  • Invoking a command shell-style: D's std.process was overhauled a few versions ago and, for the most part, it's fantastic. Easy to capture the child process's output if you want, and any other way you might want to handle the child's standard pipes is fully configurable, as well as invoking the process synchronously or asynchronously.

    But want the standard shell-script style of "synchronously run a shell command and automatically forward all the child's std pipes through the parent's"? You can do it, but for something so common, it's not very obvious how. Examining the documentation, the correct way is supposed to be spawnShell(cmd).wait();. But currently, that fails on Windows when the command's path has spaces, even when the path is properly quoted. Until that's fixed (it's actually fixed in master as of two days ago and should appear in v2.066), the only real solution, AFAICT, is to use the old system(cmd), even though it's "scheduled for deprecation".

  • Proper quoting: std.process has convenient tools for properly quoting commands and arguments. Unfortunately, it's easy to forget to call these functions whenever appropriate, which leads to easy "fails on spaces" bugs. Additionally, on Windows the quoting is done in an oddball way that's rather ugly when echoed (for tracing) and doesn't even work for command names given to system, which as explained above, is necessary for invoking commands shell-style.

  • Command tracing: Optionally echiong all the commands your script runs can be helpful, but it means writing a wrapper to use for all your std.process calls. Not a big problem, but it's one more thing to do and one more thing to clutter your script.

  • Managing imports: Adding proper imports for every part of the standard library I commonly use is perfectly fine for bigger software, but for shell-like scripts it's largely just pedantic boilerplate.

  • Safety when changing the working directory: Often you'll need to run a command in a specific directory. But as soon as you change directories, all your relative paths silently become invalid (They're relative paths, after all). Which means being extra careful to not use the wrong relative paths, and extra work and code to either maintain absolute versions of those paths or make sure everything stays in absolute form (which then results in longer, noisier command lines and tracing output...unless you go to the extra work of sanitizing that too).

    Some of that work and danger can be mitigated by using this idiom every time you change the current working directory:

    { auto saveCwd = getcwd(); chdir("some/dir"); scope(exit) chdir(saveCwd); ...do work... }

    But beyond occasional usage, that gets to be quite a bit of clutter and bookkeeping.

  • Ignoring pedantic filesystem exceptions: Sometimes you just need to make sure a directory exists, creating it if necessary. Or check whether something is a file or directory while counting "doesn't exist" as "duh, no it isn't a file/directory". The exceptions std.path throws in such cases are often a very good sanity check, guaranteeing careless errors don't get through, so I'm glad they exist. But when doing shell-like scripting, they're often more of a bother than they're worth. There are always easy ways to "do it the right way", such as checking whether a path exists before calling isFile or mkdirRecurse, but for script-like purposes these are often unnecessary concerns and only add clutter to otherwise straightforward code. Wrappers are easy to make, but that's still one more thing to do, and that much more boilerplate for your scripts.

  • Deleting an entire directory tree: Normally as easy as rm foo -rf (if you don't care about Windows compatibility), there's no similarly simple equivalent in std.file.

  • Copy and rename aren't as powerful: In a shell script, copy and move can optionally take a glob or directory as the source (instead of just a file), and a directory as the destination. But std.file's equivalents can only take individual files as both source and destination. And std.file's remove and rmdir don't accept globs either.

  • Remove isn't as powerful: Like copy and rename, remove and rmdir don't accept globs. And unlike Posix's rm, a single command can't be used to delete either files or directories. To delete a path generically, you need to first check if it's a file or a directory and call the appropriate function.

  • Accessing the filesystem with UTF-16 or UTF-32 paths: The functions in std.file only accept paths and file names in UTF-8. Although it hasn't been an issue for me personally, if you do have paths in UTF-16 or UTF-32, they need to be converted manually with to!string() or text(). This is a reasonable default since it reduces hidden allocations, but for script-like uses it could be a nuisance.

Some of these things could, and probably should, be addressed in D's standard library, Phobos. But many other issues are arguably less appropriate for Phobos for various reasons. To address all of these issues, and perhaps others that may surface, I've created a simple one-file library, Scriptlike.

At the moment, not all of the issues above are addressed by Scriptlike just yet. But the rest is ready-to-go, and you can check the issue tracker to see what work still remains. Scriptlike is also available in DUB as package scriptlike.

Here are the current features in Scriptlike:

  • A thin wrapper over std.path and std.file that provides a dedicated Path type specifically designed for managing file paths in a simple, reliable, cross-platform way. No more dealing with slashes, paths-with-spaces, calling buildPath, normalizing, or getting paths mixed up with ordinary strings.
  • Optionally enable automatic command echoing (including shell commands, changing/creating directories and deleting/copying/moving/linking/renaming both directories and files) by setting one simple flag: bool scriptlikeTraceCommands
  • Most typical Phobos modules automatically imported. Who needs rows and rows of standard lib imports for a mere script?
  • Less-pedantic filesystem operations for when you don't care whether it exists or not: existsAsFile, existsAsDir, existsAsSymlink, tryRename, trySymlink, tryCopy, tryMkdir, tryMkdirRecurse, tryRmdir, tryRmdirRecurse, tryRemove: All check whether the source path exists and return WITHOUT throwing if there's nothing to do.
  • One simple call, runShell, to run a shell command script-style (ie, synchronously with forwarded stdout/in/err) from any working directory. (Also automatically works around DMD #10863 without waiting for v2.066.)
  • One simple function, fail(string msg), to help you exit with an error message in an exception-safe way. (Does require some minor boilerplate added to your main().)
  • More to come!

Read more


SDLang-D v0.8.3 - Improved API/Docs

SDLang-D is D library to read/write SDL (Simple Declarative Language), a data language like JSON/XML/YAML, but easier and cleaner.

The main focuses of this version are a revamped API and improved/expanded GitHub-flavored documentation.

// Sample, but useful, SDL: latest-version "v0.8.3" release-date 2013/03/26 links { original-sdl "http://sdl.ikayzo.org/display/SDL/Language+Guide" sdlang-d:home "https://github.com/Abscissa/SDLang-D" // Full changelog sdlang-d:changelog "https://github.com/Abscissa/SDLang-D/blob/master/CHANGELOG.md" // How-to-Use and API Overview sdlang-d:how-to-use "https://github.com/Abscissa/SDLang-D/blob/master/HOWTO.md" }

The codename for this release is "v0.8.3".

Read more