Do you know JavaScript and want to write a shell script? Then you should give Node.js a try. It is easy to install and shell scripts are a great way to get to know it. This post explains the basics.
Accessing arguments
You access command line arguments via process.argv, an array with the following contents:
[ nodeBinary, script, arg0, arg1, ... ]
Thus, the first argument is at process.argv[2] and you can loop over all arguments via:
process.argv.slice(2).forEach(function (fileName) {
...
});
If you want to do more sophisticated argument processing, you can take a look at the Node.js modules nomnom and optimist. From now on, we’ll frequently use the file system module:
var fs = require('fs');
Reading a text file
If your file isn’t too large, you can read it in one go, into a string:
var text = fs.readFileSync(fileName, "utf8");
Afterwards, you can split the text, to process the content line by line.
text.split(/\r?\n/).forEach(function (line) {
// ...
});
For larger files, you can work with a stream to iterate over the lines. mtomis describes one solution at Stack Overflow.
Writing a text file
You can write the complete contents to a file, via a string.
fs.writeFileSync(fileName, str, 'utf8');
Or you can incrementally write strings to a stream.
var out = fs.createWriteStream(fileName, { encoding: "utf8" });
out.write(str);
out.end(); // currently the same as destroy() and destroySoon()
Cross-platform considerations
Determining the line break string.
Solution 1: Read an existing file into a string, search for a line break, assume the line break is "\n" if you can’t find one.
var EOL = fileContents.indexOf("\r\n") >= 0 ? "\r\n" : "\n";
Solution 2: Check the platform. All Windows platforms return "win32", even 64bit ones.
var EOL = (process.platform === 'win32' ? '\r\n' : '\n')
Working with paths
Use module path when working with file system paths. That ensures that the right path segment separator is used ("/" on Unix, "\" on Windows).
var path = require('path');
path.join(mydir, "foo");
Running the script
If your shell script is called myscript.js, you run it as follows:
node myscript.js arg1 arg2 ...
On Unix, you have the option to prepend a line telling the operating system how to run the script:
#!/usr/bin/env node
Then you only need to make the script executable:
chmod u+x myscript.js
Now it can be run on its own:
./myscript.js arg1 arg2 ...
Other topics
- Writing to stdout: console.log works just like in browsers. console is a global object, not a module, so there is nothing to require().
- Reading from stdin: process.stdin is a readable stream. The previously linked solution for reading lines from a stream works here, too. process is a global object.
- Execute a shell command: via child_process.exec().
19 comments:
Do you know, by chance, how to do system calls? Like backticks in Perl or os.system in Python.
I’ve added a section at the end that explains how to do that.
Also on Windows, forward and backward slashes don't matter despite the backslash being convention. The OS itself doesn't care. The bigger difference that Node's path module helps with is that all paths are relative to a root drive. "/" isn't a valid path but "C:/" is. It also helps for resolving paths when the OS's have different rules, like Windows won't understand "./" or ".." but the path module handles it for you.
Commander.js is another nice lib for handling arguments, can simplify the process a lot and it auto generates the help..
About line-breaks I would probably use a function to normalize the line breaks instead of simply checking for the existence of the EOL char (file may contain mixed line breaks), also note that you are not handling the Mac EOL format ("\r").
Cheers.
Isn't the callback juggling just overhead given performance is less often a consideration with shell scripts and they are often IO heavy?
Good stuff. Could you put the content of the *.js file into the *.cmd file? How do you start a command file foo.cmd? As "foo" or as "foo.cmd"?
Note that most of the code above is synchronous (= no callbacks).
Axel
I'm under the impression that most of node's relevant scripting libraries are asynchronous by default. Knowing the synchronous alternative, if it exists at all, constitutes cognitive overhead.
It has never been a problem for me. I’m just glad that I can write my scripts in JavaScript on a platform that has a sizable community.
A .cmd file is the same as a .bat file, though Windows treats it with a bit higher priority. .cmd, along with .bat, is labeled by default as an executable extension so you can run "foo.cmd" just using "foo". If you have foo.exe, foo.cmd., and foo.bat it does this: if you're loading from a graphical application, say explorer, it'll run the exe. If you're running from the command line, it'll run the cmd. .bat won't take priority over either. So if you have node.exe and node.cmd most of the time it's going to run the .exe so it's better to try and avoid the conflict.
You could technically put the javascript code in the cmd, in a couple ways. V8/Node has an option to eval script from the command line, so you'd do `node -e javascript.code('here)`. Obviously not super flexible. Node will also attempt to use a passed value to do a module lookup. So you could run `node myModule` and if it can find it in the normal module lookup procedure from that path then it'll run.
If you do the trick I listed with the extra flags this is a breakdown of what happens. Say I have node.cmd and nodex.exe. node.cmd simply adds the flags and forwards. So I do `node someFile.js`. This will hit node.cmd, get transformed into `nodex --harmony someFile.js`. Then Windows will evaluate `nodex` following the same files, presumably hitting your nodex.exe.
Oh I forgot. .js files actually are treated as executables by default. The problem is they're associated with Windows Script Host by default which is basically IE7 but with access to everything in Windows. If you associate .js files with Node then you can just run them straight up. It's just not going to work for say distributing stuff, in which case you're better off bootstrapping using .bat or .cmd files that hand directly off to node. Once you're in node you can do whatever magic you need to in terms of rectifying paths, loading files, etc. Batch is a pretty terrible language to attempt to program in, although you can actually do a lot of stuff with it if you work at it. A large chunk of Node's build process on Windows is handled from vcbuild.bat.
Doing Ctrl-C on a nodejs shell program kills my terminal as well (in Mac OS X). Do you experience the same problem? Is there any solution?
Use of Node purely for shell scripting is an interesting proposition for us that otherwise have no or limited overlap with JavaScript. I don't find myself drawn to the language, but on the other hand, it does seem to be becoming a mainstay that's worth knowing.
This is a very good point, after some nodejs scripting I totally agree.
Nice overview, thanks!
Any idea on how to make this work?
echo "foo bar" | ./foo.js
Specifically I’d like to know how to detect if the script is being piped to or not, and if so, how the piped value ("foo bar") can be retrieved within foo.js.
FWIW, I’ve posted this question (along with some more info) on Stack Overflow: http://stackoverflow.com/q/15466383/96656
I would do it as follows: If there is no argument, read what is coming in via stdin. Note: you first have to resume stdin, via process.stdin.resume(). That is, I don’t know how to check whether you are at the receiving end of a pipe, but you don’t necessarily have to.
The example in the readme of lazylines [1] reads everything coming in via stdin (e.g. stuff you pipe in) and prepends line numbers.
[1] https://github.com/rauschma/lazylines
Thanks!
Actually, it turns out `process.stdin.resume()` isn’t needed at all.
To quickly and synchronously detect if piped content is being passed to the current script in Node.js, the boolean `process.stdin.isTTY` property can be used:
$ node -p -e 'process.stdin.isTTY'
true
$ echo 'foo' | node -p -e 'process.stdin.isTTY'
undefined
So, the script code then becomes:
if (process.stdin.isTTY) {
handleShellArguments();
} else {
handlePipedContent();
}
`handlePipedContent()` is something like:
function handlePipedContent() {
var data = '';
var self = process.stdin;
self.on('readable', function() {
data += this.read();
});
self.on('end', function() {
doStuff(data);
});
}
Check out also ShellJS [1], "a portable implementation of Unix shell commands on top of the Node.js API"
[1]: https://github.com/arturadib/shelljs
Post a Comment