github.com/ddopson/underscore-cli @v0.2.19
Chat with this repoWhat it actually does AI analysis from the code graph — generated when you open this
README
Overview
JSON is an excellent data interchange format and rapidly becoming the preferred format for Web APIs. Thusfar, most of the tools to process it are very limited. Yet, when working in Javascript, JSON is fluid and natural.
Why can't command-line Javascript be easy?
Underscore-CLI can be a simple pretty printer:
cat data.json | underscore print --color
Or it can form the backbone of a rich, full-powered Javascript command-line, inspired by "perl -pe", and doing for structured data what sed, awk, and grep do for text.
cat example-data/earthporn.json | underscore extract 'data.children' | underscore pluck data | underscore pluck title
See [Real World Example] (#real_world_example) for the output and more examples.
Underscore-CLI is:
- FLEXIBLE - THE "swiss-army-knife" tool for processing JSON data - can be used as a simple pretty-printer, or as a full-powered Javascript command-line
- POWERFUL - Exposes the full power and functionality of [underscore.js] (http://documentcloud.github.com/underscore/) (plus [underscore.string] (https://github.com/epeli/underscore.string), json:select, and CoffeeScript)
- SIMPLE - Makes it simple to write JS one-liners similar to using "perl -pe"
- CHAINED - Multiple command invokations can be chained together to create a data processing pipeline
- MULTI-FORMAT - Rich support for input / output formats - pretty-printing, strict JSON, etc. See [Data Formats] (#data_formats)
- DOCUMENTED - Excellent command-line documentation with multiple examples for every command
A Bit More Explanation ...
Underscore-CLI is built on Node.js, which is less than a 4M download and very easy to install. Node.js is rapidly gaining mindshare as a tool for writing scalable services in Javascript.
Unfortutately, out-of-the-box, Node.js is a pretty horrible as a command-line tool. This is what it takes to simply echo stdin:
cat foo.json | node -e '
var data = "";
process.stdin.setEncoding("utf8");
process.stdin.on("data", function (d) {
data = data + d;
});
process.stdin.on("end", function () {
// put all your code here
console.log(data);
});
process.stdin.resume();
'
Ugly. Underscore-CLI handles all the verbose boilerplate, making it easy to do simple data manipulations:
echo '[1, 2, 3, 4]' | underscore process 'map(data, function (value) { return value+1 })'
If you are used to seeing "_.map", note that because we arn't worried about keeping the global namespace clean, many useful functions (including all of underscore.js) are exposed as globals.
Of course 'mapping' a function to a dataset is super common, so as a shortcut, it's exposed as a first-class command, and the expression you provide is auto-wrapped in "function (value, key, list) { return ... }".
echo '[1, 2, 3, 4]' | underscore map 'value+1'
Also, while you can pipe data in, if the data is just a string like the example above, there's a shortcut for that too:
underscore -d '[1, 2, 3, 4]' map 'value+1'
Or if it's stored in a file, and you want to write the output to another file:
underscore -i data.json map 'value+1' -o output.json
Here's what it takes to increment the minor version number for an NPM package (straight from our Makefile):
underscore -i package.json process 'vv=data.version.split("."),vv[2]++,data.version=vv.join("."),data' -o package.json
Installing Underscore-CLI
Installing Node (command-line javascript)
Installing Node is easy. It's only a 4M download:
Alternatively, if you do homebrew, you can:
brew install node
For more details on what node is, see this StackOverflow thread
Installing
npm install -g underscore-cli
underscore help
Documentation
Usage
If you run the tool without any arguments, this is what prints out:
Usage:
underscore <command> [--in <filename>|--data <JSON>|--nodata] [--infmt <format>] [--out <filename>] [--outfmt <format>] [--quiet] [--strict] [--color] [--text] [--trace] [--coffee] [--js]
Commands:
help [command] Print more detailed help and examples for a specific command
type Print the type of the input data: {object, array, number, string, boolean, null, undefined}
print Output the data without any transformations. Can be used to pretty-print JSON data.
pretty Output the data without any transformations. Can be used to pretty-print JSON data. (defaults output format to 'pretty')
run <exp> Runs arbitrary JS code. Use for CLI Javascripting.
process <exp> Run arbitrary JS against the input data. Expression Args: (data)
extract <field> Extract a field from the input data. Also supports field1.field2.field3
map <exp> Map each value from a list/object through a transformation expression whose arguments are (value, key, list).'
reduce <exp> Boil a list down to a single value by successively combining each element with a running total. Expression args: (total, value, key, list)
reduceRight <exp> Right-associative version of reduce. ie, 1 + (2 + (3 + 4)). Expression args: (total, value, key, list)
select <jselexp> Run a 'JSON Selector' query against the input data. See jsonselect.org.
find <exp> Return the first value for which the expression Return a truish value. Expression args: (value, key, list)
filter <exp> Return an array of all values that make the expression true. Expression args: (value, key, list)
reject <exp> Return an array of all values that make the expression false. Expression args: (value, key, list)
flatten Flattens a nested array (the nesting can be to any depth). If you pass '--shallow', the array will only be flattened a single level.
pluck <key> Extract a single property from a list of objects
keys Retrieve all the names of an object's properties.
values Retrieve all the values of an object's properties.
extend <object> Override properties in the input data.
defaults <object> Fill in missing properties in the input data.
any <exp> Return 'true' if any of the values in the input make the expression true. Expression args: (value, key, list)
all <exp> Return 'true' if all values in the input make the expression true. Expression args: (value, key, list)
isObject Return 'true' if the input data is an object with named properties
isArray Return 'true' if the input data is an array
isString Return 'true' if the input data is a string
isNumber Return 'true' if the input data is a number
isBoolean Return 'true' if the input data is a boolean, ie {true, false}
isNull Return 'true' if the input data is the 'null' value
isUndefined Return 'true' if the input data is undefined
template <filename> Process an underscore template and print the results. See 'help template'
Options:
-h, --help output usage information
-V, --version output the version number
-i, --in <filename> The data file to load. If not specified, defaults to stdin.
--infmt <format> The format of the input data. See 'help formats'
-o, --out <filename> The output file. If not specified, defaults to stdout.
--outfmt <format> The format of the output data. See 'help formats'
-d, --data <JSON> Input data provided in lieu of a filename
-n, --nodata Input data is 'undefined'
-q, --quiet Suppress normal output. 'console.log' will still trigger output.
--strict Use strict JSON parsing instead of more lax 'eval' syntax. To avoid security concerns, use this with ANY data from an external source.
--color Colorize output
--text Parse data as text instead of JSON. Sets input and output formats to 'text'
--trace Print stack traces when things go wrong
--coffee Interpret expression as CoffeeScript. See http://coffeescript.org/
--js Interpret expression as JavaScript. (default is "auto")
Examples:
underscore map --data '[1, 2, 3, 4]' 'value+1'
# [2, 3, 4, 5]
underscore map --data '{"a": [1, 4], "b": [2, 8]}' '_.max(value)'
# [4, 8]
echo '{"foo":1, "bar":2}' | underscore map -q 'console.log("key = ", key)'
# "key = foo\nkey = bar"
underscore pluck --data "[{name : 'moe', age : 40}, {name : 'larry', age : 50}, {name : 'curly', age : 60}]" name
# ["moe", "larry", "curly"]
underscore keys --data '{name : "larry", age : 50}'
# ["name", "age"]
underscore reduce --data '[1, 2, 3, 4]' 'total+value'
# 10
Data Formats
json
The default format. Outputs strictly correct, human-readible JSON w/ smart whitespace. This format has received a lot of love. Try the '--color' flag.
{
"num": 9,
"bool": true,
"str1": "Hello World",
"object0": { },
"object1": { "a": 1, "b": 2 },
"array0": [ ],
"array1": [1, 2, 3, 4],
"array2": [1, 2, null, null, null, 6],
"date1": "2012-06-28T22:02:25.993Z",
"date2": "2012-06-28T22:02:25.993Z",
"err1": { },
"err2": { "3": "three", "prop1": 1, "prop2": 2 },
"regex1": { },
"regex2": { "3": "three", "prop1": 1, "prop2": 2 },
"null1": null,
"deep": {
"a": [
{
"longstr": "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!",
"b": { "c": { } }
}
],
"g": {
"longstr": "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!"
}
}
}dense
Output dense JSON using JSON.stringify. Efficient, but hard to read.
{"num":9,"bool":true,"str1":"Hello World","object0":{},"object1":{"a":1,"b":2},"array0":[],"array1":[1,2,3,4],"array2":[1,2,null,null,null,6],"date1":"2012-06-28T22:02:25.993Z","date2":"2012-06-28T22:02:25.993Z","err1":{},"err2":{"3":"three","prop1":1,"prop2":2},"regex1":{},"regex2":{"3":"three","prop1":1,"prop2":2},"null1":null,"deep":{"a":[{"longstr":"This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!","b":{"c":{}}}],"g":{"longstr":"This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!"}}}stringify
Output formatted JSON using JSON.stringify. A bit too verbose.
{
"num": 9,
"bool": true,
"str1": "Hello World",
"object0": {},
"object1": {
"a": 1,
"b": 2
},
"array0": [],
"array1": [
1,
2,
3,
4
],
"array2": [
1,
2,
null,
null,
null,
6
],
"date1": "2012-06-28T22:02:25.993Z",
"date2": "2012-06-28T22:02:25.993Z",
"err1": {},
"err2": {
"3": "three",
"prop1": 1,
"prop2": 2
},
"regex1": {},
"regex2": {
"3": "three",
"prop1": 1,
"prop2": 2
},
"null1": null,
"deep": {
"a": [
{
"longstr": "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!",
"b": {
"c": {}
}
}
],
"g": {
"longstr": "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!"
}
}
}pretty
Output a richer 'inspection' syntax. When printing array-and-object graphs that can be generated by JSON.parse, the output is valid JavaScript syntax (but not strict JSON). When handling complex objects not expressable in declarative JavaScript (eg arrays that also have object properties), the output is informative, but not parseable as JavaScript.
{
num: 9,
bool: true,
str1: "Hello World",
object0: { },
object1: { a: 1, b: 2 },
array0: [ ],
array1: [1, 2, 3, 4],
array2: [1, 2, null, undefined, , 6],
date1: 2012-06-28T22:02:25.993Z,
date2: 2012-06-28T22:02:25.993Z{ "3": "three", prop1: 1, prop2: 2 },
err1: [Error: my err msg],
err2: [Error: my err msg]{ "3": "three", prop1: 1, prop2: 2 },
regex1: /^78/,
regex2: /^78/{ "3": "three", prop1: 1, prop2: 2 },
fn1: [Function],
fn2: [Function: fn_name],
fn3: [Function: fn_name],
fn4: [Function],
null1: null,
undef1: undefined,
deep: {
a: [
{
longstr: "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!",
b: { c: { } }
}
],
g: {
longstr: "This really long string will force the object containing it to line-wrap. Underscore-cli is smart about whitespace and only wraps when needed!"
}
}
}inspect
Uses Node's 'util.inspect' to print the output
{ num: 9,
bool: true,
str1: 'Hello World',Core symbols most depended-on inside this repo
withPropsShape
Languages
Modules by API surface
For agents
$ claude mcp add underscore-cli \
-- python -m otcore.mcp_server <graph>