Bayle Shanks's website: proj-oot-ootCommandlineNotes1

---

" The shell file management features. This includes redirection and pipelines. This is trickier. Much of this can be done with subprocess. But some things that are easy in the shell are unpleasant in Python. Specifically stuff like (a

b; c )

something >result. This runs two processes in parallel (with output of a as input to b), followed by a third process. The output from that sequence is run in parallel with something and the output is collected into a file named result. That's just complex to express in any other language.

---

" In UNIX, you’d have something like:

for f in *; do echo "Processing $f file..."; done

While in PowerShell?, you’d go with something similar to:

Get-ChildItem? "."

Foreach-Object { $name = $_.Name Write-Output "Processing $($name) file..." }

An equivalent functionality in Python can be achieved with:

from os import listdir

for f in listdir('.'): print('Processing {} file...'.format(f)) "

" $> cat names.txt

... To sort our list numerically, in descending order all we have to is $> cat names.txt ... if you’re using PowerShell?, cat is an alias for Get-Content and sort is an alias for Sort-Object, so the commands above can be also written as: $> cat names.txt "

namecount.py. And PowerShell? folks: $> Get-Content names.txt	python namecount.py.
namecount.py	sort -rn. And if you’re using PowerShell?: $> Get-Content names.txt	python namecount.py	Sort-Object { [int]$_.split()[-1] } -Descending
python namecount.py and $> Get-Content names.txt	python namecount.py	sort { [int]$_.split()[-1] } -Descending

"First off, the provided Unix commands _don't work_ (should have used `sort -rn -k 2`), while the provided powershell, as verbose as it is, does work."

" the entire python script along with the wrapping unix commands is encapsulated with a powershell 1-liner:

    cat ./names.txt | group | sort -d Count"

---

Ultimatt 1 day ago [-]

Skip awk, use perl....

The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE will then Evaluate an expression from the command line looping over the input line-by-line.

    alias glorp='perl -aF"/\s+/" -nE'

So now we have the command `glorp` to play with which has more familiar syntax than awk and all of CPAN available to play with!

    $ [data is generated] | glorp '/Something/ and say $F[2]'

We have access to any Perl module by putting -MModule::Name=function after the command, the following will parse a JSON record per line and glorp out what we wanted:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'say decode_json($_)->{hello};' -MJSON=decode_json
    world
    cat

Maybe you are used to using curl too. There is a nice web framework in Perl called Mojolicious (http://mojolicious.org) that provides a convenience module called 'ojo' for command line use. So grabbing the summary sentence from Wikipedia articles is as straight forward as below. Notice Mojolicious lets us use CSS selectors!

    $ echo -e 'grep\nawk\nperl' \
      | glorp 'say g("wikipedia.org/wiki/$F[0]")->dom->at("#mw-content-text > div > p")->all_text' -Mojo

vidarh 1 day ago [-]

Here's the equivalent for Ruby:

    alias glorp='ruby -ane '

    $ [data is generated] | glorp ' ~ /Something/ and puts $F[2]'

Or:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'puts JSON.load($_)["hello"] ' -rjson

(Of course Ruby got the -a autosplit-mode and the -n assumed 'while gets(); ... end' loop from Perl along with $_ and $F, so it's very intentional that they're similar)

kolodny 1 day ago [-]

Somewhat related nodejs self plug: Use nip https://github.com/kolodny/nip

    $  echo -e 'this\nis\na\nwhatever foo' | nip 'return /whatever/.test(line) && cols[1]' # foo

Ultimatt 1 day ago [-]

Awesome thanks for sharing this! I was too lazy to give a Ruby example alongside.

jedisct1 1 day ago [-]

And Ruby regexes are amazing.

omaranto 1 day ago [-]

I thought that they were amazing because they were just like Perl's. Are there any differences?

---

great article about some neat things in bash that we can probably learn from:

https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-known-about-bash/

---

 jordigh 8 hours ago [-]

Using readline is a great thing to know about too.

My favourite little-known readline command is operate-and-get-next:

https://www.gnu.org/software/bash/manual/html_node/Miscellan...

You can use it to search back in history with C-r and then execute that command with C-o and keep pressing C-o to execute the commands that followed that one in history. Very helpful for executing a whole block of history.

For some reason, this documentation is hard to find! It's not here, for example:

http://readline.kablamo.org/emacs.html

I'm a bit saddened when readline replacements don't implement C-o. For example, the Python REPLs don't have it.

lillesvin 6 hours ago [-]

I've overridden ctrl-r in my local Bash to search with fzf[0] and I'm using my history so much more now.

Didn't know about ctrl-o though, it sounds great! I hope that my ctrl-r override doesn't somehow break it.

[0]: https://github.com/junegunn/fzf

E: Fixed link.

CaptSpify? 5 hours ago [-]

That link is a 404 for me?

tokenizerrr 5 hours ago [-]

It had a trailing >, https://github.com/junegunn/fzf

---

chriswarbo 54 minutes ago [-]

For me, the biggest gotcha in bash is whether or not a sub-process/shell will be invoked, which can affect things like mutable variables and the number of open file handles. For example:

    COUNT=0
    someCommand | while read -r LINE
                  do
                    COUNT=$(( COUNT + 1 ))
                  done
    echo "$COUNT"

This will always print `0`, since the `COUNT=` line will be run in a sub-process due to the pipe, and hence it can't mutate the outer-process's `COUNT` variable. The following will count as expected, since the `<()` causes `someCommand` to run in a sub-process instead:

    COUNT=0
    while read -r LINE
    do
      COUNT=$(( COUNT + 1 ))
    done < <(someCommand)
    echo "$COUNT"

Another issue I ran into is `$()` exit codes being ignored when spliced into strings. For example, if `someCommand` errors-out then so will this:

    set -e
    FOO=$(someCommand)
    BAR="pre $FOO post"

Yet this will fail silently:

    set -e
    BAR="pre $(someCommand) post"

---

bash:

  parameter     result
  -----------   ------------------------------
  $name         polish.ostrich.racing.champion
  ${name#*.}           ostrich.racing.champion
  ${name##*.}                         champion
  ${name%%.*}   polish
  ${name%.*}    polish.ostrich.racing
  ${name%.*.*}  polish.ostrich
  ${name#*.*.}                 racing.champion
  [1]