notes-computer-incantations

ffmpeg -f concat -safe 0 -i <(printf "file '$PWD/%s'\n" ./*.wav) -c copy output.wav

---

libreoffice --headless --convert-to pdf file.xlsx

to convert csv to pdf (takes forever tho; but produces output of a decent size; but poor quality):

libreoffice --headless --infilter=text --convert-to pdf file.csv

---

pdftk in.pdf output out.pdf user_pw password

---

iconv -f utf8 -t asciiTRANSLIT /tmp/t.txt

---

qpdf -password=<your-password> -decrypt /path/to/secured.pdf out.pdf

---

sudo gem install typogen sudo xargs typogen <&0

---

dbus-monitor --system dbus-monitor --session grep . -r /sys/firmware/acpi/interrupts/

echo "disable" > /sys/firmware/acpi/interrupts/gpe50

---

from importlib import reload

---

exiftool -all= image.jpg

---

f=filename; (tar --create $f > /run/shm/$f.tar); lzip /run/shm/$f.tar; rm -vf /run/shm/$f.tar; gpg --symmetric /run/shm/$f.tar.lz; mv /run/shm/$f.tar.lz.gpg .; rm -vf /run/shm/$f.tar.lz;

tar xovf filename.tar.lz --lzip

---

find . -print0

xargs -0 cmd

find -print -exec cmd '{}' \;

---

to make a hard drive spin down (go to sleep):

to make gpg-agent forget everything (i think):

echo RELOADAGENT

gpg-connect-agent

to tell gpg-agent to reload .gnupg/gpg-agent.conf:

pkill -SIGHUP gpg-agent

---

hdparm -S 6 /dev/sdb

---

dropbox exclude

---

sudo ifconfig wlp2s0 down

sudo rmmod cfg80211 mwifiex mwifiex_pcie sudo rmmod cfg80211 mwifiex mwifiex_pcie sudo rmmod cfg80211 mwifiex mwifiex_pcie sudo rmmod cfg80211 mwifiex mwifiex_pcie

sudo modprobe cfg80211 sudo modprobe mwifiex sudo modprobe mwifiex_pcie

sudo ifconfig wlp2s0 up

---

xrdb -merge ~/.Xresources

---

to install a .deb from a slightly untrusted source but not run the postinstall (because you don't want them running anything with root access):

apt-get download <package> sudo dpkg --unpack <package>*.deb sudo rm /var/lib/dpkg/info/<package>.postinst -f sudo dpkg --configure <package> sudo apt-get install -yf #To fix dependencies

---

sudo systemctl status sudo systemctl start sudo systemctl stop sudo systemctl is-enabled sudo systemctl enable sudo systemctl disable

more at [1]

---

hugo hugo serve& cd ~/newwebsite; hugo; scp -r ~/newwebsite/public/* web:; cd -

---

in emacs, to convert OddMuse? style link markup to Asciidoc style link markup:

(we use curly braces, which is not asciidoc syntax, only temporarily so that they don't interfere with the next regexp)

(now get rid of the temporary curly braces)

also, put blank lines immediately before lists if the lists are preceded by ordinary text (instead of headers), and replace raw image URLs with image:URL[alt text], and replace '----' with

---

in emacs, to convert OddMuse? style link markup to Markup style link markup:

also, change == headings == to # headings, and put blank lines immediately before lists

---

"I use the following incantation when authorizing folks to ssh into my servers via github public keys:

    curl https://github.com/[github name].keys >> ~/.ssh/authorized_keys"

---

restarting virtualbox clipboard guest addition

kill `ps -fe

grep -i clip grep vbox head -n1awk '{print $2}'`; /usr/bin/VBoxClient? --clipboard

---

find . -path '*.txt' -print0

xargs -0 -n1 grep -H -s crossbar

---

upload to PyPI?:

pip install twine python setup.py sdist bdist_wheel twine upload dist/*

---

python -m virtualenv lets-go . ./lets-go/bin/activate

---

sudo du -schx /* --exclude=/proc --exclude=/sys

sort -h

---

force immediate clock sync/time sync/ntp update:

sudo ntpd -qg

---

monitoring bandwidth usage (that works even within virtualbox, even when iptraf doesn't in some cases):

sudo nethogs enp0s3

(where 'enp0s3' is the name of the ethernet adapter)

---

to install something with dpkg and install its dependencies:

"After using dpkg, running the following command helped me to install the dependencies :

sudo apt-get -f install " [2]

---

xrandr --dpi 135

"4.12 Display (xfce4-display-settings), which is vastly improved from 4.10?"

font size and/or DPI setting in /Settings/Appearance/Settings

https://karlgrz.com/ubuntu-unity-changing-from-high-dpi-to-dual-monitors/ http://askubuntu.com/questions/472262/adapt-ubuntu-to-a-high-dpi-resolution-screen

---

xrandr --output LVDS1 --brightness 0.9 xbacklight -set 0

---

securely wipe GNU/Linux files and hard drives:

  1. for a file: shred -u filename
  2. quicker but less secure (only overwrites with zeros, not random data): shred -n0 -z -u filename
  3. whole directories:
  4. find directoryname -print0
find directoryname -exec shred -uv '{}' \; rm -rf directoryname # because the previous command only removed files, not directories
xargs -0 -n1 shred -v -n0 -z -u
  1. for the whole harddrive /dev/sda: swapoff -a

dd if=/dev/zero of=/dev/sda bs=1m

  1. or for more security: dd if=/dev/urandom of=/dev/sda bs=1m

---

cd parent_dir (dir=dir; tar -c "$dir"

lzip -o "$dir.tar")

---

htop atop iotop iptraf dstat mtr ping traceroute dig tcpdump tshark wireshark

---

sudo clamscan -r -i --cross-fs=no / > /tmp/clamscanlog.txt

---

/l4/backup/sdcard_from_lgd520_sd_ext/data/org.mozilla.firefox/files/mozilla/uunz9v8f.default/browser.db

y'know how there's that 'reading list' in Firefox Android? You want a list of the URLs in it?

They are inside the 'browser.db' file in your profile folder; that is, in:

data/org.mozilla.firefox/files/mozilla/aery5ea9.default/browser.db

where 'aery5ea9.default' is replaced by your profile folder's name. You may or may not need root to be able to access that file, i'm not sure. Anyways, if you can get that file off using adb, then if you install sqlite3, you can do:

sqlite3 browser.db 'select url from reading_list order by added_on;'

to print out a list of URLs in the order that you saved them

---

cat >/etc/polkit-1/rules.d/99-usb-mounting.rules 1 .ssh/authorized_keys" ---

in 'screen', to save a copy of the scrollback buffer:

ctrl-a ctrl-+ to get to command mode, then

hardcopy -h <filename>

btw why do i use screen rather than tmux? tmux had problems with using the middle button to paste, over ssh


screen -X -S [session # you want to kill] kill


sudo ln -s /etc/resolvconf/run/resolv.conf


make any website editable

javascript:document.body.contentEditable='true'; document.designMode='on'; void 0


rotate and play a video:

mplayer -vf rotate=2 /tmp/p8253012.avi -vo x11 -ao none -framedrop -nobps -ni -forceidx -mc 0 -vfm ffmpeg -lavdopts fast:skiploopfilter=all

--- strip audio from a video:

ffmpeg -i /tmp/MVI_0047.AVI -an -vcodec copy /tmp/MVI_0047_nosound.AVI

another way:

avconv -i MVI_0047.AVI -an -c:v copy /tmp/MVI_0047.AVI

to do a whole directory (placing the results into new subdir 'done'):

mkdir done find *.mp4 -exec sh -c 'for f do avconv -i "$f" -an -c:v copy done/"$f"; done' sh {} +

mkdir done find *.mp4 -exec sh -c 'for f do ffmpeg -i "$f" -an -vcodec copy done/"$f"; done' sh {} +


how to increase brightness, hue on videos? the following works but is too big and can't be played on xine

mencoder -o /tmp/test.avi -oac copy -ovc raw -vf eq2=1:1:-0.06:1.20 clouds/090802/p8021536.avi


to test and then deploy djangoappengine projects:

./manage.py runserver ./manage.py deploy


GET http://google.com


When changing email addresses, use mairix, mutt, and a standing emacs session to quickly, semiautomatedly send email to some people who have emailed my old address in the last six months:

put in .mairixrc:


  base=~/Mail
  maildir=...
  mfolder=~/mairixResults
  database=~/Mail/mairix_database

initialize mairix database:

  mairix

create a search folder containing every message sent to my old email address by someone other than myself in the last six months:

  mairix d:6m- t:old_email@inet.org f:~my_own_email@inet.org

(replace old_email@inet.org and my_own_email@inet.org with real addresses)

put in .muttrc:

  macro index <F12> "<reply>^M^Unew email: new_email@inet.org^Mny"

(replace new_email@inet.org with the real address)

also, make sure that your sent mail is being saved in .muttrc, with something like:

  set record= =sent-mail

in emacs, type something like this into any buffer, then kill it (and yank it, if you want):


Hi,

My current email address is now

  new_email@inet.org

You are receiving this automated email because you have recently sent an email to my old email address, old_email@inet.org, which will stop working soon -- please update your address book. Thanks, bayle


restart mutt. change to folder ~/Mail/mairixResults. go down the list until you find someone to whom you'd like to send an update, making sure their message is highlighted. hit F12. this starts a reply to the highlighted message with the subject heading as in the F12 macro definition above. now you land in emacs to type the body of the message. press F3 (begin macro definition). Yank the text of the message. Kill the buffer. You return to mutt, and the mutt macro completes execution by automatically sending the message.

Go down the list until you find someone else you wish to notify. This time, in emacs, just press F4 twice. The first press ends the macro defn, the second press executes the macro (which now includes yanking the text and closing the buffer).

For subsequent notifications, just press F4 (sometimes that didn't work for me and i had to press it twice for some reason).

Be careful not to highlight or copy or kill any text while you're doing this, or emacs might start yanking that into your outgoing message body rather than the text that you want. i'd check your sent-mail after sending the first message to make sure it went out OK (or even send the first one to yourself by sending yourself a test message, highlighting it in Mutt when your receive it, and pressing F12). I'd check sent-mail again when you are done to make sure the rest went out ok.


http://www.google.com/trends/viz?q=mercurial,+git&date=all&geo=all&graph=weekly_img&sort=0&sa=N

links:

http://www.commandlinefu.com/commands/browse/sort-by-votes


ruby debugging

require 'ruby-debug'

debugger

SOME_VARIABLE.public_methods - Object.public_methods


screen -X hardcopy -h /tmp/t

---

to kill an unresponsive ssh session:

enter, ~, .

---

if you accidentally hit ctrl-S, things might freeze, use ctrl-Q to unfreeze

-- nearlyfreespeech commandline incremental awstats

wc /home/logs/access_log ulimit -t 300 maxlines=10000 startline=0 n=$startline; while true; do echo starting at $n; tail -n +$n /home/logs/access_log

head -n $maxlines perl /usr/local/www/awstats/cgi-bin/awstats.pl -config=nfsn -update -LogFile?=-; n=$(($n + $maxlines)); done
  1. todo break when no more records perl /usr/local/www/awstats/cgi-bin/awstats.pl -config=nfsn -update

if it doesn't say anything for more than five minutes, kill it and reduce maxlines. Supposedly awstats is already already flushing every 20000 lines but i don't see that happening.

--

sync heroku and github: http://stackoverflow.com/a/3373611/171761

multiple accounts on heroku: http://stackoverflow.com/a/13518904/171761

--

to get a later version of nodejs on ubuntu:

https://github.com/mishoo/UglifyJS/issues/479

sudo add-apt-repository ppa:chris-lea/node.js sudo apt-get update sudo apt-get install nodejs

---

to quickly view an HTML file from github: use rawgithub.com: just click 'raw' in github, then edit the URL to remove the first dot in raw.github.com

--

https://raw.github.com/rakudo/rakudo/nom/docs/architecture.html

---

first, use firefox to go to http://www.pinterest.com/bayleshanks/pins/ . Hold down Page Down to scroll all the way to the bottom to get everything to load. File->Save Page As.

cat 'Bayle Shanks on Pinterest.html'

perl -lane 'if (/"(https:\/\/www.pinterest.com\/pin\/[\d]+\/)"/) {print $1;}' > pin_urls.txt

wget -p -i pin_urls.txt -H -D pinimg.com

or, for secret boards, go to each board in Firefox and save it as above, then filter the file to create a list of URLs as above, install the Export Cookies extension in Firefox, then restart Firefox and Tools->Export Cookies, then do:

NOTE: this no longer works! it only saves a few images near what you are currently looking at (about 60-90, for me). so you'll have to page thru while saving multiple times from your browser.

wget --load-cookies /tmp/cookies.txt -p -i pin_urls.txt -H -D pinimg.com

---

to sum a column of numbers (thanks http://stackoverflow.com/questions/450799/shell-command-to-sum-integers-one-per-line ):

awk '{s+=$1} END {print s}' mydatafile

--

to use xz to compress things which aren't already .xz compressed:

  1. find /l3/bshanks-salk-cp -not -name \*.xz
find /l3/bshanks-salk-cp -not -name \*.xz -print0
xargs -n1 xz
xargs -0 xz
  1. find /l3/bshanks-salk-cp -not -name \*.xz -exec xz \{\} \;

--

when an ssh connection freezes, here's how to kill it: type enter, then ~, then .

for a listing of other ssh escape commands, type enter, then ~, then ?

--

sudo apt-get install timidity rosegarden

sudo adduser ____ audio (and relogin) sudo adduser ____ pulseaudio-access (and relogin)

consider installing ubuntustudio packages

jackd -R -P4 -dalsa -r44100 -p512 -n4 -D -Chw:0 -Phw:0 &

timidity -Oj -iA &

rosegarden &

Studio->Manage MIDI devices on top right, click on Available Output "128:0 TiMidity? port 0 (write)" (sometimes you have to try 129:0 though)

sometimes have to quit and re-start jackd, timidity, rosegarden

turn volume half-way up (in your system volume setting)

click to create segment Segment -> Edit with Matrix

keyboard shortcut to play: cntl-enter keyboard shortcut to rewind to beginning: home keyboard shortcut to rewind one measure: pageup

a tutorial: http://www.rosegardenmusic.com/wiki/doc:companion-en

---

to quote a single quote within a single-quoted string in the shell: use '\

eg: echo 'Don'\t do that'

(technically, you are not putting a single quote 'within' a single quoted string, rather, you are ending the single-quoted string, then inserting the ' which is quoted via backslash, then beginning the single-quoted string againg. Thanks http://www.grymoire.com/Unix/Quote.html )

---

  1. (exc_type, exc_value, exc_traceback) = sys.exc_info()

---

echoing with sudo

echo "echo 12000 > /proc/sys/vm/dirty_writeback_centisecs"

sudo sh

---

git

DigitalSea? 2 days ago

I just wanted to say thank you Linus for giving us Git. Before Git I was using SVN and while the wounds are healing, I will bear these SVN scars for many years to come. As a developer Git has made my life so much easier and not only that, thanks to Github I can help collaborate on open source projects as well as my own with ease.

reply

plongeur 2 days ago

99% of the time I use:

I guess that's "the basics" - what command should I learn next?

reply

mbell 2 days ago

Command to learn: - git rebase (if you learn everything you can do with this command, you'll pretty much know git inside and out)

Other useful stuff: - git diff - git diff --cached - git log --graph --decorate --pretty=oneline --abbrev-commit --all - git cherry-pick - git stash

reply

joe_inferno 2 days ago

git bisect is a powerful one i use to identify when a given bug was introduced

reply

icefox 2 days ago

Install git achievements (https://github.com/icefox/git-achievements) and get free internet points to expore them all.

Disclaimer: wrote this a few years ago and it has way more users than I ever would have thought.

reply

abvdasker 2 days ago

I use this command constantly:

http://blog.kfish.org/2010/04/git-lola.html

Being able to see the graph structure made it far easier for me to get a grasp git's general structure and flow on a higher level.

reply

Aga 2 days ago

Looking at the history tree with all the branches and remote branches is a great way really understand Git.

More powerful history browsers to "lola": * gitk (graphical, developed in git's own repo) https://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to... * tig (console) http://jonas.nitro.dk/tig/manual.html

reply

--

vi /etc/default/rcS and uncomment FSCKFIX and change to FSCKFIX=yes touch /forcefsck

--

    make --eval="print-%: ; @echo $*=$($*)" print-SOURCE_FILES

--

before logging in to wordpress, ssh in and do:

chmod -R 775 /home/public/blog/wp-login.php

and afterwards: chmod -R 750 /home/public/blog/wp-login.php

---

in nsfn, you can 'reposses files' through their admin UI, from user 'web' to you

--

in nsfn, the "Identica tools" plugin, when active, upon page load, makes tons of files whose names are prefixed with "sno" in the /tmp directory.

--

search-and-replace in-place

perl -pe '$_ =~ s/before/after/g' -i filename.txt

--

total sum of known bad blocks plus not-yet-scraped bad blocks in a ddrescue logfile (requires perl and pyp to be installed):

perl -lane 'if (($F[2] eq "/")

($F[2] eq "-")) {print $F[1];}' ddrescue_logfile pyp 'int(p, base=16)sum(pp)'

but it's probably better just to use:

ddrescuelog --show-status ddrescue_logfile

some other ddrescue commands (be suer to always use a logfile with ddrescue!!! using a logfile really should be the default):

sudo ddrescue -vvvv -f /dev/sdc1 /dev/sdb1 ddrescue_logfile_1 sudo ddrescue --data-preview --direct --sector-size=4096 -vvvv -f /dev/sdc1 /dev/sdb1 ddrescue_logfile sudo ddrescue --data-preview --try-again --retrim --no-scrape -vvvv -f /dev/sdc1 /dev/sdb1 ddrescue_logfile

other lists of data recovery programs: https://en.wikipedia.org/wiki/Ddrescue#References

--

ok, so you've used ddrescue, and it couldn't get all the blocks, so how to find out which files are damaged?

the easiest method, and the ones recommended in the ddrescue documentation, is apparently to (in the copy) fill the damaged blocks with some string that doesn't appear anywhere else, and then grep for it. But i'm concerned b/c i want to know if/how many of the filesystem/directory-containing blocks are damaged.

http://tim.purewhite.id.au/2011/04/disk-recovery-which-files-are-damaged/ presents another way, using sleuthkit

sudo apt-get install sleuthkit

i'll assume that the copy of the damaged filesystem is at /dev/sdb1.

First, get the 'fragment size', which is probably 4k:

$ sudo dumpe2fs -h /dev/sdb1

dumpe2fs 1.42.9 (4-Feb-2014) Fragment size: 4096
grep 'Fragment size'

4096 is written '1000' in hex.

Look in the logfile for a sector, eg:

$ head ddrescue_logfile

  1. Rescue Logfile. Created by GNU ddrescue version 1.20-pre1
  2. Command line: ddrescue --reverse --direct --sector-size=4096 --data-preview --try-again --retrim --no-scrape -vvvv -f /dev/sdg1 /dev/sdb1 ddrescue_logfile ...
  3. pos size status ... 0x14C924A5000 0x00001000 ?

divide the position by the fragment size (in hex) to get the fragment:

$ echo "ibase=16; 14C924A5000/1000"

86390678
bc

use 'blkstat'

$ sudo blkstat /dev/sdb1 86390678 Fragment: 86390678 Not Allocated Group: 2636

if it was allocated, though, then use:

$ ifind rescued.img -d 86390678

which can take a few minutes. Then use 'istat' to see which file is using that inode.

to do this in batch:

grep '.*-$' ddrescue_logfile


to query a dns server 200 times and return the RMS of the latency (RMS = root mean square, the sqrt of the average of the squares of the latency):

(for i in `seq 1 200`; do dig @208.67.222.222 google.com

grep 'Query time'; done)awk '{s+=$4*$4} END {print sqrt(s/NR)}'

---

a JS bookmarklet that enables zooming:

    javascript:document.querySelector('meta%5Bname=viewport%5D').setAttribute('content','width=device-width,initial-scale=1.0,maximum-scale=10.0,user-scalable=1');

[3]

---

todo: see if this helps with my hd probs:

Disabling auto suspending USB for i in /sys/bus/usb/devices/*/power/autosuspend; do echo 2 > $i; done

Disable USB autosuspend for foo in /sys/bus/usb/devices/*/power/level; do echo on > $foo; done

---

'sudo blkid' shows you the UUIDs and locations and labels of various disk drives (you need the 'sudo' otherwise some will be hidden)

---

unbuffer

http://unix.stackexchange.com/a/25375/131295

---

atop iotop

---

REISUB upon system freeze:

http://askubuntu.com/questions/4408/what-should-i-do-when-ubuntu-freezes/36717#36717 https://en.m.wikipedia.org/wiki/Magic_SysRq_key

---

to forcibly sync clock:

systemctl restart systemd-timesyncd

(according to [4])

---

youtube-dl URL -f best

---

f, filepath = tempfile.mkstemp(); os.close(f)

  1. note: that's OS.close(f), NOT just close(f), which will silently fail

---

  1. os.close(1001); os.close(1002); os.close(1003); os.close(1004); # close some files so that we can import psutil, assuming you already have too many open files

import psutil psutil.Process().open_files()

---

append 'activity/saved' to a Pinterest URL to see repins

---

https://soundcloud.com/tags/

---

http://shallowsky.com/blog/tags/forward/

---

"Yes, you can delete 'undeletable' apps through ADB, without rooting the device.

  pm uninstall -k --user 0 <name of package>"

---


Footnotes:

1. EOF polkit.addRule(function(action, subject) { if (action.id == "org.freedesktop.udisks2.filesystem-mount-system" && subject.isInGroup("users")) { return polkit.Result.YES; } }); EOF

---

  1. set difference, see http://unix.stackexchange.com/questions/120416/set-difference-from-two-files-in-unix

grep -vxF -f setA setB

--

git merge-tree `git merge-base master bob/master` master bob/master

---

echo powersave

echo 0 > /sys/devices/system/cpu/cpu2/online echo 0 > /sys/devices/system/cpu/cpu3/online cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq echo 0 > /sys/devices/system/cpu/cpufreq/boost
tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor > /dev/null
tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq > /dev/null

sensors

i7z

lspci lsusb lshw free vmstat top atop

ls /proc/cpuinfo

---

to disable auto shutdown due to high CPU temp (almost always a bad idea eg THIS CAN FRY YOUR COMPUTER):

https://burnz.wordpress.com/2009/02/23/ubuntu-auto-shutdown-due-to-high-cpu-temperature/ http://ubuntuforums.org/showthread.php?t=1710107&page=2

---

:w !sudo tee % Save a file you edited in vim without the needed permissions

--- cat ~/.ssh/id_rsa.pub

ssh user@server "cat