What are some useful Unix utilities you've been using lately?

Something that's entered my toolbox in the last few months is `just`.
https://github.com/casey/just

They're kind of like Makefiles but simpler, because it's not trying to be a build utility. It just runs commands. After a few months of use, I have justfiles sprinkled all through my system to run various project-specific shell commands. It hits a sweet spot between one liners and dedicated scripts, and it has really enhanced various workflows for me. Things that I wouldn't have automated before (because it feels too small), I *do* automate now, because these justfiles feel like an appropriate place for those little tasks.

nano, ever heard of it?

I skimmed the info manual for nano recently, and it had a lot more features than I realized. I was pleasantly surprised that it even had very basic syntax highlighting for org-mode files.

Dick Stallman is very sneaky

fzf. Life was truly terrible before fzf.

>>3
tbh I didn't know nano had syntax highlighting. Then again I rarely need nano or syntax highlighting.

Hmm...
My .bash_history is 90% yt-dlp calls and various ffmpeg invocations. gifski has been super useful for making .gif files that don't suck, too.

One thing I found that was useful was (is this a bashism? oh well):
comm <(sort file1) <(sort file2)
This was me checking a script full of video URLs to download against another one to see if I had any duplicates between them. With some more fiddling, I was easily able to generate one big script that didn't have any dupes thanks to comm.

I've used qrencode so I don't have to retype a long/ugly URL from my desktop onto my phone, I can just scan it:
qrencode -t ansi "https://www.youtube.com/watch?v=iNkfGHrXixo"

detox unfucks filenames to make them shell friendly. Love it.
Don't blindly run "detox *" in a directory, lol.

I also do have an actual usecase for cmatrix (which just shows a cool matrix screensaver in the terminal), since it's very useful as a live indicator for your connection quality over SSH.
Several of my machines on the wifi will just suffer and die horribly with 1000ms+ pings, although really, I should just go fucking set them up on a hard-wired network.
It also just looks cool.

...god, that is a LOT of yt-dlp and ffmpeg calls.

>>6
I didn't know about comm. I've used diff for similar tasks, but I like comm's 3 column output that makes it clear how 2 lists differ.
https://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html

I also didn't know about detox. That's something I could use a lot. I dislike having filenames with spaces, but renaming them manually is a pain.
https://github.com/dharple/detox

I don't get to use this often, but something I really like is inotifywait. You can use it to observe a directory for file system events, and run a script when those events happen. The most common use is for doing automatic rebuilds when a source file changes, but I've also used it for other kinds of file processing pipelines.
https://github.com/inotify-tools/inotify-tools

I feel like I should have a use for inotifywait, I just end up thinking of another solution before I get to it, and I don't do the task enough that it's worthwhile anyway to instantly operate on the contents of a directory...

I use ghq almost exclusively for cloning git repos.
https://github.com/x-motemen/ghq

I also use it to quickly change to the directory of a git repo with this zsh function.
https://miyagawa.co/blog/ghq-peco-percol

# peco for git repo navigation via CTRL+]
function peco-src () {
local selected_dir=$(ghq list --full-path | peco --query "$LBUFFER")
if [ -n "$selected_dir" ]; then
BUFFER="cd ${selected_dir}"
zle accept-line
fi
zle clear-screen
}
zle -N peco-src
bindkey '^]' peco-src

I've been using watch a lot lately.
All it does is run a command repeatedly, but it's nice for basic monitoring of sensors or a remote control program (eg, what song is playing).
example: watch -n 2 audtool current-song
will show me the current song being played in audacious right now, updating every 2 seconds


Also, one thing I do pretty regularly is less `which script`, where you'd replace script with the name of some script you wrote and I forgot what it's for/what arguments yo slapped together. ` is the backtick, and even if you should be using $(which script) instead, ` is easier to type lol.

>>6
I don't like detox. This is my filename canonicalization script:
sed 's/([0-9])''''/\1_inches/g
s/\#([0-9])/number_\1/g
s/\\[0-9][0-9][0-9]//g
s/%/_percent_/g
s/\+/_plus_/g
s/=/_equals_/g
s/\&/_and_/g
s/@/_at_/g' |
tr A-Z'\-([~)];.,' a-z_________ | tr -d '`''!?$' | tr -s ' ' _ |
sed 's/^_//
s/_$//'

>>11
I totally forgot about watch. I haven't needed it lately, but I like it too. As for which, I use it often in exactly the way you described. It's very useful.

I just found out about a neat trick.

diff <(lsusb) <(sleep 3; lsusb)

Run this command, plug in a USB device within 3 seconds, and you will get the information from the newly plugged in USB device. This would have been useful for me a lot of times in the past... ( ´ω｀)

>>14
That's really clever.

>>14
You could also use

paste -d '\n' <(lsusb) <(sleep 3; lsusb) | uniq -u

which i think is semantically more fitting.

I habitually use the trash-cli tools because I hate dealing with footguns.
no more "oh I fucked everything up" with rm -rf /some/important/dir

instead, if I do trash-put /some/important/dir
and realize, oh shit, that was wrong
I can just trash-restore it

If you wrangle a lot of JSON, duckdb is an amazing powertool. I used to love jq for this purpose, but duckdb obsoleted it for me. Being able to treat JSON files as tables and writing SQL queries against them has been a game changer for me. It's so much faster to both write the queries and execute them in duckdb compared to piping and filtering with jq.
https://duckdb.org/
(It can read a lot of different formats including its own database format, but I'm mostly using it with JSON.)

I also use it together with ob-duckdb in Emacs which makes it even more amazing.
https://github.com/gggion/ob-duckdb/

Highly recommended.

Please remember that GNU Parallel exists. GNU Parallel has completely replaced quite a few other solutions I have used over the ages. GNU Parallel is wonderful.
There are a lot of times when I will do a for loop on a bunch of files but like, I shouldn't even bother with that.

converting a bunch of video files?
$ ls *.mkv | parallel my-conversion-script.sh
(where my-conversion-script usually has some fucking ffmpeg command I do really often)

everything is done at maximum fucking speed

downloading a bunch of files?
$ parallel -j 4 wget < list-of-URLs.txt
replace 4 with however many files you want to download at once

{} is the default filename placeholder, so if you need to put it somewhere specific in your command, use that
eg,
$ parallel some-script.sh {} -o {}-converted.ext < files-to-process.txt

and there are nice features like getting the filename without the extension:
$ parallel some-script.sh {} -o {.}.ext < files-to-process.txt

>>19
Isn't this just a more chatty version of xargs -P?

>>20
honestly, I will admit yes

a big thing about parallel is that it does properly batch output (eg, if you're piping stdout into something) and supports shit like running tasks across several machines and whatnot

but if you don't need that and already use xargs, you don't really need parallel over xargs

xzgrep, xzcat, xzless, and xdiff are a huge help for me. I feel really silly for having decompressed logs before instead of just using those. Emacs uses it by default for recursive grep, which is also how I stumbled upon it. And they are part of the GNU coreutils too!