dx/README.md

dx

Counts what lies in a directory, and how heavy it is — one level deep, one glance, done at once.

$ dx ~/src -r -t 8
/Users/mike/src        2489     1.1G
  Wetter/               297     316M  ██▉         29%
  gblog/                 99     135M  █▎          12%
  gbld/                  82     131M  █▏          12%
  goca/                  80     108M  █           10%
  futterturm/            62      68M  ▋            6%
  BugSim/               323      68M  ▋            6%
  gvm/                   37      57M  ▌            5%
  mJournal/             204      42M  ▍            4%
  … 16 more            1305     179M  █▋          16%

On the left the name, then the number of files inside, then the sum of their sizes, then the share of the whole. The header line is the directory itself. Directories are walked through completely, but not unfolded — for everything beyond that there is ncdu.

In a terminal the output is coloured, after the palette from mwxcol.

Building and installing

cd ~/src/dx
./build.sh
cp bin/dx ~/bin/dx

build.sh builds into ./bin, one binary each for macOS and Linux on arm64 and amd64, and puts bin/dx there as a symlink to the one for this machine. For a single platform:

PLATFORMS="linux/amd64" ./build.sh

Every run counts the patch number in version.txt up by one and builds it into all the binaries via -ldflags — so dx --version tells which build is running. A bare go build works too, but then reports the default from main.go instead of the real number.

Needs Go ≥ 1.26 and github.com/fatih/color. The Perl original (dx, 1.0.2, 2016) lies next to it unchanged and is not touched by the build; it has no colours and no bars, but otherwise does the same thing.

Updating itself

dx --check-update   # only look
dx --update         # fetch and replace

dx fetches the newest release from git.micw.org/mike/dx — the URL sits fixed in the program, there is nothing to configure. What it needs is one release per version, whose tag is the bare number (2.1.6), with the files from ./bin as its assets; the one looked for is the one matching GOOS/GOARCH of this machine.

What gets replaced is the running file itself. If the dx that was called is a symlink — say ~/bin/dx pointing at ~/src/dx/bin/dx-darwin-arm64 — the target behind it is renewed, not the link. Before the swap, what was freshly fetched is called once with --version; if it does not report the expected number, everything stays as it was. The swap itself is a rename within the same directory, hence atomic: either the old file or the new one, never half of one.

If the binary lies somewhere the user may not write to (/usr/local/bin), dx --update says so and does nothing — then sudo.

Once a day, by itself

Without being asked, dx looks once a day and afterwards says so at the end of the output, on stderr:

dx 2.1.9 is available, run 'dx --update'

The run in the foreground never touches the network for this. It only reads a note — ~/Library/Caches/dx/update.json, on Linux ~/.cache/dx/update.json — and when that one is older than a day, it starts dx --update-refresh on the side: the same runner once more, detached, without output, only to ask. Nobody waits for its answer; it will be in the note at the next call. dx thereby stays exactly as fast as before, even when the server happens to be silent.

The timestamp moves on before the asking. Two simultaneous runs therefore start one query, not two, and a server that does not answer is asked again tomorrow rather than on every call. If the note cannot be written, the question is dropped entirely — otherwise a write-protected cache directory would mean one process per call.

Asking and speaking happen only when stderr hangs on a terminal. In a pipe, in a script and under cron there is quiet, and DX_NO_UPDATE_CHECK=1 turns it off altogether.

This costs space: net/http together with TLS turns a 1.8 MB binary into one of 6.2 MB. For a tool that sits on the disk anyway, that is the price of being able to renew itself — whoever will not pay it leaves selfupdate.go out.

Reusing it

selfupdate.go hangs on nothing in the rest of the program and gets by with the standard library. For another program, copy the file, adjust the block at the top —

var selfUpdate = selfUpdater{
	repo:     "https://git.micw.org/mike/foo",
	asset:    "foo",
	current:  version,
	verify:   []string{"--version"},
	every:    24 * time.Hour,
	quietEnv: "FOO_NO_UPDATE_CHECK",
}

— plus three lines in the options part: --update, --check-update and the hidden updateRefreshFlag, and one at the end of the program:

if hint := selfUpdate.daily(); hint != "" {
	fmt.Fprintln(os.Stderr, hint)
}

daily delivers only the text; what it looks like is for the program to decide. Everything else in the file is called selfUpdate… or update… and therefore does not collide. Leave verify empty when the program has no cheap version call, and the trial run is skipped; every: 0 turns off the looking by itself and leaves just the two options. What is expected is a Gitea instance (tested with 1.27); GitHub knows the same route under different field names and is not served.

Options

-n	--number	sort by number of files instead of by size
-d	--dirs	count only directories, leave out top-level files
-r	--reverse	largest first
-t N	--top=N	show only the N largest, sum the rest into one line
-h	--help	help
	--bar, --no-bar	force bar and percentage, or switch them off
	--color=auto|always|never	force colour, or switch it off
	--version	version
	--check-update	look whether a newer release is there
	--update	fetch and install the newest release

Short options can be bundled (-rt 3, -dn), the value of -t may stick (-t5) or follow (-t 5). A -- separates options from a path beginning with -. Without a path, the current directory applies.

Sorting is ascending, so the largest stands right above the prompt and there is no need to scroll. -r turns that around, for pipes and long lists.

With -t, the summary line always sits at the small end: at the top under normal sorting, at the bottom with -r.

Exit
0	everything read
1	path does not exist or is not a directory
2	option used wrongly

Colour and bars

Both apply at the terminal and fall away in a pipe — dx | sort gets plain text. --color=always switches both back on, for less -R. Whoever wants only one of them says so explicitly with --bar / --no-bar. NO_COLOR is respected.

The colours follow the file_type roles from themes/eza/theme.yml, so that the same file looks the same in eza and in dx:

directory	blue, bold		archives	orange
executable	green, bold		images	violet
symlink	pink		video	pink
source, configuration	blue		audio	green
README, Makefile, go.mod	yellow		keys, certificates	red
documents	white		compiled (.o, .dylib)	grey
other files	white		.tmp, .bak, .DS_Store	darker

The number column is grey, a zero dark. The size column carries the size ramp of the eza theme — up to K green, M yellow, G orange, T red — and the bar takes on the colour of the column whose share it shows: that of the size, with -n that of the file count.

The bar is ten cells wide and reckons in eighths (▏▎▍▌▋▊▉█). Whatever amounts to less than half a percent gets no stroke and stands there as <1% instead of a flattering 0%. That way exactly those lines stay visible that matter.

Sizes

The thresholds are those of the original: from 1024 of the unit in question it switches over. Below 10, one decimal is added, because 2G could otherwise mean anything between 1.5 and 2.4 gigabytes. Whole bytes stay whole.

8   4.9K   316M   1.1G

What exactly gets counted

What gets counted are regular files, as with find -type f. Directories, sockets, fifos and devices do not count, nor do the directories themselves.

• A symlink to a directory is not followed and therefore stands there with 0 — coloured pink, so that the zero is explained.
• A symlink to a file at the top level counts with the size of its target, likewise pink.
• Hardlinks count as often as they occur. In a tree made with rsync --link-dest or Time Machine the numbers come out too high; dx is not meant for that sort of thing.
• What is measured is the file size, not the space occupied. Sparse files and APFS clones appear larger than they are on the disk.
• What cannot be read — locked directories, dead symlinks — is skipped and reported at the end on stderr, so that the table stays intact.
• -d leaves out top-level files entirely, from the sum as well.

Subdirectories are walked concurrently, as many at a time as there are cores. The original started one find process per directory; on ~/go that makes 0.19 s instead of 0.34 s.

Differences from the Perl original

1. Ties sort by name. Perl left them in the order of the hash, and dx -n gave a different order on every call.
2. An invalid path is an error. Perl silently listed the current directory — plausible numbers, but the wrong ones.
3. The name column is one place wider when the longest entry is a directory. Perl did not count the appended / towards the column width, and the longest name stuck to the block of numbers.
4. Errors go to stderr, instead of appearing as find: … Permission denied in the middle of the output.
5. Sizes have one decimal, see above. That is the only deviation in the old output format; without colour and bars the rest is identical byte for byte.

New are -r, -t, the bar column, the colours and the long options.

Layout

File
main.go	counting, sorting, formatting, options
filetypes.go	extension and file name → colour
selfupdate.go	--update, self-contained and meant for copying
build.sh	build for all platforms, counts version.txt up
version.txt	the version built last
dx	the Perl original from 2016

The palette sits in main.go twice — once as numeric values, once in the comments as a role. The source of truth remains ~/src/mwxcol/mwxcol.go; if it changes there, the eleven rgb{…} here have to be brought along.