mike/mgsh
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

Compare commits

base: mike/mgsh:4.0.24
Branches Tags
mike/mgsh:master
mike/mgsh:4.0.70 mike/mgsh:4.0.69 mike/mgsh:4.0.66 mike/mgsh:4.0.65 mike/mgsh:4.0.58 mike/mgsh:4.0.57 mike/mgsh:4.0.55 mike/mgsh:4.0.50 mike/mgsh:4.0.46 mike/mgsh:4.0.42 mike/mgsh:4.0.29 mike/mgsh:4.0.24
..
compare: mike/mgsh:4.0.70
Branches Tags
mike/mgsh:master
mike/mgsh:4.0.70 mike/mgsh:4.0.69 mike/mgsh:4.0.66 mike/mgsh:4.0.65 mike/mgsh:4.0.58 mike/mgsh:4.0.57 mike/mgsh:4.0.55 mike/mgsh:4.0.50 mike/mgsh:4.0.46 mike/mgsh:4.0.42 mike/mgsh:4.0.29 mike/mgsh:4.0.24
24 Commits
4.0.24 .. 4.0.70
Author SHA1 Message Date
mike 6181ff53c0 [mike@mwxm4] 2026-08-11 16:19:18 +02:00
mike 55f42ec50d [mike@mwxm4] 2026-08-11 16:14:55 +02:00
mike 34be410a97 [mike@mwxm4] 2026-08-11 15:12:28 +02:00
mike 682ac2a4a4 [mike@mwxm4] 2026-08-11 15:07:21 +02:00
mike 39f5b48d94 [mike@mwxm4] 2026-08-11 11:40:32 +02:00
mike 113527f220 [mike@mwxm4] 2026-07-28 15:25:54 +02:00
mike 1e3ae9a4c7 [mike@maginot] 2026-07-28 06:55:46 +02:00
mikeandClaude Opus 5 b4797b056e Fix y/n questions being skipped after the first one 
Every y/n prompt after the first in a session answered itself with its
default and left the keypress queued for the next command line.

MIN and TIME are not part of the canonical/non-canonical switch: they
live in their own slots of the control-character array and survive
`stty icanon`. drainTTY left them at "min 0 time 0" — return whatever is
buffered, do not wait — and the restore named only icanon and echo, so
the next read returned zero bytes without ever waiting for a key.

Set MIN and TIME explicitly on the way in, and restore the terminal from
the state captured with `stty -g` instead of naming the flags we changed;
mgsh now hands the terminal back exactly as it found it, where before
"min 0" outlived mgsh itself and broke the next program's single-key
reads too.

getkey also reports whether it got a key at all. When it did not, the
answer is no whatever the default says: a question nobody saw must not
be taken as consent. All five call sites default to no, so the bug never
destroyed anything — it only made agreeing impossible.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-27 21:38:32 +02:00
mike 9bc17513ed [mike@mwxm4] 2026-07-27 16:05:01 +02:00
mikeandClaude Opus 5 8c4d8da4e9 Attach ./bin and ./assets to a release 
`release` now uploads every file in the project's ./bin and ./assets when
those directories exist. Nothing to configure, and nothing happens for a
project that has neither.

This is the part deliberately left out when `release` was written,
because it is where the three providers stop resembling each other:

  Gitea   multipart POST to .../releases/<id>/assets?name=<name>
  GitHub  raw POST to the separate upload host the release object names
          in upload_url, whose RFC 6570 template suffix has to go first
  GitLab  a release stores links, not files: the file goes into the
          project's generic package registry and the release gets a
          package link pointing at it

So findRelease and createRelease now return a releaseRef carrying the id
and, for GitHub, that upload host -- the id alone cannot address an
upload. Uploads stream from disk rather than buffering: these are whole
binaries, and the Gitea multipart body is assembled through a pipe.

Only regular files directly in those directories are taken. Symlinks are
skipped, which matters here: build.sh leaves bin/mgsh pointing at one of
its siblings, and uploading the same 9M twice under two names helps
nobody. A name present in both directories is used from bin and reported
for assets. Re-releasing a tag replaces same-named assets rather than
failing on them, since rebuilding and publishing again is the normal
reason to do it, and a file that fails does not stop the rest.

Each provider's request shape is pinned down against the recording
stand-in, and the whole chain was run once end to end -- real repository,
real binaries, a fake Gitea that also serves git-http-backend so the tag
push is real too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 20:13:09 +02:00
mikeandClaude Opus 5 f3d8cbe281 Cross-compile into ./bin for the usual platforms 
build.sh produced one binary for the machine it ran on. It now builds
darwin/arm64, darwin/amd64, linux/amd64 and linux/arm64 into ./bin, all
from the single version that run bumped, so the four never disagree about
what they are. PLATFORMS="linux/amd64" ./build.sh narrows it.

bin/mgsh is a symlink to the host's build, so there is still one stable
path to "the binary for this machine". The old ./mgsh in the repo root is
no longer written, and the script deletes it once if it finds it -- a
stale binary on a path people have in their fingers is worse than a
missing one.

CGO_ENABLED=0 throughout: the cross builds then need no toolchain per
target and the binaries are static. That is only safe because os/user
still resolves the current user without cgo, which mgsh needs for the
"[user@host]" stamp on every commit -- checked on darwin before relying
on it.

Windows is left out on purpose: mgsh shells out to stty and /bin/sh, so
it would compile there and then not work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 19:53:29 +02:00
mikeandClaude Opus 5 cd5ab1a2bd Remove the view command and the editor setting with it 
`view` was the last user of `editor`, so leaving the setting behind would
have made it exactly what `gitkey` was until recently: documented,
parsed, and doing nothing. It is gone from the struct, the template, the
environment, `config -k` and the settings table.

Both names are free for aliases now, as `open` already was.

The tests that used `editor` as their example of a project-overridable
setting use `gitkey` instead, which is the same kind of thing and still
exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 19:11:06 +02:00
mikeandClaude Opus 5 65342bcd7c Remove the open command 
`open` and `view` shared one implementation and differed in a single
line: `open` also made the project the active one. Only `view` is left,
with the behaviour it always had.

Dropping it from builtinCmds is the part worth noting: a reserved word
cannot be shadowed by an alias, so `open` is now free for one --
`alias open '!xdg-open $1'` works, which it could not before. That also
made a completion test wrong, since it used `open` as its example of a
name a builtin owns; it uses `status` now.

The `runInDir(d, "open", ...)` calls stay: those are macOS's open(1),
which is how an Xcode workspace gets opened.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 18:35:45 +02:00
mikeandClaude Opus 5 0d0d28560e Complete aliases that expand to a shell escape 
`alias ll '!ls -la'` makes everything after `ll` a shell argument just as
surely as typing the `!` does, but Tab there still went to the builtin
command tree and found nothing. The dispatch now asks what a line will
turn into rather than how it starts: a '!' escape, or a name that is not a
builtin and resolves to an alias whose body starts with '!'.

Only the arguments complete — the command word is fixed by the alias
body, so `ll vi` offers the file, never the editor. An alias to a builtin
stays with the builtin tree.

Only the alias itself is inspected, not what its expansion might expand
to in turn: an alias chain can rewrite its own arguments, and guessing at
that would offer candidates for a command line other than the one being
built.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 18:28:35 +02:00
mikeandClaude Opus 5 2acca170e6 Complete ! shell escapes like a shell 
A line starting with '!' was not in the completer tree at all, so Tab did
nothing there -- just where the paths are longest. It now completes the
way a shell does: the command word against the executables on PATH, the
arguments against the filesystem, resolved relative to the active project
because that is where forwardShell runs the line. Directories complete
with their trailing slash, `~/` and absolute paths work, and dot entries
stay hidden until the prefix asks for one.

readline's completer is a tree of fixed words, which cannot express "a
prefix that is not a word", so this is a small AutoCompleter that
dispatches on the '!' and hands everything else to the existing tree. Its
contract is easy to get subtly wrong -- candidates are the suffixes still
missing, and the length is counted in runes, not bytes -- so the
conversion has its own test, as does completing only the basename inside
a directory, which is what keeps the candidate list readable.

Only the basename is offered inside a directory, PATH is scanned once per
session, and a bare '!' offers nothing rather than every executable on
the machine.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 18:18:01 +02:00
mikeandClaude Opus 5 2a622046f2 Define mirror targets one way: remote.<name>.<field> 
There were two spellings for the same thing -- a flat
remoteurl/remotekey/remotetype/remotevisibility set for a single server,
and remote.<name>.* blocks for several. The flat one is gone; every
target, including a lone one, is now a named block with the fields url,
key, type and visibility.

An existing ~/.mgshrc is converted on the next start. Only the key is
rewritten, so values, comments, alignment, commented-out lines and the
file's 0600 mode survive untouched, and mgsh prints each rename rather
than doing it quietly. The target is named "public", which is what the
old settings called the git remote they created, so a converted setup
keeps pushing to the same place under the same remote name. A file that
carries both spellings keeps what the new one says.

The environment follows the same shape: MGSH_REMOTEURL and friends are
replaced by MGSH_REMOTE_<NAME>_<FIELD>, so MGSH_REMOTE_GITLAB_KEY sets
remote.gitlab.key. The field is read from the end of the variable name,
which leaves target names free to contain underscores.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 18:00:26 +02:00
mikeandClaude Opus 5 61a7059f61 Fold the unpublished projects into the overview table 
They were a list underneath the table, which meant reading the same
project names in two different shapes. They are rows now, with the action
in an "init" column that only appears when some row needs it, and they
sort to the bottom as their own group: an un-inited directory is a
different kind of task and should not push the daily ones down.

Every directory under the base gets a row, not just the repositories --
`init` is exactly what turns a plain directory into a project, so leaving
those out would have hidden the ones the column is for. Such a row has no
git state to show and costs no subprocesses either, since projectStatus
now checks for .git before running any.

The count line gained "N to init"; the projects count still counts
repositories, so the two numbers stay meaningful side by side. An
unreachable server marks nothing at all, as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 17:49:59 +02:00
mikeandClaude Opus 5 8ca05d6ad2 Drop the "not cloned here" line from overview 
It answered a question `list` already answers, and it did so on every
run: the point of the overview is the state of the projects you have,
not a second listing of the server. reportInventory became
reportUnpublished and now reports one thing -- the local projects the
server has never seen, which are the ones `init` is for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 17:45:02 +02:00
mikeandClaude Opus 5 cb1ff98c3d Lay overview out as a table 
The status field was not a column: "*", "↑2", "✓" and "✓ (no upstream)"
are four different widths, so everything after them started somewhere
else on every line and the eye had to hunt along each row instead of
going down one.

Each field now has its own measured column: name (with the branch
appended when it is not master/main), a one-character dirty marker, the
sync state, host and age, then the mirrors. "(no upstream)" was fifteen
columns wide for something that is not even a problem, and is now "–".
Colour weights the row rather than decorating it -- a project that is
clean and in sync goes grey, the arrows and the dirty marker keep their
colour -- and the rows needing action sort to the top, alphabetically
within each group so positions stay predictable.

padRight counted bytes, which was fine while everything it padded was
ASCII; the arrows and check marks are three bytes and one column, so it
counts runes now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 17:42:37 +02:00
mikeandClaude Opus 5 86fb898df4 Fix list breaking when the server's du fails 
Two mistakes in the size support, reported from a real server.

The remote command used "2>/dev/null" to silence du. That is sh syntax,
and the git user's login shell need not be sh: in csh it parses as an
argument "2" followed by a redirection of stdout, so du was handed a
file named "2", complained, and exited non-zero. The redirection is
gone -- without it there is no bogus argument to trip over, and the
command now uses nothing that differs between sh and csh.

Worse, the exit status of the chain is the *last* command's, so that
failing du made sshOut return an error and `list` threw away a listing
that had arrived perfectly intact. It now reports a failure only when
nothing usable came back at all; a listing that parsed is shown whatever
the exit status, simply without the size column.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 17:35:50 +02:00
mikeandClaude Opus 5 3a420093d1 Show repository sizes in list 
`list -a` had sizes because archives are files; repositories are
directories, and a long listing reports the inode size for those -- 4096
for every single one. Taking that number would have filled the column
with the same meaningless value, so the real disk usage is asked of `du`
instead, appended to the same remote command so it still costs one round
trip.

The column is dropped entirely when no usable sizes come back, rather
than showing a column of zeroes, so a server without a working `du`
degrades to the previous output. The summary line carries the total.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 17:30:12 +02:00
mikeandClaude Opus 5 8c68b28dc2 Format list as an aligned table 
The name is what the eye looks for, but it came last, behind a ragged
date column, so nothing lined up. Worse, colorRepoLine rebuilt the line
with strings.Fields and single spaces, which destroyed the alignment ls
had produced -- the output was aligned only when colour was off.

The listing line is now parsed properly instead of being split at the
size field: name, date and size come out as fields, the date is re-padded
to a fixed twelve columns so "Sep 28  2016" and "Jan  3 14:32" agree, and
the name leads in a column sized to the longest entry. Colour decorates
that layout without changing it, which a test now checks by stripping the
escapes and comparing. `list -a` shows archive sizes, which were parsed
and thrown away before.

An empty result says so instead of printing nothing, which was
indistinguishable from a failure, and the count line matches the rest of
mgsh. The pattern now filters on the repository name rather than the
whole listing line: matching the owner or the date was never intended and
`list 2016` quietly did it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 16:53:54 +02:00
mikeandClaude Opus 5 cad7a4ec2c Turn overview into an inventory across local and server 
overview showed dirty and ahead/behind per project, which git can do on
its own. mgsh is the only thing that sees both the local base directory
and the ssh server, and joining those answers the questions git cannot:
which projects were never pushed to the server (candidates for `init`),
and which exist there but not on this machine (candidates for `clone`).
Both lists are printed after the summary. An unreachable server is
reported as such, rather than as "everything is missing".

Each row also names the machine that made the last commit and how long
ago. That costs nothing: `push` has always stamped "[user@host]" into
the commit message, and nothing ever read it back. On a setup spanning
several machines it is usually the piece one actually wanted. Rows also
show which mirror targets the repository has a remote for, which is
local git config and therefore free.

The walk is now concurrent and cheaper per project: `git status
--porcelain=v2 --branch` yields branch, upstream, ahead/behind and dirty
in one subprocess where three were used before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 16:17:40 +02:00
mikeandClaude Opus 5 59da8f376c Check the staged diff for credentials before push commits 
`push` runs `git add --all .`, so anything lying in the project gets
committed, and with `mirror = true` it reaches a public server in the
same breath. It is the one action in mgsh that cannot be undone: a
deleted server repository comes back from an archive, a published
credential does not.

The staged diff is now scanned before the commit is made -- private keys,
GitHub/GitLab/Slack/AWS/PyPI tokens, and credential-shaped assignments --
and a hit is shown with file and line before asking whether to continue.
Declining leaves the changes staged but uncommitted, so removing the file
and adding a .gitignore entry is all it takes.

The hard part is not detection but silence. A scanner that cries wolf
gets answered with a reflexive "y" and stops being a safety net, so
values that are plainly environment references, dotted identifiers,
constant names, template slots or masked stand-ins are filtered out. A
test scans mgsh's own README and mgshrc.example -- both full of
credential-shaped text -- and fails if either would trip the check. It
caught the documentation for this very feature, which is why the README
describes the sample output instead of reproducing it.

For a line that legitimately looks like a credential there is
`mgsh:allow`, which suppresses that one line; `secretscan = off` turns
the check off entirely. Only an explicit "off" does that -- a typo in the
setting leaves the safety net in place, which is what the new falsy()
is for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-26 16:17:30 +02:00
34 changed files with 5168 additions and 476 deletions
Expand all files Collapse all files
.gitignore
+1
View File
@@ -8,4 +8,5 @@
.TemporaryItems .TemporaryItems
.Trashes .Trashes
mgsh mgsh
bin/
.mgshrc .mgshrc
README.md
+377 -61
View File
@@ -7,8 +7,11 @@ directory. Go port of the original Perl `mgsh` (`mgsh.perl`).
## Contents ## Contents
- [Build](#build) · [Usage](#usage) · [Commands](#commands) · [Aliases](#aliases) - [Build](#build) · [Usage](#usage) · [Commands](#commands) · [Aliases](#aliases)
- [Overview](#overview) · [Credential check](#credential-check)
- [Public mirror (`pushremote`)](#public-mirror-pushremote) · - [Public mirror (`pushremote`)](#public-mirror-pushremote) ·
[Deleting a mirror (`deleteremote`)](#deleting-a-mirror-deleteremote) ·
[Releases](#releases) [Releases](#releases)
- [Updating itself](#updating-itself)
- [Configuration](#configuration) · [Settings reference](#settings-reference) · - [Configuration](#configuration) · [Settings reference](#settings-reference) ·
[Per-project configuration](#per-project-configuration) [Per-project configuration](#per-project-configuration)
- [Git server layout](#git-server-layout) - [Git server layout](#git-server-layout)
@@ -16,14 +19,36 @@ directory. Go port of the original Perl `mgsh` (`mgsh.perl`).
## Build ## Build
```sh ```sh
./build.sh # builds ./mgsh and bumps the patch version by 0.0.1 ./build.sh # all platforms into ./bin, bumps the patch version
go build -o mgsh . # plain build, keeps the default version PLATFORMS="linux/amd64" ./build.sh # just one
go build -o mgsh . # plain build, keeps the default version
``` ```
`build.sh` reads `version.txt`, increments the patch component, injects it via `build.sh` cross-compiles for `darwin/arm64`, `darwin/amd64`, `linux/amd64` and
`-ldflags -X main.VERSION`, and writes it back — so `version.txt` always holds `linux/arm64` into `./bin`:
the version of the binary just built. Dependencies are fetched via Go modules
(`go.mod` / `go.sum`) on first build. ```
bin/mgsh -> mgsh-darwin-arm64 (this machine)
bin/mgsh-darwin-amd64
bin/mgsh-darwin-arm64
bin/mgsh-linux-amd64
bin/mgsh-linux-arm64
```
`bin/mgsh` is a symlink to the build for the host, so there is one stable path
to "the binary for this machine". `bin/` is git-ignored. Windows is deliberately
absent: mgsh shells out to `stty` and `/bin/sh`, so it would compile there and
then not work.
Everything is built with `CGO_ENABLED=0`, which makes the cross builds need no
toolchain per target and the binaries static; `os/user` resolves the current
user without cgo on both darwin and linux.
It reads `version.txt`, increments the patch component, injects it via
`-ldflags -X main.VERSION` into **all** platforms of that run, and writes it
back — so `version.txt` always holds the version of the binaries just built, and
they all carry the same one. Dependencies are fetched via Go modules (`go.mod` /
`go.sum`) on first build.
Run the tests with `go test ./...`. Run the tests with `go test ./...`.
@@ -39,11 +64,11 @@ stand in. Outside `base` no project is selected. `mgsh <project>` starts the
interactive shell with that project preselected. interactive shell with that project preselected.
The commands available directly from the shell are `clone`, `init`, `log`, The commands available directly from the shell are `clone`, `init`, `log`,
`push`, `pushremote`, `release`, `list`, `tag`, `archive`, `show`, `open`, `push`, `pushremote`, `deleteremote`, `release`, `list`, `tag`, `archive`,
`pull`, `fetch`, `status`, `diff`, `overview`, `config`, `count`, `login` and `show`, `pull`, `fetch`, `status`, `diff`, `overview`, `config`, `count`,
`cloneall`; every other command is interactive-only. `login` and `cloneall`; every other command is interactive-only.
The interactive prompt is colored (Catppuccin Mocha) and shows the active The interactive prompt is colored (Catppuccin-flavored) and shows the active
project, its git branch and a `*` dirty marker: project, its git branch and a `*` dirty marker:
``` ```
@@ -51,9 +76,10 @@ project, its git branch and a `*` dirty marker:
``` ```
Features: command history (`~/.mgsh_history`), Tab completion (commands, local Features: command history (`~/.mgsh_history`), Tab completion (commands, local
projects for `cd`/`open`, server repos for `clone`/`show`, branches/tags for projects for `cd`, server repos for `clone`/`show`, branches/tags for
`checkout`/`tag`, mirror targets for `pushremote`, filesystem paths for `dist`), `checkout`/`tag`, mirror targets for `pushremote`/`release`, filesystem paths for
and colored `list`/`log`/error output. `dist`, and shell-style completion after `!` and for aliases that expand to
one), and colored `list`/`log`/error output.
The server repository list is fetched once per session on the first Tab that The server repository list is fetched once per session on the first Tab that
needs it; `rescan` refreshes it (and reloads the configuration). needs it; `rescan` refreshes it (and reloads the configuration).
@@ -69,6 +95,34 @@ prefix it with `!`:
< src/myproject > !ls -la < src/myproject > !ls -la
``` ```
It runs in the active project's directory. Tab completion works there the way it
does in a shell: the word after the `!` completes against the executables on
`PATH`, everything after it against the filesystem — relative to the project,
with `~/` and absolute paths understood, and directories completing with their
trailing slash so the next Tab walks into them. Dot entries stay out of the way
until the prefix asks for one.
```
< src/myproject > !vi ma<Tab> -> !vi main
< src/myproject > !vi <Tab> -> Makefile main.go main_test.go src/
< src/myproject > !gre<Tab> -> grep gresource
```
An alias that expands to a shell escape completes the same way, because its
arguments end up as shell arguments:
```
alias ll '!ls -la'
< src/myproject > ll ma<Tab> -> ll main
```
Only the alias's arguments complete, never its first word — the command is
fixed by the alias body. An alias to a builtin (`alias co 'checkout $1'`) is not
a shell line and is left alone.
Word splitting for completion is by whitespace only; quotes and backslash
escapes are left to the shell that runs the line.
### Commands ### Commands
Run `help` for the full list. Highlights: Run `help` for the full list. Highlights:
@@ -78,13 +132,14 @@ Run `help` for the full list. Highlights:
| `cd [project]` | change project (no argument: back to the base) | | `cd [project]` | change project (no argument: back to the base) |
| `push [comment]` | commit everything and push to the server | | `push [comment]` | commit everything and push to the server |
| `pushremote [desc]` | mirror the repo to a public server (gitea/github/gitlab) | | `pushremote [desc]` | mirror the repo to a public server (gitea/github/gitlab) |
| `deleteremote <@name\|host>` | delete the repo on that public server (asks first) |
| `pull` / `fetch` | pull / fetch from the server | | `pull` / `fetch` | pull / fetch from the server |
| `status [-a]` / `diff` | short git status (`-a`: overview of all projects) | | `status [-a]` / `diff` | short git status (`-a`: overview of all projects) |
| `overview` | dirty / ahead-behind summary of all projects | | `overview` | inventory of all projects, local and on the server |
| `log` | show the project log | | `log` | show the project log |
| `edit [n]` | interactive rebase of the last n commits | | `edit [n]` | interactive rebase of the last n commits |
| `clone [-a] <repo>` | clone a repository (or archive) from the server | | `clone [-a] <repo>` | clone a repository (or archive) from the server |
| `list [-a] [pattern]` | list repositories on the server | | `list [-a] [pattern]` | list repositories on the server (`-a`: archives, with sizes) |
| `show <repo>` | show a repository log directly on the server | | `show <repo>` | show a repository log directly on the server |
| `archive [comment]` | snapshot the server-side repo into `./archive` | | `archive [comment]` | snapshot the server-side repo into `./archive` |
| `init` | make a new repository from the current directory | | `init` | make a new repository from the current directory |
@@ -94,6 +149,7 @@ Run `help` for the full list. Highlights:
| `unalias <name>` | remove a command alias | | `unalias <name>` | remove a command alias |
| `config [-k]` | show the effective configuration and its sources | | `config [-k]` | show the effective configuration and its sources |
| `rescan` | reload the config, refresh the cached repo list | | `rescan` | reload the config, refresh the cached repo list |
| `update [-c]` | update mgsh to the newest release (`-c`: only look) |
| `!<command>` | run `<command>` in the shell | | `!<command>` | run `<command>` in the shell |
Commands that touch a repository (`push`, `pull`, `log`, `diff`, `tag`, `dist`, Commands that touch a repository (`push`, `pull`, `log`, `diff`, `tag`, `dist`,
@@ -114,7 +170,6 @@ project /Users/me/src/myproject/.mgshrc
gitport 22 gitport 22
gituser git gituser git
gitpath /home/git gitpath /home/git
editor code (.mgshrc)
remotes hub (.mgshrc) remotes hub (.mgshrc)
clone url ssh://git@git.example.com:22/home/git clone url ssh://git@git.example.com:22/home/git
@@ -125,6 +180,111 @@ pushremote targets (in push order):
Tokens are masked, so the output is safe to paste into a bug report. Tokens are masked, so the output is safe to paste into a bug report.
`config -k` prints just the setting names, one per line. `config -k` prints just the setting names, one per line.
### Listing the server
`list` shows what is on the git server, name first and aligned, ordered by
modification time — `push` touches the bare repository, so the most recently
worked-on project sits closest to the prompt:
```
< src > list
Betaflight3.0.0 Sep 28 2016 181M
website Mar 3 2024 2.1M
notes Jan 3 14:32 876K
3 repositories · 184M
```
The size is the repository's real disk usage on the server, asked of `du` in
the same round trip as the listing — a long listing reports the inode size for
a directory, which is the same number for every repository and says nothing. If
the server produces no usable sizes the column is left out rather than filled
with zeroes.
`list -a` lists the archives instead, whose sizes come from the listing itself;
a pattern filters by name (`list note`).
### Overview
`overview` (or `status -a`) is the one view that needs mgsh: it is the only
thing that sees the local base directory *and* the git server at once.
```
< src > overview
notes * ↑2 laptop 3h
website * ✓ desktop 2d → hub
Betaflight3.0.0 ✓ workstation 20d → gitea hub
experiments init
sandbox (wip) – laptop 1h init
4 projects · 2 dirty · 2 in sync · 2 to init
```
Every field sits in its own column, so the eye can go down one instead of
hunting along each line. The projects that need something done come first, the
settled ones next, and the directories the git server does not have yet come
last — those are a different kind of task. Within each group the order stays
alphabetical, so positions do not jump around.
| column | meaning |
|---|---|
| name | the project, with its branch appended when it is not `master`/`main` |
| `*` | uncommitted changes |
| `↑n` `↓n` | commits ahead of / behind the upstream (`↑2↓1` when both) |
| `✓` | in sync with the upstream |
| `–` | the branch tracks nothing — never pushed |
| `init` | the git server does not have this one; run `init` |
| host, age | who last committed and when, from the `[user@host]` stamp |
| `→` | mirror targets this repository has a remote for |
Every directory under the base gets a row, including those that are not
repositories at all — `init` is exactly what turns one into a project, so it
belongs in the table rather than in a list underneath it. Such a row simply has
no git state to show.
The host and age come from the commit itself — `push` writes `[user@host]` into
every message, so `overview` can say where a project was last worked on without
storing anything. On a setup spanning a laptop and a workstation that is usually
the piece of information you actually wanted.
The `init` column is the join no git command can do. If the server cannot be
reached, no row is marked and mgsh says so — not knowing is not the same as
knowing they are missing. The other direction, repositories on the server that
are not here, is what `list` shows.
### Credential check
`push` runs `git add --all .`, so anything lying in the project gets committed —
and with `mirror = true` it reaches a public server in the same breath. That is
the only action in mgsh that cannot be undone: a deleted server repository comes
back from an archive, a published credential does not.
So before anything is committed, the staged diff is checked for private keys,
GitHub/GitLab/Slack/AWS/PyPI tokens and credential-shaped assignments:
```
< src/notes > push new notes
2 possible credential(s) in what is about to be committed:
.env:3 credential assignment
API_KEY="<the offending line is shown in full here>"
deploy_key:1 private key
<the BEGIN … PRIVATE KEY header is shown here>
(set 'secretscan = off' to skip this check)
push anyway? y/N ?
```
Declining stops the push with nothing committed; the changes stay staged, so
`git restore --staged <file>` and a `.gitignore` entry are all it takes.
For a line that only *looks* like a credential and is meant to stay, put
`mgsh:allow` in it — a comment on that line is enough. That is better than
turning the whole check off for one false positive.
This is not a complete secret scanner and does not try to be one. It aims for a
high hit rate on what actually leaks, with few enough false alarms that the
prompt still means something: values that are plainly environment references,
constants, template slots (`<token>`, `${VAR}`) or masked stand-ins are ignored —
a test checks that mgsh's own README and `mgshrc.example`, both full of
credential-shaped text, stay quiet. Switch it off with `secretscan = off`.
### Aliases ### Aliases
`alias <name> '<command>'` defines a reusable shortcut, persisted to `alias <name> '<command>'` defines a reusable shortcut, persisted to
@@ -153,38 +313,57 @@ alias ec '!echo $1' # ec hello -> echo hello (shell)
Besides the internal ssh git server, `pushremote` mirrors the active project to Besides the internal ssh git server, `pushremote` mirrors the active project to
one or more public hosting servers (Gitea, GitHub or GitLab) over their REST one or more public hosting servers (Gitea, GitHub or GitLab) over their REST
API. A single server is configured flat: API. Each server is one `remote.<name>.<field>` block:
```ini ```ini
remoteurl = https://git.example.com # base URL of the server remote.gitea.url = https://git.example.com
remotekey = <personal-access-token> # API token remote.gitea.key = <personal-access-token>
# remotetype = gitea # optional; auto-detected from remoteurl remote.gitea.type = gitea # optional; auto-detected from the url
# remotevisibility = private # visibility of created repos (default private) remote.gitea.visibility = private # or public (default private)
# mirror = true # `push` also mirrors via pushremote
remote.gitlab.url = https://gitlab.example.com
remote.gitlab.key = <personal-access-token>
remote.gitlab.type = gitlab
remote.gitlab.visibility = public
remote.gitlab.active = false # only used when named (default true)
# remotes = gitea, gitlab # optional: restrict and order the set
# mirror = true # `push` also mirrors via pushremote
``` ```
Several servers get one `remote.<name>.*` block each: `<name>` is yours to pick; there is no other spelling. Older versions had a flat
`remoteurl`/`remotekey` pair for a single server — mgsh converts those to
```ini `remote.public.*` in place on the next start and says so, keeping the git remote
remote.gitea.url = https://git.example.com name those versions used.
remote.gitea.key = <personal-access-token>
remote.hub.url = https://github.com
remote.hub.key = <personal-access-token>
remote.hub.visibility = public
# remotes = gitea, hub # optional: restrict and order the set
```
| command | pushes to | | command | pushes to |
|------------------------|-----------------------------------------------| |------------------------|-----------------------------------------------|
| `pushremote` | every configured target, in order | | `pushremote` | every **active** target, in order |
| `pushremote @hub` | only `hub` | | `pushremote @gitea` | only `gitea`, active or not |
| `pushremote @hub @gitea` | those two | | `pushremote @gitea @gitlab` | those two |
| `pushremote a fix` | every target, description "a fix" | | `pushremote a fix` | every active target, description "a fix" |
Each target owns a git remote of the same name in the repository (the flat form #### Active and inactive targets
uses `public`, as before), so `git push hub` keeps working outside mgsh. A
`remote.<name>.active = false` takes a target out of the default set: it is
skipped by a bare `pushremote`, by `release` without a `@name`, and by the
automatic mirror of `mirror = true`. Naming it uses it exactly as before —
`pushremote @gitea`, `release @gitea v1.2`, or by host, `pushremote @git.example.com`.
That is the difference to the alternatives: removing the block loses the token,
and leaving the target in `remotes = …` keeps it on the automatic path. An
inactive target stays configured, keeps its credentials, is listed by `config`
(a shade dimmer, marked `inactive`) and completes with Tab — it just no longer
goes along by default. Useful for a server that is behind a VPN, rate-limited,
or only occasionally interesting.
Unset means active, so a configuration written before this flag existed keeps
mirroring where it always did. When *every* configured target is inactive, a
bare `pushremote` says so and names one instead of claiming nothing is
configured.
Each target owns a git remote of the same name in the repository, so
`git push gitlab` keeps working outside mgsh. A
target that fails does not stop the others; with more than one target target that fails does not stop the others; with more than one target
`pushremote` prints an `n/m remotes updated` summary. `remotes = …` restricts `pushremote` prints an `n/m remotes updated` summary. `remotes = …` restricts
and orders the set, which is mostly useful in a project `.mgshrc` — see below. and orders the set, which is mostly useful in a project `.mgshrc` — see below.
@@ -198,9 +377,56 @@ The token is sent as a one-shot HTTP auth header: it is never written into the
repo's git config, and it reaches git through the environment rather than the repo's git config, and it reaches git through the environment rather than the
command line, so it does not show up in the process table. Because `~/.mgshrc` command line, so it does not show up in the process table. Because `~/.mgshrc`
then holds a credential, mgsh creates it mode `600` and warns at startup if an then holds a credential, mgsh creates it mode `600` and warns at startup if an
existing file is readable by others. The provider is auto-detected from `remoteurl` (`github.com` → existing file is readable by others. The provider is auto-detected from the url
GitHub, `gitlab*` → GitLab, otherwise Gitea) and can be forced with (`github.com` → GitHub, `gitlab*` → GitLab, otherwise Gitea) and can be forced
`remotetype`. Set `mirror = true` to have every `push` mirror automatically. with `remote.<name>.type`. Set `mirror = true` to have every `push` mirror
automatically.
### Deleting a mirror (`deleteremote`)
`deleteremote` is the counterpart: it deletes the active project's repository on
a public server, through the same API and with the same token.
```
deleteremote <@name|host> [...]
```
```
< src/myproject > deleteremote git.example.com
remote gitea https://git.example.com (as mike)
this deletes https://git.example.com/mike/myproject.git with its issues,
releases and history, and cannot be undone
delete mike/myproject on gitea? y/N ? y
deleted mike/myproject on gitea
removed git remote gitea
```
A target can be named either way: `@gitea` (its configured name) or
`git.example.com` (the host of its url) select the same server, and the `@` is
optional here. Both spellings work for `pushremote` and `release` too.
`active = false` changes nothing for this command — it always needs the server
named anyway.
The guard rails, because nothing on the other end can be undone from here:
- **the server must be named.** Unlike `pushremote`, a bare `deleteremote` does
*not* mean "every configured target" — it lists them and stops.
- the repository is looked up first, so a typo is reported as "nothing to
delete" instead of becoming a request nobody meant to send.
- each deletion is confirmed on its own, naming owner/repo and server, and the
default answer is **no**.
- the **local clone is never touched**. Only the git remote of that name is
dropped, and only while it still points at the repository that was deleted.
Only the mirror is affected — the repository on the internal ssh git server, and
everything `list`/`archive` deal with, stays where it is.
Deletion needs more from the token than pushing does: the `delete_repo` scope on
GitHub, `write:repository` on Gitea, and the `api` scope plus the Owner role on
GitLab. When it is missing the server answers 403, and mgsh names the scope
rather than passing the bare status on. GitLab may only *schedule* the deletion
(HTTP 202): depending on the plan the project stays visible until its retention
period is over.
### Releases ### Releases
@@ -217,7 +443,7 @@ release [@name ...] <tag> [notes]
remote hub released https://github.com/mike/mgsh.git remote hub released https://github.com/mike/mgsh.git
``` ```
Without `@name` it releases to every configured mirror target, exactly like Without `@name` it releases to every active mirror target, exactly like
`pushremote`. Everything after the tag becomes the release notes *and* the tag's `pushremote`. Everything after the tag becomes the release notes *and* the tag's
annotation. annotation.
@@ -236,11 +462,103 @@ on different servers. `release` also refuses when the repository is not on the
mirror yet and tells you to run `pushremote` first, rather than creating it as a mirror yet and tells you to run `pushremote` first, rather than creating it as a
side effect. side effect.
**No binary assets.** The three providers handle uploads in three incompatible #### Binaries and assets
ways — Gitea attaches them to the release, GitHub uses a separate upload host,
and GitLab does not host them at all but expects a link into its package If the project has a `./bin` or `./assets` directory, every file in it is
registry. mgsh publishes source releases with notes; if you need binaries, attached to the release — nothing to configure, and nothing happens for a
upload them with the provider's own tooling. project that has neither:
```
< src/mgsh > release v4.1.0
attaching 5 assets, 38M from ./bin and ./assets
remote gitea released https://git.example.com/mike/mgsh.git
uploading logo.png 2.0K
uploading mgsh-darwin-amd64 9.8M
uploading mgsh-darwin-arm64 9.2M
uploading mgsh-linux-amd64 9.7M
uploading mgsh-linux-arm64 8.9M
```
Only regular files directly in those directories are taken: subdirectories are
not descended into, and symlinks are skipped — `bin/mgsh` points at one of its
own siblings, and uploading the same binary twice under two names helps nobody.
A name present in both directories is used from `bin` and reported for
`assets`, since one asset name can only mean one file.
Re-releasing the same tag replaces same-named assets instead of failing or
piling up duplicates, because rebuilding and publishing again is the normal
reason to do it. A file that fails to upload does not stop the rest.
This is where the providers stop resembling each other, and mgsh papers over it:
| | how the bytes get there |
|---|---|
| Gitea | multipart `POST` to `…/releases/<id>/assets?name=<name>` |
| GitHub | raw `POST` to the separate upload host named by the release's `upload_url` |
| GitLab | a release stores links, not files: the file goes into the project's generic **package registry** and the release gets a `package` link pointing at it |
The GitLab route needs the package registry enabled on the project — it is on by
default, but a self-hosted instance can turn it off.
## Updating itself
```
update -c # only look
update # fetch and replace
```
and from outside the shell, `mgsh update` / `mgsh --update` — the dashed
spelling is the one that also works before mgsh is configured, where every
other command exits with "not configured".
mgsh fetches the newest release from
[git.micw.org/mike/mgsh](https://git.micw.org/mike/mgsh) — the URL sits fixed in
the program, there is nothing to configure. What it needs is one release per
version, whose tag is the bare number (`4.1.0`), with the files from `./bin` as
its assets; the one looked for is the one matching `GOOS`/`GOARCH` of this
machine. That is exactly what `./build.sh` produces and `release` publishes, so
`./build.sh && release 4.1.0` is the whole publishing side.
What gets replaced is the running file itself. If the `mgsh` that was called is
a symlink — say `~/bin/mgsh` pointing at `~/src/mgsh/bin/mgsh-darwin-arm64` —
the target behind it is renewed, not the link. Before the swap, what was freshly
fetched is called once with `--version`; if it does not report the expected
number, everything stays as it was. The swap itself is a `rename` within the
same directory, hence atomic: either the old file or the new one, never half of
one. If the binary lies somewhere you may not write to (`/usr/local/bin`),
`update` says so and does nothing — then `sudo`.
### Once a day, by itself
Without being asked, mgsh looks once a day and says so on stderr — at the start
of an interactive session, and after the output of a one-shot command:
```
mgsh 4.1.0 is available, run 'mgsh --update'
```
The run in the foreground never touches the network for this. It only reads a
note — `~/Library/Caches/mgsh/update.json`, on Linux `~/.cache/mgsh/update.json`
— and when that one is older than a day, it starts `mgsh --update-refresh` on
the side: the same binary once more, detached, without output, only to ask.
Nobody waits for its answer; it will be in the note at the next call. mgsh
thereby stays exactly as fast as before, even when the server happens to be
silent.
The timestamp moves on *before* the asking. Two simultaneous runs therefore
start one query, not two, and a server that does not answer is asked again
tomorrow rather than on every call. If the note cannot be written, the question
is dropped entirely — otherwise a write-protected cache directory would mean one
process per call.
Asking and speaking happen only when stderr hangs on a terminal. In a pipe, in a
script and under cron there is quiet, and `MGSH_NO_UPDATE_CHECK=1` turns it off
altogether.
`selfupdate.go` is a copy from [dx](https://git.micw.org/mike/dx) and hangs on
nothing in the rest of mgsh: standard library only, every name it brings starts
with `selfUpdate` or `update`, and the block at the top of the file is all there
is to adjust when it moves on to the next program.
## Configuration ## Configuration
@@ -274,7 +592,6 @@ gitpath = /home/git
gitname = Your Name gitname = Your Name
gitemail = you@example.com gitemail = you@example.com
pushdefault = matching pushdefault = matching
editor = code # fallback opener for `open`
alias co 'checkout $1' alias co 'checkout $1'
``` ```
@@ -284,8 +601,10 @@ alias co 'checkout $1'
### Settings reference ### Settings reference
Every setting can also be given as an environment variable named `MGSH_<KEY>` Every setting can also be given as an environment variable named `MGSH_<KEY>`
(e.g. `MGSH_GITHOST`), which wins over both files. "Scope" says whether a (e.g. `MGSH_GITHOST`), which wins over both files; a mirror field is
project `.mgshrc` may override the setting. `MGSH_REMOTE_<NAME>_<FIELD>`, so `MGSH_REMOTE_GITLAB_KEY` sets
`remote.gitlab.key`. "Scope" says whether a project `.mgshrc` may override the
setting.
| setting | scope | meaning | | setting | scope | meaning |
|---|---|---| |---|---|---|
@@ -298,17 +617,14 @@ project `.mgshrc` may override the setting.
| `gitname` | global | `user.name` written to the **global** git config at startup | | `gitname` | global | `user.name` written to the **global** git config at startup |
| `gitemail` | global | `user.email` written to the global git config | | `gitemail` | global | `user.email` written to the global git config |
| `pushdefault` | global | `push.default` written to the global git config | | `pushdefault` | global | `push.default` written to the global git config |
| `editor` | project | opener used by `open`/`view` when the project has no Xcode workspace (default `coda`) | | `remote.<name>.url` | project | base URL of the mirror target `<name>` |
| `remoteurl` | project | base URL of a single mirror server (target name `public`) |
| `remotekey` | project | API token for `remoteurl` |
| `remotetype` | project | `gitea`\|`github`\|`gitlab`; auto-detected from the URL when unset |
| `remotevisibility` | project | `private` (default) or `public` for repositories created by `pushremote` |
| `remote.<name>.url` | project | base URL of the named mirror target |
| `remote.<name>.key` | project | API token for that target | | `remote.<name>.key` | project | API token for that target |
| `remote.<name>.type` | project | provider override for that target | | `remote.<name>.type` | project | `gitea`\|`github`\|`gitlab`; auto-detected from the url when unset |
| `remote.<name>.visibility` | project | visibility for that target | | `remote.<name>.visibility` | project | `private` (default) or `public` for repositories `pushremote` creates |
| `remote.<name>.active` | project | `false` keeps the target out of the default set; it is then only used when named (`pushremote @<name>`). Default `true` |
| `remotes` | project | comma- or space-separated list restricting and ordering the mirror targets | | `remotes` | project | comma- or space-separated list restricting and ordering the mirror targets |
| `mirror` | project | truthy (`1`/`true`/`yes`/`on`) → every `push` also mirrors | | `mirror` | project | truthy (`1`/`true`/`yes`/`on`) → every `push` also mirrors |
| `secretscan` | project | `off` disables the credential check `push` runs before committing (on by default; only an explicit `off` disables it) |
The three settings written to the global git config are applied at startup, and The three settings written to the global git config are applied at startup, and
only when they actually differ, so a plain `mgsh status` does not rewrite only when they actually differ, so a plain `mgsh status` does not rewrite
@@ -318,7 +634,7 @@ only when they actually differ, so a plain `mgsh status` does not rewrite
A project may carry its own `.mgshrc`, which overrides the global settings while A project may carry its own `.mgshrc`, which overrides the global settings while
that project is active — a project on a different git server, with a different that project is active — a project on a different git server, with a different
editor, or mirrored to a different place: ssh identity, or mirrored to a different place:
```ini ```ini
# ~/src/myproject/.mgshrc # ~/src/myproject/.mgshrc
alias.go
+5 -5
View File
@@ -44,10 +44,10 @@ var builtinCmds = map[string]bool{
"": true, "quit": true, "exit": true, "help": true, "rescan": true, "": true, "quit": true, "exit": true, "help": true, "rescan": true,
"dist": true, "list": true, "show": true, "log": true, "status": true, "dist": true, "list": true, "show": true, "log": true, "status": true,
"diff": true, "pull": true, "fetch": true, "push": true, "edit": true, "diff": true, "pull": true, "fetch": true, "push": true, "edit": true,
"pushremote": true, "overview": true, "archive": true, "init": true, "pushremote": true, "deleteremote": true, "overview": true, "archive": true, "init": true,
"login": true, "cd": true, "checkout": true, "clone": true, "cloneall": true, "login": true, "cd": true, "checkout": true, "clone": true, "cloneall": true,
"open": true, "view": true, "count": true, "tag": true, "alias": true, "count": true, "tag": true, "alias": true,
"unalias": true, "config": true, "release": true, "unalias": true, "config": true, "release": true, "update": true,
} }
func isBuiltin(name string) bool { return builtinCmds[name] } func isBuiltin(name string) bool { return builtinCmds[name] }
@@ -241,7 +241,7 @@ func handleUnalias(name string) {
func listAliases() { func listAliases() {
if len(aliases) == 0 { if len(aliases) == 0 {
fmt.Println(col(cGray, "no aliases defined")) fmt.Println(col(cDark, "no aliases defined"))
return return
} }
for _, n := range aliasNames() { for _, n := range aliasNames() {
@@ -251,7 +251,7 @@ func listAliases() {
// formatAlias renders `name = 'body'` with color. // formatAlias renders `name = 'body'` with color.
func formatAlias(name, body string) string { func formatAlias(name, body string) string {
return col(cGreen, name) + col(cGray, " = ") + col(cYellow, "'"+body+"'") return col(cGreen, name) + col(cDark, " = ") + col(cYellow, "'"+body+"'")
} }
// expandAlias substitutes positional parameters in an alias body. With no // expandAlias substitutes positional parameters in an alias body. With no
assets.go
+323
View File
@@ -0,0 +1,323 @@
package main
// assets.go — attaching files to a release.
//
// `release` uploads everything in the project's ./bin and ./assets when those
// directories exist. This is where the three providers stop resembling each
// other, so each gets its own path:
//
// Gitea multipart POST to .../releases/<id>/assets?name=<name>
// GitHub raw POST to the upload host named by the release object's upload_url
// GitLab no asset hosting on a release at all: the file goes into the generic
// package registry, and the release gets a link pointing at it
//
// Re-releasing the same tag replaces same-named assets rather than failing or
// piling up duplicates, because rebuilding and publishing again is the normal
// reason to do it.
import (
"encoding/json"
"fmt"
"io"
"mime"
"mime/multipart"
"net/http"
"net/url"
"os"
"path/filepath"
"sort"
"strconv"
)
// assetDirs are the project directories whose files a release carries.
var assetDirs = []string{"bin", "assets"}
// releaseAsset is one file to attach to a release.
type releaseAsset struct {
name string
path string
size int64
}
// collectAssets gathers the files to attach: every regular file directly in
// the project's ./bin and ./assets. Subdirectories are not descended into, and
// symlinks are skipped — bin/mgsh points at one of its own siblings, and
// uploading the same binary twice under two names helps nobody.
func collectAssets(dir string) (assets []releaseAsset, skipped []string) {
seen := map[string]bool{}
for _, sub := range assetDirs {
path := filepath.Join(dir, sub)
entries, err := os.ReadDir(path)
if err != nil {
continue // the directory simply is not there
}
for _, e := range entries {
if e.IsDir() || e.Type()&os.ModeSymlink != 0 {
continue
}
fi, err := e.Info()
if err != nil || !fi.Mode().IsRegular() {
continue
}
if seen[e.Name()] {
// bin/x and assets/x would fight over one asset name
skipped = append(skipped, filepath.Join(sub, e.Name()))
continue
}
seen[e.Name()] = true
assets = append(assets, releaseAsset{
name: e.Name(),
path: filepath.Join(path, e.Name()),
size: fi.Size(),
})
}
}
sort.Slice(assets, func(i, j int) bool { return assets[i].name < assets[j].name })
return assets, skipped
}
// contentType guesses a type from the file name, falling back to the one for
// "some bytes" — which is what a compiled binary is.
func contentType(name string) string {
if t := mime.TypeByExtension(filepath.Ext(name)); t != "" {
return t
}
return "application/octet-stream"
}
// doUpload sends a request with a raw body of a known size. The JSON helper
// cannot express these: an asset upload is bytes, not an object, and the
// providers insist on a Content-Length rather than a chunked body.
func (r *remoteAPI) doUpload(method, endpoint, ctype string, body io.Reader, size int64) (int, []byte, error) {
req, err := http.NewRequest(method, endpoint, body)
if err != nil {
return 0, nil, err
}
req.ContentLength = size
req.Header.Set("Content-Type", ctype)
hk, hv := r.authHeader()
req.Header.Set(hk, hv)
req.Header.Set("Accept", "application/json")
resp, err := r.http.Do(req)
if err != nil {
return 0, nil, err
}
defer resp.Body.Close()
data, _ := io.ReadAll(resp.Body)
return resp.StatusCode, data, nil
}
// uploadAssets attaches every file to the release, reporting each one as it
// goes: these are megabytes over a network, and silence for a minute looks like
// a hang. A file that fails does not stop the rest.
func (r *remoteAPI) uploadAssets(owner, repo string, ref releaseRef, assets []releaseAsset) error {
failed := 0
for _, a := range assets {
fmt.Printf(" %s %s %s\n", col(cDark, "uploading"),
col(cGreen, padRight(a.name, 28)), col(cDark, humanSize(a.size)))
if err := r.uploadAsset(owner, repo, ref, a); err != nil {
errorln(" " + a.name + ": " + err.Error())
failed++
}
}
if failed > 0 {
return fmt.Errorf("%d of %d assets failed", failed, len(assets))
}
return nil
}
// uploadAsset attaches one file, replacing an asset of the same name that is
// already on the release.
func (r *remoteAPI) uploadAsset(owner, repo string, ref releaseRef, a releaseAsset) error {
if err := r.removeAsset(owner, repo, ref, a.name); err != nil {
return err
}
switch r.kind {
case kindGitLab:
return r.uploadAssetGitLab(owner, repo, ref, a)
case kindGitHub:
return r.uploadAssetGitHub(ref, a)
default:
return r.uploadAssetGitea(owner, repo, ref, a)
}
}
// uploadAssetGitea posts the file as a multipart form, streamed from disk
// rather than buffered: these are whole binaries.
func (r *remoteAPI) uploadAssetGitea(owner, repo string, ref releaseRef, a releaseAsset) error {
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
pr, pw := io.Pipe()
mw := multipart.NewWriter(pw)
go func() {
part, err := mw.CreateFormFile("attachment", a.name)
if err == nil {
_, err = io.Copy(part, f)
}
if err != nil {
pw.CloseWithError(err)
return
}
pw.CloseWithError(mw.Close())
}()
ep := r.releasePath(owner, repo) + "/" + ref.id + "/assets?name=" + url.QueryEscape(a.name)
// the multipart length is not known up front, so this one is chunked
code, data, err := r.doUpload("POST", ep, mw.FormDataContentType(), pr, -1)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
}
return nil
}
// uploadAssetGitHub posts the raw bytes to the host the release object named.
func (r *remoteAPI) uploadAssetGitHub(ref releaseRef, a releaseAsset) error {
if ref.uploadURL == "" {
return fmt.Errorf("the release carries no upload_url")
}
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
ep := ref.uploadURL + "?name=" + url.QueryEscape(a.name)
code, data, err := r.doUpload("POST", ep, contentType(a.name), f, a.size)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
}
return nil
}
// uploadAssetGitLab puts the file into the project's generic package registry
// and links the release to it, since a GitLab release stores links, not files.
func (r *remoteAPI) uploadAssetGitLab(owner, repo string, ref releaseRef, a releaseAsset) error {
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
pkg := r.packagePath(owner, repo, ref.tag, a.name)
code, data, err := r.doUpload("PUT", pkg, contentType(a.name), f, a.size)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("package upload failed (HTTP %d): %s", code, firstLine(data))
}
code, data, err = r.do("POST", r.releaseByTagPath(owner, repo, ref.tag)+"/assets/links",
map[string]any{"name": a.name, "url": pkg, "link_type": "package"})
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("linking the package failed (HTTP %d): %s", code, firstLine(data))
}
return nil
}
// packagePath is where a release asset lives in GitLab's generic registry: one
// package named after the repository, one version per tag.
func (r *remoteAPI) packagePath(owner, repo, tag, name string) string {
return r.repoPath(owner, repo) + "/packages/generic/" +
url.PathEscape(repo) + "/" + url.PathEscape(tag) + "/" + url.PathEscape(name)
}
// removeAsset deletes an asset of the given name from the release when one is
// there, so re-releasing a tag after a rebuild replaces the files instead of
// failing or leaving two of each.
func (r *remoteAPI) removeAsset(owner, repo string, ref releaseRef, name string) error {
listEP, delEP := r.assetEndpoints(owner, repo, ref)
code, data, err := r.do("GET", listEP, nil)
if err != nil {
return err
}
if code == 404 {
return nil // nothing uploaded yet
}
if code != 200 {
return fmt.Errorf("listing assets failed (HTTP %d): %s", code, firstLine(data))
}
for _, id := range assetIDsNamed(data, name) {
if code, data, err := r.do("DELETE", delEP+"/"+id, nil); err != nil {
return err
} else if code != 200 && code != 202 && code != 204 {
return fmt.Errorf("deleting the previous %s failed (HTTP %d): %s", name, code, firstLine(data))
}
}
return nil
}
// assetEndpoints returns where a release's assets are listed and deleted.
// GitHub deletes an asset through the repository rather than the release.
func (r *remoteAPI) assetEndpoints(owner, repo string, ref releaseRef) (list, del string) {
switch r.kind {
case kindGitLab:
base := r.releaseByTagPath(owner, repo, ref.tag) + "/assets/links"
return base, base
case kindGitHub:
return r.releasePath(owner, repo) + "/" + ref.id + "/assets",
r.releasePath(owner, repo) + "/assets"
default: // Gitea
base := r.releasePath(owner, repo) + "/" + ref.id + "/assets"
return base, base
}
}
// assetIDsNamed picks the ids of the listed assets called name. All three
// providers answer with an array of objects carrying "id" and "name".
func assetIDsNamed(data []byte, name string) []string {
var list []struct {
ID json.Number `json:"id"`
Name string `json:"name"`
}
if err := json.Unmarshal(data, &list); err != nil {
return nil
}
var ids []string
for _, a := range list {
if a.Name == name && a.ID.String() != "" {
ids = append(ids, a.ID.String())
}
}
return ids
}
// assetSummary describes what will be uploaded, for the line `release` prints
// before it starts.
func assetSummary(assets []releaseAsset) string {
var total int64
for _, a := range assets {
total += a.size
}
unit := "assets"
if len(assets) == 1 {
unit = "asset"
}
return strconv.Itoa(len(assets)) + " " + unit + ", " + humanSize(total)
}
// assetDirList names the directories that were actually found, for the same line.
func assetDirList(dir string) []string {
var found []string
for _, sub := range assetDirs {
if isDir(filepath.Join(dir, sub)) {
found = append(found, "./"+sub)
}
}
return found
}
assets_test.go
+284
View File
@@ -0,0 +1,284 @@
package main
// assets_test.go — attaching files to a release.
//
// The three providers upload assets in three unrelated ways, and the difference
// is invisible until a real server rejects the request. The recording stand-in
// lets each one be pinned down: where the bytes go, how they are wrapped, and
// what else has to happen around them.
import (
"os"
"path/filepath"
"strings"
"testing"
)
// assetTree builds a project with ./bin and ./assets and returns its path.
func assetTree(t *testing.T, files map[string]string) string {
t.Helper()
dir := t.TempDir()
for rel, content := range files {
p := filepath.Join(dir, rel)
if err := os.MkdirAll(filepath.Dir(p), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(content), 0755); err != nil {
t.Fatal(err)
}
}
return dir
}
func TestCollectAssets(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/mgsh-linux-amd64": "ELF...",
"bin/mgsh-darwin-arm64": "MACH...",
"assets/logo.png": "PNG",
"assets/notes.txt": "text",
"bin/sub/nested": "not descended into",
"src/main.go": "not an asset directory",
})
// bin/mgsh is a symlink to one of its siblings, as build.sh leaves it
if err := os.Symlink("mgsh-linux-amd64", filepath.Join(dir, "bin", "mgsh")); err != nil {
t.Fatal(err)
}
assets, skipped := collectAssets(dir)
var names []string
for _, a := range assets {
names = append(names, a.name)
}
want := "logo.png,mgsh-darwin-arm64,mgsh-linux-amd64,notes.txt"
if strings.Join(names, ",") != want {
t.Errorf("collected %v, want %s", names, want)
}
if len(skipped) != 0 {
t.Errorf("skipped = %v, want none", skipped)
}
for _, a := range assets {
if a.size == 0 {
t.Errorf("%s has no size", a.name)
}
if _, err := os.Stat(a.path); err != nil {
t.Errorf("%s: %v", a.name, err)
}
}
// a project with neither directory contributes nothing at all
if got, _ := collectAssets(t.TempDir()); len(got) != 0 {
t.Errorf("a project without bin/ or assets/ produced %v", got)
}
}
// TestCollectAssetsNameCollision: one asset name can only mean one file, so the
// second directory's copy is reported rather than silently overwriting.
func TestCollectAssetsNameCollision(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/tool": "the binary",
"assets/tool": "something else with the same name",
})
assets, skipped := collectAssets(dir)
if len(assets) != 1 || assets[0].name != "tool" ||
!strings.HasSuffix(assets[0].path, "bin/tool") {
t.Errorf("assets = %+v, want only bin/tool", assets)
}
if len(skipped) != 1 || !strings.Contains(skipped[0], "tool") {
t.Errorf("skipped = %v, want the assets/ copy", skipped)
}
}
// TestUploadAssetGitea: a multipart POST under the field name Gitea expects,
// with the asset name in the query.
func TestUploadAssetGitea(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/mgsh-linux-amd64": "binary-bytes-here"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "gitea")
ref := releaseRef{tag: "v1.0", id: "7"}
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
req := f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets")
if req == nil {
t.Fatalf("no upload, requests: %v", f.paths())
}
if !strings.Contains(req.query, "name=mgsh-linux-amd64") {
t.Errorf("query = %q, want the asset name", req.query)
}
if !strings.HasPrefix(req.ctype, "multipart/form-data") {
t.Errorf("content type = %q, want multipart", req.ctype)
}
body := string(req.raw)
if !strings.Contains(body, `name="attachment"`) {
t.Errorf("form field is not 'attachment': %q", body)
}
if !strings.Contains(body, "binary-bytes-here") {
t.Errorf("the file contents did not make it: %q", body)
}
}
// TestUploadAssetGitHub: the bytes go raw to the host the release object named,
// not to the API host.
func TestUploadAssetGitHub(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "raw-bytes"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
f.route("POST /uploads/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "github")
// the template suffix from the release object has to be stripped
ref := api.parseReleaseRef("v1.0", []byte(`{"id":7,"upload_url":"`+
f.URL+`/uploads/repos/mike/mgsh/releases/7/assets{?name,label}"}`))
if ref.id != "7" || strings.Contains(ref.uploadURL, "{") {
t.Fatalf("release ref = %+v, want id 7 and a bare upload url", ref)
}
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
req := f.find("POST /uploads/repos/mike/mgsh/releases/7/assets")
if req == nil {
t.Fatalf("no upload to the upload host, requests: %v", f.paths())
}
if string(req.raw) != "raw-bytes" {
t.Errorf("body = %q, want the file verbatim", req.raw)
}
if !strings.Contains(req.query, "name=tool") {
t.Errorf("query = %q, want the asset name", req.query)
}
}
// TestUploadAssetGitHubWithoutUploadURL: without one there is nowhere to put
// the bytes, and that has to be said rather than guessed at.
func TestUploadAssetGitHubWithoutUploadURL(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "x"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
api := newRemoteAPI(f.URL, "tok", "github")
err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0])
if err == nil || !strings.Contains(err.Error(), "upload_url") {
t.Errorf("error = %v, want it to name the missing upload_url", err)
}
}
// TestUploadAssetGitLab: a GitLab release stores links, not files, so the file
// goes into the generic package registry first and the release then points at
// it.
func TestUploadAssetGitLab(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "package-bytes"})
assets, _ := collectAssets(dir)
const pkg = "/api/v4/projects/mike%2Fmgsh/packages/generic/mgsh/v1.0/tool"
const links = "/api/v4/projects/mike%2Fmgsh/releases/v1.0/assets/links"
f := newFakeProvider(t)
f.route("GET "+links, 200, `[]`)
f.route("PUT "+pkg, 201, `{"message":"201 Created"}`)
f.route("POST "+links, 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "gitlab")
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1.0", id: "v1.0"}, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
up := f.find("PUT " + pkg)
if up == nil {
t.Fatalf("no package upload, requests: %v", f.paths())
}
if string(up.raw) != "package-bytes" {
t.Errorf("package body = %q, want the file verbatim", up.raw)
}
link := f.find("POST " + links)
if link == nil {
t.Fatalf("the release was not linked to the package, requests: %v", f.paths())
}
if link.body["name"] != "tool" || link.body["link_type"] != "package" {
t.Errorf("link body = %v", link.body)
}
if u, _ := link.body["url"].(string); !strings.HasSuffix(u, pkg) {
t.Errorf("link url = %q, want it to point at the package", u)
}
}
// TestUploadAssetReplacesPrevious: re-releasing after a rebuild has to replace
// the old file, not fail on it or leave two of each.
func TestUploadAssetReplacesPrevious(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "v2"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200,
`[{"id":41,"name":"other"},{"id":42,"name":"tool"}]`)
f.route("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42", 204, ``)
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":43}`)
api := newRemoteAPI(f.URL, "tok", "gitea")
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42") == nil {
t.Errorf("the previous asset was not removed: %v", f.paths())
}
// only the one with the matching name
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/41") != nil {
t.Errorf("an unrelated asset was deleted: %v", f.paths())
}
if f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets") == nil {
t.Errorf("the replacement was not uploaded: %v", f.paths())
}
}
// TestUploadAssetsReportsFailures: one bad file must not stop the others, and
// the caller has to hear about it.
func TestUploadAssetsReportsFailures(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/good": "ok", "bin/bad": "boom"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
// the fake answers 404 for anything unrouted, so the POST fails
api := newRemoteAPI(f.URL, "tok", "gitea")
out := captureStdout(t, func() {
if err := api.uploadAssets("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets); err == nil {
t.Error("failed uploads were not reported")
}
})
// both were attempted, and each was announced before it started
for _, name := range []string{"good", "bad"} {
if !strings.Contains(out, name) {
t.Errorf("%s was not attempted or not announced:\n%s", name, out)
}
}
}
func TestAssetSummaryAndDirList(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/a": strings.Repeat("x", 1024),
"assets/b": strings.Repeat("y", 512),
"src/main.go": "not counted",
})
assets, _ := collectAssets(dir)
if got := assetSummary(assets); got != "2 assets, 1.5K" {
t.Errorf("assetSummary = %q, want \"2 assets, 1.5K\"", got)
}
if got := strings.Join(assetDirList(dir), ","); got != "./bin,./assets" {
t.Errorf("assetDirList = %q", got)
}
// singular reads properly, and a project without the directories lists none
if got := assetSummary(assets[:1]); !strings.HasPrefix(got, "1 asset,") {
t.Errorf("assetSummary(one) = %q", got)
}
if got := assetDirList(t.TempDir()); len(got) != 0 {
t.Errorf("assetDirList of a bare project = %v", got)
}
}
build.sh
+41 -4
View File
@@ -1,12 +1,22 @@
#!/bin/sh #!/bin/sh
# Build mgsh, auto-incrementing the patch version by 0.0.1 on every build. # Build mgsh for the usual platforms into ./bin, auto-incrementing the patch
# version by 0.0.1 on every build.
# #
# version.txt holds the currently built version. Each run increments the patch # version.txt holds the currently built version. Each run increments the patch
# component, then builds with that version injected via -ldflags, and writes it # component, then builds every platform with that one version injected via
# back. So version.txt always reflects the version of the binary just built. # -ldflags, and writes it back. So version.txt always reflects the version of
# the binaries just built, and all of them carry the same one.
#
# Override the platform list to build just one:
# PLATFORMS="linux/amd64" ./build.sh
#
# Windows is deliberately absent: mgsh shells out to stty and /bin/sh, so it
# would compile there and then not work.
set -e set -e
cd "$(dirname "$0")" cd "$(dirname "$0")"
PLATFORMS=${PLATFORMS:-"darwin/arm64 darwin/amd64 linux/amd64 linux/arm64"}
V=$(cat version.txt 2>/dev/null || echo 4.0.0) V=$(cat version.txt 2>/dev/null || echo 4.0.0)
# split MAJOR.MINOR.PATCH and increment PATCH (no carry: 4.0.9 -> 4.0.10) # split MAJOR.MINOR.PATCH and increment PATCH (no carry: 4.0.9 -> 4.0.10)
@@ -17,7 +27,34 @@ PATCH=${REST#*.}
PATCH=$((PATCH + 1)) PATCH=$((PATCH + 1))
NV="$MAJOR.$MINOR.$PATCH" NV="$MAJOR.$MINOR.$PATCH"
go build -ldflags "-X main.VERSION=$NV" -o mgsh . # earlier versions built ./mgsh in the repo root; drop it so nothing keeps
# running a stale binary from a path that is no longer written
if [ -f mgsh ]; then
rm -f mgsh
echo "removed stale ./mgsh (the build now writes ./bin)"
fi
mkdir -p bin
HOST="$(go env GOOS)/$(go env GOARCH)"
for p in $PLATFORMS; do
os=${p%/*}
arch=${p#*/}
out="bin/mgsh-$os-$arch"
# CGO_ENABLED=0 throughout: it makes the cross builds work without a
# toolchain per target and the binaries static, and os/user still resolves
# the current user without cgo on both darwin and linux.
CGO_ENABLED=0 GOOS="$os" GOARCH="$arch" \
go build -ldflags "-X main.VERSION=$NV" -o "$out" .
if [ "$p" = "$HOST" ]; then
ln -sf "mgsh-$os-$arch" bin/mgsh # the one for this machine
echo " $out -> bin/mgsh"
else
echo " $out"
fi
done
echo "$NV" > version.txt echo "$NV" > version.txt
echo "built mgsh v$NV" echo "built mgsh v$NV"
colors.go
+61 -24
View File
@@ -2,24 +2,29 @@ package main
import ( import (
"fmt" "fmt"
"strconv"
"strings" "strings"
"unicode/utf8"
) )
// Colors for the prompt, banner and output, using the Catppuccin Mocha palette // Colors for the prompt, banner and output, as 24-bit truecolor escapes: the
// as 24-bit truecolor escapes // Catppuccin Mocha accents with a warm white and three greys of its own.
// (https://terminalcolors.com/themes/catppuccin/mocha/).
const ( const (
cReset = "\033[0m" cReset = "\033[0m"
cBold = "\033[1m" cBold = "\033[1m"
cDim = "\033[2m" cDim = "\033[2m"
cGreen = "\033[38;2;166;227;161m" // Green #a6e3a1 cYellow = "\033[38;2;249;226;175m" // #f9e2af
cYellow = "\033[38;2;249;226;175m" // Yellow #f9e2af cOrange = "\033[38;2;250;179;135m" // #fab387
cRed = "\033[38;2;243;139;168m" // Red #f38ba8 cRed = "\033[38;2;243;139;168m" // #f38ba8
cCyan = "\033[38;2;148;226;213m" // Teal #94e2d5 cGreen = "\033[38;2;148;226;213m" // #94e2d5
cPurple = "\033[38;2;203;166;247m" // Mauve #cba6f7 cBlue = "\033[38;2;180;190;254m" // #b4befe
cWhite = "\033[38;2;205;214;244m" // Text #cdd6f4 cPink = "\033[38;2;245;178;247m" // #f5b2f7
cGray = "\033[38;2;108;112;134m" // Overlay0 #6c7086 cViolet = "\033[38;2;203;166;247m" // #cba6f7
cWhite = "\033[38;2;240;240;234m" // #f0f0ea
cGrey = "\033[38;2;170;170;187m" // #aaaabb
cDark = "\033[38;2;119;119;136m" // #777788
cDarker = "\033[38;2;68;68;85m" // #444455
) )
// col wraps s in color c, but only when color output is enabled. // col wraps s in color c, but only when color output is enabled.
@@ -35,25 +40,57 @@ func errorln(msg string) {
fmt.Println(col(cRed, msg)) fmt.Println(col(cRed, msg))
} }
// padRight pads an ASCII string with trailing spaces to width n. // padRight pads s with trailing spaces to a width of n columns. It counts
// runes, not bytes: the overview pads fields holding ↑ ↓ ✓, each of which is
// one column wide but three bytes long.
func padRight(s string, n int) string { func padRight(s string, n int) string {
if len(s) < n { if l := utf8.RuneCountInString(s); l < n {
return s + strings.Repeat(" ", n-len(s)) return s + strings.Repeat(" ", n-l)
} }
return s return s
} }
// colorRepoLine colors a `list` entry: the leading `ls -ltr` date (3 fields) in // formatRepoList renders the server listing for `list`: the name first, in a
// yellow and the repository name in green. // column wide enough for the longest one, then the date, and for archives the
func colorRepoLine(s string) string { // size. Names come first because that is what the eye scans for; putting the
if !useColor { // ragged date there instead is what made the old output hard to read.
return s //
// The order is left as it arrives: `ls -ltr` sorts by modification time, and
// `push` touches the bare repository, so the most recently worked-on project
// ends up closest to the prompt.
func formatRepoList(entries []lsEntry, withSize bool) string {
width := 0
for _, e := range entries {
if len(e.name) > width {
width = len(e.name)
}
} }
parts := strings.Fields(s) var b strings.Builder
if len(parts) >= 4 { for _, e := range entries {
date := strings.Join(parts[:3], " ") fmt.Fprintf(&b, " %s %s", col(cGreen, padRight(e.name, width)), col(cYellow, e.date))
name := strings.Join(parts[3:], " ") if withSize {
return col(cYellow, date) + " " + col(cGreen, name) fmt.Fprintf(&b, " %s", col(cDark, fmt.Sprintf("%7s", humanSize(e.size))))
}
b.WriteByte('\n')
} }
return col(cGreen, s) return b.String()
}
// humanSize renders a byte count compactly, the way `ls -h` does: a decimal
// only while it still carries information, so "3.2M" but "512K".
func humanSize(n int64) string {
const unit = 1024
if n < unit {
return strconv.FormatInt(n, 10) + "B"
}
div, exp := int64(unit), 0
for v := n / unit; v >= unit && exp < 4; v /= unit {
div *= unit
exp++
}
v := float64(n) / float64(div)
if v < 10 {
return fmt.Sprintf("%.1f%c", v, "KMGTP"[exp])
}
return fmt.Sprintf("%.0f%c", v, "KMGTP"[exp])
} }
commands.go
+191 -81
View File
@@ -14,32 +14,94 @@ import (
var ( var (
optRe = regexp.MustCompile(`^-(\w)$`) optRe = regexp.MustCompile(`^-(\w)$`)
numRe = regexp.MustCompile(`^\d+$`) numRe = regexp.MustCompile(`^\d+$`)
// a `ls -ltr` long-listing line: mode, link count, owner, group, size, then // a `ls -ltr` long-listing line: mode, link count, owner, group, size, the
// the date columns and the name. Owner and group are matched as opaque // three date columns, then the name. Owner and group are matched as opaque
// fields — the bare repositories need not belong to a user or group // fields — the bare repositories need not belong to a user or group
// literally named "git". // literally named "git".
lsEntryRe = regexp.MustCompile(`^\S+\s+\d+\s+\S+\s+\S+\s+\d+\s+(.*)$`) lsEntryRe = regexp.MustCompile(`^\S+\s+\d+\s+\S+\s+\S+\s+(\d+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(.*)$`)
gitDirRe = regexp.MustCompile(`^(.*)\.git$`) gitDirRe = regexp.MustCompile(`^(.*)\.git$`)
sanRe = regexp.MustCompile(`[,;:\\/='"|?><-]+`) // '!' is in here for the server's sake: csh expands history even inside
wsRe = regexp.MustCompile(`\s+`) // single quotes, so a comment like "fix!now" would reach it as something
// else entirely (see remoteRejected).
sanRe = regexp.MustCompile(`[!,;:\\/='"|?><-]+`)
wsRe = regexp.MustCompile(`\s+`)
) )
// lsEntry extracts the "<date columns> <name>" tail of a `ls -ltr` line whose // listMarker separates the two sections of the combined listing command, so
// entry name ends in suffix, with the suffix removed. It returns "" for any // `list` gets both the long listing and the disk usage in one round trip.
// other line (the leading "total" line, entries of a different kind). const listMarker = "---mgsh---"
func lsEntry(line, suffix string) string {
// duRe matches one `du -sk` line: kilobytes, then the path.
var duRe = regexp.MustCompile(`^(\d+)\s+(.*)$`)
// splitAtMarker divides the remote output into the part before and after the
// marker line. Everything is in the first section when the marker is absent —
// which is what happens when only a plain listing was asked for.
func splitAtMarker(lines []string, marker string) (before, after []string) {
for i, ln := range lines {
if strings.TrimSpace(ln) == marker {
return lines[:i], lines[i+1:]
}
}
return lines, nil
}
// parseDuSizes turns `du -sk` output into a name -> bytes map. A long listing
// reports the inode size for a directory — the same number for every bare
// repository — so this is the only way to say how large one actually is.
// A symlinked repository reports the size of the link, not of its target.
func parseDuSizes(lines []string) map[string]int64 {
out := map[string]int64{}
for _, ln := range lines {
m := duRe.FindStringSubmatch(strings.TrimRight(ln, "\r"))
if m == nil {
continue
}
kb, err := strconv.ParseInt(m[1], 10, 64)
if err != nil {
continue
}
// du echoes the path it was given, and find gives it an absolute one:
// key on the last element, which is what the listing calls the entry
name := strings.TrimSpace(m[2])
if i := strings.LastIndexByte(name, '/'); i >= 0 {
name = name[i+1:]
}
out[name] = kb * 1024
}
return out
}
// lsEntry is one parsed entry of the server's listing.
type lsEntry struct {
name string // with the ".git" / ".git.tar.gz" suffix removed
date string // the ls date columns, normalised to a fixed 12 columns
size int64
}
// parseLsEntry reads one `ls -ltr` line whose entry name ends in suffix. It
// returns false for anything else: the leading "total" line, entries of another
// kind, or output that does not look like a long listing at all.
func parseLsEntry(line, suffix string) (lsEntry, bool) {
m := lsEntryRe.FindStringSubmatch(strings.TrimSpace(line)) m := lsEntryRe.FindStringSubmatch(strings.TrimSpace(line))
if m == nil { if m == nil {
return "" return lsEntry{}, false
} }
name := m[1] name := m[5]
if i := strings.Index(name, " -> "); i >= 0 { if i := strings.Index(name, " -> "); i >= 0 {
name = name[:i] // a symlinked bare repo lists as "link.git -> target.git" name = name[:i] // a symlinked bare repo lists as "link.git -> target.git"
} }
if !strings.HasSuffix(name, suffix) { if !strings.HasSuffix(name, suffix) {
return "" return lsEntry{}, false
} }
return strings.TrimSuffix(name, suffix) size, _ := strconv.ParseInt(m[1], 10, 64)
return lsEntry{
name: strings.TrimSuffix(name, suffix),
// ls pads these itself, but only in its own column widths; re-pad so
// "Sep 28 2016" and "Jan 3 14:32" line up at 12 either way
date: fmt.Sprintf("%s %2s %5s", m[2], m[3], m[4]),
size: size,
}, true
} }
// validProject reports whether name is usable as a project name: a single path // validProject reports whether name is usable as a project name: a single path
@@ -138,11 +200,20 @@ func runCommandDepth(line string, depth int) bool {
case "config": // show the effective configuration and where it came from case "config": // show the effective configuration and where it came from
if opt["k"] { if opt["k"] {
fmt.Println(strings.Join(configKeys(), "\n")) fmt.Println(strings.Join(configKeys(), "\n"))
fmt.Println("remote.<name>.url|key|type|visibility") fmt.Println("remote.<name>.url|key|type|visibility|active")
break break
} }
showConfig() showConfig()
case "update": // replace this binary with the newest release (-c: only look)
run := selfUpdate.install
if opt["c"] {
run = selfUpdate.check
}
if err := run(os.Stdout); err != nil {
errorln("update: " + err.Error())
}
case "rescan": // reload the configuration and the cached server repo list case "rescan": // reload the configuration and the cached server repo list
reloadConfig() reloadConfig()
rescanServer() rescanServer()
@@ -184,20 +255,80 @@ func runCommandDepth(line string, depth int) bool {
if opt["a"] { if opt["a"] {
path, suffix = "./archive", ".git.tar.gz" path, suffix = "./archive", ".git.tar.gz"
} }
pat := word(words, 1) one, many := "repository", "repositories"
lines, err := sshOut("/bin/ls -ltr " + shq(path)) if opt["a"] {
if err != nil { one, many = "archive", "archives"
errorln("could not list repositories on the git server")
break
} }
for _, ln := range lines { pat := strings.ToLower(word(words, 1))
if pat != "" && !strings.Contains(strings.ToLower(ln), strings.ToLower(pat)) { remote := "/bin/ls -ltr " + shq(serverPath(path))
if !opt["a"] {
// archives are files and carry a real size; repositories are
// directories, whose listed size is the inode's, so ask du in the
// same round trip. Nothing shell-specific here on purpose: the
// login shell may be csh, where "2>/dev/null" is not a redirection
// but an argument followed by one.
//
// The pattern goes to find, quoted, and never to the shell. A glob
// that matches nothing is a fatal error in a non-interactive zsh —
// "no matches found: *.git" — which aborts the command and turned an
// empty server into "could not list". sh would have handed the
// literal "*.git" to du instead, which is not much better.
remote += "; echo " + shq(listMarker) +
"; find " + shq(serverPath(path)) + " -maxdepth 1 -name " +
shq("*"+suffix) + " -exec du -sk {} +"
}
lines, err := sshOut(remote)
lsLines, duLines := splitAtMarker(lines, listMarker)
sizes := parseDuSizes(duLines)
var entries []lsEntry
var total int64
for _, ln := range lsLines {
e, ok := parseLsEntry(ln, suffix)
// the pattern filters the name, not the whole listing line — an
// accidental match on the date or the owner helps nobody
if !ok || (pat != "" && !strings.Contains(strings.ToLower(e.name), pat)) {
continue continue
} }
if name := lsEntry(ln, suffix); name != "" { if !opt["a"] {
fmt.Println(colorRepoLine(name)) e.size = sizes[e.name+suffix] // 0 when du said nothing
} }
total += e.size
entries = append(entries, e)
} }
// The exit status belongs to the last command in the chain, so a `du`
// that fails must not discard a listing that arrived intact. Only
// complain when nothing usable came back at all.
if len(entries) == 0 {
if err != nil {
what := "could not list " + many + " on the git server"
if opt["a"] {
// by far the likeliest reason, and one the wording used to
// hide behind something that sounded like a dead connection
what += " — is there an 'archive' directory in " + cfg.GitPath + "?"
}
errorln(what)
break
}
what := "no " + many + " on the git server"
if pat != "" {
what = "no " + many + " matching '" + word(words, 1) + "'"
}
fmt.Println(col(cDark, what))
break
}
// no size column when the server gave no usable sizes, rather than a
// column of zeroes
fmt.Print(formatRepoList(entries, total > 0))
label := many
if len(entries) == 1 {
label = one
}
summary := fmt.Sprintf("%d %s", len(entries), label)
if total > 0 {
summary += " · " + humanSize(total)
}
fmt.Println(col(cDark, summary))
case "show": // show a repository's log directly on the server case "show": // show a repository's log directly on the server
prj := PRJ prj := PRJ
@@ -217,8 +348,8 @@ func runCommandDepth(line string, depth int) bool {
errorln("repository not found") errorln("repository not found")
break break
} }
logLines, _ := sshOut("cd " + shq(cfg.GitPath+"/"+prj+".git") + logLines, _ := sshOut("git --git-dir=" + shq(serverPath(prj+".git")) +
" && git log --reverse --format='%h %ct %s'") " log --reverse --format='%h %ct %s'")
repolog(logLines) repolog(logLines)
case "log": case "log":
@@ -272,19 +403,29 @@ func runCommandDepth(line string, depth int) bool {
} }
comment := strings.Join(fields[1:], " ") comment := strings.Join(fields[1:], " ")
git(DIR, "add", "--all", ".") git(DIR, "add", "--all", ".")
// last look before anything is committed: `add --all` sweeps up whatever
// is lying around, and with mirroring on it goes straight to a public
// server. Nothing has been committed yet, so declining costs nothing.
if !secretsApproved(DIR) {
errorln("push cancelled — your changes are staged but not committed")
break
}
msg := strings.TrimSpace(fmt.Sprintf("[%s@%s] %s", USER, HOST, comment)) msg := strings.TrimSpace(fmt.Sprintf("[%s@%s] %s", USER, HOST, comment))
git(DIR, "commit", "-m", msg) // may be "nothing to commit"; continue anyway git(DIR, "commit", "-m", msg) // may be "nothing to commit"; continue anyway
if !gitOK(DIR, "push") { if !gitOK(DIR, "push") {
break break
} }
sshOK("touch " + shq(cfg.GitPath+"/"+PRJ+".git")) sshOK("touch " + shq(serverPath(PRJ+".git")))
if targets, _ := cfg.mirrorTargets(); truthy(cfg.Mirror) && len(targets) > 0 { if targets, _ := cfg.mirrorTargets(); truthy(cfg.Mirror) && len(activeRemotes(targets)) > 0 {
handlePushRemote("") // auto-mirror to every configured server handlePushRemote("") // auto-mirror to every active server
} }
case "pushremote": // mirror the repo to a public git server via its API case "pushremote": // mirror the repo to a public git server via its API
handlePushRemote(strings.Join(fields[1:], " ")) handlePushRemote(strings.Join(fields[1:], " "))
case "deleteremote": // delete the repo on a public git server via its API
handleDeleteRemote(strings.Join(fields[1:], " "))
case "release": // tag a commit and publish it as a release on the mirrors case "release": // tag a commit and publish it as a release on the mirrors
handleRelease(strings.Join(fields[1:], " ")) handleRelease(strings.Join(fields[1:], " "))
@@ -324,13 +465,17 @@ func runCommandDepth(line string, depth int) bool {
if comment != "" { if comment != "" {
name = PRJ + "_" + z + "_" + comment name = PRJ + "_" + z + "_" + comment
} }
if !sshOK("cp -r " + shq(PRJ+".git") + " " + shq("archive/"+name+".git")) { // the archive directory is mgsh's own convention, so make it rather than
// fail on a server where nobody has created it yet
if !sshOK("mkdir -p " + shq(serverPath("archive")) +
" && cp -r " + shq(serverPath(PRJ+".git")) + " " + shq(serverPath("archive/"+name+".git"))) {
break break
} }
if !sshOK("cd archive && tar cvzf " + shq(name+".git.tar.gz") + " " + shq(name+".git")) { if !sshOK("tar cvzf " + shq(serverPath("archive/"+name+".git.tar.gz")) +
" -C " + shq(serverPath("archive")) + " " + shq(name+".git")) {
break break
} }
sshOK("rm -rf " + shq("archive/"+name+".git")) sshOK("rm -rf " + shq(serverPath("archive/"+name+".git")))
case "init": // create a new repository from the current directory case "init": // create a new repository from the current directory
if !requireProject() { if !requireProject() {
@@ -350,11 +495,13 @@ func runCommandDepth(line string, depth int) bool {
if exists && !yesno("overwrite existing repository "+PRJ+" on the server?", false) { if exists && !yesno("overwrite existing repository "+PRJ+" on the server?", false) {
break break
} }
remote := shq(cfg.GitPath + "/" + PRJ + ".git") remote := shq(serverPath(PRJ + ".git"))
if !sshOK("rm -rf " + remote) { if !sshOK("rm -rf " + remote) {
break break
} }
if !sshOK("mkdir " + remote + " && cd " + remote + " && git --bare init") { // `git init --bare <path>` makes the directory itself: one command, and
// none of it depends on a `cd` having worked
if !sshOK("git init --bare " + remote) {
break break
} }
gi := DIR + "/.gitignore" gi := DIR + "/.gitignore"
@@ -437,13 +584,14 @@ func runCommandDepth(line string, depth int) bool {
break break
} }
} else { } else {
if !sshOK("cd archive && tar xvzf " + shq(prj+".git.tar.gz")) { if !sshOK("tar xvzf " + shq(serverPath("archive/"+prj+".git.tar.gz")) +
" -C " + shq(serverPath("archive"))) {
break break
} }
if !gitOK(BASE, "clone", URL+"/archive/"+prj+".git") { if !gitOK(BASE, "clone", URL+"/archive/"+prj+".git") {
break break
} }
sshOK("rm -rf " + shq("archive/"+prj+".git")) sshOK("rm -rf " + shq(serverPath("archive/"+prj+".git")))
} }
if isDir(BASE + "/" + prj) { if isDir(BASE + "/" + prj) {
PRJ = prj PRJ = prj
@@ -454,7 +602,7 @@ func runCommandDepth(line string, depth int) bool {
if opt["a"] { if opt["a"] {
path = "./archive" path = "./archive"
} }
lines, err := sshOut("/bin/ls " + shq(path)) lines, err := sshOut("/bin/ls " + shq(serverPath(path)))
if err != nil { if err != nil {
errorln("could not list repositories on the git server") errorln("could not list repositories on the git server")
break break
@@ -467,45 +615,6 @@ func runCommandDepth(line string, depth int) bool {
} }
} }
case "open", "view": // open project in Xcode / editor
prj := PRJ
if w := word(words, 1); w != "" {
prj = w
}
d := BASE + "/" + prj
if !validProject(prj) || !isDir(d) {
errorln("not found")
break
}
xws, xprj := "", ""
if entries, err := os.ReadDir(d); err == nil {
for _, e := range entries {
if strings.HasSuffix(e.Name(), ".xcworkspace") {
xws = e.Name()
}
if strings.HasSuffix(e.Name(), ".xcodeproj") {
xprj = e.Name()
}
}
}
// prefer the workspace over the project; fall back to the editor unless
// one of them is really openable (a name match on a plain file is not).
switch {
case xws != "" && isDir(d+"/"+xws):
runInDir(d, "open", xws)
case xprj != "" && isDir(d+"/"+xprj):
runInDir(d, "open", xprj)
default:
editor := cfg.Editor
if editor == "" {
editor = "coda"
}
runInDir(d, editor, d)
}
if words[0] == "open" {
PRJ = prj
}
case "count": // count source lines in the project case "count": // count source lines in the project
if !requireProject() { if !requireProject() {
break break
@@ -543,7 +652,7 @@ func runCommandDepth(line string, depth int) bool {
default: // unknown command — no longer forwarded to the shell default: // unknown command — no longer forwarded to the shell
fmt.Println(col(cRed, "unknown command: "+words[0]) + fmt.Println(col(cRed, "unknown command: "+words[0]) +
col(cGray, " (prefix with '!' to run a shell command)")) col(cDark, " (prefix with '!' to run a shell command)"))
} }
return true return true
@@ -604,9 +713,9 @@ func formatLog(lines []string, now time.Time) string {
n++ n++
} }
fmt.Fprintf(&full, "%s %s %s\n", col(cPurple, hash), col(cYellow, z), subj) fmt.Fprintf(&full, "%s %s %s\n", col(cViolet, hash), col(cYellow, z), subj)
fmt.Fprintf(&short, "%s %s %s\n", col(cPurple, hash), col(cYellow, zs), subj) fmt.Fprintf(&short, "%s %s %s\n", col(cViolet, hash), col(cYellow, zs), subj)
fmt.Fprintf(&tiny, "%s %s %s\n", col(cPurple, hash), col(cYellow, zss), subj) fmt.Fprintf(&tiny, "%s %s %s\n", col(cViolet, hash), col(cYellow, zss), subj)
} }
switch { switch {
@@ -703,15 +812,15 @@ const gitignore = `.DS_Store
var helpItems = []struct{ cmd, desc string }{ var helpItems = []struct{ cmd, desc string }{
{"cd [project]", "change project (no argument: back to the base)"}, {"cd [project]", "change project (no argument: back to the base)"},
{"open [project]", "open project"},
{"init", "make new repository from current directory"}, {"init", "make new repository from current directory"},
{"push [comment]", "push changes to git server"}, {"push [comment]", "push changes to git server"},
{"pushremote [@name] [desc]", "mirror repo to the public server(s) (gitea/github/gitlab)"}, {"pushremote [@name] [desc]", "mirror repo to the public server(s) (gitea/github/gitlab)"},
{"deleteremote <@name|host>", "delete the repo on that public server (asks first)"},
{"release <tag> [notes]", "tag and publish a release on the public server(s)"}, {"release <tag> [notes]", "tag and publish a release on the public server(s)"},
{"pull", "pull changes from git server"}, {"pull", "pull changes from git server"},
{"fetch", "fetch changes from git server"}, {"fetch", "fetch changes from git server"},
{"status [-a]", "short git status (-a: overview of all projects)"}, {"status [-a]", "short git status (-a: overview of all projects)"},
{"overview", "status of all projects (dirty, ahead/behind)"}, {"overview", "inventory of all projects, local and on the server"},
{"diff [args]", "show git diff"}, {"diff [args]", "show git diff"},
{"edit [number]", "edit last [number] commits (default is 10)"}, {"edit [number]", "edit last [number] commits (default is 10)"},
{"clone [-a] <repository>", "clone repository from git server (-a for archive)"}, {"clone [-a] <repository>", "clone repository from git server (-a for archive)"},
@@ -732,6 +841,7 @@ var helpItems = []struct{ cmd, desc string }{
{"unalias <name>", "remove an alias"}, {"unalias <name>", "remove an alias"},
{"config [-k]", "show effective configuration (-k: list all setting names)"}, {"config [-k]", "show effective configuration (-k: list all setting names)"},
{"rescan", "reload config and refresh cached server repository list"}, {"rescan", "reload config and refresh cached server repository list"},
{"update [-c]", "update mgsh to the newest release (-c: only look)"},
{"!<command>", "run <command> in the shell"}, {"!<command>", "run <command> in the shell"},
{"quit", "exit mgsh"}, {"quit", "exit mgsh"},
} }
completion.go
+45 -14
View File
@@ -4,19 +4,54 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"unicode/utf8"
"github.com/chzyer/readline" "github.com/chzyer/readline"
) )
// completer wires up Tab completion. Command names complete at the start of the // completer wires up Tab completion. A line headed for a shell — a '!' escape,
// line; cd/open/view complete local project names; clone/show complete // or an alias that expands to one — is completed the way a shell would:
// executables for the command, paths for its arguments. Everything else goes to
// the builtin command tree.
func completer() readline.AutoCompleter {
return &mgshCompleter{builtin: builtinCompleter()}
}
// mgshCompleter dispatches between the two completion worlds.
type mgshCompleter struct{ builtin *readline.PrefixCompleter }
func (c *mgshCompleter) Do(line []rune, pos int) ([][]rune, int) {
if pos > len(line) {
pos = len(line)
}
if cands, prefix, ok := completeShellLine(string(line[:pos])); ok {
return runeSuffixes(cands, prefix)
}
return c.builtin.Do(line, pos)
}
// runeSuffixes converts full candidate words into what readline wants: the part
// still missing after the prefix already typed, plus that prefix's length.
func runeSuffixes(cands []string, prefix string) ([][]rune, int) {
n := utf8.RuneCountInString(prefix)
out := make([][]rune, 0, len(cands))
for _, c := range cands {
r := []rune(c)
if len(r) >= n {
out = append(out, r[n:])
}
}
return out, n
}
// builtinCompleter is the command tree. Command names complete at the start of
// the line; cd completes local project names; clone/show complete
// repository names cached from the git server; checkout/tag complete branch and // repository names cached from the git server; checkout/tag complete branch and
// tag names; dist completes filesystem paths. // tag names; pushremote/release complete mirror targets; dist completes
func completer() *readline.PrefixCompleter { // filesystem paths.
func builtinCompleter() *readline.PrefixCompleter {
return readline.NewPrefixCompleter( return readline.NewPrefixCompleter(
readline.PcItem("cd", readline.PcItemDynamic(dynLocalProjects)), readline.PcItem("cd", readline.PcItemDynamic(dynLocalProjects)),
readline.PcItem("open", readline.PcItemDynamic(dynLocalProjects)),
readline.PcItem("view", readline.PcItemDynamic(dynLocalProjects)),
readline.PcItem("clone", readline.PcItem("clone",
readline.PcItem("-a", readline.PcItemDynamic(dynServerArchives)), readline.PcItem("-a", readline.PcItemDynamic(dynServerArchives)),
readline.PcItemDynamic(dynServerRepos), readline.PcItemDynamic(dynServerRepos),
@@ -26,6 +61,7 @@ func completer() *readline.PrefixCompleter {
readline.PcItem("list", readline.PcItem("-a")), readline.PcItem("list", readline.PcItem("-a")),
readline.PcItem("push"), readline.PcItem("push"),
readline.PcItem("pushremote", readline.PcItemDynamic(dynRemoteNames)), readline.PcItem("pushremote", readline.PcItemDynamic(dynRemoteNames)),
readline.PcItem("deleteremote", readline.PcItemDynamic(dynRemoteNames)),
readline.PcItem("release", readline.PcItemDynamic(dynRemoteNames)), readline.PcItem("release", readline.PcItemDynamic(dynRemoteNames)),
readline.PcItem("pull"), readline.PcItem("pull"),
readline.PcItem("fetch"), readline.PcItem("fetch"),
@@ -49,6 +85,7 @@ func completer() *readline.PrefixCompleter {
readline.PcItem("unalias", readline.PcItemDynamic(dynAliasNames)), readline.PcItem("unalias", readline.PcItemDynamic(dynAliasNames)),
readline.PcItem("config", readline.PcItem("-k")), readline.PcItem("config", readline.PcItem("-k")),
readline.PcItem("rescan"), readline.PcItem("rescan"),
readline.PcItem("update", readline.PcItem("-c")),
readline.PcItem("help"), readline.PcItem("help"),
readline.PcItem("quit"), readline.PcItem("quit"),
readline.PcItem("exit"), readline.PcItem("exit"),
@@ -82,21 +119,15 @@ func fetchServerRepos() {
if serverFetched { if serverFetched {
return return
} }
lines, err := sshOut("/bin/ls .") repos, err := serverRepoNames()
if err != nil { if err != nil {
// a transient failure (server down, no network) must not cache an // a transient failure (server down, no network) must not cache an
// empty list for the rest of the session — the next Tab tries again // empty list for the rest of the session — the next Tab tries again
return return
} }
var repos []string
for _, ln := range lines {
if m := gitDirRe.FindStringSubmatch(strings.TrimSpace(ln)); m != nil {
repos = append(repos, m[1])
}
}
// a missing ./archive is a permanent, unremarkable state: still cache // a missing ./archive is a permanent, unremarkable state: still cache
var archives []string var archives []string
if lines, err := sshOut("/bin/ls archive"); err == nil { if lines, err := sshOut("/bin/ls " + shq(serverPath("archive"))); err == nil {
for _, ln := range lines { for _, ln := range lines {
t := strings.TrimSpace(ln) t := strings.TrimSpace(ln)
if strings.HasSuffix(t, ".git.tar.gz") { if strings.HasSuffix(t, ".git.tar.gz") {
config.go
+165 -40
View File
@@ -6,6 +6,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"regexp" "regexp"
"slices"
"sort" "sort"
"strings" "strings"
) )
@@ -32,12 +33,8 @@ type Config struct {
GitName string // git user.name to set globally ("" = leave alone) GitName string // git user.name to set globally ("" = leave alone)
GitEmail string // git user.email to set globally ("" = leave alone) GitEmail string // git user.email to set globally ("" = leave alone)
PushDefault string // git push.default to set globally ("" = leave alone) PushDefault string // git push.default to set globally ("" = leave alone)
Editor string // editor/opener used as fallback by `open` ("" = coda)
RemoteURL string // public mirror server base URL (Gitea/GitHub/GitLab)
RemoteKey string // API token for the mirror server (used by `pushremote`)
RemoteType string // "gitea"|"github"|"gitlab" (auto-detected when empty)
RemoteVis string // visibility of created repos: "private" (default)|"public"
Mirror string // truthy -> `push` also mirrors via `pushremote` Mirror string // truthy -> `push` also mirrors via `pushremote`
SecretScan string // falsy -> `push` skips the credential scan
Remotes []RemoteTarget Remotes []RemoteTarget
RemoteNames string // "remotes": explicit, ordered subset of targets to use RemoteNames string // "remotes": explicit, ordered subset of targets to use
} }
@@ -46,29 +43,42 @@ type Config struct {
// `remote.<name>.<field>` block. Name doubles as the git remote name created in // `remote.<name>.<field>` block. Name doubles as the git remote name created in
// the repository, so several targets can coexist side by side. // the repository, so several targets can coexist side by side.
type RemoteTarget struct { type RemoteTarget struct {
Name string Name string
URL string URL string
Key string Key string
Type string // "gitea"|"github"|"gitlab" (auto-detected when empty) Type string // "gitea"|"github"|"gitlab" (auto-detected when empty)
Vis string // "private" (default) | "public" Vis string // "private" (default) | "public"
Active string // falsy -> taken along only when named (default: on)
} }
// legacyRemoteName is the target name for the flat remoteurl/remotekey pair, // isActive reports whether the target takes part in a command that was given no
// matching the git remote that earlier versions created. // target of its own — `pushremote`, `release`, an auto-mirroring `push`. An
// inactive one is not disabled, it is merely off the automatic path: naming it
// (`pushremote @gitea`) uses it as it always did.
//
// Unset means active. The setting exists to take a server *out* of the default
// set, so a configuration written before it existed must keep behaving.
func (t RemoteTarget) isActive() bool { return !falsy(t.Active) }
// legacyRemoteName is the target the pre-4.1 flat remoteurl/remotekey settings
// are migrated to. It matches the git remote those versions created, so a
// converted configuration keeps pushing to the same place.
const legacyRemoteName = "public" const legacyRemoteName = "public"
// legacyRemoteKeys maps the old flat spelling onto the named-target form. A
// mirror target is defined one way now, not two.
var legacyRemoteKeys = map[string]string{
"remoteurl": "remote." + legacyRemoteName + ".url",
"remotekey": "remote." + legacyRemoteName + ".key",
"remotetype": "remote." + legacyRemoteName + ".type",
"remotevisibility": "remote." + legacyRemoteName + ".visibility",
}
// mirrorTargets returns the usable mirror targets in configured order, plus the // mirrorTargets returns the usable mirror targets in configured order, plus the
// names of targets that are defined but unusable (missing url or key) so the // names of targets that are defined but unusable (missing url or key) so the
// caller can complain about them instead of silently skipping. // caller can complain about them instead of silently skipping.
func (c Config) mirrorTargets() (usable []RemoteTarget, incomplete []string) { func (c Config) mirrorTargets() (usable []RemoteTarget, incomplete []string) {
var all []RemoteTarget all := c.Remotes
if c.RemoteURL != "" || c.RemoteKey != "" {
all = append(all, RemoteTarget{
Name: legacyRemoteName, URL: c.RemoteURL, Key: c.RemoteKey,
Type: c.RemoteType, Vis: c.RemoteVis,
})
}
all = append(all, c.Remotes...)
// `remotes = a, b` narrows and orders the set — a project .mgshrc uses it // `remotes = a, b` narrows and orders the set — a project .mgshrc uses it
// to mirror to only some of the globally configured servers. // to mirror to only some of the globally configured servers.
@@ -146,11 +156,54 @@ func loadConfig() Config {
m := parseConfig(string(data)) m := parseConfig(string(data))
applyConfig(&c, m) applyConfig(&c, m)
warnConfigPerms(path, m) warnConfigPerms(path, m)
migrateRemoteKeys(path, string(data))
} }
applyEnv(&c) applyEnv(&c)
return c return c
} }
// migrateRemoteKeys converts the pre-4.1 flat remote settings in a config file
// to the remote.<name>.<field> spelling, so a mirror target is defined one way
// and not two. Only the key is rewritten: values, comments, blank lines and the
// file's permissions stay exactly as they are, and commented-out lines are left
// alone. Reports what it changed rather than doing it silently.
func migrateRemoteKeys(path, data string) {
lines := strings.Split(data, "\n")
var renamed []string
for i, ln := range lines {
trimmed := strings.TrimLeft(ln, " \t")
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
continue
}
sep := strings.IndexAny(trimmed, "=:")
if sep < 0 {
continue
}
key := strings.TrimRight(trimmed[:sep], " \t")
dotted, ok := legacyRemoteKeys[strings.ToLower(key)]
if !ok {
continue
}
indent := ln[:len(ln)-len(trimmed)]
gap := trimmed[len(key):sep] // whatever alignment was there
lines[i] = indent + dotted + gap + trimmed[sep:]
renamed = append(renamed, key+" → "+dotted)
}
if len(renamed) == 0 {
return
}
if err := os.WriteFile(path, []byte(strings.Join(lines, "\n")), configMode); err != nil {
errorln("could not update " + path + ": " + err.Error())
return
}
fmt.Println(col(cDark, path+": mirror settings renamed to the remote.<name>.* form"))
for _, r := range renamed {
fmt.Println(col(cDark, " "+r))
}
}
// projectGlobalOnly lists settings a project-level .mgshrc must not change: // projectGlobalOnly lists settings a project-level .mgshrc must not change:
// `base` decides where projects live in the first place, and the git identity // `base` decides where projects live in the first place, and the git identity
// keys are written to the user's *global* git config at startup — applying // keys are written to the user's *global* git config at startup — applying
@@ -276,15 +329,17 @@ func writeConfigTemplate(path string) {
b.WriteString("# gitname = Your Name\n") b.WriteString("# gitname = Your Name\n")
b.WriteString("# gitemail = you@example.com\n") b.WriteString("# gitemail = you@example.com\n")
b.WriteString("# pushdefault = matching\n") b.WriteString("# pushdefault = matching\n")
b.WriteString("# editor = code\n\n") b.WriteString("\n")
b.WriteString("# --- public mirrors for `pushremote` ---\n") b.WriteString("# --- public mirrors for `pushremote` ---\n")
b.WriteString("# One block per server; `pushremote` pushes to all of them,\n") b.WriteString("# One 'remote.<name>.*' block per server. `pushremote` pushes to all\n")
b.WriteString("# `pushremote @hub` to a single one.\n") b.WriteString("# of them, `pushremote @gitlab` to a single one. <name> is also the\n")
b.WriteString("# remote.hub.url = https://github.com\n") b.WriteString("# git remote created in the repository.\n")
b.WriteString("# remote.hub.key = <personal-access-token>\n") b.WriteString("# remote.gitlab.url = https://gitlab.example.com\n")
b.WriteString("# remote.hub.visibility = public\n") b.WriteString("# remote.gitlab.key = <personal-access-token>\n")
b.WriteString("# remotes = hub # optional: restrict/order the set\n") b.WriteString("# remote.gitlab.type = gitlab # optional; detected from the url\n")
b.WriteString("# mirror = true # `push` also mirrors\n\n") b.WriteString("# remote.gitlab.visibility = private # or public (default private)\n")
b.WriteString("# remotes = gitlab # optional: restrict/order the set\n")
b.WriteString("# mirror = true # `push` also mirrors\n\n")
b.WriteString("# A project may override any of these (except base and the git\n") b.WriteString("# A project may override any of these (except base and the git\n")
b.WriteString("# identity) in its own <project>/.mgshrc.\n") b.WriteString("# identity) in its own <project>/.mgshrc.\n")
@@ -363,18 +418,40 @@ func applyConfig(c *Config, m map[string]string) {
set("gitname", &c.GitName) set("gitname", &c.GitName)
set("gitemail", &c.GitEmail) set("gitemail", &c.GitEmail)
set("pushdefault", &c.PushDefault) set("pushdefault", &c.PushDefault)
set("editor", &c.Editor)
set("remoteurl", &c.RemoteURL)
set("remotekey", &c.RemoteKey)
set("remotetype", &c.RemoteType)
set("remotevisibility", &c.RemoteVis)
set("remotes", &c.RemoteNames) set("remotes", &c.RemoteNames)
set("mirror", &c.Mirror) set("mirror", &c.Mirror)
applyRemoteTargets(c, m) set("secretscan", &c.SecretScan)
applyRemoteTargets(c, foldLegacyRemoteKeys(m))
}
// foldLegacyRemoteKeys rewrites the pre-4.1 flat remote settings into the
// named-target form, so a configuration that has not been converted yet still
// works while it is being read. The file itself is converted by
// migrateRemoteKeys; this only makes the current run behave.
func foldLegacyRemoteKeys(m map[string]string) map[string]string {
folded, copied := m, false
for old, dotted := range legacyRemoteKeys {
v, ok := m[old]
if !ok || v == "" {
continue
}
if _, taken := m[dotted]; taken {
continue // an explicit new-style setting always wins
}
if !copied { // copy on first write, never touch the caller's map
folded = make(map[string]string, len(m))
for k, val := range m {
folded[k] = val
}
copied = true
}
folded[dotted] = v
}
return folded
} }
// remoteFieldRe matches a named mirror target setting: remote.<name>.<field>. // remoteFieldRe matches a named mirror target setting: remote.<name>.<field>.
var remoteFieldRe = regexp.MustCompile(`^remote\.([a-z0-9_.-]+)\.(url|key|type|visibility)$`) var remoteFieldRe = regexp.MustCompile(`^remote\.([a-z0-9_.-]+)\.(url|key|type|visibility|active)$`)
// applyRemoteTargets merges `remote.<name>.<field>` settings into c.Remotes. // applyRemoteTargets merges `remote.<name>.<field>` settings into c.Remotes.
// An already known target is updated field by field, so a project .mgshrc can // An already known target is updated field by field, so a project .mgshrc can
@@ -405,6 +482,8 @@ func applyRemoteTargets(c *Config, m map[string]string) {
t.Type = v t.Type = v
case "visibility": case "visibility":
t.Vis = v t.Vis = v
case "active":
t.Active = v
} }
} }
} }
@@ -436,11 +515,57 @@ func applyEnv(c *Config) {
env("MGSH_GITNAME", &c.GitName) env("MGSH_GITNAME", &c.GitName)
env("MGSH_GITEMAIL", &c.GitEmail) env("MGSH_GITEMAIL", &c.GitEmail)
env("MGSH_PUSHDEFAULT", &c.PushDefault) env("MGSH_PUSHDEFAULT", &c.PushDefault)
env("MGSH_EDITOR", &c.Editor)
env("MGSH_REMOTEURL", &c.RemoteURL)
env("MGSH_REMOTEKEY", &c.RemoteKey)
env("MGSH_REMOTETYPE", &c.RemoteType)
env("MGSH_REMOTEVISIBILITY", &c.RemoteVis)
env("MGSH_REMOTES", &c.RemoteNames) env("MGSH_REMOTES", &c.RemoteNames)
env("MGSH_MIRROR", &c.Mirror) env("MGSH_MIRROR", &c.Mirror)
env("MGSH_SECRETSCAN", &c.SecretScan)
applyRemoteEnv(c)
}
// remoteFields are the settings a mirror target is made of.
var remoteFields = []string{"url", "key", "type", "visibility", "active"}
// applyRemoteEnv reads MGSH_REMOTE_<NAME>_<FIELD>, the environment spelling of
// a remote.<name>.<field> setting — MGSH_REMOTE_GITLAB_KEY for
// remote.gitlab.key. The field is taken from the end, so a target name may
// contain underscores itself.
func applyRemoteEnv(c *Config) {
const prefix = "MGSH_REMOTE_"
// sorted, so a target these variables introduce lands in the push order the
// same way on every run
envs := os.Environ()
sort.Strings(envs)
for _, kv := range envs {
eq := strings.IndexByte(kv, '=')
if eq < 0 {
continue
}
name, value := kv[:eq], kv[eq+1:]
if value == "" || !strings.HasPrefix(name, prefix) {
continue
}
rest := name[len(prefix):]
us := strings.LastIndexByte(rest, '_')
if us <= 0 {
continue
}
target, field := strings.ToLower(rest[:us]), strings.ToLower(rest[us+1:])
if !slices.Contains(remoteFields, field) {
continue // MGSH_REMOTES and anything else that merely starts alike
}
t := &c.Remotes[c.remoteIndex(target)]
switch field {
case "url":
t.URL = value
case "key":
t.Key = value
case "type":
t.Type = value
case "visibility":
t.Vis = value
case "active":
t.Active = value
}
}
} }
deleteremote.go
+184
View File
@@ -0,0 +1,184 @@
package main
// deleteremote.go — the `deleteremote` command: remove the active project's
// repository from a public mirror server via that server's REST API.
//
// This is the only mgsh command that destroys something on a server it does not
// own, and nothing here can undo it. Four things keep it in check:
//
// - the server has to be named. `pushremote` without one means "every
// configured target", which is convenient there and catastrophic here.
// - the repository is looked up first, so a typo comes back as "not on that
// server" instead of as a DELETE nobody meant to send.
// - every deletion is confirmed on its own, spelling out owner/repo and the
// server it lives on, and the default answer is no.
// - the local clone is never touched. Only the git remote `pushremote`
// created is dropped, and only while it still points at what was deleted.
import (
"fmt"
"strings"
)
// deleteRepo removes owner/repo from the server.
//
// The providers disagree on what success looks like: Gitea and GitHub answer
// 204, GitLab 202 — there the deletion is merely scheduled, and depending on
// the plan the project stays visible until its retention period is over.
func (r *remoteAPI) deleteRepo(owner, repo string) error {
code, data, err := r.do("DELETE", r.repoPath(owner, repo), nil)
if err != nil {
return err
}
switch code {
case 200, 202, 204:
return nil
case 403:
// the common failure by far: the token can push and create, but was
// never given the separate permission a deletion needs
return fmt.Errorf("not allowed to delete %s/%s — the token needs %s (HTTP 403): %s",
owner, repo, r.deletePermission(), firstLine(data))
default:
return fmt.Errorf("deleting repository failed (HTTP %d): %s", code, firstLine(data))
}
}
// deletePermission names what a provider wants from a token before it will
// delete a repository, because a bare "403" sends people to the wrong setting.
func (r *remoteAPI) deletePermission() string {
switch r.kind {
case kindGitHub:
return "the 'delete_repo' scope"
case kindGitLab:
return "the 'api' scope and the Owner role"
default: // Gitea
return "the 'write:repository' scope"
}
}
// parseRemoteSelectors splits `deleteremote <name|host> ...` into the targets
// it names. The '@' that `pushremote` requires is accepted but optional here:
// the command takes nothing but selectors, so there is no description a bare
// word could be mistaken for.
func parseRemoteSelectors(args string) []string {
var out []string
for _, f := range strings.Fields(args) {
if s := strings.TrimPrefix(f, "@"); s != "" {
out = append(out, s)
}
}
return out
}
// handleDeleteRemote implements `deleteremote <@name|host> ...`: it deletes the
// active project's repository on each named mirror server.
func handleDeleteRemote(args string) {
if !requireProject() {
return
}
targets, incomplete := cfg.mirrorTargets()
for _, n := range incomplete {
errorln("remote " + n + ": url or key missing — skipped")
}
if len(targets) == 0 {
errorln("deleteremote needs a 'remote.<name>.url' and 'remote.<name>.key' in " + configFile())
return
}
// no selector is never "all of them" — that is the whole point of the
// command taking one
sels := parseRemoteSelectors(args)
if len(sels) == 0 {
errorln("deleteremote needs the server to delete from: " + remoteChoices(targets))
return
}
targets = pickRemotes(targets, sels)
if len(targets) == 0 {
return // pickRemotes already named the selectors it did not recognise
}
repo := PRJ // like pushremote: the project names the repository
done := 0
for _, t := range targets {
if deleteOnRemote(t, repo) {
done++
}
}
if len(targets) > 1 {
fmt.Printf("%s %d/%d repositories deleted\n", col(cDark, "deleteremote:"), done, len(targets))
}
}
// remoteChoices lists the configured targets the way they may be selected, for
// the message a `deleteremote` without a target earns.
func remoteChoices(targets []RemoteTarget) string {
out := make([]string, 0, len(targets))
for _, t := range targets {
if h := remoteHost(t.URL); h != "" && !strings.EqualFold(h, t.Name) {
out = append(out, "@"+t.Name+" ("+h+")")
continue
}
out = append(out, "@"+t.Name)
}
return strings.Join(out, ", ")
}
// deleteOnRemote deletes repo on one target, after asking. It reports whether
// something was actually deleted — a repository that is not there, and a
// question answered with no, are not failures, and neither stops the remaining
// targets.
func deleteOnRemote(t RemoteTarget, repo string) bool {
api := newRemoteAPI(t.URL, t.Key, t.Type)
owner, err := api.authUser()
if err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
fmt.Printf("%s %s %s (as %s)\n",
col(cDark, "remote"), col(cYellow, t.Name), col(cBlue, api.url), col(cGreen, owner))
exists, err := api.repoExists(owner, repo)
if err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
if !exists {
fmt.Println(col(cDark, "no repository "+owner+"/"+repo+" there — nothing to delete"))
return false
}
fmt.Println(col(cOrange, "this deletes "+api.repoWebURL(owner, repo)+
" with its issues, releases and history, and cannot be undone"))
if !yesno("delete "+owner+"/"+repo+" on "+t.Name+"?", false) {
fmt.Println(col(cDark, "kept"))
return false
}
if err := api.deleteRepo(owner, repo); err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
fmt.Println(col(cRed, "deleted ") + col(cWhite, owner+"/"+repo) + col(cDark, " on "+t.Name))
dropGitRemote(t.Name, api.repoWebURL(owner, repo))
return true
}
// dropGitRemote removes the local git remote named name, but only while it
// still points at web — the repository just deleted. A remote the user has
// since re-aimed somewhere else is theirs, not ours, and a `git push` failing
// against a repository that no longer exists is worse than no remote at all.
func dropGitRemote(name, web string) {
if !isDir(DIR + "/.git") {
return
}
url, err := gitCapture(DIR, "remote", "get-url", name)
if err != nil || strings.TrimSpace(url) != web {
return
}
if gitOK(DIR, "remote", "remove", name) {
fmt.Println(col(cDark, "removed git remote "+name))
}
}
deleteremote_test.go
+266
View File
@@ -0,0 +1,266 @@
package main
// deleteremote_test.go — the command that deletes a repository on a server mgsh
// does not own.
//
// Nothing it does can be undone from here, so the tests are less about the happy
// path than about the ways it must refuse: no target named, an unknown target, a
// question answered with no, a repository that is not there. Each of those has
// to leave the server untouched, which is asserted on the requests the fake
// provider received — the DELETE either went out or it did not.
import (
"strings"
"testing"
)
// mirrorServer is a fake Gitea holding one repository owned by "mike".
func mirrorServer(t *testing.T, project string) *fakeProvider {
t.Helper()
f := newFakeProvider(t)
f.route("GET /api/v1/user", 200, `{"login":"mike"}`)
f.route("GET /api/v1/repos/mike/"+project, 200, `{}`)
f.route("DELETE /api/v1/repos/mike/"+project, 204, "")
return f
}
// mirrorProject points the globals at a project with f as its single mirror
// target, and answers every y/n question with answer. It returns the project
// directory and the recorded questions.
func mirrorProject(t *testing.T, f *fakeProvider, name string, answer bool) (string, *[]string) {
t.Helper()
dir := useProject(t, name)
cfg.Remotes = []RemoteTarget{{Name: "gitea", URL: f.URL, Key: "tok", Type: "gitea"}}
return dir, fakeAnswers(t, answer)
}
// deleted reports whether the DELETE for project actually went to the server.
func deleted(f *fakeProvider, project string) bool {
return f.find("DELETE /api/v1/repos/mike/"+project) != nil
}
// TestDeleteRepoPerProvider pins the request each provider expects. GitLab
// answers 202 — the deletion is only scheduled — which still counts as success.
func TestDeleteRepoPerProvider(t *testing.T) {
for _, c := range []struct {
typ, path string
code int
}{
{"gitea", "/api/v1/repos/mike/mgsh", 204},
{"github", "/api/v3/repos/mike/mgsh", 204},
{"gitlab", "/api/v4/projects/mike%2Fmgsh", 202},
} {
f := newFakeProvider(t)
f.route("DELETE "+c.path, c.code, "")
if err := newRemoteAPI(f.URL, "tok", c.typ).deleteRepo("mike", "mgsh"); err != nil {
t.Fatalf("%s: deleteRepo: %v (requests: %v)", c.typ, err, f.paths())
}
if f.find("DELETE "+c.path) == nil {
t.Errorf("%s: expected DELETE %s, got %v", c.typ, c.path, f.paths())
}
}
}
// TestDeleteRepoNamesTheMissingPermission: deleting needs more of a token than
// pushing does, so a 403 here is the likeliest failure of all. Passing the bare
// status on would send people to the wrong setting.
func TestDeleteRepoNamesTheMissingPermission(t *testing.T) {
f := newFakeProvider(t)
f.route("DELETE /api/v3/repos/mike/mgsh", 403, `{"message":"Must have admin rights"}`)
err := newRemoteAPI(f.URL, "tok", "github").deleteRepo("mike", "mgsh")
if err == nil || !strings.Contains(err.Error(), "delete_repo") {
t.Errorf("403 error = %v, want it to name the delete_repo scope", err)
}
}
// TestDeleteRepoTreatsUnroutedAsFailure: anything the server did not accept must
// surface as an error, or a repository that is still there looks deleted.
func TestDeleteRepoReportsFailure(t *testing.T) {
f := newFakeProvider(t) // everything 404s
if err := newRemoteAPI(f.URL, "tok", "gitea").deleteRepo("mike", "mgsh"); err == nil {
t.Error("deleteRepo accepted a 404 as success")
}
}
// TestDeleteRemoteNeedsATarget is the one that matters most: `pushremote` with
// no @name means "every configured server", and inheriting that here would wipe
// the project off all of them at once.
func TestDeleteRemoteNeedsATarget(t *testing.T) {
f := mirrorServer(t, "notes")
_, asked := mirrorProject(t, f, "notes", true) // even a standing "yes"
out := captureStdout(t, func() { runCommand("deleteremote") })
if len(f.got) != 0 {
t.Fatalf("deleteremote talked to the server without being told where: %v", f.paths())
}
if len(*asked) != 0 {
t.Errorf("asked %q although no target was named", *asked)
}
if !strings.Contains(out, "@gitea") {
t.Errorf("output does not say which targets exist: %q", out)
}
}
// TestDeleteRemoteRejectsUnknownTarget: a mistyped server name must not fall
// back to some other target.
func TestDeleteRemoteRejectsUnknownTarget(t *testing.T) {
f := mirrorServer(t, "notes")
_, asked := mirrorProject(t, f, "notes", true)
out := captureStdout(t, func() { runCommand("deleteremote @gitae") })
if len(f.got) != 0 {
t.Fatalf("an unknown target still reached a server: %v", f.paths())
}
if len(*asked) != 0 {
t.Errorf("asked %q for an unknown target", *asked)
}
if !strings.Contains(out, "gitae") {
t.Errorf("unknown target not reported: %q", out)
}
}
// TestDeleteRemoteKeepsRepoWhenDeclined: the question is the last guard, so a
// "no" has to stop the DELETE, not just the message about it.
func TestDeleteRemoteKeepsRepoWhenDeclined(t *testing.T) {
f := mirrorServer(t, "notes")
_, asked := mirrorProject(t, f, "notes", false)
captureStdout(t, func() { runCommand("deleteremote @gitea") })
if len(*asked) != 1 {
t.Fatalf("questions asked = %q, want exactly one", *asked)
}
if !strings.Contains((*asked)[0], "mike/notes") {
t.Errorf("question %q does not name the repository being deleted", (*asked)[0])
}
if deleted(f, "notes") {
t.Fatalf("deleted the repository after the user declined: %v", f.paths())
}
}
// TestDeleteRemoteWithoutRepositoryAsksNothing: a project that was never
// mirrored (or a typo in the project name) is a no-op, not a question.
func TestDeleteRemoteWithoutRepositoryAsksNothing(t *testing.T) {
f := newFakeProvider(t)
f.route("GET /api/v1/user", 200, `{"login":"mike"}`) // the repo lookup 404s
_, asked := mirrorProject(t, f, "notes", true)
out := captureStdout(t, func() { runCommand("deleteremote @gitea") })
if len(*asked) != 0 {
t.Errorf("asked %q about a repository that is not there", *asked)
}
if deleted(f, "notes") {
t.Error("sent a DELETE for a repository the server does not have")
}
if !strings.Contains(out, "nothing to delete") {
t.Errorf("output = %q, want it to say there is nothing to delete", out)
}
}
// TestDeleteRemoteByHostAndGitRemoteCleanup covers the whole accepted path: the
// server picked by its host rather than its configured name, the DELETE sent,
// and the now-dangling git remote dropped — while an unrelated remote stays.
func TestDeleteRemoteByHostAndGitRemoteCleanup(t *testing.T) {
f := mirrorServer(t, "notes")
dir, asked := mirrorProject(t, f, "notes", true)
mustGit(t, dir, "init", "-q")
mustGit(t, dir, "remote", "add", "gitea", f.URL+"/mike/notes.git")
mustGit(t, dir, "remote", "add", "origin", "git@git.example:notes.git")
captureStdout(t, func() { runCommand("deleteremote " + remoteHost(f.URL)) })
if len(*asked) != 1 {
t.Fatalf("questions asked = %q, want exactly one", *asked)
}
if !deleted(f, "notes") {
t.Fatalf("no DELETE sent, requests: %v", f.paths())
}
remotes, err := gitCapture(dir, "remote")
if err != nil {
t.Fatal(err)
}
names := splitLines(strings.TrimSpace(remotes))
for _, n := range names {
if n == "gitea" {
t.Errorf("git remote gitea survived the deletion: %q", names)
}
}
if len(names) != 1 || names[0] != "origin" {
t.Errorf("git remotes = %q, want origin left alone", names)
}
}
// TestDeleteRemoteKeepsARepointedGitRemote: the local remote is only dropped
// while it still points at what was deleted. One the user has since aimed
// somewhere else is theirs.
func TestDeleteRemoteKeepsARepointedGitRemote(t *testing.T) {
f := mirrorServer(t, "notes")
dir, _ := mirrorProject(t, f, "notes", true)
mustGit(t, dir, "init", "-q")
mustGit(t, dir, "remote", "add", "gitea", "https://elsewhere.example/mike/notes.git")
captureStdout(t, func() { runCommand("deleteremote @gitea") })
if !deleted(f, "notes") {
t.Fatalf("no DELETE sent, requests: %v", f.paths())
}
url, err := gitCapture(dir, "remote", "get-url", "gitea")
if err != nil {
t.Fatalf("git remote gitea was removed although it pointed elsewhere: %v", err)
}
if strings.TrimSpace(url) != "https://elsewhere.example/mike/notes.git" {
t.Errorf("git remote url = %q, want it untouched", strings.TrimSpace(url))
}
}
// TestRemoteMatches: a target answers to its configured name and to the host of
// its url, in any case and with or without the '@'.
func TestRemoteMatches(t *testing.T) {
tgt := RemoteTarget{Name: "gitea", URL: "https://git.example.com:3000/root"}
for _, c := range []struct {
sel string
want bool
}{
{"gitea", true},
{"GITEA", true},
{"@gitea", true},
{"git.example.com", true},
{"https://git.example.com/mike/x", true},
{"git@git.example.com:mike/x.git", true},
{"example.com", false}, // a suffix is not the host
{"other", false},
{"@", false},
{"", false},
} {
if got := remoteMatches(tgt, c.sel); got != c.want {
t.Errorf("remoteMatches(%q) = %v, want %v", c.sel, got, c.want)
}
}
}
// TestParseRemoteSelectors: everything on the line selects a server, '@' or not
// — the command takes nothing else that a bare word could be confused with.
func TestParseRemoteSelectors(t *testing.T) {
for _, c := range []struct {
in string
want []string
}{
{"", nil},
{"@gitea", []string{"gitea"}},
{"git.example.com", []string{"git.example.com"}},
{"@gitea git.example.com", []string{"gitea", "git.example.com"}},
{"@", nil},
} {
got := parseRemoteSelectors(c.in)
if strings.Join(got, ",") != strings.Join(c.want, ",") {
t.Errorf("parseRemoteSelectors(%q) = %v, want %v", c.in, got, c.want)
}
}
}
git.go
+63 -3
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"errors"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -127,8 +128,30 @@ var sshExec = func(remote string, capture bool) ([]byte, error) {
return out, err return out, err
} }
// errRemoteBang is what a command carrying a '!' comes back with, so a caller
// sees a failure rather than a command that quietly did something else.
var errRemoteBang = errors.New("remote command contains '!'")
// remoteRejected reports whether a command must not be sent at all, and says
// why. '!' is the one character shq cannot protect: csh expands history *before*
// it looks at quotes, and it does so non-interactively too — `echo 'fix!now'`
// answers "Event not found" on the tcsh server. No spelling survives both csh
// and sh, so the only safe move is not to send one. This is the place every
// remote command passes.
func remoteRejected(remote string) bool {
if !strings.ContainsRune(remote, '!') {
return false
}
errorln("not sending a command with '!' in it — the server's login shell " +
"would expand it instead of passing it on: " + remote)
return true
}
// ssh runs a single remote command over ssh with inherited stdio. // ssh runs a single remote command over ssh with inherited stdio.
func ssh(remote string) error { func ssh(remote string) error {
if remoteRejected(remote) {
return errRemoteBang
}
_, err := sshExec(remote, false) _, err := sshExec(remote, false)
return err return err
} }
@@ -144,6 +167,9 @@ func sshOK(remote string) bool {
// sshOut runs a remote command and returns its stdout split into lines. // sshOut runs a remote command and returns its stdout split into lines.
func sshOut(remote string) ([]string, error) { func sshOut(remote string) ([]string, error) {
if remoteRejected(remote) {
return nil, errRemoteBang
}
out, err := sshExec(remote, true) out, err := sshExec(remote, true)
lines := strings.Split(string(out), "\n") lines := strings.Split(string(out), "\n")
for len(lines) > 0 && lines[len(lines)-1] == "" { for len(lines) > 0 && lines[len(lines)-1] == "" {
@@ -152,11 +178,45 @@ func sshOut(remote string) ([]string, error) {
return lines, err return lines, err
} }
// serverPath anchors a path on the git server at the configured gitpath.
//
// Every remote command has to name its target outright, because the login
// directory of the git user is not necessarily the directory holding the bare
// repositories. With `gituser = git` and `gitpath = /home/git` the two are the
// same place and a bare "." worked by luck; with `gituser = root` and
// `gitpath = /root/mgsh` it lists the home directory, where there is nothing to
// find.
func serverPath(rel string) string {
base := strings.TrimRight(cfg.GitPath, "/")
rel = strings.TrimPrefix(strings.TrimSpace(rel), "./")
if rel == "" || rel == "." {
return base
}
return base + "/" + rel
}
// serverRepoNames lists the bare repositories on the git server, without the
// ".git" suffix.
func serverRepoNames() ([]string, error) {
lines, err := sshOut("/bin/ls " + shq(serverPath(".")))
if err != nil {
return nil, err
}
var out []string
for _, ln := range lines {
if m := gitDirRe.FindStringSubmatch(strings.TrimSpace(ln)); m != nil {
out = append(out, m[1])
}
}
return out, nil
}
// serverEntryExists reports whether entry is present in the remote directory // serverEntryExists reports whether entry is present in the remote directory
// path (relative to the git user's home). The error is returned rather than // path (relative to gitpath, which serverPath resolves). The error is returned
// folded into the bool so a failed lookup is never mistaken for "not there". // rather than folded into the bool so a failed lookup is never mistaken for
// "not there".
func serverEntryExists(path, entry string) (bool, error) { func serverEntryExists(path, entry string) (bool, error) {
lines, err := sshOut("/bin/ls " + shq(path)) lines, err := sshOut("/bin/ls " + shq(serverPath(path)))
if err != nil { if err != nil {
return false, err return false, err
} }
input.go
+70 -22
View File
@@ -1,5 +1,11 @@
package main package main
// input.go — the y/n prompts that guard the destructive commands.
//
// These questions are the only thing standing between `init` and a wiped
// server repository, so the terminal handling here has to be exactly right: a
// question that cannot be answered is worse than no question at all.
import ( import (
"fmt" "fmt"
"os" "os"
@@ -13,50 +19,86 @@ import (
// variable because it guards the destructive operations: tests replace it to // variable because it guards the destructive operations: tests replace it to
// drive those paths without a terminal, and to assert that the question was // drive those paths without a terminal, and to assert that the question was
// asked at all. // asked at all.
//
// When the keypress cannot be read at all the answer is no, whatever the
// default says — a question nobody saw must never be taken as consent.
var yesno = func(prompt string, def bool) bool { var yesno = func(prompt string, def bool) bool {
suffix := " y/N ? " suffix := " y/N ? "
if def { if def {
suffix = " Y/n ? " suffix = " Y/n ? "
} }
ans := strings.ToLower(strings.TrimSpace(getkey(prompt + suffix))) key, ok := getkey(prompt + suffix)
if !ok {
errorln("could not read an answer from the terminal — assuming no")
return false
}
ans := strings.ToLower(strings.TrimSpace(key))
if ans == "" { if ans == "" {
return def return def
} }
return ans == "y" return ans == "y"
} }
// getkey reads a single keypress from the terminal without echo. It reads a // keyModeArgs put the terminal into single-key input: one keypress is delivered
// single byte directly from stdin; readline is not reading at this point (we // as it is typed, and it is not echoed.
// are inside command execution), so there is no reader to desync with. //
// MIN and TIME are set explicitly, and that is not decoration. They are not
// part of the canonical/non-canonical switch — they live in their own slots of
// the control-character array and survive `stty icanon`. drainTTY leaves them
// at "min 0 time 0" ("return what is buffered, do not wait"), so without this
// the *second* question of a session read zero bytes, answered itself with its
// default, and left the keypress queued for the next prompt line.
var keyModeArgs = []string{"-icanon", "-echo", "min", "1", "time", "0"}
// getkey reads a single keypress from the terminal without echo, and reports
// whether it got one. It reads a single byte directly from stdin; readline only
// reads while it is inside Readline(), and we are inside command execution
// here, so there is no reader to desync with.
// //
// Anything else already typed on the same line is discarded: answering "yes" // Anything else already typed on the same line is discarded: answering "yes"
// to a y/n prompt must not leave "es\n" queued for the next readline call, // to a y/n prompt must not leave "es\n" queued for the next readline call,
// where it would come back as a bogus command. // where it would come back as a bogus command.
func getkey(prompt string) string { func getkey(prompt string) (string, bool) {
fmt.Print(prompt) fmt.Print(prompt)
tty := readline.IsTerminal(int(os.Stdin.Fd()))
if tty { restore := func() {}
stty("-icanon", "-echo") if stdinIsTTY() {
restore = singleKeyMode()
} }
var buf [1]byte var buf [1]byte
n, err := os.Stdin.Read(buf[:]) n, err := os.Stdin.Read(buf[:])
if tty { restore()
drainTTY()
stty("icanon", "echo") if err != nil || n == 0 {
} fmt.Println()
key := "" return "", false
if err == nil && n > 0 {
key = strings.Trim(string(buf[:n]), "\r\n\t")
} }
key := strings.Trim(string(buf[:n]), "\r\n\t")
fmt.Println(key) fmt.Println(key)
return key return key, true
}
// singleKeyMode switches the terminal to single-key input and returns the
// function that puts it back. The previous settings are restored verbatim from
// `stty -g` rather than by naming the flags we changed: naming them is how the
// MIN/TIME above were left behind in the first place, and mgsh should hand the
// terminal back exactly as it found it.
func singleKeyMode() func() {
saved, err := sttyRun("-g")
saved = strings.TrimSpace(saved)
sttyRun(keyModeArgs...)
if err != nil || saved == "" {
return func() { drainTTY(); sttyRun("icanon", "echo") } // best effort
}
return func() { drainTTY(); sttyRun(strings.Fields(saved)...) }
} }
// drainTTY discards input already queued on the terminal. `min 0 time 0` makes // drainTTY discards input already queued on the terminal. `min 0 time 0` makes
// a read return whatever is buffered without waiting, so this cannot block when // a read return whatever is buffered without waiting, so this cannot block when
// nothing is pending. // nothing is pending. Its caller restores the terminal afterwards.
func drainTTY() { func drainTTY() {
stty("-icanon", "-echo", "min", "0", "time", "0") sttyRun("-icanon", "-echo", "min", "0", "time", "0")
buf := make([]byte, 256) buf := make([]byte, 256)
for { for {
n, err := os.Stdin.Read(buf) n, err := os.Stdin.Read(buf)
@@ -66,10 +108,16 @@ func drainTTY() {
} }
} }
func stty(args ...string) { // stdinIsTTY reports whether keypresses come from a terminal. A variable so the
// tests can exercise the terminal path without one.
var stdinIsTTY = func() bool { return readline.IsTerminal(int(os.Stdin.Fd())) }
// sttyRun runs stty on the terminal and returns its output. Errors are silent:
// every caller has a fallback, and a stray "stty: ..." line in the middle of a
// half-printed question helps nobody.
var sttyRun = func(args ...string) (string, error) {
c := exec.Command("stty", args...) c := exec.Command("stty", args...)
c.Stdin = os.Stdin c.Stdin = os.Stdin
c.Stdout = os.Stdout out, err := c.Output()
c.Stderr = os.Stderr return string(out), err
c.Run()
} }
input_test.go
+226
View File
@@ -0,0 +1,226 @@
package main
// input_test.go — the y/n prompt is the last thing between `init` and a wiped
// server repository, so the terminal handling around it is pinned here.
import (
"os"
"strings"
"sync"
"testing"
"time"
)
// fakeTTY stands in for the terminal driver. It models the one detail that made
// the skipped-question bug possible: MIN and TIME are not part of the
// canonical/non-canonical switch, so they survive `stty icanon` and carry over
// into the next prompt.
type fakeTTY struct {
mu sync.Mutex
state map[string]string
calls [][]string
ready chan struct{} // signalled once single-key mode is in effect
}
func newFakeTTY() *fakeTTY {
return &fakeTTY{
state: map[string]string{"icanon": "on", "echo": "on", "min": "1", "time": "0"},
ready: make(chan struct{}, 4),
}
}
func (f *fakeTTY) run(args ...string) (string, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, args)
switch {
case len(args) == 1 && args[0] == "-g":
return f.serializeLocked(), nil
case len(args) == 1 && strings.HasPrefix(args[0], "saved:"):
for _, kv := range strings.Split(strings.TrimPrefix(args[0], "saved:"), ",") {
if k, v, ok := strings.Cut(kv, "="); ok {
f.state[k] = v
}
}
return "", nil
}
for i := 0; i < len(args); i++ {
switch a := args[i]; a {
case "min", "time":
if i+1 < len(args) {
f.state[a] = args[i+1]
i++
}
default:
f.state[strings.TrimPrefix(a, "-")] = boolWord(!strings.HasPrefix(a, "-"))
}
}
if strings.Join(args, " ") == strings.Join(keyModeArgs, " ") {
f.ready <- struct{}{} // single-key mode is set; the read comes next
}
return "", nil
}
func boolWord(on bool) string {
if on {
return "on"
}
return "off"
}
// serializeLocked renders the settings as one token, the way `stty -g` does.
func (f *fakeTTY) serializeLocked() string {
return "saved:icanon=" + f.state["icanon"] + ",echo=" + f.state["echo"] +
",min=" + f.state["min"] + ",time=" + f.state["time"]
}
func (f *fakeTTY) get(k string) string {
f.mu.Lock()
defer f.mu.Unlock()
return f.state[k]
}
func (f *fakeTTY) snapshot() string {
f.mu.Lock()
defer f.mu.Unlock()
return f.serializeLocked()
}
// installFakeTTY points getkey at the fake terminal for the duration of a test.
func installFakeTTY(t *testing.T) *fakeTTY {
t.Helper()
f := newFakeTTY()
oldRun, oldIsTTY, oldStdin := sttyRun, stdinIsTTY, os.Stdin
sttyRun = f.run
stdinIsTTY = func() bool { return true }
t.Cleanup(func() {
sttyRun, stdinIsTTY, os.Stdin = oldRun, oldIsTTY, oldStdin
})
return f
}
// askOnce runs one getkey against the fake terminal, answering with keys once
// the terminal is actually in single-key mode. It reports the MIN in effect at
// the moment of the read — the value the old code got wrong.
func askOnce(t *testing.T, f *fakeTTY, keys string) (key string, ok bool, minAtRead string) {
t.Helper()
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
os.Stdin = r
done := make(chan struct{})
go func() {
key, ok = getkey("question? ")
close(done)
}()
select {
case <-f.ready:
case <-time.After(5 * time.Second):
t.Fatal("the terminal was never put into single-key mode")
}
minAtRead = f.get("min")
w.WriteString(keys)
w.Close() // so the drain that follows the keypress sees EOF instead of blocking
<-done
r.Close()
return key, ok, minAtRead
}
// TestEveryQuestionWaitsForAnAnswer is the regression test for the bug where the
// second y/n question of a session was skipped: drainTTY left the terminal at
// "min 0" ("return what is buffered, do not wait"), the restore only named
// icanon and echo, and so the next read returned zero bytes and answered the
// question with its default.
func TestEveryQuestionWaitsForAnAnswer(t *testing.T) {
f := installFakeTTY(t)
for i, want := range []string{"y", "n", "y"} {
key, ok, minAtRead := askOnce(t, f, want)
if !ok || key != want {
t.Fatalf("question %d: got (%q, %v), want (%q, true)", i+1, key, ok, want)
}
if minAtRead != "1" {
t.Errorf("question %d read the terminal at min=%s, want min=1 — "+
"min 0 returns without waiting and answers the question by itself",
i+1, minAtRead)
}
}
}
// TestPromptHandsTheTerminalBackUnchanged: mgsh must not leave the terminal in
// a mode it chose. The old restore named only icanon and echo and left "min 0"
// behind, which outlived mgsh itself and broke the next program's single-key
// reads too.
func TestPromptHandsTheTerminalBackUnchanged(t *testing.T) {
f := installFakeTTY(t)
initial := f.snapshot()
if _, ok, _ := askOnce(t, f, "y"); !ok {
t.Fatal("getkey did not read the key")
}
if got := f.snapshot(); got != initial {
t.Errorf("terminal left as %s, want it back at %s", got, initial)
}
}
// TestPromptRestoresFromTheSavedState pins *how* the terminal is restored: from
// the state captured with `stty -g`, not by naming the flags we changed.
func TestPromptRestoresFromTheSavedState(t *testing.T) {
f := installFakeTTY(t)
if _, ok, _ := askOnce(t, f, "y"); !ok {
t.Fatal("getkey did not read the key")
}
f.mu.Lock()
defer f.mu.Unlock()
if len(f.calls) == 0 {
t.Fatal("stty was never called")
}
if first := f.calls[0]; len(first) != 1 || first[0] != "-g" {
t.Errorf("first stty call was %v, want [-g]: the state has to be captured "+
"before it is changed", first)
}
last := f.calls[len(f.calls)-1]
if len(last) != 1 || !strings.HasPrefix(last[0], "saved:") {
t.Errorf("last stty call was %v, want the saved state played back", last)
}
}
// TestSingleKeyModeSetsMinExplicitly: MIN and TIME are inherited, so they have
// to be named on the way in. Without that, a terminal left at "min 0" by an
// earlier program (or an earlier mgsh) skips the question.
func TestSingleKeyModeSetsMinExplicitly(t *testing.T) {
args := strings.Join(keyModeArgs, " ")
for _, want := range []string{"-icanon", "-echo", "min 1", "time 0"} {
if !strings.Contains(args, want) {
t.Errorf("single-key mode is %q, missing %q", args, want)
}
}
}
// TestQuestionSkippedByAPoisonedTerminalIsNotAYes: even if the terminal is
// already in the broken state when mgsh starts, the answer must not be the
// default — nobody saw the question, so nobody agreed to anything.
func TestQuestionSkippedByAPoisonedTerminalIsNotAYes(t *testing.T) {
oldRun, oldIsTTY, oldStdin := sttyRun, stdinIsTTY, os.Stdin
defer func() { sttyRun, stdinIsTTY, os.Stdin = oldRun, oldIsTTY, oldStdin }()
sttyRun = func(args ...string) (string, error) { return "", nil }
stdinIsTTY = func() bool { return true }
r, w, _ := os.Pipe()
w.Close() // a terminal that returns nothing at all
os.Stdin = r
defer r.Close()
if key, ok := getkey("question? "); ok {
t.Fatalf("getkey reported a key %q from a terminal that gave none", key)
}
if yesno("destroy everything?", true) {
t.Error("an unanswerable question was taken as yes")
}
}
main.go
+55 -2
View File
@@ -45,8 +45,20 @@ var (
) )
func main() { func main() {
setup() // Answered before setup(), which exits when nothing is configured yet: an
// update has to work on a machine that has never run mgsh, and `--version`
// is what the freshly downloaded binary is probed with, right there.
if updateFlags() {
return
}
// Colour first, then setup(): the configuration warnings setup() prints are
// the first thing on screen, and they were the only messages coming out
// uncoloured — col() saw useColor still false while they were written.
// Nothing about the terminal depends on the configuration, so the question
// can just as well be asked one line earlier.
useColor = readline.IsTerminal(int(os.Stdout.Fd())) useColor = readline.IsTerminal(int(os.Stdout.Fd()))
setup()
fmt.Println(banner()) fmt.Println(banner())
@@ -68,12 +80,52 @@ func main() {
} }
updateDirState() updateDirState()
runCommand(cmdline) runCommand(cmdline)
updateNote() // after the output: a footer, not a headline
return return
} }
updateNote() // before the prompt: a session starts here, not when it ends
runInteractive() runInteractive()
} }
// updateFlags answers the self-update options and reports whether it did. They
// are deliberately spelled with dashes and kept out of parseArgs: `mgsh update`
// is the command for everyday use, and these are what works when there is no
// configuration to read yet.
func updateFlags() bool {
if len(os.Args) < 2 {
return false
}
switch os.Args[1] {
case "--version":
fmt.Printf("mgsh %s\n", VERSION)
case "--update":
if err := selfUpdate.install(os.Stdout); err != nil {
fmt.Fprintf(os.Stderr, "mgsh: %v\n", err)
os.Exit(1)
}
case "--check-update":
if err := selfUpdate.check(os.Stdout); err != nil {
fmt.Fprintf(os.Stderr, "mgsh: %v\n", err)
os.Exit(1)
}
case updateRefreshFlag: // the background run, not in the help
selfUpdate.refresh()
default:
return false
}
return true
}
// updateNote prints the once-a-day hint, when there is one. It costs nothing:
// the line comes from the note in the cache directory, and the asking behind it
// happens in the background, at most once a day.
func updateNote() {
if hint := selfUpdate.daily(); hint != "" {
fmt.Fprintln(os.Stderr, col(cDark, hint))
}
}
// runInteractive drives the colored, history- and completion-enabled REPL. // runInteractive drives the colored, history- and completion-enabled REPL.
func runInteractive() { func runInteractive() {
home, _ := os.UserHomeDir() home, _ := os.UserHomeDir()
@@ -145,9 +197,10 @@ func parseArgs() (int, string, bool) {
} }
cls := map[string]int{ cls := map[string]int{
"clone": 2, "init": 2, "log": 2, "clone": 2, "init": 2, "log": 2,
"push": 1, "pushremote": 1, "list": 1, "tag": 1, "archive": 1, "show": 1, "open": 1, "push": 1, "pushremote": 1, "deleteremote": 1, "list": 1, "tag": 1, "archive": 1, "show": 1,
"pull": 1, "fetch": 1, "status": 1, "diff": 1, "overview": 1, "pull": 1, "fetch": 1, "status": 1, "diff": 1, "overview": 1,
"config": 1, "count": 1, "login": 1, "cloneall": 1, "release": 1, "config": 1, "count": 1, "login": 1, "cloneall": 1, "release": 1,
"update": 1,
} }
if c, ok := cls[a0]; ok { if c, ok := cls[a0]; ok {
return c, strings.Join(os.Args[1:], " "), false return c, strings.Join(os.Args[1:], " "), false
mgsh_test.go
+379 -47
View File
@@ -9,6 +9,7 @@ import (
"strings" "strings"
"testing" "testing"
"time" "time"
"unicode/utf8"
) )
func TestSanitizeComment(t *testing.T) { func TestSanitizeComment(t *testing.T) {
@@ -74,19 +75,65 @@ func TestFormatLogRecentCompact(t *testing.T) {
} }
} }
func TestColorRepoLine(t *testing.T) { func TestFormatRepoList(t *testing.T) {
useColor = false useColor = false
in := "Sep 28 2016 Betaflight3.0.0" entries := []lsEntry{
if got := colorRepoLine(in); got != in { {name: "short", date: "Sep 28 2016", size: 4096},
t.Errorf("colorRepoLine with color off changed input: %q", got) {name: "a-much-longer-name", date: "Jan 3 14:32", size: 1536},
} }
useColor = true out := formatRepoList(entries, false)
got := colorRepoLine(in) lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
if !strings.Contains(got, "Betaflight3.0.0") || !strings.Contains(got, cGreen) || !strings.Contains(got, cYellow) { if len(lines) != 2 {
t.Errorf("colorRepoLine did not color parts: %q", got) t.Fatalf("expected 2 lines, got %d: %q", len(lines), out)
} }
// order is preserved: `ls -ltr` already sorted by modification time
if !strings.Contains(lines[0], "short") || !strings.Contains(lines[1], "a-much-longer-name") {
t.Errorf("order not preserved: %q", out)
}
// the date starts at the same column on every line
if strings.Index(lines[0], "Sep") != strings.Index(lines[1], "Jan") {
t.Errorf("date column not aligned:\n%s", out)
}
if strings.Contains(out, "4.0K") {
t.Errorf("size shown for repositories: %q", out)
}
if withSize := formatRepoList(entries, true); !strings.Contains(withSize, "4.0K") ||
!strings.Contains(withSize, "1.5K") {
t.Errorf("archive sizes missing: %q", withSize)
}
// colour must decorate the layout, never change it
useColor = true
colored := formatRepoList(entries, false)
useColor = false useColor = false
strip := func(s string) string {
for _, c := range []string{cReset, cGreen, cYellow, cDark} {
s = strings.ReplaceAll(s, c, "")
}
return s
}
if strip(colored) != out {
t.Errorf("colour changed the layout:\n%q\n%q", strip(colored), out)
}
}
func TestHumanSize(t *testing.T) {
cases := []struct {
n int64
want string
}{
{0, "0B"}, {512, "512B"}, {1024, "1.0K"}, {1536, "1.5K"},
{1024 * 1024, "1.0M"}, {3 * 1024 * 1024 * 1024, "3.0G"},
// past 10 the decimal carries nothing, as with `ls -h`
{512 * 1024, "512K"}, {99 * 1024 * 1024, "99M"},
}
for _, c := range cases {
if got := humanSize(c.n); got != c.want {
t.Errorf("humanSize(%d) = %q, want %q", c.n, got, c.want)
}
}
} }
func TestParseConfig(t *testing.T) { func TestParseConfig(t *testing.T) {
@@ -95,7 +142,7 @@ func TestParseConfig(t *testing.T) {
githost = 10.0.0.1 githost = 10.0.0.1
GitPort: 22 GitPort: 22
gituser = "deploy" gituser = "deploy"
editor = 'code' gitkey = 'mgit_rsa'
ignored line without separator ignored line without separator
base=/tmp/src base=/tmp/src
` `
@@ -104,7 +151,7 @@ base=/tmp/src
"githost": "10.0.0.1", "githost": "10.0.0.1",
"gitport": "22", "gitport": "22",
"gituser": "deploy", "gituser": "deploy",
"editor": "code", "gitkey": "mgit_rsa",
"base": "/tmp/src", "base": "/tmp/src",
} }
for k, want := range checks { for k, want := range checks {
@@ -119,7 +166,7 @@ base=/tmp/src
func TestParseConfigInlineComments(t *testing.T) { func TestParseConfigInlineComments(t *testing.T) {
rc := ` rc := `
editor = code # fallback opener for ` + "`open`" + ` gitkey = mgit_rsa # fallback opener comment
mirror = true # ` + "`push`" + ` also mirrors via pushremote mirror = true # ` + "`push`" + ` also mirrors via pushremote
gitport = 22 # ssh port gitport = 22 # ssh port
remotekey = abc#123 remotekey = abc#123
@@ -129,7 +176,7 @@ gitemail = # value is only a comment
` `
m := parseConfig(rc) m := parseConfig(rc)
checks := map[string]string{ checks := map[string]string{
"editor": "code", "gitkey": "mgit_rsa",
"mirror": "true", "mirror": "true",
"gitport": "22", "gitport": "22",
"remotekey": "abc#123", // '#' not preceded by space stays part of the value "remotekey": "abc#123", // '#' not preceded by space stays part of the value
@@ -147,38 +194,49 @@ gitemail = # value is only a comment
} }
} }
func TestLsEntry(t *testing.T) { func TestParseLsEntry(t *testing.T) {
cases := []struct{ line, suffix, want string }{ cases := []struct {
// ownership is not assumed: any user/group must list line, suffix string
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"}, name, date string
{"drwxr-xr-x 7 deploy deploy 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"}, size int64
{"drwxr-xr-x 7 mike staff 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"}, ok bool
{"drwxr-xr-x. 7 git users 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"}, }{
// archives only match the archive suffix, and vice versa // ownership is not assumed: any user/group must parse
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git.tar.gz", "Sep 28 2016 myproj"}, {"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git", ""}, {"drwxr-xr-x 7 deploy deploy 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git.tar.gz", ""}, {"drwxr-xr-x. 7 git users 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
// a recent entry carries a time instead of a year, and still lines up
{"drwxr-xr-x 7 mike staff 224 Jan 3 14:32 myproj.git", ".git", "myproj", "Jan 3 14:32", 224, true},
// a symlinked bare repo lists its target too — only the link name counts
{"lrwxrwxrwx 1 git git 14 Sep 28 2016 myproj.git -> /srv/other.git", ".git", "myproj", "Sep 28 2016", 14, true},
// archives carry a size worth showing
{"-rw-r--r-- 1 git git 524288 Sep 28 2016 myproj.git.tar.gz", ".git.tar.gz", "myproj", "Sep 28 2016", 524288, true},
// suffixes must not cross over
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git", "", "", 0, false},
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git.tar.gz", "", "", 0, false},
{"lrwxrwxrwx 1 git git 5 Sep 28 2016 notes -> x.git", ".git", "", "", 0, false},
// non-entries // non-entries
{"total 48", ".git", ""}, {"total 48", ".git", "", "", 0, false},
{"", ".git", ""}, {"", ".git", "", "", 0, false},
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 notes", ".git", ""}, {"drwxr-xr-x 7 git git 4096 Sep 28 2016 notes", ".git", "", "", 0, false},
} }
for _, c := range cases { for _, c := range cases {
if got := lsEntry(c.line, c.suffix); got != c.want { e, ok := parseLsEntry(c.line, c.suffix)
t.Errorf("lsEntry(%q, %q) = %q, want %q", c.line, c.suffix, got, c.want) if ok != c.ok {
t.Errorf("parseLsEntry(%q, %q) ok = %v, want %v", c.line, c.suffix, ok, c.ok)
continue
}
if !ok {
continue
}
if e.name != c.name || e.size != c.size {
t.Errorf("parseLsEntry(%q) = %+v, want name %q size %d", c.line, e, c.name, c.size)
}
// every date renders to the same width, whichever form ls used
if e.date != c.date || len(e.date) != 12 {
t.Errorf("parseLsEntry(%q) date = %q (len %d), want %q at 12",
c.line, e.date, len(e.date), c.date)
} }
}
}
func TestLsEntrySymlink(t *testing.T) {
// a symlinked bare repo lists its target too — only the link name counts
in := "lrwxrwxrwx 1 git git 14 Sep 28 2016 myproj.git -> /srv/other.git"
if got := lsEntry(in, ".git"); got != "Sep 28 2016 myproj" {
t.Errorf("lsEntry(symlink) = %q, want %q", got, "Sep 28 2016 myproj")
}
// and a symlink to something that is not a repo must not match
if got := lsEntry("lrwxrwxrwx 1 git git 5 Sep 28 2016 notes -> x.git", ".git"); got != "" {
t.Errorf("lsEntry(non-repo symlink) = %q, want empty", got)
} }
} }
@@ -410,7 +468,7 @@ remote.broken.url = https://nowhere.example # no key -> unusable
} }
func TestMirrorTargetsLegacyAndSelection(t *testing.T) { func TestMirrorTargetsLegacyAndSelection(t *testing.T) {
// the flat remoteurl/remotekey pair stays supported, as target "public" // the pre-4.1 flat pair still loads, folded onto the target "public"
var c Config var c Config
applyConfig(&c, parseConfig("remoteurl = https://git.example.com\nremotekey = tok\n")) applyConfig(&c, parseConfig("remoteurl = https://git.example.com\nremotekey = tok\n"))
usable, _ := c.mirrorTargets() usable, _ := c.mirrorTargets()
@@ -441,6 +499,77 @@ remotes = hub, public
} }
} }
// TestInactiveRemoteIsOnlyUsedWhenNamed: `active = false` takes a target off the
// automatic path — not used when no name is given — while naming it, by name or
// by host, still reaches it. It is not the same as removing the target, and not
// the same as an unusable one: it stays in the configuration, listed and ready.
func TestInactiveRemoteIsOnlyUsedWhenNamed(t *testing.T) {
rc := `
remote.gitea.url = https://git.example.com
remote.gitea.key = tok
remote.gitea.active = false
remote.hub.url = https://github.com
remote.hub.key = tok2
`
var c Config
applyConfig(&c, parseConfig(rc))
all, incomplete := c.mirrorTargets()
if len(all) != 2 || len(incomplete) != 0 {
t.Fatalf("mirrorTargets = %+v (incomplete %v), want both targets", all, incomplete)
}
if got := pickRemotes(all, nil); len(got) != 1 || got[0].Name != "hub" {
t.Errorf("selection without a name = %+v, want only the active hub", got)
}
for _, sel := range []string{"gitea", "@gitea", "git.example.com"} {
if got := pickRemotes(all, []string{sel}); len(got) != 1 || got[0].Name != "gitea" {
t.Errorf("pickRemotes(%q) = %+v, want the inactive gitea", sel, got)
}
}
// MGSH_REMOTE_<NAME>_ACTIVE is the environment spelling
t.Setenv("MGSH_REMOTE_HUB_ACTIVE", "false")
var c2 Config
applyConfig(&c2, parseConfig(rc))
applyEnv(&c2)
all2, _ := c2.mirrorTargets()
if got := pickRemotes(all2, nil); len(got) != 0 {
t.Errorf("MGSH_REMOTE_HUB_ACTIVE=false left %+v in the default set", got)
}
}
// TestRemoteActiveDefaultsToOn: the setting exists to take a server *out* of the
// default set, so anything but a deliberate "off" — unset, "true", a typo — has
// to leave it in. A configuration written before the flag existed must not
// suddenly stop mirroring.
func TestRemoteActiveDefaultsToOn(t *testing.T) {
for _, v := range []string{"", "true", "yes", "1", "on", "sometimes"} {
if !(RemoteTarget{Active: v}).isActive() {
t.Errorf("active = %q switched the target off", v)
}
}
for _, v := range []string{"false", "no", "0", "off", " FALSE "} {
if (RemoteTarget{Active: v}).isActive() {
t.Errorf("active = %q did not switch the target off", v)
}
}
}
// TestPushRemoteWithOnlyInactiveTargets: the message has to name the reason.
// "needs a remote.<name>.url" would send the user looking for a setting that is
// sitting right there in the file.
func TestPushRemoteWithOnlyInactiveTargets(t *testing.T) {
dir := useProject(t, "notes")
mustGit(t, dir, "init", "-q")
cfg.Remotes = []RemoteTarget{{Name: "gitea", URL: "https://git.example.com", Key: "tok", Active: "false"}}
out := captureStdout(t, func() { runCommand("pushremote") })
if !strings.Contains(out, "inactive") || !strings.Contains(out, "@gitea") {
t.Errorf("output = %q, want it to report the inactive target and how to name it", out)
}
}
func TestParsePushRemoteArgs(t *testing.T) { func TestParsePushRemoteArgs(t *testing.T) {
cases := []struct { cases := []struct {
in string in string
@@ -481,7 +610,7 @@ func TestResolveProjectConfig(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
global := Config{ global := Config{
Base: "/base", GitHost: "global.example", GitPort: "22", GitUser: "git", Base: "/base", GitHost: "global.example", GitPort: "22", GitUser: "git",
GitPath: "/home/git", GitName: "Global Name", Editor: "vi", GitPath: "/home/git", GitName: "Global Name", GitKey: "global_rsa",
Remotes: []RemoteTarget{{Name: "gitea", URL: "https://gitea.example", Key: "tok"}}, Remotes: []RemoteTarget{{Name: "gitea", URL: "https://gitea.example", Key: "tok"}},
} }
@@ -492,7 +621,7 @@ func TestResolveProjectConfig(t *testing.T) {
rc := ` rc := `
githost = project.example githost = project.example
editor = code gitkey = project_rsa
base = /somewhere/else base = /somewhere/else
gitname = Project Name gitname = Project Name
remote.hub.url = https://github.com remote.hub.url = https://github.com
@@ -504,8 +633,8 @@ remote.gitea.visibility = public
} }
got := resolveConfig(global, dir) got := resolveConfig(global, dir)
if got.GitHost != "project.example" || got.Editor != "code" { if got.GitHost != "project.example" || got.GitKey != "project_rsa" {
t.Errorf("project overrides not applied: host=%q editor=%q", got.GitHost, got.Editor) t.Errorf("project overrides not applied: host=%q gitkey=%q", got.GitHost, got.GitKey)
} }
// base and the git identity stay global // base and the git identity stay global
if got.Base != "/base" { if got.Base != "/base" {
@@ -749,6 +878,7 @@ func TestTruthy(t *testing.T) {
func TestFormatProjStatus(t *testing.T) { func TestFormatProjStatus(t *testing.T) {
useColor = false useColor = false
defer func() { useColor = false }() defer func() { useColor = false }()
w := overviewWidths{label: 12, sync: 5, host: 7}
cases := []struct { cases := []struct {
s projStatus s projStatus
contains []string contains []string
@@ -763,10 +893,16 @@ func TestFormatProjStatus(t *testing.T) {
{projStatus{name: "d", branch: "feature", dirty: true}, {projStatus{name: "d", branch: "feature", dirty: true},
[]string{"d", "*", "(feature)"}, nil}, []string{"d", "*", "(feature)"}, nil},
{projStatus{name: "e", branch: "master"}, // clean, no upstream {projStatus{name: "e", branch: "master"}, // clean, no upstream
[]string{"e", "no upstream"}, []string{"*"}}, []string{"e", "–"}, []string{"*", "✓"}},
{projStatus{name: "f", branch: "master", hasUpstream: true, ahead: 1, behind: 2},
[]string{"f", "↑1↓2"}, []string{"✓"}}, // diverged shows both
{projStatus{name: "g", branch: "master", hasUpstream: true, lastHost: "laptop",
mirrors: []string{"hub", "gitea"}},
[]string{"g", "laptop", "→ hub gitea"}, nil},
} }
for _, c := range cases { for _, c := range cases {
got := formatProjStatus(c.s, 8) c.s.isRepo = true // these all describe real repositories
got := formatProjStatus(c.s, w)
for _, sub := range c.contains { for _, sub := range c.contains {
if !strings.Contains(got, sub) { if !strings.Contains(got, sub) {
t.Errorf("formatProjStatus(%+v) = %q, missing %q", c.s, got, sub) t.Errorf("formatProjStatus(%+v) = %q, missing %q", c.s, got, sub)
@@ -780,6 +916,62 @@ func TestFormatProjStatus(t *testing.T) {
} }
} }
// TestOverviewColumnsAlign is the point of the table: every field has to start
// at the same column on every row, whatever the name lengths or the multi-byte
// status glyphs do.
func TestOverviewColumnsAlign(t *testing.T) {
useColor = false
// host names must not occur anywhere else in a row, or the index search
// below would find them inside a project or branch name instead
rows := []projStatus{
{name: "a", branch: "master", hasUpstream: true, ahead: 12, behind: 3, lastHost: "workstation"},
{name: "a-very-long-project-name", branch: "wip", dirty: true, lastHost: "buildbox"},
{name: "mid", branch: "main", hasUpstream: true, lastHost: "laptop"},
}
w := measureOverview(rows)
var widths []int
for _, r := range rows {
line := formatProjStatus(r, w)
// the host column starts right after the padded sync field
idx := strings.Index(line, r.lastHost)
if idx < 0 {
t.Fatalf("host %q missing from %q", r.lastHost, line)
}
widths = append(widths, utf8.RuneCountInString(line[:idx]))
}
for i := 1; i < len(widths); i++ {
if widths[i] != widths[0] {
t.Errorf("host column starts at %d on row %d, %d on row 0:\n%s",
widths[i], i, widths[0], strings.Join([]string{
formatProjStatus(rows[0], w), formatProjStatus(rows[i], w)}, "\n"))
}
}
}
func TestAttentionRank(t *testing.T) {
ranks := []struct {
s projStatus
want int
}{
{projStatus{isRepo: true, dirty: true}, 0},
{projStatus{isRepo: true, ahead: 1}, 0},
{projStatus{isRepo: true, behind: 1}, 0},
{projStatus{isRepo: true, hasUpstream: true}, 1},
{projStatus{isRepo: true}, 1}, // clean, no upstream
// not on the server is a different kind of task and goes last, even
// when the working tree is dirty — it cannot be pushed anyway
{projStatus{isRepo: true, notOnServer: true}, 2},
{projStatus{isRepo: true, dirty: true, notOnServer: true}, 2},
{projStatus{notOnServer: true}, 2},
}
for _, c := range ranks {
if got := attentionRank(c.s); got != c.want {
t.Errorf("attentionRank(%+v) = %d, want %d", c.s, got, c.want)
}
}
}
func TestDetectRemoteKind(t *testing.T) { func TestDetectRemoteKind(t *testing.T) {
cases := []struct { cases := []struct {
url, override string url, override string
@@ -886,3 +1078,143 @@ func TestExpandAlias(t *testing.T) {
} }
} }
} }
func TestSplitAtMarker(t *testing.T) {
lines := []string{"a", "b", "---mgsh---", "c", "d"}
before, after := splitAtMarker(lines, "---mgsh---")
if strings.Join(before, ",") != "a,b" || strings.Join(after, ",") != "c,d" {
t.Errorf("split = %v / %v", before, after)
}
// no marker: everything is the first section, so a server that produced no
// du output simply yields no sizes
before, after = splitAtMarker([]string{"a", "b"}, "---mgsh---")
if strings.Join(before, ",") != "a,b" || after != nil {
t.Errorf("split without marker = %v / %v", before, after)
}
}
func TestParseDuSizes(t *testing.T) {
lines := []string{
"185432\tBetaflight3.0.0.git",
"2144\twebsite.git",
"876 spaced-with-blanks.git", // some du implementations use spaces
"1024\t./with-dot-slash.git",
"1500\tmy project.git", // a name with a space survives
"garbage",
"",
}
got := parseDuSizes(lines)
want := map[string]int64{
"Betaflight3.0.0.git": 185432 * 1024,
"website.git": 2144 * 1024,
"spaced-with-blanks.git": 876 * 1024,
"with-dot-slash.git": 1024 * 1024,
"my project.git": 1500 * 1024,
}
if len(got) != len(want) {
t.Fatalf("parseDuSizes = %v, want %d entries", got, len(want))
}
for k, v := range want {
if got[k] != v {
t.Errorf("parseDuSizes[%q] = %d, want %d", k, got[k], v)
}
}
}
// TestMigrateRemoteKeys rewrites the pre-4.1 flat spelling in place. Only the
// key changes: values, comments and everything else stay byte for byte.
func TestMigrateRemoteKeys(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, ".mgshrc")
before := `# my config
base = /home/me/src
remoteurl = https://git.example.com # the mirror
remotekey = s3cr3t-token
remotevisibility: public
# remotetype = gitea (commented out, must stay put)
alias co 'checkout $1'
`
if err := os.WriteFile(path, []byte(before), 0600); err != nil {
t.Fatal(err)
}
out := captureStdout(t, func() { migrateRemoteKeys(path, before) })
got := readFile(t, path)
for _, want := range []string{
"remote.public.url = https://git.example.com # the mirror",
"remote.public.key = s3cr3t-token",
" remote.public.visibility: public",
"# remotetype = gitea (commented out, must stay put)",
"base = /home/me/src",
"alias co 'checkout $1'",
} {
if !strings.Contains(got, want) {
t.Errorf("migrated file missing %q:\n%s", want, got)
}
}
if strings.Contains(got, "\nremoteurl") || strings.Contains(got, "\nremotekey") {
t.Errorf("old spelling left behind:\n%s", got)
}
if !strings.Contains(out, "remoteurl → remote.public.url") {
t.Errorf("migration was not reported: %q", out)
}
// the file keeps its private mode
if fi, err := os.Stat(path); err != nil {
t.Fatal(err)
} else if fi.Mode().Perm() != 0o600 {
t.Errorf("mode after migration = %04o, want 0600", fi.Mode().Perm())
}
// running again changes nothing and says nothing
second := captureStdout(t, func() { migrateRemoteKeys(path, readFile(t, path)) })
if strings.TrimSpace(second) != "" {
t.Errorf("a converted file was migrated again: %q", second)
}
if readFile(t, path) != got {
t.Error("a second migration changed the file")
}
}
// TestLegacyKeysDoNotOverrideExplicitOnes: a config carrying both spellings must
// keep what the new one says.
func TestLegacyKeysDoNotOverrideExplicitOnes(t *testing.T) {
var c Config
applyConfig(&c, parseConfig(
"remoteurl = https://old.example\nremote.public.url = https://new.example\n"+
"remote.public.key = tok\n"))
targets, _ := c.mirrorTargets()
if len(targets) != 1 || targets[0].URL != "https://new.example" {
t.Errorf("targets = %+v, want the remote.public.url value", targets)
}
}
// TestRemoteEnvOverrides: MGSH_REMOTE_<NAME>_<FIELD> is the environment
// spelling of remote.<name>.<field>.
func TestRemoteEnvOverrides(t *testing.T) {
t.Setenv("MGSH_REMOTE_GITLAB_URL", "https://gitlab.example")
t.Setenv("MGSH_REMOTE_GITLAB_KEY", "env-token")
t.Setenv("MGSH_REMOTE_GITLAB_VISIBILITY", "public")
t.Setenv("MGSH_REMOTE_MY_HUB_URL", "https://hub.example") // name with an underscore
t.Setenv("MGSH_REMOTE_MY_HUB_KEY", "hub-token")
t.Setenv("MGSH_REMOTES", "") // must not be mistaken for a target field
var c Config
applyConfig(&c, parseConfig("remote.gitlab.url = https://from-file.example\nremote.gitlab.key = file-token\n"))
applyEnv(&c)
targets, incomplete := c.mirrorTargets()
if len(incomplete) != 0 {
t.Fatalf("incomplete targets: %v", incomplete)
}
byName := map[string]RemoteTarget{}
for _, tg := range targets {
byName[tg.Name] = tg
}
if g := byName["gitlab"]; g.URL != "https://gitlab.example" || g.Key != "env-token" || g.Vis != "public" {
t.Errorf("env did not override the file: %+v", g)
}
// the field is taken from the end, so the name may contain underscores
if h := byName["my_hub"]; h.URL != "https://hub.example" || h.Key != "hub-token" {
t.Errorf("MGSH_REMOTE_MY_HUB_* = %+v, want target my_hub", h)
}
}
mgshrc.example
+29 -17
View File
@@ -20,27 +20,39 @@ gitpath = /home/git
# gitname = Your Name # gitname = Your Name
# gitemail = you@example.com # gitemail = you@example.com
# pushdefault = matching # pushdefault = matching
# editor = code
# --- pushremote: mirror to public servers (gitea/github/gitlab) via their API --- # --- pushremote: mirror to public servers (gitea/github/gitlab) via their API ---
# A single server, the flat form (this target is named "public"): # One "remote.<name>.<field>" block per server, with the fields url, key, type
# remoteurl = https://git.example.com # and visibility. <name> is yours to pick and becomes the git remote created in
# remotekey = <personal-access-token> # the repository, so `git push gitlab` keeps working outside mgsh.
# remotetype = gitea # optional; auto-detected from remoteurl
# remotevisibility = private # visibility of created repos (default private)
# #
# Or any number of named servers. `pushremote` pushes to all of them in the # `pushremote` pushes to every active server in the order given,
# order given, `pushremote @hub` to a single one. Each target gets a git remote # `pushremote @gitlab` to a single one (active or not).
# of the same name in the repository.
# remote.gitea.url = https://git.example.com
# remote.gitea.key = <personal-access-token>
# remote.hub.url = https://github.com
# remote.hub.key = <personal-access-token>
# remote.hub.type = github # optional; auto-detected from the url
# remote.hub.visibility = public # default private
# remotes = gitea, hub # optional: restrict and order the set
# #
# mirror = true # `push` also mirrors via pushremote # `deleteremote @gitlab` (or `deleteremote gitlab.example.com`) deletes the
# project's repository there again, after asking. It always needs the server
# named, and the token needs permission to delete: delete_repo on GitHub,
# write:repository on Gitea, api + Owner on GitLab.
#
# remote.gitea.url = https://git.example.com
# remote.gitea.key = <personal-access-token>
# remote.gitea.type = gitea # optional; auto-detected from the url
# remote.gitea.visibility = private # or public (default private)
#
# remote.gitlab.url = https://gitlab.example.com
# remote.gitlab.key = <personal-access-token>
# remote.gitlab.type = gitlab
# remote.gitlab.visibility = public
# remote.gitlab.active = false # default true; false = only used when
# # named, e.g. `pushremote @gitlab`
#
# remotes = gitea, gitlab # optional: restrict and order the set
# mirror = true # `push` also mirrors via pushremote
# --- safety ---
# `push` checks the staged diff for private keys and API tokens before it
# commits, and asks before continuing. Only an explicit "off" disables it.
# secretscan = off
# --- per-project overrides --- # --- per-project overrides ---
# A <project>/.mgshrc overrides all of the above for that project only, except # A <project>/.mgshrc overrides all of the above for that project only, except
overview.go
+317 -55
View File
@@ -1,25 +1,49 @@
package main package main
// overview.go — the `overview` command (also reachable as `status -a`): a // overview.go — the `overview` command (also reachable as `status -a`).
// one-line-per-project summary of every git project under BASE, showing the //
// dirty state and how far each branch is ahead/behind its upstream. // mgsh is the only thing that sees all three places a project can live: the
// local base directory, the internal ssh server, and the public mirrors. Joining
// those answers the questions plain git cannot — which projects were never
// pushed to the server, and which machine last touched each one (every `push`
// stamps "[user@host]" into the commit message, so that comes for free).
import ( import (
"fmt" "fmt"
"os" "os"
"regexp"
"sort"
"strconv"
"strings" "strings"
"sync"
"time"
"unicode/utf8"
) )
// projStatus is the collected state of one project for the overview. // projStatus is the collected state of one project for the overview.
type projStatus struct { type projStatus struct {
name string name string
branch string branch string
isRepo bool // has a .git of its own
dirty bool dirty bool
ahead, behind int ahead, behind int
hasUpstream bool hasUpstream bool
notOnServer bool // known to be missing from the git server
lastHost string // machine that made the last commit, from "[user@host]"
lastWhen time.Time // when that was
mirrors []string // configured mirror remotes present in this repo
} }
// overviewAll prints a status summary for all git projects under BASE. // commitHostRe pulls the host out of the "[user@host] subject" line that `push`
// writes, so the overview can say where a project was last worked on.
var commitHostRe = regexp.MustCompile(`^\[[^@\]]*@([^\]]+)\]`)
// overviewScanLimit bounds how many projects are inspected at once. The work is
// all subprocess latency, so some concurrency helps a lot and more does not.
const overviewScanLimit = 8
// overviewAll prints a status summary for all git projects under BASE, plus the
// projects that exist on only one side of the local/server divide.
func overviewAll() { func overviewAll() {
entries, err := os.ReadDir(BASE) entries, err := os.ReadDir(BASE)
if err != nil { if err != nil {
@@ -27,30 +51,52 @@ func overviewAll() {
return return
} }
var rows []projStatus // ask the server while the local tree is being walked
width := 0 type serverList struct {
names []string
err error
}
srvCh := make(chan serverList, 1)
go func() {
names, err := serverRepoNames()
srvCh <- serverList{names, err}
}()
// every directory gets a row, repository or not: one that is not a
// repository yet is exactly what `init` is for, and putting it in the table
// beats a separate list underneath
var local []string
for _, e := range entries { for _, e := range entries {
if !e.IsDir() || strings.HasPrefix(e.Name(), ".") { if e.IsDir() && !strings.HasPrefix(e.Name(), ".") {
continue local = append(local, e.Name())
}
dir := BASE + "/" + e.Name()
if !isDir(dir + "/.git") {
continue
}
rows = append(rows, projectStatus(e.Name(), dir))
if len(e.Name()) > width {
width = len(e.Name())
} }
} }
rows := scanProjects(local)
srv := <-srvCh
markUnpublished(rows, srv.names, srv.err)
// what needs doing first, alphabetical within each group
sort.SliceStable(rows, func(i, j int) bool {
return attentionRank(rows[i]) < attentionRank(rows[j])
})
if len(rows) == 0 { if len(rows) == 0 {
fmt.Println(col(cGray, "no git projects under "+BASE)) fmt.Println(col(cDark, "nothing under "+BASE))
return return
} }
dirtyN, syncN := 0, 0 w := measureOverview(rows)
repoN, dirtyN, syncN, initN := 0, 0, 0, 0
for _, r := range rows { for _, r := range rows {
fmt.Println(formatProjStatus(r, width)) fmt.Println(formatProjStatus(r, w))
if r.notOnServer {
initN++
}
if !r.isRepo {
continue
}
repoN++
if r.dirty { if r.dirty {
dirtyN++ dirtyN++
} }
@@ -58,51 +104,267 @@ func overviewAll() {
syncN++ syncN++
} }
} }
fmt.Printf("%s\n", col(cGray, fmt.Sprintf("%d projects · %d dirty · %d in sync", len(rows), dirtyN, syncN)))
summary := fmt.Sprintf("%d projects · %d dirty · %d in sync", repoN, dirtyN, syncN)
if initN > 0 {
summary += fmt.Sprintf(" · %d to init", initN)
}
fmt.Println(col(cDark, summary))
if srv.err != nil {
fmt.Println(col(cDark, " git server not reachable — local view only"))
}
} }
// projectStatus gathers the git state of a single project directory. // scanProjects collects the state of every project concurrently. Each project
// costs two git subprocesses, and serially that is the slowest thing mgsh does.
func scanProjects(names []string) []projStatus {
rows := make([]projStatus, len(names))
sem := make(chan struct{}, overviewScanLimit)
var wg sync.WaitGroup
for i, n := range names {
wg.Add(1)
go func(i int, n string) {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
rows[i] = projectStatus(n, BASE+"/"+n)
}(i, n)
}
wg.Wait()
return rows
}
// markUnpublished flags the rows the git server has never seen — the ones
// `init` is for. A listing that failed leaves every row unmarked: not knowing
// is not the same as knowing they are missing, and marking all of them would
// tell the user to re-init their whole base directory.
func markUnpublished(rows []projStatus, server []string, err error) {
if err != nil {
return
}
onServer := map[string]bool{}
for _, n := range server {
onServer[n] = true
}
for i := range rows {
rows[i].notOnServer = !onServer[rows[i].name]
}
}
// projectStatus gathers the git state of a single project directory. A
// directory without a repository is reported as it is, and costs no
// subprocesses at all.
func projectStatus(name, dir string) projStatus { func projectStatus(name, dir string) projStatus {
s := projStatus{name: name, branch: "-"} s := projStatus{name: name, branch: "-"}
if out, err := gitCapture(dir, "rev-parse", "--abbrev-ref", "HEAD"); err == nil { if s.isRepo = isDir(dir + "/.git"); !s.isRepo {
s.branch = strings.TrimSpace(out) return s
}
if out, err := gitCapture(dir, "status", "--porcelain"); err == nil && strings.TrimSpace(out) != "" {
s.dirty = true
}
// left/right counts against the upstream: "<behind>\t<ahead>"
if out, err := gitCapture(dir, "rev-list", "--left-right", "--count", "@{upstream}...HEAD"); err == nil {
if _, e := fmt.Sscanf(strings.TrimSpace(out), "%d\t%d", &s.behind, &s.ahead); e == nil {
s.hasUpstream = true
}
} }
readStatus(&s, dir)
readLastCommit(&s, dir)
s.mirrors = configuredMirrors(dir)
return s return s
} }
// formatProjStatus renders one aligned overview row. // readStatus fills in branch, upstream, ahead/behind and dirty from a single
func formatProjStatus(s projStatus, width int) string { // `git status` — the porcelain v2 header carries all four.
var marks []string func readStatus(s *projStatus, dir string) {
if s.dirty { out, err := gitCapture(dir, "status", "--porcelain=v2", "--branch")
marks = append(marks, col(cYellow, "*")) if err != nil {
return
} }
if s.ahead > 0 { for _, ln := range splitLines(out) {
marks = append(marks, col(cGreen, fmt.Sprintf("↑%d", s.ahead))) if !strings.HasPrefix(ln, "# ") {
} s.dirty = true // any entry line means the tree is not clean
if s.behind > 0 { continue
marks = append(marks, col(cRed, fmt.Sprintf("↓%d", s.behind))) }
} f := strings.Fields(ln)
state := strings.Join(marks, " ") if len(f) < 3 {
if state == "" { continue
if s.hasUpstream { }
state = col(cGreen, "✓") switch f[1] {
} else { case "branch.head":
state = col(cGray, "✓ (no upstream)") s.branch = f[2]
case "branch.upstream":
s.hasUpstream = true
case "branch.ab":
if len(f) >= 4 {
s.ahead, _ = strconv.Atoi(strings.TrimPrefix(f[2], "+"))
s.behind, _ = strconv.Atoi(strings.TrimPrefix(f[3], "-"))
}
} }
} }
}
line := " " + col(cGreen, padRight(s.name, width+2)) + state
if s.branch != "master" && s.branch != "main" && s.branch != "-" { // readLastCommit records when the project was last committed to and from which
line += col(cGray, " ("+s.branch+")") // machine, taken from the "[user@host]" prefix `push` writes.
} func readLastCommit(s *projStatus, dir string) {
return line out, err := gitCapture(dir, "log", "-1", "--format=%ct%x00%s")
if err != nil {
return
}
parts := strings.SplitN(strings.TrimSpace(out), "\x00", 2)
if len(parts) != 2 {
return
}
if epoch, err := strconv.ParseInt(parts[0], 10, 64); err == nil {
s.lastWhen = time.Unix(epoch, 0)
}
if m := commitHostRe.FindStringSubmatch(parts[1]); m != nil {
s.lastHost = m[1]
}
}
// configuredMirrors returns the mirror targets this repository actually has a
// remote for — free to determine, since it is only local git config.
func configuredMirrors(dir string) []string {
targets, _ := cfg.mirrorTargets()
if len(targets) == 0 {
return nil
}
out, err := gitCapture(dir, "remote")
if err != nil {
return nil
}
have := map[string]bool{}
for _, r := range splitLines(out) {
have[strings.TrimSpace(r)] = true
}
var found []string
for _, t := range targets {
if have[t.Name] {
found = append(found, t.Name)
}
}
return found
}
// overviewWidths are the column widths of the overview table, measured from the
// rows so every field starts at the same place. Ragged columns were what made
// the old one-line-per-project output hard to read.
type overviewWidths struct {
label, sync, host int
hint bool // any row carries an action hint
}
// hintWidth is the width of the action column, sized for its only word.
const hintWidth = 4
// measureOverview sizes the columns for a set of rows.
func measureOverview(rows []projStatus) overviewWidths {
var w overviewWidths
for _, r := range rows {
w.label = max(w.label, utf8.RuneCountInString(projLabel(r)))
w.sync = max(w.sync, utf8.RuneCountInString(syncState(r)))
w.host = max(w.host, utf8.RuneCountInString(r.lastHost))
w.hint = w.hint || r.notOnServer
}
return w
}
// projLabel is the first column: the project, with its branch appended when it
// is not the usual one. Keeping the branch attached to the name costs no extra
// column and keeps the table narrow.
func projLabel(s projStatus) string {
if s.branch != "" && s.branch != "-" && s.branch != "master" && s.branch != "main" {
return s.name + " (" + s.branch + ")"
}
return s.name
}
// syncState renders the relation to the upstream as one short field: ahead,
// behind, both, in sync, or "–" for a branch that tracks nothing. The old
// spelled-out "(no upstream)" was fifteen columns wide and pushed every
// following field out of line.
func syncState(s projStatus) string {
switch {
case !s.isRepo:
return "" // nothing to compare: there is no repository here yet
case s.ahead > 0 && s.behind > 0:
return fmt.Sprintf("↑%d↓%d", s.ahead, s.behind)
case s.ahead > 0:
return fmt.Sprintf("↑%d", s.ahead)
case s.behind > 0:
return fmt.Sprintf("↓%d", s.behind)
case s.hasUpstream:
return "✓"
default:
return "–"
}
}
// syncColor weights a row visually: anything needing action is coloured, a
// project that is clean and in sync recedes into grey.
func syncColor(s projStatus) string {
switch {
case s.behind > 0:
return cRed
case s.ahead > 0:
return cGreen
default:
return cDark
}
}
// attentionRank groups the rows: work in progress at the top, then everything
// that is settled, and last the directories the server does not have yet. With
// many projects, scanning the whole list for the two dirty ones is the actual
// work — and an un-inited directory is a different kind of task, not something
// to push past the daily ones.
func attentionRank(s projStatus) int {
switch {
case s.notOnServer:
return 2
case s.dirty || s.ahead > 0 || s.behind > 0:
return 0
default:
return 1
}
}
// formatProjStatus renders one row of the overview table.
func formatProjStatus(s projStatus, w overviewWidths) string {
dirty := " "
if s.dirty {
dirty = "*"
}
var b strings.Builder
b.WriteString(" ")
b.WriteString(col(cGreen, padRight(projLabel(s), w.label)))
b.WriteString(" " + col(cYellow, dirty) + " ")
b.WriteString(col(syncColor(s), padRight(syncState(s), w.sync)))
if w.host > 0 {
age := ""
if !s.lastWhen.IsZero() {
age = shortAge(time.Since(s.lastWhen))
}
b.WriteString(" " + col(cDark, padRight(s.lastHost, w.host)))
b.WriteString(" " + col(cDark, fmt.Sprintf("%4s", age)))
}
if w.hint {
hint := ""
if s.notOnServer {
hint = "init"
}
b.WriteString(" " + col(cYellow, padRight(hint, hintWidth)))
}
if len(s.mirrors) > 0 {
b.WriteString(col(cDark, " → "+strings.Join(s.mirrors, " ")))
}
return strings.TrimRight(b.String(), " ")
}
// shortAge renders a duration compactly: 90s -> "1m", 36h -> "1d".
func shortAge(d time.Duration) string {
switch {
case d < time.Minute:
return "now"
case d < time.Hour:
return fmt.Sprintf("%dm", int(d.Minutes()))
case d < 24*time.Hour:
return fmt.Sprintf("%dh", int(d.Hours()))
default:
return fmt.Sprintf("%dd", int(d.Hours()/24))
}
} }
overview_test.go
+182
View File
@@ -0,0 +1,182 @@
package main
// overview_test.go — the inventory view: what mgsh knows that plain git cannot.
import (
"errors"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// TestReadStatusParsesPorcelainV2 covers the single `git status` call that
// replaced three separate ones: branch, upstream, ahead/behind and dirty all
// come out of its header.
func TestReadStatusParsesPorcelainV2(t *testing.T) {
dir := t.TempDir()
bare := filepath.Join(t.TempDir(), "o.git")
mustGit(t, "", "init", "--bare", "-q", bare)
mustGit(t, dir, "init", "-q")
mustGit(t, dir, "config", "user.name", "t")
mustGit(t, dir, "config", "user.email", "t@e")
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "[mike@laptop] work")
mustGit(t, dir, "remote", "add", "origin", bare)
mustGit(t, dir, "push", "-q", "-u", "origin", "HEAD")
var s projStatus
readStatus(&s, dir)
if !s.hasUpstream || s.ahead != 0 || s.behind != 0 || s.dirty {
t.Errorf("clean synced repo = %+v", s)
}
if s.branch == "" || s.branch == "-" {
t.Errorf("branch not read: %q", s.branch)
}
// one unstaged file and one unpushed commit
if err := os.WriteFile(filepath.Join(dir, "x"), []byte("x"), 0644); err != nil {
t.Fatal(err)
}
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "[mike@desktop] more")
s = projStatus{}
readStatus(&s, dir)
if !s.dirty {
t.Error("untracked file did not register as dirty")
}
if s.ahead != 1 {
t.Errorf("ahead = %d, want 1", s.ahead)
}
// and the host stamp `push` writes is picked up
var l projStatus
readLastCommit(&l, dir)
if l.lastHost != "desktop" {
t.Errorf("lastHost = %q, want desktop", l.lastHost)
}
if l.lastWhen.IsZero() {
t.Error("lastWhen not read")
}
}
// TestCommitHostRe: only mgsh's own "[user@host]" stamp counts.
func TestCommitHostRe(t *testing.T) {
cases := map[string]string{
"[mike@laptop] fixed a thing": "laptop",
"[mike@build-01] ": "build-01",
"[@host] no user": "host",
"fixed a thing": "",
"[not a stamp] text": "",
"see [a@b] mid-line": "",
}
for subj, want := range cases {
got := ""
if m := commitHostRe.FindStringSubmatch(subj); m != nil {
got = m[1]
}
if got != want {
t.Errorf("host of %q = %q, want %q", subj, got, want)
}
}
}
// TestMarkUnpublishedUnreachableServer: a listing that failed must leave every
// row unmarked. Not knowing is not the same as knowing they are missing —
// marking all of them would tell the user to re-init their whole base.
func TestMarkUnpublishedUnreachableServer(t *testing.T) {
rows := []projStatus{{name: "a"}, {name: "b"}}
markUnpublished(rows, nil, errors.New("network is unreachable"))
for _, r := range rows {
if r.notOnServer {
t.Errorf("%s marked as missing although the server could not be listed", r.name)
}
}
}
// TestMarkUnpublished flags only what the server really does not have.
func TestMarkUnpublished(t *testing.T) {
rows := []projStatus{{name: "both"}, {name: "onlyhere"}}
markUnpublished(rows, []string{"both", "onlythere"}, nil)
if rows[0].notOnServer {
t.Error("a project present on both sides was marked")
}
if !rows[1].notOnServer {
t.Error("a local-only project was not marked")
}
}
// TestUnpublishedRowsCarryTheHint: the entries live in the table now, with the
// action in their own column, rather than in a list underneath it.
func TestUnpublishedRowsCarryTheHint(t *testing.T) {
useColor = false
rows := []projStatus{
{name: "published", isRepo: true, hasUpstream: true},
{name: "fresh", isRepo: true, notOnServer: true},
{name: "notarepo", notOnServer: true},
}
w := measureOverview(rows)
if !w.hint {
t.Fatal("hint column not reserved although rows need it")
}
got := []string{}
for _, r := range rows {
got = append(got, formatProjStatus(r, w))
}
if strings.Contains(got[0], "init") {
t.Errorf("a published project was hinted: %q", got[0])
}
for _, i := range []int{1, 2} {
if !strings.Contains(got[i], "init") {
t.Errorf("row %d missing the init hint: %q", i, got[i])
}
}
// a directory that is not a repository has no sync state to report
if strings.ContainsAny(got[2], "✓–↑↓") {
t.Errorf("non-repository row claims a git state: %q", got[2])
}
// with nothing to hint the column disappears entirely
if measureOverview(rows[:1]).hint {
t.Error("hint column reserved although no row needs it")
}
}
func TestShortAge(t *testing.T) {
cases := []struct {
d time.Duration
want string
}{
{30 * time.Second, "now"},
{90 * time.Second, "1m"},
{2 * time.Hour, "2h"},
{36 * time.Hour, "1d"},
{20 * 24 * time.Hour, "20d"},
}
for _, c := range cases {
if got := shortAge(c.d); got != c.want {
t.Errorf("shortAge(%v) = %q, want %q", c.d, got, c.want)
}
}
}
// captureStdout collects everything a function prints.
func captureStdout(t *testing.T, fn func()) string {
t.Helper()
old := os.Stdout
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
os.Stdout = w
done := make(chan string)
go func() {
var b strings.Builder
io.Copy(&b, r)
done <- b.String()
}()
fn()
w.Close()
os.Stdout = old
return <-done
}
prompt.go
+5 -5
View File
@@ -45,21 +45,21 @@ func plainPrompt() string {
func coloredPrompt() string { func coloredPrompt() string {
var b strings.Builder var b strings.Builder
b.WriteString(cBold + cPurple + "< " + cReset) b.WriteString(cBold + cViolet + "< " + cReset)
b.WriteString(cCyan + filepath.Base(BASE) + cReset) b.WriteString(cBlue + filepath.Base(BASE) + cReset)
if PRJ != "" && isDir(BASE+"/"+PRJ) { if PRJ != "" && isDir(BASE+"/"+PRJ) {
b.WriteString(cBold + cWhite + "/" + cReset + cBold + cGreen + PRJ + cReset) b.WriteString(cBold + cWhite + "/" + cReset + cBold + cGreen + PRJ + cReset)
if BPLSTATE != "" { if BPLSTATE != "" {
b.WriteString(cRed + BPLSTATE + cReset) b.WriteString(cRed + BPLSTATE + cReset)
} }
if BRANCH != "" { if BRANCH != "" {
b.WriteString(cGray + " (" + cReset + cCyan + BRANCH + cReset) b.WriteString(cDark + " (" + cReset + cBlue + BRANCH + cReset)
if DIRTY { if DIRTY {
b.WriteString(cRed + "*" + cReset) b.WriteString(cRed + "*" + cReset)
} }
b.WriteString(cGray + ")" + cReset) b.WriteString(cDark + ")" + cReset)
} }
} }
b.WriteString(" " + cBold + cPurple + ">" + cReset + " ") b.WriteString(" " + cBold + cViolet + ">" + cReset + " ")
return b.String() return b.String()
} }
release.go
+71 -33
View File
@@ -56,46 +56,68 @@ func (r *remoteAPI) releaseByTagPath(owner, repo, tag string) string {
return r.releasePath(owner, repo) + "/tags/" + url.PathEscape(tag) return r.releasePath(owner, repo) + "/tags/" + url.PathEscape(tag)
} }
// findRelease returns the provider's id for the release of tag and whether it // releaseRef identifies a release for the calls that follow creating it.
// exists at all. GitLab addresses releases by tag, so there the tag is the id. type releaseRef struct {
func (r *remoteAPI) findRelease(owner, repo, tag string) (id string, found bool, err error) { tag string
id string // provider id; the tag itself on GitLab
// GitHub uploads assets to a different host than its API, and names it in
// the release object as an RFC 6570 template
uploadURL string
}
// parseReleaseRef reads the identifying fields out of a release object.
func (r *remoteAPI) parseReleaseRef(tag string, data []byte) releaseRef {
ref := releaseRef{tag: tag, id: tag}
if r.kind == kindGitLab {
return ref // GitLab addresses a release by its tag throughout
}
var res struct {
ID int64 `json:"id"`
UploadURL string `json:"upload_url"`
}
json.Unmarshal(data, &res)
ref.id = strconv.FormatInt(res.ID, 10)
// ".../assets{?name,label}" -> ".../assets"
if i := strings.IndexByte(res.UploadURL, '{'); i >= 0 {
ref.uploadURL = res.UploadURL[:i]
} else {
ref.uploadURL = res.UploadURL
}
return ref
}
// findRelease returns the release of tag and whether it exists at all.
func (r *remoteAPI) findRelease(owner, repo, tag string) (ref releaseRef, found bool, err error) {
code, data, err := r.do("GET", r.releaseByTagPath(owner, repo, tag), nil) code, data, err := r.do("GET", r.releaseByTagPath(owner, repo, tag), nil)
if err != nil { if err != nil {
return "", false, err return releaseRef{}, false, err
} }
switch code { switch code {
case 404: case 404:
return "", false, nil return releaseRef{}, false, nil
case 200: case 200:
if r.kind == kindGitLab { return r.parseReleaseRef(tag, data), true, nil
return tag, true, nil
}
var res struct {
ID int64 `json:"id"`
}
json.Unmarshal(data, &res)
return strconv.FormatInt(res.ID, 10), true, nil
default: default:
return "", false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data)) return releaseRef{}, false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data))
} }
} }
// createRelease publishes a new release for an already pushed tag. // createRelease publishes a new release for an already pushed tag.
func (r *remoteAPI) createRelease(owner, repo string, rel release) error { func (r *remoteAPI) createRelease(owner, repo string, rel release) (releaseRef, error) {
code, data, err := r.do("POST", r.releasePath(owner, repo), r.releaseBody(rel, true)) code, data, err := r.do("POST", r.releasePath(owner, repo), r.releaseBody(rel, true))
if err != nil { if err != nil {
return err return releaseRef{}, err
} }
if code != 200 && code != 201 { if code != 200 && code != 201 {
return fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data)) return releaseRef{}, fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data))
} }
return nil return r.parseReleaseRef(rel.Tag, data), nil
} }
// updateRelease rewrites the name and notes of an existing release. Gitea and // updateRelease rewrites the name and notes of an existing release. Gitea and
// GitHub patch it by numeric id, GitLab puts it by tag. // GitHub patch it by numeric id, GitLab puts it by tag.
func (r *remoteAPI) updateRelease(owner, repo, id string, rel release) error { func (r *remoteAPI) updateRelease(owner, repo string, ref releaseRef, rel release) error {
method, ep := "PATCH", r.releasePath(owner, repo)+"/"+id method, ep := "PATCH", r.releasePath(owner, repo)+"/"+ref.id
if r.kind == kindGitLab { if r.kind == kindGitLab {
method, ep = "PUT", r.releaseByTagPath(owner, repo, rel.Tag) method, ep = "PUT", r.releaseByTagPath(owner, repo, rel.Tag)
} }
@@ -147,15 +169,13 @@ func handleRelease(args string) {
return return
} }
targets, incomplete := cfg.mirrorTargets() configured, incomplete := cfg.mirrorTargets()
for _, n := range incomplete { for _, n := range incomplete {
errorln("remote " + n + ": url or key missing — skipped") errorln("remote " + n + ": url or key missing — skipped")
} }
targets = pickRemotes(targets, names) targets := pickRemotes(configured, names)
if len(targets) == 0 { if len(targets) == 0 {
if len(names) == 0 { // an unknown @name already reported itself reportNoTargets("release", configured, names)
errorln("release needs a mirror target — see 'config'")
}
return return
} }
@@ -164,14 +184,25 @@ func handleRelease(args string) {
body = releaseNotes(DIR, tag) body = releaseNotes(DIR, tag)
} }
// ./bin and ./assets ride along when they are there; nothing to configure,
// and nothing happens for a project that has neither
assets, skipped := collectAssets(DIR)
for _, s := range skipped {
errorln("skipping " + s + ": another directory already contributes that name")
}
if len(assets) > 0 {
fmt.Printf("%s %s %s\n", col(cDark, "attaching"), col(cYellow, assetSummary(assets)),
col(cDark, "from "+strings.Join(assetDirList(DIR), " and ")))
}
done := 0 done := 0
for _, t := range targets { for _, t := range targets {
if publishRelease(t, PRJ, tag, body) { if publishRelease(t, PRJ, tag, body, assets) {
done++ done++
} }
} }
if len(targets) > 1 { if len(targets) > 1 {
fmt.Printf("%s %d/%d remotes released\n", col(cGray, "release:"), done, len(targets)) fmt.Printf("%s %d/%d remotes released\n", col(cDark, "release:"), done, len(targets))
} }
} }
@@ -223,7 +254,7 @@ func releaseNotes(dir, tag string) string {
// publishRelease pushes the tag to one mirror target and turns it into a // publishRelease pushes the tag to one mirror target and turns it into a
// release there. It returns success, so one unreachable server does not stop // release there. It returns success, so one unreachable server does not stop
// the remaining ones. // the remaining ones.
func publishRelease(t RemoteTarget, repo, tag, body string) bool { func publishRelease(t RemoteTarget, repo, tag, body string, assets []releaseAsset) bool {
api := newRemoteAPI(t.URL, t.Key, t.Type) api := newRemoteAPI(t.URL, t.Key, t.Type)
owner, err := api.authUser() owner, err := api.authUser()
@@ -248,17 +279,17 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
} }
rel := release{Tag: tag, Name: tag, Body: body, Prerelease: preReleaseRe.MatchString(tag)} rel := release{Tag: tag, Name: tag, Body: body, Prerelease: preReleaseRe.MatchString(tag)}
id, found, err := api.findRelease(owner, repo, tag) ref, found, err := api.findRelease(owner, repo, tag)
if err != nil { if err != nil {
errorln(t.Name + ": " + err.Error()) errorln(t.Name + ": " + err.Error())
return false return false
} }
if found { if found {
if err := api.updateRelease(owner, repo, id, rel); err != nil { if err := api.updateRelease(owner, repo, ref, rel); err != nil {
errorln(t.Name + ": " + err.Error()) errorln(t.Name + ": " + err.Error())
return false return false
} }
} else if err := api.createRelease(owner, repo, rel); err != nil { } else if ref, err = api.createRelease(owner, repo, rel); err != nil {
errorln(t.Name + ": " + err.Error()) errorln(t.Name + ": " + err.Error())
return false return false
} }
@@ -270,7 +301,14 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
if rel.Prerelease { if rel.Prerelease {
what += " (pre-release)" what += " (pre-release)"
} }
fmt.Printf("%s %s %s %s\n", col(cGray, "remote"), col(cYellow, t.Name), fmt.Printf("%s %s %s %s\n", col(cDark, "remote"), col(cYellow, t.Name),
col(cGreen, what), col(cCyan, api.repoWebURL(owner, repo))) col(cGreen, what), col(cBlue, api.repoWebURL(owner, repo)))
if len(assets) > 0 {
if err := api.uploadAssets(owner, repo, ref, assets); err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
}
return true return true
} }
release_test.go
+15 -11
View File
@@ -9,6 +9,7 @@ package main
import ( import (
"encoding/json" "encoding/json"
"io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
@@ -20,6 +21,8 @@ import (
// recordedReq is one request the fake provider received. // recordedReq is one request the fake provider received.
type recordedReq struct { type recordedReq struct {
method, path, query string method, path, query string
ctype string
raw []byte // the body as sent, for the asset uploads
body map[string]any body map[string]any
} }
@@ -42,11 +45,12 @@ func newFakeProvider(t *testing.T) *fakeProvider {
body string body string
}{}} }{}}
f.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { f.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery} rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery,
ctype: r.Header.Get("Content-Type")}
if r.Body != nil { if r.Body != nil {
var m map[string]any raw, _ := io.ReadAll(r.Body)
json.NewDecoder(r.Body).Decode(&m) rec.raw = raw
rec.body = m json.Unmarshal(raw, &rec.body)
} }
f.got = append(f.got, rec) f.got = append(f.got, rec)
@@ -106,7 +110,7 @@ func TestCreateReleasePerProvider(t *testing.T) {
f.route("POST "+c.wantPath, 201, `{"id":7}`) f.route("POST "+c.wantPath, 201, `{"id":7}`)
api := newRemoteAPI(f.URL, "tok", c.typ) api := newRemoteAPI(f.URL, "tok", c.typ)
if err := api.createRelease("mike", "mgsh", rel); err != nil { if _, err := api.createRelease("mike", "mgsh", rel); err != nil {
t.Fatalf("%s: createRelease: %v (requests: %v)", c.typ, err, f.paths()) t.Fatalf("%s: createRelease: %v (requests: %v)", c.typ, err, f.paths())
} }
@@ -145,11 +149,11 @@ func TestFindReleaseAndUpdate(t *testing.T) {
f.route(c.wantMethod+" "+c.wantUpdate, 200, `{}`) f.route(c.wantMethod+" "+c.wantUpdate, 200, `{}`)
api := newRemoteAPI(f.URL, "tok", c.typ) api := newRemoteAPI(f.URL, "tok", c.typ)
id, found, err := api.findRelease("mike", "mgsh", "v1.2") ref, found, err := api.findRelease("mike", "mgsh", "v1.2")
if err != nil || !found { if err != nil || !found {
t.Fatalf("%s: findRelease = %q,%v,%v", c.typ, id, found, err) t.Fatalf("%s: findRelease = %+v,%v,%v", c.typ, ref, found, err)
} }
if err := api.updateRelease("mike", "mgsh", id, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil { if err := api.updateRelease("mike", "mgsh", ref, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil {
t.Fatalf("%s: updateRelease: %v (requests: %v)", c.typ, err, f.paths()) t.Fatalf("%s: updateRelease: %v (requests: %v)", c.typ, err, f.paths())
} }
req := f.find(c.wantMethod + " " + c.wantUpdate) req := f.find(c.wantMethod + " " + c.wantUpdate)
@@ -168,9 +172,9 @@ func TestFindReleaseAndUpdate(t *testing.T) {
func TestFindReleaseMissing(t *testing.T) { func TestFindReleaseMissing(t *testing.T) {
f := newFakeProvider(t) // everything 404s f := newFakeProvider(t) // everything 404s
api := newRemoteAPI(f.URL, "tok", "gitea") api := newRemoteAPI(f.URL, "tok", "gitea")
id, found, err := api.findRelease("mike", "mgsh", "v9") ref, found, err := api.findRelease("mike", "mgsh", "v9")
if err != nil || found || id != "" { if err != nil || found || ref.id != "" {
t.Fatalf("findRelease on empty server = %q,%v,%v", id, found, err) t.Fatalf("findRelease on empty server = %+v,%v,%v", ref, found, err)
} }
} }
remote.go
+82 -27
View File
@@ -4,23 +4,18 @@ package main
// git hosting server (Gitea, GitHub or GitLab), creating the repository via the // git hosting server (Gitea, GitHub or GitLab), creating the repository via the
// server's REST API when it does not exist yet. // server's REST API when it does not exist yet.
// //
// Configuration (in ~/.mgshrc, a project .mgshrc, or MGSH_* env) — either a // Configuration (in ~/.mgshrc, a project .mgshrc, or MGSH_* env): one
// single flat target: // remote.<name>.<field> block per server, which `pushremote` mirrors to in turn.
// //
// remoteurl = https://git.example.com base URL of the server // remote.gitlab.url = https://gitlab.example.com
// remotekey = <api-token> personal access token // remote.gitlab.key = <api-token>
// remotetype = gitea|github|gitlab optional; auto-detected from the URL // remote.gitlab.type = gitlab optional; detected from the url
// remote.gitlab.visibility = public or private (the default)
// remotes = gitlab optional: restrict/order the set
// //
// or any number of named ones, which `pushremote` mirrors to in turn: // Each target owns a git remote of its own name in the repository. There is no
// // second spelling: the pre-4.1 flat remoteurl/remotekey pair is migrated to
// remote.gitea.url = https://git.example.com // remote.public.* on load.
// remote.gitea.key = <api-token>
// remote.hub.url = https://github.com
// remote.hub.key = <api-token>
// remote.hub.visibility = public
// remotes = gitea, hub optional: restrict/order the set
//
// Each target owns a git remote of the same name in the repository.
// //
// The token is used for the API calls and, via an HTTP Basic auth header, for // The token is used for the API calls and, via an HTTP Basic auth header, for
// the git push. It is never written into the repository's git config, and it is // the git push. It is never written into the repository's git config, and it is
@@ -243,18 +238,64 @@ func parsePushRemoteArgs(args string) (names []string, description string) {
return names, strings.Join(fields[i:], " ") return names, strings.Join(fields[i:], " ")
} }
// remoteHost reduces a configured url — or whatever a user typed in its place —
// to its bare host: scheme, credentials, port and path removed.
func remoteHost(s string) string {
s = strings.TrimSpace(s)
if i := strings.Index(s, "://"); i >= 0 {
s = s[i+3:]
}
if i := strings.IndexByte(s, '@'); i >= 0 { // user[:pass]@host
s = s[i+1:]
}
if i := strings.IndexAny(s, "/:"); i >= 0 {
s = s[:i]
}
return s
}
// remoteMatches reports whether sel picks target t. A target answers both to its
// configured name and to the host of its url: the name is what the config calls
// the server, the host is what the user sees in the browser, and `@gitea` should
// not be the only way to say `git.example.com`.
func remoteMatches(t RemoteTarget, sel string) bool {
sel = strings.TrimPrefix(strings.TrimSpace(sel), "@")
if sel == "" {
return false
}
if strings.EqualFold(t.Name, sel) {
return true
}
h := remoteHost(t.URL)
return h != "" && strings.EqualFold(h, remoteHost(sel))
}
// activeRemotes keeps the targets that take part when a command was given no
// target of its own (see RemoteTarget.isActive).
func activeRemotes(all []RemoteTarget) []RemoteTarget {
out := make([]RemoteTarget, 0, len(all))
for _, t := range all {
if t.isActive() {
out = append(out, t)
}
}
return out
}
// pickRemotes narrows all to the explicitly requested names, complaining about // pickRemotes narrows all to the explicitly requested names, complaining about
// any that are not configured. With no names given, all targets are used. // any that are not configured. With no names given the active targets are used
// — `active = false` takes a server off that automatic path, and naming it is
// what puts it back on.
func pickRemotes(all []RemoteTarget, names []string) []RemoteTarget { func pickRemotes(all []RemoteTarget, names []string) []RemoteTarget {
if len(names) == 0 { if len(names) == 0 {
return all return activeRemotes(all)
} }
var out []RemoteTarget var out []RemoteTarget
seen := map[string]bool{} seen := map[string]bool{}
for _, n := range names { for _, n := range names {
found := false found := false
for _, t := range all { for _, t := range all {
if !strings.EqualFold(t.Name, n) { if !remoteMatches(t, n) {
continue continue
} }
if !seen[t.Name] { // `@hub @hub` must not push twice if !seen[t.Name] { // `@hub @hub` must not push twice
@@ -271,8 +312,24 @@ func pickRemotes(all []RemoteTarget, names []string) []RemoteTarget {
return out return out
} }
// reportNoTargets explains an empty selection. There are three ways to end up
// with nothing to push to, and they need different answers: nothing configured
// at all, everything configured switched to `active = false`, or a name nobody
// recognised — which pickRemotes has already complained about by itself.
func reportNoTargets(cmd string, configured []RemoteTarget, names []string) {
switch {
case len(names) > 0:
return
case len(configured) > 0:
errorln("every configured remote is inactive — name one, e.g. '" +
cmd + " @" + configured[0].Name + "'")
default:
errorln(cmd + " needs a 'remote.<name>.url' and 'remote.<name>.key' in " + configFile())
}
}
// handlePushRemote implements `pushremote [@name ...] [description]`. Without // handlePushRemote implements `pushremote [@name ...] [description]`. Without
// a @name it mirrors to every configured target; the description, if given, is // a @name it mirrors to every active target; the description, if given, is
// set on the repository when it is created. // set on the repository when it is created.
func handlePushRemote(args string) { func handlePushRemote(args string) {
if !requireRepo() { if !requireRepo() {
@@ -280,15 +337,13 @@ func handlePushRemote(args string) {
} }
names, description := parsePushRemoteArgs(args) names, description := parsePushRemoteArgs(args)
targets, incomplete := cfg.mirrorTargets() configured, incomplete := cfg.mirrorTargets()
for _, n := range incomplete { for _, n := range incomplete {
errorln("remote " + n + ": url or key missing — skipped") errorln("remote " + n + ": url or key missing — skipped")
} }
targets = pickRemotes(targets, names) targets := pickRemotes(configured, names)
if len(targets) == 0 { if len(targets) == 0 {
if len(names) == 0 { // an unknown @name already reported itself reportNoTargets("pushremote", configured, names)
errorln("pushremote needs 'remoteurl'/'remotekey' or a 'remote.<name>.*' block in " + configFile())
}
return return
} }
@@ -301,7 +356,7 @@ func handlePushRemote(args string) {
} }
} }
if len(targets) > 1 { if len(targets) > 1 {
fmt.Printf("%s %d/%d remotes updated\n", col(cGray, "pushremote:"), done, len(targets)) fmt.Printf("%s %d/%d remotes updated\n", col(cDark, "pushremote:"), done, len(targets))
} }
} }
@@ -318,7 +373,7 @@ func pushToRemote(t RemoteTarget, repo, description string) bool {
return false return false
} }
fmt.Printf("%s %s %s (as %s)\n", fmt.Printf("%s %s %s (as %s)\n",
col(cGray, "remote"), col(cYellow, t.Name), col(cCyan, api.url), col(cGreen, owner)) col(cDark, "remote"), col(cYellow, t.Name), col(cBlue, api.url), col(cGreen, owner))
exists, err := api.repoExists(owner, repo) exists, err := api.repoExists(owner, repo)
if err != nil { if err != nil {
@@ -348,7 +403,7 @@ func pushToRemote(t RemoteTarget, repo, description string) bool {
return false return false
} }
gitPushHeader(DIR, t.Name, header, "--tags") gitPushHeader(DIR, t.Name, header, "--tags")
fmt.Println(col(cGreen, "pushed to ") + col(cCyan, web)) fmt.Println(col(cGreen, "pushed to ") + col(cBlue, web))
return true return true
} }
remotecmd_test.go
+166 -9
View File
@@ -101,7 +101,7 @@ func TestInitKeepsServerRepoWhenDeclined(t *testing.T) {
if c := findCmd(*sent, "rm -rf"); c != "" { if c := findCmd(*sent, "rm -rf"); c != "" {
t.Fatalf("init destroyed the server repository after the user declined: %q", c) t.Fatalf("init destroyed the server repository after the user declined: %q", c)
} }
if c := findCmd(*sent, "git --bare init"); c != "" { if c := findCmd(*sent, "git init --bare"); c != "" {
t.Fatalf("init re-created the repository after the user declined: %q", c) t.Fatalf("init re-created the repository after the user declined: %q", c)
} }
} }
@@ -147,7 +147,7 @@ func TestInitOnFreshProjectCreatesRepo(t *testing.T) {
if findCmd(*sent, "rm -rf "+shq("/home/git/notes.git")) == "" { if findCmd(*sent, "rm -rf "+shq("/home/git/notes.git")) == "" {
t.Errorf("init did not clear the target path, sent: %q", *sent) t.Errorf("init did not clear the target path, sent: %q", *sent)
} }
if c := findCmd(*sent, "git --bare init"); !strings.Contains(c, shq("/home/git/notes.git")) { if c := findCmd(*sent, "git init --bare"); !strings.Contains(c, shq("/home/git/notes.git")) {
t.Errorf("init did not create the bare repository at the configured path: %q", c) t.Errorf("init did not create the bare repository at the configured path: %q", c)
} }
} }
@@ -224,25 +224,110 @@ func TestArchiveNamesAndQuotesSnapshot(t *testing.T) {
} }
cp, tar, rm := (*sent)[1], (*sent)[2], (*sent)[3] cp, tar, rm := (*sent)[1], (*sent)[2], (*sent)[3]
// name is <project>_<stamp>_<sanitised comment> // name is <project>_<stamp>_<sanitised comment>, and every path is spelled
// out from gitpath — the login directory is not necessarily the same place
stamp := archiveStamp() stamp := archiveStamp()
name := "notes_" + stamp + "_before_rewrite" name := "notes_" + stamp + "_before_rewrite"
if !strings.HasPrefix(cp, "cp -r "+shq("notes.git")+" ") { // the archive directory is made first: on a server where nobody created it,
// the copy used to fail with a raw cp error
if !strings.HasPrefix(cp, "mkdir -p "+shq("/home/git/archive")+" && cp -r "+shq("/home/git/notes.git")+" ") {
t.Errorf("cp command = %q", cp) t.Errorf("cp command = %q", cp)
} }
if !strings.Contains(cp, shq("archive/"+name+".git")) { if !strings.Contains(cp, shq("/home/git/archive/"+name+".git")) {
t.Errorf("cp target = %q, want it to contain %q", cp, name) t.Errorf("cp target = %q, want it to contain %q", cp, name)
} }
// `cd archive && tar`, not `cd archive;tar`: a failed cd must not let tar // `tar -C <dir>`, not `cd <dir> && tar`: on the tcsh server `cd` is aliased
// run in the login directory // to `cd !*;echo $cwd`, which turns the guard into `cd X; echo && tar` —
if !strings.HasPrefix(tar, "cd archive && tar ") || !strings.Contains(tar, shq(name+".git.tar.gz")) { // tar then runs in the login directory even when the cd failed, and the
// whole command still reports success
if !strings.HasPrefix(tar, "tar cvzf "+shq("/home/git/archive/"+name+".git.tar.gz")+" -C "+shq("/home/git/archive")+" ") ||
!strings.Contains(tar, shq(name+".git")) {
t.Errorf("tar command = %q", tar) t.Errorf("tar command = %q", tar)
} }
if rm != "rm -rf "+shq("archive/"+name+".git") { if rm != "rm -rf "+shq("/home/git/archive/"+name+".git") {
t.Errorf("cleanup command = %q", rm) t.Errorf("cleanup command = %q", rm)
} }
} }
// TestServerCommandsAreAnchoredAtGitPath: mgsh used to address the server
// through the login directory, which only worked because `gituser = git` and
// `gitpath = /home/git` happen to be the same place. With `gituser = root` and
// `gitpath = /root/mgsh` every command went to /root instead — `list` came back
// empty and `archive` had nothing to copy.
func TestServerCommandsAreAnchoredAtGitPath(t *testing.T) {
useProject(t, "notes")
cfg.GitPath = "/root/mgsh" // the repositories are NOT in the login directory
sent := fakeServer(t, func(cmd string) (string, error) {
if strings.HasPrefix(cmd, "/bin/ls") {
return "notes.git\n", nil
}
return "", nil
})
captureStdout(t, func() {
runCommand("list")
runCommand("list -a")
runCommand("show notes")
runCommand("archive")
})
if len(*sent) == 0 {
t.Fatal("no remote commands recorded")
}
for _, c := range *sent {
if !strings.Contains(c, "/root/mgsh") {
t.Errorf("remote command not anchored at gitpath: %q", c)
}
// No `cd` either. On the tcsh server it is aliased to `cd !*;echo $cwd`,
// which splits `cd X && Y` into `cd X; echo $cwd && Y`: Y runs even when
// the cd failed, and the command still exits 0. Every tool mgsh uses can
// be told its directory instead — tar -C, git --git-dir, find <path>.
if strings.HasPrefix(c, "cd ") || strings.Contains(c, " cd ") {
t.Errorf("remote command relies on cd: %q", c)
}
}
}
// TestRemoteCommandsRefuseExclamationMark: '!' is the one character shq cannot
// protect. csh expands history before it looks at quotes, and does so
// non-interactively too, so `cp -r '/home/git/wei!rd.git' …` arrives as
// something else. Such a command must not be sent at all.
func TestRemoteCommandsRefuseExclamationMark(t *testing.T) {
useProject(t, "wei!rd")
sent := fakeServer(t, func(cmd string) (string, error) {
if strings.HasPrefix(cmd, "/bin/ls") {
return "wei!rd.git\n", nil // the repository is there, so both proceed
}
return "", nil
})
out := captureStdout(t, func() {
runCommand("show wei!rd")
runCommand("archive")
})
for _, c := range *sent {
if strings.Contains(c, "!") {
t.Errorf("sent a command containing '!': %q", c)
}
}
if c := findCmd(*sent, "--git-dir"); c != "" {
t.Errorf("show sent %q despite the '!' in the name", c)
}
if c := findCmd(*sent, "cp -r"); c != "" {
t.Errorf("archive sent %q despite the '!' in the name", c)
}
if !strings.Contains(out, "not sending") {
t.Errorf("output = %q, want the refusal to say what it did not do", out)
}
// the other way in is an archive comment, which is why sanitizeComment
// drops the character before it ever becomes part of a name
if got := sanitizeComment("fix!now"); strings.ContainsRune(got, '!') {
t.Errorf("sanitizeComment(%q) = %q, want the '!' gone", "fix!now", got)
}
}
// TestCloneRefusesUnknownRepository: `clone` must not start a git clone for a // TestCloneRefusesUnknownRepository: `clone` must not start a git clone for a
// repository the server does not list. // repository the server does not list.
func TestCloneRefusesUnknownRepository(t *testing.T) { func TestCloneRefusesUnknownRepository(t *testing.T) {
@@ -329,3 +414,75 @@ func mustGit(t *testing.T, dir string, args ...string) {
t.Fatalf("git %v: %v\n%s", args, err, out) t.Fatalf("git %v: %v\n%s", args, err, out)
} }
} }
// TestListSurvivesFailingDu: `list` chains the listing and `du` into one remote
// command, and the exit status is the *last* command's. A server whose du fails
// — a shell that mis-parses the arguments, a du that is not there, a permission
// problem — must still get its repositories listed.
func TestListSurvivesFailingDu(t *testing.T) {
useProject(t, "x")
fakeServer(t, func(cmd string) (string, error) {
return "total 4\n" +
"drwxr-xr-x 7 git git 4096 Jan 3 14:32 notes.git\n" +
"drwxr-xr-x 7 git git 4096 Sep 28 2016 website.git\n" +
listMarker + "\n",
errors.New("exit status 1") // du blew up, ls did not
})
out := captureStdout(t, func() { runCommand("list") })
if strings.Contains(out, "could not list") {
t.Errorf("a failing du discarded a good listing:\n%s", out)
}
for _, want := range []string{"notes", "website", "2 repositories"} {
if !strings.Contains(out, want) {
t.Errorf("listing missing %q:\n%s", want, out)
}
}
// without sizes there must be no size column, not a column of zeroes
if strings.Contains(out, "0B") {
t.Errorf("zero sizes shown when du produced none:\n%s", out)
}
}
// TestListReportsATrulyFailedListing: when nothing usable came back, the error
// still has to surface.
func TestListReportsATrulyFailedListing(t *testing.T) {
useProject(t, "x")
fakeServer(t, func(cmd string) (string, error) {
return "", errors.New("ssh: connect failed")
})
out := captureStdout(t, func() { runCommand("list") })
if !strings.Contains(out, "could not list") {
t.Errorf("a failed listing was not reported:\n%s", out)
}
}
// TestListSendsNoShellSpecificSyntax guards the bug this replaced: the remote
// command is run by the git user's login shell, which may be csh, where
// "2>/dev/null" is an argument followed by a redirection rather than a
// redirection of stderr.
func TestListSendsNoShellSpecificSyntax(t *testing.T) {
useProject(t, "x")
sent := fakeServer(t, func(cmd string) (string, error) { return "", nil })
captureStdout(t, func() { runCommand("list") })
if len(*sent) == 0 {
t.Fatal("list sent nothing")
}
for _, c := range *sent {
if strings.Contains(c, "2>") || strings.Contains(c, "&>") {
t.Errorf("remote command uses sh-only redirection: %q", c)
}
// A glob is expanded by that same login shell, and a non-interactive
// zsh that finds nothing to match does not pass the pattern on like sh
// does — it fails the command outright ("no matches found: *.git"),
// which is how an empty server came to be reported as unreachable.
for _, idx := range indexesOf(c, "*") {
if idx == 0 || c[idx-1] != '\'' {
t.Errorf("unquoted glob in remote command: %q", c)
break
}
}
}
}
secrets.go
+206
View File
@@ -0,0 +1,206 @@
package main
// secrets.go — a last look at what `push` is about to commit.
//
// `push` runs `git add --all .`, so anything lying in the project — an .env, a
// stray key file, a token pasted into a config — is committed and pushed, and
// with `mirror = true` it reaches a *public* server in the same breath. That is
// the one action in mgsh that cannot be undone: a deleted server repository can
// come back from an archive, a published credential is burnt.
//
// So the staged diff is scanned for a small set of high-signal patterns before
// the commit is made. This is not a complete secret scanner and does not try to
// be one; it aims for a high hit rate on the things that actually leak, with
// few enough false alarms that the prompt still means something. Turn it off
// with `secretscan = off`.
import (
"fmt"
"regexp"
"strconv"
"strings"
)
// secretHit is one suspicious added line.
type secretHit struct {
file string
lineNo int
kind string
text string
}
// secretPattern matches one kind of credential. `certain` patterns are
// unmistakable and are reported as they are; the others match a shape that
// merely looks like a secret and are filtered through looksLikePlaceholder.
type secretPattern struct {
kind string
re *regexp.Regexp
certain bool
}
var secretPatterns = []secretPattern{
{"private key", regexp.MustCompile(`-----BEGIN (?:[A-Z]+ )?PRIVATE KEY-----`), true},
{"GitHub token", regexp.MustCompile(`\bgh[pousr]_[A-Za-z0-9]{20,}`), true},
{"GitLab token", regexp.MustCompile(`\bglpat-[A-Za-z0-9_-]{16,}`), true},
{"AWS access key", regexp.MustCompile(`\b(?:AKIA|ASIA)[0-9A-Z]{16}\b`), true},
{"Slack token", regexp.MustCompile(`\bxox[baprs]-[A-Za-z0-9-]{10,}`), true},
{"PyPI token", regexp.MustCompile(`\bpypi-AgEIcHlwaS5vcmc[A-Za-z0-9_-]{10,}`), true},
{"credential assignment", regexp.MustCompile(
`(?i)\b(?:password|passwd|secret|api[_-]?key|apikey|access[_-]?key|auth[_-]?token|token)\b` +
`\s*[:=]\s*(?:"([^"\s]{12,})"|'([^'\s]{12,})'|([^\s"';,]{20,}))\s*;?\s*$`), false},
}
var (
diffFileRe = regexp.MustCompile(`^\+\+\+ b/(.*)$`)
diffHunkRe = regexp.MustCompile(`^@@ -\d+(?:,\d+)? \+(\d+)`)
// a value that is plainly a reference or a stand-in, not a credential
constRefRe = regexp.MustCompile(`^[A-Z][A-Z0-9_]*$`)
dottedRefRe = regexp.MustCompile(`^[\w-]+(?:\.[\w-]+)+$`)
maskedRe = regexp.MustCompile(`^[*x•.]+$`)
)
// placeholderWords are the values people write when they mean "fill this in".
var placeholderWords = map[string]bool{
"changeme": true, "change_me": true, "password": true, "secret": true,
"token": true, "your_token": true, "your-token": true, "yourtoken": true,
"todo": true, "none": true, "null": true, "example": true, "redacted": true,
}
// looksLikePlaceholder reports whether a matched value is obviously not a real
// credential: a template slot, an environment reference, a constant name, or a
// masked stand-in. Documentation and example files are full of these, and every
// one of them that reaches the prompt makes the prompt worth less.
func looksLikePlaceholder(v string) bool {
v = strings.Trim(v, `"'`)
if v == "" {
return true
}
if strings.ContainsAny(v, "<>${}()") { // <token>, ${VAR}, $(cmd), {{ tpl }}
return true
}
if maskedRe.MatchString(v) || constRefRe.MatchString(v) || dottedRefRe.MatchString(v) {
return true
}
if placeholderWords[strings.ToLower(v)] {
return true
}
// a value made of one repeated character carries no information
if strings.Count(v, string(v[0])) == len(v) {
return true
}
return false
}
// scanDiff finds suspicious added lines in a unified diff. Only added lines are
// examined: removing a secret is what we want people to do.
func scanDiff(diff string) []secretHit {
var hits []secretHit
file := ""
lineNo := 0
for _, ln := range strings.Split(diff, "\n") {
switch {
case strings.HasPrefix(ln, "+++ "):
file = ""
if m := diffFileRe.FindStringSubmatch(ln); m != nil {
file = m[1]
}
continue
case strings.HasPrefix(ln, "@@"):
if m := diffHunkRe.FindStringSubmatch(ln); m != nil {
lineNo, _ = strconv.Atoi(m[1])
}
continue
case strings.HasPrefix(ln, "---") || strings.HasPrefix(ln, "diff ") ||
strings.HasPrefix(ln, "index ") || strings.HasPrefix(ln, "new file") ||
strings.HasPrefix(ln, "deleted file") || strings.HasPrefix(ln, "similarity "):
continue
case strings.HasPrefix(ln, "-"):
continue // removed line: not our problem
case !strings.HasPrefix(ln, "+"):
lineNo++ // context line
continue
}
text := ln[1:]
if kind := matchSecret(text); kind != "" {
hits = append(hits, secretHit{file: file, lineNo: lineNo, kind: kind, text: text})
}
lineNo++
}
return hits
}
// allowMarker suppresses the check for one line. Any scanner needs a per-line
// escape: a project will eventually hold something credential-shaped on
// purpose, and switching the whole check off for that is far too blunt.
const allowMarker = "mgsh:allow"
// matchSecret returns the kind of credential a line appears to contain, or "".
func matchSecret(text string) string {
if strings.Contains(text, allowMarker) {
return ""
}
for _, p := range secretPatterns {
m := p.re.FindStringSubmatch(text)
if m == nil {
continue
}
if p.certain {
return p.kind
}
// the first non-empty capture group is the value that was assigned
value := ""
for _, g := range m[1:] {
if g != "" {
value = g
break
}
}
if !looksLikePlaceholder(value) {
return p.kind
}
}
return ""
}
// secretScanEnabled reports whether the scan runs. It is on unless explicitly
// switched off, so a typo in the setting leaves the safety net in place.
func secretScanEnabled() bool { return !falsy(cfg.SecretScan) }
// secretsApproved scans what `push` has staged. With nothing suspicious found
// it returns true silently; otherwise it shows the findings and asks. Returns
// false when the push should stop.
func secretsApproved(dir string) bool {
if !secretScanEnabled() {
return true
}
diff, err := gitCapture(dir, "diff", "--cached", "-U0", "--no-color")
if err != nil {
return true // nothing staged, or no HEAD yet: not our call to block
}
hits := scanDiff(diff)
if len(hits) == 0 {
return true
}
fmt.Println(col(cRed, fmt.Sprintf("%d possible credential(s) in what is about to be committed:", len(hits))))
for _, h := range hits {
where := h.file
if h.lineNo > 0 {
where += ":" + strconv.Itoa(h.lineNo)
}
fmt.Printf(" %s %s\n %s\n",
col(cYellow, where), col(cDark, h.kind), col(cRed, ellipsis(strings.TrimSpace(h.text), 100)))
}
fmt.Println(col(cDark, " (set 'secretscan = off' to skip this check)"))
return yesno("push anyway?", false)
}
// ellipsis shortens s to at most n characters.
func ellipsis(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n-1] + "…"
}
secrets_test.go
+212
View File
@@ -0,0 +1,212 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// TestMatchSecretCatchesRealCredentials: the shapes that actually leak.
func TestMatchSecretCatchesRealCredentials(t *testing.T) {
lines := []string{
"-----BEGIN OPENSSH PRIVATE KEY-----",
"-----BEGIN RSA PRIVATE KEY-----",
" ghp_aB3dEfGh1jKlMn0pQrStUvWxYz012345678",
"GITLAB=glpat-aB3dEfGh1jKlMn0pQrSt",
`aws_access_key_id = AKIAIOSFODNN7EXAMPLE`,
"slack: xoxb-1234567890-abcdefghij",
`API_KEY="s3cr3tV4lu3W1thStuff"`,
"password = hunter2hunter2hunter2",
"token: 'aB3dEfGh1jKlMn0pQrSt'",
"auth-token=9f8e7d6c5b4a39281706abcdef123456",
}
for _, ln := range lines {
if matchSecret(ln) == "" {
t.Errorf("missed a credential in %q", ln)
}
}
}
// TestMatchSecretIgnoresNoise: everyday code and documentation must not trip
// the prompt, or people learn to answer "yes" without reading it.
func TestMatchSecretIgnoresNoise(t *testing.T) {
lines := []string{
"remotekey = <personal-access-token>", // our own README
"# remotekey = <personal-access-token>", // and mgshrc.example
"token = process.env.GITHUB_TOKEN", // reference, not a value
"const token = getToken()", // call
"password = ${DB_PASSWORD}", // template
`api_key = "changeme"`, // placeholder
"secret: TODO", //
"key gh***************xk", // masked, from `config`
"password = xxxxxxxxxxxxxxxxxxxxxxx", // masked
"// the token is never persisted in the repo", // prose
"apiKey := os.Getenv(\"MGSH_REMOTEKEY\")", // lookup
"token = SOME_CONSTANT_NAME", // constant
"secret = my.config.value", // dotted reference
"+++ b/token.go", // diff furniture
"password = short", // too short to be one
"Authorization: Basic <base64(owner:token)>", // documentation
"remote.hub.key = <personal-access-token>",
}
for _, ln := range lines {
if kind := matchSecret(ln); kind != "" {
t.Errorf("false positive (%s) on %q", kind, ln)
}
}
}
// TestScanDiffReportsFileAndLine: the report has to point at the right place,
// and must ignore removed lines — deleting a secret is the desired action.
func TestScanDiffReportsFileAndLine(t *testing.T) {
diff := `diff --git a/.env b/.env
new file mode 100644
--- /dev/null
+++ b/.env
@@ -0,0 +1,3 @@
+HOME=/tmp
+API_KEY="s3cr3tV4lu3W1thStuff"
+DEBUG=1
diff --git a/old.txt b/old.txt
--- a/old.txt
+++ b/old.txt
@@ -7,1 +7,0 @@
-password = hunter2hunter2hunter2
`
hits := scanDiff(diff)
if len(hits) != 1 {
t.Fatalf("expected exactly one hit, got %d: %+v", len(hits), hits)
}
h := hits[0]
if h.file != ".env" {
t.Errorf("file = %q, want .env", h.file)
}
if h.lineNo != 2 {
t.Errorf("lineNo = %d, want 2", h.lineNo)
}
if !strings.Contains(h.text, "API_KEY") {
t.Errorf("text = %q", h.text)
}
}
// TestScanDiffCountsLinesAcrossHunks keeps the line numbers honest when a file
// is edited in several places.
func TestScanDiffCountsLinesAcrossHunks(t *testing.T) {
diff := `+++ b/config.yml
@@ -1,0 +1,1 @@
+harmless: yes
@@ -40,0 +41,2 @@
+also fine
+aws_key = AKIAIOSFODNN7EXAMPLE
`
hits := scanDiff(diff)
if len(hits) != 1 || hits[0].lineNo != 42 {
t.Fatalf("hits = %+v, want one at line 42", hits)
}
}
// TestSecretsApprovedBlocksThePush drives the real thing: a staged .env, the
// scan, and the answer deciding whether push continues.
func TestSecretsApprovedBlocksThePush(t *testing.T) {
dir := t.TempDir()
mustGit(t, dir, "init", "-q")
mustGit(t, dir, "config", "user.name", "t")
mustGit(t, dir, "config", "user.email", "t@e")
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "base")
old := cfg
defer func() { cfg = old }()
cfg = Config{}
// clean tree: no prompt, no interference
asked := fakeAnswers(t, false)
if !secretsApproved(dir) {
t.Fatal("a clean tree must not block the push")
}
if len(*asked) != 0 {
t.Fatalf("asked about a clean tree: %v", *asked)
}
if err := os.WriteFile(filepath.Join(dir, ".env"),
[]byte("API_KEY=\"s3cr3tV4lu3W1thStuff\"\n"), 0600); err != nil {
t.Fatal(err)
}
mustGit(t, dir, "add", "--all", ".")
declined := fakeAnswers(t, false)
if secretsApproved(dir) {
t.Error("a staged credential must stop the push when declined")
}
if len(*declined) == 0 {
t.Error("the user was never asked")
}
accepted := fakeAnswers(t, true)
if !secretsApproved(dir) {
t.Error("an explicit yes must let the push through")
}
if len(*accepted) == 0 {
t.Error("the user was never asked")
}
// and the escape hatch really switches it off
cfg.SecretScan = "off"
never := fakeAnswers(t, false)
if !secretsApproved(dir) {
t.Error("secretscan = off must not block")
}
if len(*never) != 0 {
t.Errorf("secretscan = off still asked: %v", *never)
}
}
// TestSecretScanDefaultsToOn: only a deliberate "off" disables it, so a typo
// leaves the safety net in place.
func TestSecretScanDefaultsToOn(t *testing.T) {
old := cfg
defer func() { cfg = old }()
for _, v := range []string{"", "true", "on", "yes", "wharrgarbl", "1"} {
cfg = Config{SecretScan: v}
if !secretScanEnabled() {
t.Errorf("secretscan = %q disabled the scan", v)
}
}
for _, v := range []string{"off", "0", "false", "no", " OFF "} {
cfg = Config{SecretScan: v}
if secretScanEnabled() {
t.Errorf("secretscan = %q did not disable the scan", v)
}
}
}
// TestOwnDocsDoNotTripTheScanner: mgsh's own README and example config are full
// of credential-shaped text; committing mgsh itself must stay quiet.
func TestOwnDocsDoNotTripTheScanner(t *testing.T) {
for _, f := range []string{"README.md", "mgshrc.example"} {
data, err := os.ReadFile(f)
if err != nil {
t.Fatal(err)
}
for i, ln := range strings.Split(string(data), "\n") {
if kind := matchSecret(ln); kind != "" {
t.Errorf("%s:%d would trip the scanner (%s): %q", f, i+1, kind, ln)
}
}
}
}
// TestAllowMarkerSuppressesOneLine: the per-line escape for something that only
// looks like a credential and is meant to stay.
func TestAllowMarkerSuppressesOneLine(t *testing.T) {
line := `API_KEY="s3cr3tV4lu3W1thStuff"`
if matchSecret(line) == "" {
t.Fatal("test line is not detected at all")
}
if kind := matchSecret(line + " # mgsh:allow — sample value"); kind != "" {
t.Errorf("mgsh:allow did not suppress the hit (%s)", kind)
}
if kind := matchSecret("-----BEGIN OPENSSH PRIVATE KEY----- mgsh:allow"); kind != "" {
t.Errorf("mgsh:allow did not suppress a certain pattern (%s)", kind)
}
}
selfupdate.go
+484
View File
@@ -0,0 +1,484 @@
// selfupdate.go — updating oneself from the releases of a Gitea instance.
//
// The file is meant to be copied: take it into another program, adjust the
// configuration block below, hang `--update` and `--check-update` into the
// options — done. It needs nothing but the standard library, and apart from
// that block it brings no names that do not begin with "selfUpdate" or
// "update".
//
// It assumes the layout build.sh produces: one release per version, whose tag
// is the bare number (4.0.64, a leading "v" is allowed), holding one asset
// "<name>-<goos>-<goarch>" each — that is, exactly the files from ./bin. Under
// /api/v1/repos/<owner>/<repo>/releases/latest Gitea hands out the newest
// release that is neither a draft nor a prerelease; GitHub speaks the same
// route with different field names and is therefore not covered.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"os/exec"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
)
// ------------------------------------------------------------ Configuration
var selfUpdate = selfUpdater{
repo: "https://git.micw.org/mike/mgsh",
asset: "mgsh",
current: VERSION, // from main.go, set by -ldflags
verify: []string{"--version"},
every: 24 * time.Hour,
quietEnv: "MGSH_NO_UPDATE_CHECK",
}
type selfUpdater struct {
repo string // repo URL as in the browser: https://host/owner/repo
asset string // base name of the assets, "-<goos>-<goarch>" is added
current string // the running version
verify []string // trial run of the download; empty skips it
every time.Duration // how often to look on its own; 0 turns that off
quietEnv string // this environment variable set: keep quiet as well
}
// updateRefreshFlag is the option the program calls itself with, in the
// background. It is deliberately absent from the help.
const updateRefreshFlag = "--update-refresh"
// ------------------------------------------------------------ Looking by itself
// daily is the hook for the ordinary run of the program. It costs nothing: in
// the foreground the network is never touched. What comes back is the line
// pointing at a new version — or "", when there is nothing to say; what it
// looks like is up to the caller. Should the note be older than `every`, daily
// starts a background run on the side, whose answer the next call will find
// waiting.
func (u selfUpdater) daily() string {
if u.every <= 0 || os.Getenv(u.quietEnv) != "" || !updateOnTerminal() {
return ""
}
st := u.loadState() // no file: the zero value, hence due at once
if time.Since(st.Checked) >= u.every {
// The timestamp moves on before the asking, not after: otherwise two
// simultaneous runs start two queries, and a server that is not in the
// mood would get a new one on every call. If the note does not stay
// put, nothing is asked either — else an unwritable cache directory
// would mean one process per call.
st.Checked = time.Now()
if u.saveState(st) == nil {
u.spawnRefresh()
}
}
if st.Latest == "" || updateCompare(st.Latest, u.current) <= 0 {
return ""
}
return fmt.Sprintf("%s %s is available, run '%s --update'", u.asset, st.Latest, u.asset)
}
// refresh is the background run: ask, write it down, stay quiet. The writing
// down is done by latest; if the query fails, the old state remains.
func (u selfUpdater) refresh() {
_, _ = u.latest()
}
// spawnRefresh calls this program once more, only to ask, and does not wait.
// Without a Wait the child is adopted by init when this process ends — it thus
// outlives the call, and the call's output stays untouched by it.
func (u selfUpdater) spawnRefresh() {
exe, err := os.Executable()
if err != nil {
return
}
cmd := exec.Command(exe, updateRefreshFlag)
cmd.Stdin, cmd.Stdout, cmd.Stderr = nil, nil, nil // everything to /dev/null
if cmd.Start() == nil {
cmd.Process.Release()
}
}
// The hint is meant for the person sitting there. Running in a pipe, in a
// script or under cron, the program neither asks nor says anything.
func updateOnTerminal() bool {
st, err := os.Stderr.Stat()
return err == nil && st.Mode()&os.ModeCharDevice != 0
}
// --------------------------------------------------------------------- Note
// updateState is what is left between two calls: when the last question was
// asked and what came of it.
type updateState struct {
Checked time.Time `json:"checked"`
Latest string `json:"latest"`
}
// The note lives in the cache directory, not in the configuration: if it gets
// lost, the only cost is asking once too early.
func (u selfUpdater) statePath() (string, error) {
dir, err := os.UserCacheDir()
if err != nil {
return "", err
}
return filepath.Join(dir, u.asset, "update.json"), nil
}
func (u selfUpdater) loadState() updateState {
var st updateState
path, err := u.statePath()
if err != nil {
return st
}
b, err := os.ReadFile(path)
if err != nil {
return st
}
json.Unmarshal(b, &st) // a broken file counts as none
return st
}
func (u selfUpdater) saveState(st updateState) error {
path, err := u.statePath()
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
b, err := json.Marshal(st)
if err != nil {
return err
}
// By way of a file alongside, so that a simultaneous run never comes upon
// half a JSON.
tmp := path + ".new"
if err := os.WriteFile(tmp, b, 0o644); err != nil {
return err
}
if err := os.Rename(tmp, path); err != nil {
os.Remove(tmp)
return err
}
return nil
}
// ------------------------------------------------------------------ The work
// check only looks and touches nothing.
func (u selfUpdater) check(w io.Writer) error {
rel, err := u.latest()
if err != nil {
return err
}
if updateCompare(rel.TagName, u.current) <= 0 {
fmt.Fprintf(w, "%s %s is up to date\n", u.asset, u.current)
return nil
}
fmt.Fprintf(w, "%s %s is available, running %s\n %s\n run '%s --update' to install it\n",
u.asset, rel.TagName, u.current, rel.HTMLURL, u.asset)
return nil
}
// install fetches the newest release and replaces the running file with it.
func (u selfUpdater) install(w io.Writer) error {
rel, err := u.latest()
if err != nil {
return err
}
if updateCompare(rel.TagName, u.current) <= 0 {
fmt.Fprintf(w, "%s %s is up to date\n", u.asset, u.current)
return nil
}
want := fmt.Sprintf("%s-%s-%s", u.asset, runtime.GOOS, runtime.GOARCH)
var src *updateAsset
for i := range rel.Assets {
if rel.Assets[i].Name == want {
src = &rel.Assets[i]
break
}
}
if src == nil {
names := make([]string, len(rel.Assets))
for i, a := range rel.Assets {
names[i] = a.Name
}
return fmt.Errorf("release %s has no %q (only %s)", rel.TagName, want, strings.Join(names, ", "))
}
exe, err := os.Executable()
if err != nil {
return fmt.Errorf("cannot locate the running binary: %w", err)
}
// An installed mgsh is often a symlink into ./bin. What should be replaced
// is the file behind it, not the link.
if real, err := filepath.EvalSymlinks(exe); err == nil {
exe = real
}
mode := os.FileMode(0o755)
if st, err := os.Stat(exe); err == nil {
mode = st.Mode().Perm()
}
fmt.Fprintf(w, "downloading %s %s (%s)\n", want, rel.TagName, updateSize(src.Size))
tmp, err := u.download(src, exe, mode)
if err != nil {
return err
}
defer os.Remove(tmp) // only bites when the renaming below falls through
if err := u.probe(tmp, rel.TagName); err != nil {
return err
}
if err := updateReplace(tmp, exe); err != nil {
return err
}
fmt.Fprintf(w, "%s %s → %s, at %s\n", u.asset, u.current, rel.TagName, exe)
return nil
}
func (u selfUpdater) download(a *updateAsset, exe string, mode os.FileMode) (string, error) {
// The new file comes into being next to the old one: same filesystem, so
// the renaming at the end is one atomic step and not half a copy. It also
// comes into being before the first byte — a missing write permission ought
// to show up before a few megabytes have gone down the wire.
dir := filepath.Dir(exe)
f, err := os.CreateTemp(dir, "."+filepath.Base(exe)+".new")
if err != nil {
var pe *os.PathError // the path is in the message already
if errors.As(err, &pe) {
err = pe.Err
}
return "", fmt.Errorf("cannot write to %s: %w", dir, err)
}
tmp := f.Name()
resp, err := updateGet(context.Background(), a.URL)
if err != nil {
f.Close()
os.Remove(tmp)
return "", err
}
defer resp.Body.Close()
n, err := io.Copy(f, resp.Body)
if cerr := f.Close(); err == nil {
err = cerr
}
if err == nil && a.Size > 0 && n != a.Size {
err = fmt.Errorf("got %d of %d bytes from %s", n, a.Size, a.URL)
}
if err == nil {
err = os.Chmod(tmp, mode)
}
if err != nil {
os.Remove(tmp)
return "", err
}
return tmp, nil
}
// probe calls the freshly fetched runner once. That catches a file that is
// truncated, built for the wrong platform, or not executable in the first
// place, before it replaces the running one.
func (u selfUpdater) probe(path, tag string) error {
if len(u.verify) == 0 {
return nil
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
out, err := exec.CommandContext(ctx, path, u.verify...).CombinedOutput()
if err != nil {
return fmt.Errorf("the downloaded binary does not run: %w", err)
}
if !strings.Contains(string(out), strings.TrimPrefix(tag, "v")) {
return fmt.Errorf("the downloaded binary reports %q, expected %s",
strings.TrimSpace(string(out)), tag)
}
return nil
}
// updateReplace swaps the running file for the new one.
func updateReplace(tmp, exe string) error {
if err := os.Rename(tmp, exe); err == nil {
return nil
}
// Unix overwrites the file of a running program without complaint, Windows
// does not: there the old one has to be got out of the way first. Deleting
// it becomes possible when this process ends at the earliest — so the
// tidying up is allowed to fail.
old := exe + ".old"
os.Remove(old)
if err := os.Rename(exe, old); err != nil {
return fmt.Errorf("cannot replace %s: %w", exe, err)
}
if err := os.Rename(tmp, exe); err != nil {
os.Rename(old, exe) // back to how it was
return fmt.Errorf("cannot replace %s: %w", exe, err)
}
os.Remove(old)
return nil
}
// -------------------------------------------------------------------- Gitea
type updateRelease struct {
TagName string `json:"tag_name"`
HTMLURL string `json:"html_url"`
Assets []updateAsset `json:"assets"`
}
type updateAsset struct {
Name string `json:"name"`
Size int64 `json:"size"`
URL string `json:"browser_download_url"`
}
func (u selfUpdater) latest() (updateRelease, error) {
base, err := u.apiBase()
if err != nil {
return updateRelease{}, err
}
// The question is a small one; if it hangs, it does not hang for long. The
// generous time limit of updateClient is meant for the download.
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
resp, err := updateGet(ctx, base+"/releases/latest")
if err != nil {
return updateRelease{}, err
}
defer resp.Body.Close()
var rel updateRelease
if err := json.NewDecoder(resp.Body).Decode(&rel); err != nil {
return updateRelease{}, fmt.Errorf("unexpected answer from %s: %w", base, err)
}
if rel.TagName == "" {
return updateRelease{}, fmt.Errorf("%s has no releases", u.repo)
}
// Every question that succeeds fills the note — no matter whether it came
// from --update, from --check-update or from the background run.
u.saveState(updateState{Checked: time.Now(), Latest: rel.TagName})
return rel, nil
}
// apiBase turns https://host/owner/repo into the API root of the repo.
func (u selfUpdater) apiBase() (string, error) {
bad := fmt.Errorf("repo %q: expected https://host/owner/repo", u.repo)
ref, err := url.Parse(strings.TrimSuffix(strings.TrimSuffix(u.repo, "/"), ".git"))
if err != nil || ref.Host == "" {
return "", bad
}
parts := strings.Split(strings.Trim(ref.Path, "/"), "/")
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
return "", bad
}
return fmt.Sprintf("%s://%s/api/v1/repos/%s/%s", ref.Scheme, ref.Host, parts[0], parts[1]), nil
}
// One time limit for all of it: the look costs a few hundred milliseconds, the
// download a few megabytes — both may hang, but not forever.
var updateClient = &http.Client{Timeout: 5 * time.Minute}
func updateGet(ctx context.Context, target string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, target, nil)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", "selfupdate.go (+"+runtime.GOOS+"/"+runtime.GOARCH+")")
resp, err := updateClient.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("GET %s: %s", target, resp.Status)
}
return resp, nil
}
// ------------------------------------------------------------------ Numbers
// updateCompare compares two versions component by component, numerically, so
// that 2.1.10 lands behind 2.1.9 and not in front of it. A leading "v" does not
// count, missing places count as 0 (2.1 == 2.1.0), and a suffix on the number
// makes the version older, not newer (2.1.6-rc1 < 2.1.6). The result is the one
// of strings.Compare: -1, 0, 1.
func updateCompare(a, b string) int {
as := strings.Split(strings.TrimPrefix(a, "v"), ".")
bs := strings.Split(strings.TrimPrefix(b, "v"), ".")
for i := 0; i < len(as) || i < len(bs); i++ {
x, y := "0", "0"
if i < len(as) {
x = as[i]
}
if i < len(bs) {
y = bs[i]
}
if c := updateComparePart(x, y); c != 0 {
return c
}
}
return 0
}
func updateComparePart(a, b string) int {
na, ra := updateSplitNum(a)
nb, rb := updateSplitNum(b)
switch {
case na != nb:
if na < nb {
return -1
}
return 1
case ra == rb:
return 0
case ra == "": // 2.1.6 is finished, 2.1.6-rc1 is not yet
return 1
case rb == "":
return -1
}
return strings.Compare(ra, rb)
}
// updateSplitNum separates "10-rc1" into 10 and "-rc1".
func updateSplitNum(s string) (int, string) {
i := 0
for i < len(s) && s[i] >= '0' && s[i] <= '9' {
i++
}
n, _ := strconv.Atoi(s[:i])
return n, s[i:]
}
// updateSize is deliberately a small formatting of its own and not humanSize
// from colors.go — the file is meant to stand on its own.
func updateSize(b int64) string {
const k = 1024
switch {
case b > k*k:
return fmt.Sprintf("%.1f MB", float64(b)/k/k)
case b > k:
return fmt.Sprintf("%.1f KB", float64(b)/k)
default:
return fmt.Sprintf("%d B", b)
}
}
selfupdate_test.go
+190
View File
@@ -0,0 +1,190 @@
package main
// selfupdate_test.go — the parts of selfupdate.go that can be checked without
// replacing the running binary: the version arithmetic, the URL it derives, and
// what it makes of a Gitea release.
//
// selfupdate.go itself is a file copied between programs and stays as it is;
// the tests live here so the copy keeps working when it lands in the next one.
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
)
// cacheInTemp points os.UserCacheDir at a temporary directory, so the note
// selfupdate writes never touches the real one.
func cacheInTemp(t *testing.T) {
t.Helper()
dir := t.TempDir()
t.Setenv("HOME", dir) // darwin: ~/Library/Caches
t.Setenv("XDG_CACHE_HOME", filepath.Join(dir, "xdg")) // linux
}
// fakeGitea serves one /releases/latest answer, the way Gitea does.
func fakeGitea(t *testing.T, body string) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/v1/repos/mike/mgsh/releases/latest" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(body))
}))
t.Cleanup(srv.Close)
return srv
}
func testUpdater(srv *httptest.Server, current string) selfUpdater {
return selfUpdater{
repo: srv.URL + "/mike/mgsh",
asset: "mgsh",
current: current,
every: 0, // no background run from a test
quietEnv: "MGSH_NO_UPDATE_CHECK",
}
}
// TestUpdateCompare is the one piece of arithmetic in the file, and the reason
// it exists: a plain string comparison puts 4.0.9 after 4.0.10 and would offer
// an update backwards forever.
func TestUpdateCompare(t *testing.T) {
for _, c := range []struct {
a, b string
want int
}{
{"4.0.10", "4.0.9", 1}, // numerically, not alphabetically
{"4.0.9", "4.0.10", -1},
{"4.0.64", "4.0.64", 0},
{"v4.1.0", "4.0.64", 1}, // a leading v does not count
{"4.1", "4.1.0", 0}, // missing places are zeroes
{"4.2", "4.1.9", 1},
{"4.0.64-rc1", "4.0.64", -1}, // a suffix is not yet the release
{"4.0.64", "4.0.64-rc1", 1},
{"4.0.64-rc2", "4.0.64-rc1", 1},
{"5.0.0", "4.99.99", 1},
} {
if got := updateCompare(c.a, c.b); got != c.want {
t.Errorf("updateCompare(%q, %q) = %d, want %d", c.a, c.b, got, c.want)
}
}
}
// TestUpdateAPIBase: the browser URL of the repo is all the file is configured
// with, so everything depends on it becoming the right API root.
func TestUpdateAPIBase(t *testing.T) {
for _, c := range []struct{ repo, want string }{
{"https://git.micw.org/mike/mgsh", "https://git.micw.org/api/v1/repos/mike/mgsh"},
{"https://git.micw.org/mike/mgsh/", "https://git.micw.org/api/v1/repos/mike/mgsh"},
{"https://git.micw.org/mike/mgsh.git", "https://git.micw.org/api/v1/repos/mike/mgsh"},
{"http://localhost:3000/mike/mgsh", "http://localhost:3000/api/v1/repos/mike/mgsh"},
} {
got, err := selfUpdater{repo: c.repo}.apiBase()
if err != nil || got != c.want {
t.Errorf("apiBase(%q) = %q, %v, want %q", c.repo, got, err, c.want)
}
}
// anything that is not host/owner/repo has to say so rather than build a
// URL that 404s later
for _, bad := range []string{"", "git.micw.org/mike/mgsh", "https://git.micw.org/mike", "https://git.micw.org/a/b/c"} {
if _, err := (selfUpdater{repo: bad}).apiBase(); err == nil {
t.Errorf("apiBase(%q) accepted a malformed repo URL", bad)
}
}
}
// TestSelfUpdateCheck covers both answers of `update -c`, and that a successful
// look leaves the note behind that the daily hint later reads.
func TestSelfUpdateCheck(t *testing.T) {
cacheInTemp(t)
srv := fakeGitea(t, `{"tag_name":"4.1.0","html_url":"https://git.micw.org/mike/mgsh/releases/tag/4.1.0",
"assets":[{"name":"mgsh-darwin-arm64","size":9,"browser_download_url":"x"}]}`)
var out strings.Builder
if err := testUpdater(srv, "4.0.64").check(&out); err != nil {
t.Fatalf("check: %v", err)
}
if !strings.Contains(out.String(), "4.1.0 is available") {
t.Errorf("check output = %q, want the new version offered", out.String())
}
out.Reset()
if err := testUpdater(srv, "4.1.0").check(&out); err != nil {
t.Fatalf("check: %v", err)
}
if !strings.Contains(out.String(), "up to date") {
t.Errorf("check output = %q, want 'up to date'", out.String())
}
// the same version must not be offered as an update to itself
out.Reset()
if err := testUpdater(srv, "4.2.0").check(&out); err != nil {
t.Fatalf("check: %v", err)
}
if strings.Contains(out.String(), "available") {
t.Errorf("a newer running version was offered an update: %q", out.String())
}
cache, err := os.UserCacheDir()
if err != nil {
t.Fatal(err)
}
note := filepath.Join(cache, "mgsh", "update.json")
b, err := os.ReadFile(note)
if err != nil {
t.Fatalf("no note written to %s: %v", note, err)
}
if !strings.Contains(string(b), `"latest":"4.1.0"`) {
t.Errorf("note = %s, want the looked-up version in it", b)
}
}
// TestSelfUpdateNoReleases: an empty repository must produce a plain message,
// not a nil release that gets compared against the running version.
func TestSelfUpdateNoReleases(t *testing.T) {
cacheInTemp(t)
srv := fakeGitea(t, `{}`)
err := testUpdater(srv, "4.0.64").check(&strings.Builder{})
if err == nil || !strings.Contains(err.Error(), "no releases") {
t.Errorf("check against a release-less repo = %v, want a 'no releases' error", err)
}
}
// TestSelfUpdateWithoutMatchingAsset: a release built for other platforms must
// not be installed, and the message has to name what was looked for — that is
// what tells you the release is incomplete rather than the machine unsupported.
func TestSelfUpdateWithoutMatchingAsset(t *testing.T) {
cacheInTemp(t)
srv := fakeGitea(t, `{"tag_name":"4.1.0","assets":[{"name":"mgsh-plan9-mips","size":1,"browser_download_url":"x"}]}`)
err := testUpdater(srv, "4.0.64").install(&strings.Builder{})
if err == nil {
t.Fatal("install accepted a release without a binary for this platform")
}
if !strings.Contains(err.Error(), "mgsh-") || !strings.Contains(err.Error(), "mgsh-plan9-mips") {
t.Errorf("error = %v, want the wanted and the available asset names", err)
}
}
// TestUpdateDailyStaysQuiet: the automatic look is for the person sitting
// there. Under MGSH_NO_UPDATE_CHECK, and with `every` at zero, it says nothing
// and starts nothing.
func TestUpdateDailyStaysQuiet(t *testing.T) {
cacheInTemp(t)
u := selfUpdater{asset: "mgsh", current: "4.0.0", every: 0, quietEnv: "MGSH_NO_UPDATE_CHECK"}
if hint := u.daily(); hint != "" {
t.Errorf("daily with every=0 = %q, want silence", hint)
}
t.Setenv("MGSH_NO_UPDATE_CHECK", "1")
u.every = 1
if hint := u.daily(); hint != "" {
t.Errorf("daily under MGSH_NO_UPDATE_CHECK = %q, want silence", hint)
}
}
shellcomplete.go
+181
View File
@@ -0,0 +1,181 @@
package main
// shellcomplete.go — Tab completion for the '!' shell escape.
//
// `!vi <Tab>` should behave like it does in a shell: the first word completes
// against the executables on PATH, everything after it against the filesystem.
// Paths resolve relative to the active project directory, because that is where
// forwardShell runs the command.
//
// Word splitting here is whitespace only. Quoting and backslash escapes are the
// shell's business at execution time; getting them right for completion too
// would buy little for a one-off escape hatch.
import (
"os"
"path/filepath"
"sort"
"strings"
"sync"
)
// shellCommandList supplies the executable names for the command position. A
// variable so tests can hand over a fixed set instead of whatever happens to be
// installed on the machine running them.
var shellCommandList = pathExecutables
// completeShellLine returns the candidates and the prefix they replace for a
// line that is headed for a shell. ok is false for any other line, which is
// then left to the builtin command tree.
func completeShellLine(typed string) (cands []string, prefix string, ok bool) {
body, allowCommand, ok := shellLine(typed)
if !ok {
return nil, "", false
}
cands, prefix = shellCandidates(body, allowCommand)
return cands, prefix, true
}
// shellLine works out which part of a typed line will reach a shell, and
// whether its command word is still open for completion. Two things get there:
// a '!' escape, and an alias that expands to one — `alias ll '!ls -la'` makes
// everything after `ll` a shell argument just as surely.
//
// Only the alias itself is inspected, not what its expansion might expand to
// again: an alias chain can rewrite its arguments, and guessing at that would
// offer candidates for a command line that is not the one being built.
func shellLine(typed string) (body string, allowCommand, ok bool) {
trimmed := strings.TrimLeft(typed, " \t")
if rest, found := strings.CutPrefix(trimmed, "!"); found {
return rest, true, true
}
// the alias name has to be complete — while it is still being typed there
// is no way to know what it will turn out to be
sep := strings.IndexAny(trimmed, " \t")
if sep < 0 {
return "", false, false
}
name := trimmed[:sep]
if isBuiltin(name) { // a builtin can never be shadowed by an alias
return "", false, false
}
expansion, defined := aliases[name]
if !defined || !strings.HasPrefix(strings.TrimSpace(expansion), "!") {
return "", false, false
}
// the command comes from the alias body, so only arguments are left to complete
return trimmed[sep:], false, true
}
// shellCandidates completes the last word of a shell command line. allowCommand
// says whether its first word may still be completed against PATH.
func shellCandidates(body string, allowCommand bool) (cands []string, prefix string) {
word := body[strings.LastIndexAny(body, " \t")+1:]
inCommand := allowCommand && strings.TrimLeft(body[:len(body)-len(word)], " \t") == ""
// a command word without a separator names something on PATH; with one it
// is a path like ./script, exactly as a shell reads it
if inCommand && !strings.ContainsRune(word, '/') {
if word == "" {
return nil, "" // every executable on the machine helps nobody
}
return matchPrefix(shellCommandList(), word), word
}
dir, base := splitPathToken(word)
return matchPrefix(pathEntries(dir), base), base
}
// splitPathToken splits a path token into the directory part, kept exactly as
// typed, and the basename being completed. Completing only the basename is what
// keeps the candidate list readable: "src/ma<Tab>" offers "main.go", not the
// whole path again.
func splitPathToken(word string) (dir, base string) {
if i := strings.LastIndexByte(word, '/'); i >= 0 {
return word[:i+1], word[i+1:]
}
return "", word
}
// pathEntries lists what a directory token points at. Directories come back
// with a trailing slash, so completing one leads straight into it.
func pathEntries(dir string) []string {
root := DIR
switch {
case strings.HasPrefix(dir, "~/"):
home, err := os.UserHomeDir()
if err != nil {
return nil
}
root, dir = home, dir[2:]
case strings.HasPrefix(dir, "/"):
root = ""
}
entries, err := os.ReadDir(filepath.Join(root, dir))
if err != nil {
return nil
}
out := make([]string, 0, len(entries))
for _, e := range entries {
name := e.Name()
if e.IsDir() {
name += "/"
}
out = append(out, name)
}
return out
}
// matchPrefix keeps the candidates starting with prefix, sorted and without
// duplicates. A hidden entry only shows up once the prefix asks for it, as in a
// shell.
func matchPrefix(cands []string, prefix string) []string {
wantHidden := strings.HasPrefix(prefix, ".")
seen := map[string]bool{}
var out []string
for _, c := range cands {
if !strings.HasPrefix(c, prefix) || seen[c] {
continue
}
if !wantHidden && strings.HasPrefix(c, ".") {
continue
}
seen[c] = true
out = append(out, c)
}
sort.Strings(out)
return out
}
// pathExecutables lists the executable names on PATH. The scan happens once per
// session: PATH cannot change from inside mgsh, and a few thousand directory
// entries are not worth walking on every Tab.
var pathExecutables = sync.OnceValue(func() []string {
var out []string
seen := map[string]bool{}
for _, dir := range filepath.SplitList(os.Getenv("PATH")) {
if dir == "" {
dir = "."
}
entries, err := os.ReadDir(dir)
if err != nil {
continue
}
for _, e := range entries {
name := e.Name()
if seen[name] {
continue // the first one on PATH is the one that would run
}
fi, err := e.Info()
if err != nil || fi.IsDir() || fi.Mode().Perm()&0o111 == 0 {
continue
}
seen[name] = true
out = append(out, name)
}
}
sort.Strings(out)
return out
})
shellcomplete_test.go
+260
View File
@@ -0,0 +1,260 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// complete runs the real dispatch a Tab press goes through.
func complete(typed string) ([]string, string) {
cands, prefix, _ := completeShellLine(typed)
return cands, prefix
}
// fakeCommands installs a fixed set of PATH executables for the test.
func fakeCommands(t *testing.T, names ...string) {
t.Helper()
old := shellCommandList
shellCommandList = func() []string { return names }
t.Cleanup(func() { shellCommandList = old })
}
// shellTree lays out a directory to complete against and points DIR at it.
func shellTree(t *testing.T) string {
t.Helper()
dir := t.TempDir()
for _, d := range []string{"src", "src/deep", ".hidden"} {
if err := os.MkdirAll(filepath.Join(dir, d), 0755); err != nil {
t.Fatal(err)
}
}
for _, f := range []string{"main.go", "main_test.go", "Makefile", ".env", "src/util.go"} {
if err := os.WriteFile(filepath.Join(dir, f), []byte("x"), 0644); err != nil {
t.Fatal(err)
}
}
old := DIR
DIR = dir
t.Cleanup(func() { DIR = old })
return dir
}
func TestShellCandidatesCommandWord(t *testing.T) {
fakeCommands(t, "vi", "vim", "view", "grep", "git")
shellTree(t)
cands, prefix := complete("!vi")
if prefix != "vi" {
t.Errorf("prefix = %q, want vi", prefix)
}
if strings.Join(cands, ",") != "vi,view,vim" {
t.Errorf("candidates = %v, want vi,view,vim sorted", cands)
}
// a bare '!' must not dump every executable on the machine
if cands, _ := complete("!"); len(cands) != 0 {
t.Errorf("bare '!' offered %d candidates", len(cands))
}
// leading blanks are allowed, as runCommand allows them
if cands, _ := complete(" !gi"); strings.Join(cands, ",") != "git" {
t.Errorf("indented escape = %v, want git", cands)
}
// a command word with a separator is a path, not a PATH lookup
if cands, prefix := complete("!./ma"); prefix != "ma" ||
strings.Join(cands, ",") != "main.go,main_test.go" {
t.Errorf("./ma = %v (prefix %q), want the local files", cands, prefix)
}
}
func TestShellCandidatesArguments(t *testing.T) {
fakeCommands(t, "vi")
shellTree(t)
// paths resolve against the project directory, where `!` commands run
cands, prefix := complete("!vi ma")
if prefix != "ma" || strings.Join(cands, ",") != "main.go,main_test.go" {
t.Errorf("candidates = %v (prefix %q)", cands, prefix)
}
// an empty argument lists the directory — dot entries stay out of the way
cands, prefix = complete("!vi ")
if prefix != "" {
t.Errorf("prefix = %q, want empty", prefix)
}
if strings.Join(cands, ",") != "Makefile,main.go,main_test.go,src/" {
t.Errorf("directory listing = %v", cands)
}
// ... until the prefix asks for them
if cands, _ := complete("!vi ."); strings.Join(cands, ",") != ".env,.hidden/" {
t.Errorf("dot prefix = %v, want the hidden entries", cands)
}
// a directory completes with its slash, so the next Tab walks into it
cands, prefix = complete("!vi sr")
if prefix != "sr" || strings.Join(cands, ",") != "src/" {
t.Errorf("directory candidate = %v (prefix %q)", cands, prefix)
}
// inside a directory only the basename is completed, which is what keeps
// the candidate list readable
cands, prefix = complete("!vi src/ut")
if prefix != "ut" || strings.Join(cands, ",") != "util.go" {
t.Errorf("nested candidate = %v (prefix %q), want util.go / ut", cands, prefix)
}
// later arguments complete the same way as the first
if cands, _ := complete("!diff main.go ma"); strings.Join(cands, ",") != "main.go,main_test.go" {
t.Errorf("second argument = %v", cands)
}
}
func TestShellCandidatesAbsoluteAndHome(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
if err := os.WriteFile(filepath.Join(home, "notes.txt"), []byte("x"), 0644); err != nil {
t.Fatal(err)
}
shellTree(t)
fakeCommands(t, "vi")
if cands, prefix := complete("!vi ~/no"); prefix != "no" ||
strings.Join(cands, ",") != "notes.txt" {
t.Errorf("~/ completion = %v (prefix %q)", cands, prefix)
}
abs := filepath.Join(home, "no")
if cands, prefix := complete("!vi " + abs); prefix != "no" ||
strings.Join(cands, ",") != "notes.txt" {
t.Errorf("absolute completion = %v (prefix %q)", cands, prefix)
}
}
// TestShellCompleterDoContract is the part that would corrupt the line if it
// were wrong: readline replaces the last `length` runes with a candidate, so
// the candidates must be suffixes and the length must count runes.
func TestShellCompleterDoContract(t *testing.T) {
fakeCommands(t, "vim", "view")
shellTree(t)
c := completer()
line := []rune("!vi")
got, length := c.Do(line, len(line))
if length != 2 { // "vi" — the '!' is not part of the word
t.Fatalf("length = %d, want 2", length)
}
// rebuilding the line from prefix + candidate must give the full word
for i, g := range got {
full := string(line[:len(line)-length]) + string(line[len(line)-length:]) + string(g)
if full != "!vim" && full != "!view" {
t.Errorf("candidate %d rebuilds to %q", i, full)
}
}
// a non-'!' line still goes to the builtin command tree (which appends its
// own trailing space on a unique match)
line = []rune("stat")
got, length = c.Do(line, len(line))
if length != 4 || len(got) == 0 || !strings.HasPrefix(string(got[0]), "us") {
t.Errorf("builtin completion = %q, %d; want a candidate starting \"us\" at 4", got, length)
}
// a multi-byte prefix must be measured in runes, not bytes
if _, n := runeSuffixes([]string{"übermorgen"}, "üb"); n != 2 {
t.Errorf("runeSuffixes length = %d, want 2 runes", n)
}
}
// TestPathExecutablesFindsRealBinaries checks the PATH scan against a directory
// it controls: only files with an execute bit, no directories.
func TestPathExecutablesFindsRealBinaries(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "runnable"), []byte("#!/bin/sh\n"), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "plainfile"), []byte("x"), 0644); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Join(dir, "subdir"), 0755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", dir)
// pathExecutables caches for the session, so exercise the scan directly
var names []string
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
for _, e := range entries {
fi, err := e.Info()
if err != nil || fi.IsDir() || fi.Mode().Perm()&0o111 == 0 {
continue
}
names = append(names, e.Name())
}
if strings.Join(names, ",") != "runnable" {
t.Errorf("executable scan = %v, want only runnable", names)
}
}
// withAliases installs a fixed alias set for the test.
func withAliases(t *testing.T, m map[string]string) {
t.Helper()
old := aliases
aliases = m
t.Cleanup(func() { aliases = old })
}
// TestShellCandidatesThroughAlias: an alias that expands to a '!' escape turns
// everything after its name into shell arguments, so it completes as such.
func TestShellCandidatesThroughAlias(t *testing.T) {
shellTree(t)
fakeCommands(t, "vi", "ls")
withAliases(t, map[string]string{
"ll": "!ls -la",
"e": "!vi $1",
"co": "checkout $1", // expands to a builtin, not a shell command
"status": "!git status", // shadows a builtin: must not count
})
// arguments of a shell alias complete against the filesystem
if cands, prefix := complete("ll ma"); prefix != "ma" ||
strings.Join(cands, ",") != "main.go,main_test.go" {
t.Errorf("alias argument = %v (prefix %q)", cands, prefix)
}
// including the empty one, which lists the directory
if cands, _ := complete("e "); strings.Join(cands, ",") != "Makefile,main.go,main_test.go,src/" {
t.Errorf("empty alias argument = %v", cands)
}
// and paths inside it
if cands, prefix := complete("ll src/ut"); prefix != "ut" ||
strings.Join(cands, ",") != "util.go" {
t.Errorf("nested alias argument = %v (prefix %q)", cands, prefix)
}
// the command word of an alias is fixed by its body, so PATH is never
// offered — `ll vi` means the file "vi", not the editor
if cands, _ := complete("ll vi"); len(cands) != 0 {
t.Errorf("alias argument matched PATH: %v", cands)
}
// an alias to a builtin is not a shell line at all
if _, _, ok := shellLine("co ma"); ok {
t.Error("an alias expanding to a builtin was treated as a shell line")
}
// nor is a name that a builtin owns, since runCommand never expands those
if _, _, ok := shellLine("status ma"); ok {
t.Error("a builtin name was resolved through an alias")
}
// nor an undefined name
if _, _, ok := shellLine("nosuch ma"); ok {
t.Error("an undefined alias was treated as a shell line")
}
// while the alias name itself is still being typed there is nothing to know
if _, _, ok := shellLine("ll"); ok {
t.Error("an incomplete alias name was resolved")
}
}
show_config.go
+20 -15
View File
@@ -24,11 +24,11 @@ func showConfig() {
project = DIR + "/" + projectRC project = DIR + "/" + projectRC
} }
fmt.Printf("%s %s\n", col(cGray, "global "), col(cCyan, global)) fmt.Printf("%s %s\n", col(cDark, "global "), col(cBlue, global))
if project != "" { if project != "" {
fmt.Printf("%s %s\n", col(cGray, "project"), col(cCyan, project)) fmt.Printf("%s %s\n", col(cDark, "project"), col(cBlue, project))
} else if PRJ != "" { } else if PRJ != "" {
fmt.Printf("%s %s\n", col(cGray, "project"), col(cGray, "no "+projectRC+" in "+PRJ)) fmt.Printf("%s %s\n", col(cDark, "project"), col(cDark, "no "+projectRC+" in "+PRJ))
} }
fmt.Println() fmt.Println()
@@ -42,8 +42,8 @@ func showConfig() {
{"gitname", cfg.GitName}, {"gitname", cfg.GitName},
{"gitemail", cfg.GitEmail}, {"gitemail", cfg.GitEmail},
{"pushdefault", cfg.PushDefault}, {"pushdefault", cfg.PushDefault},
{"editor", cfg.Editor},
{"mirror", cfg.Mirror}, {"mirror", cfg.Mirror},
{"secretscan", cfg.SecretScan},
{"remotes", cfg.RemoteNames}, {"remotes", cfg.RemoteNames},
} }
@@ -72,13 +72,13 @@ func showConfig() {
} }
if k := sshKeyPath(); k != "" { if k := sshKeyPath(); k != "" {
fmt.Printf(" %s%s\n", col(cGray, padRight("ssh identity", 14)), col(cGray, k)) fmt.Printf(" %s%s\n", col(cDark, padRight("ssh identity", 14)), col(cDark, k))
} }
fmt.Printf(" %s%s\n", col(cGray, padRight("clone url", 14)), col(cGray, URL)) fmt.Printf(" %s%s\n", col(cDark, padRight("clone url", 14)), col(cDark, URL))
// the project's real origin: it can differ from what the current settings // the project's real origin: it can differ from what the current settings
// would produce, e.g. after moving the server or editing a project .mgshrc // would produce, e.g. after moving the server or editing a project .mgshrc
if o := originURL(); o != "" { if o := originURL(); o != "" {
fmt.Printf(" %s%s\n", col(cGray, padRight("origin", 14)), col(cGray, o)) fmt.Printf(" %s%s\n", col(cDark, padRight("origin", 14)), col(cDark, o))
} }
showRemotes() showRemotes()
@@ -90,11 +90,11 @@ func showRemotes() {
targets, incomplete := cfg.mirrorTargets() targets, incomplete := cfg.mirrorTargets()
fmt.Println() fmt.Println()
if len(targets) == 0 && len(incomplete) == 0 { if len(targets) == 0 && len(incomplete) == 0 {
fmt.Println(col(cGray, "no pushremote targets configured")) fmt.Println(col(cDark, "no pushremote targets configured"))
return return
} }
fmt.Println(col(cGray, "pushremote targets (in push order):")) fmt.Println(col(cDark, "pushremote targets (in push order):"))
for _, t := range targets { for _, t := range targets {
vis := "private" vis := "private"
if strings.EqualFold(strings.TrimSpace(t.Vis), "public") { if strings.EqualFold(strings.TrimSpace(t.Vis), "public") {
@@ -104,9 +104,15 @@ func showRemotes() {
if kind == "" { if kind == "" {
kind = remoteKindName(detectRemoteKind(t.URL, "")) + " (detected)" kind = remoteKindName(detectRemoteKind(t.URL, "")) + " (detected)"
} }
fmt.Printf(" %s%s %s\n", // an inactive target is still a target, just not an automatic one: it
col(cGreen, padRight("@"+t.Name, 14)), t.URL, // steps back a shade instead of dropping out of the listing
col(cGray, kind+", "+vis+", key "+maskSecret(t.Key))) name, state := col(cGreen, padRight("@"+t.Name, 14)), ""
if !t.isActive() {
name = col(cGrey, padRight("@"+t.Name, 14))
state = ", inactive (only when named)"
}
fmt.Printf(" %s%s %s\n", name, t.URL,
col(cDark, kind+", "+vis+", key "+maskSecret(t.Key)+state))
} }
for _, n := range incomplete { for _, n := range incomplete {
fmt.Printf(" %s%s\n", col(cRed, padRight("@"+n, 14)), col(cRed, "incomplete: url or key missing")) fmt.Printf(" %s%s\n", col(cRed, padRight("@"+n, 14)), col(cRed, "incomplete: url or key missing"))
@@ -145,9 +151,8 @@ func envName(key string) string { return "MGSH_" + strings.ToUpper(key) }
func configKeys() []string { func configKeys() []string {
keys := []string{ keys := []string{
"base", "githost", "gitport", "gituser", "gitpath", "gitkey", "base", "githost", "gitport", "gituser", "gitpath", "gitkey",
"gitname", "gitemail", "pushdefault", "editor", "gitname", "gitemail", "pushdefault",
"remoteurl", "remotekey", "remotetype", "remotevisibility", "remotes", "mirror", "secretscan",
"remotes", "mirror",
} }
sort.Strings(keys) sort.Strings(keys)
return keys return keys
util.go
+11
View File
@@ -39,6 +39,17 @@ func truthy(s string) bool {
return false return false
} }
// falsy reports whether a config string explicitly means "off". It is not the
// negation of truthy: for a setting that defaults to on, an unset value or a
// typo must leave it on, and only a deliberate "off" may switch it off.
func falsy(s string) bool {
switch strings.ToLower(strings.TrimSpace(s)) {
case "0", "false", "no", "off":
return true
}
return false
}
func fileExists(p string) bool { func fileExists(p string) bool {
fi, err := os.Stat(p) fi, err := os.Stat(p)
return err == nil && !fi.IsDir() return err == nil && !fi.IsDir()
version.txt
+1 -1
View File
@@ -1 +1 @@
4.0.24 4.0.70