Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a622046f2 | ||
|
|
61a7059f61 | ||
|
|
8ca05d6ad2 | ||
|
|
cb1ff98c3d | ||
|
|
86fb898df4 | ||
|
|
3a420093d1 | ||
|
|
8c68b28dc2 | ||
|
|
cad7a4ec2c | ||
|
|
59da8f376c |
@@ -7,6 +7,7 @@ directory. Go port of the original Perl `mgsh` (`mgsh.perl`).
|
|||||||
## Contents
|
## Contents
|
||||||
|
|
||||||
- [Build](#build) · [Usage](#usage) · [Commands](#commands) · [Aliases](#aliases)
|
- [Build](#build) · [Usage](#usage) · [Commands](#commands) · [Aliases](#aliases)
|
||||||
|
- [Overview](#overview) · [Credential check](#credential-check)
|
||||||
- [Public mirror (`pushremote`)](#public-mirror-pushremote) ·
|
- [Public mirror (`pushremote`)](#public-mirror-pushremote) ·
|
||||||
[Releases](#releases)
|
[Releases](#releases)
|
||||||
- [Configuration](#configuration) · [Settings reference](#settings-reference) ·
|
- [Configuration](#configuration) · [Settings reference](#settings-reference) ·
|
||||||
@@ -80,11 +81,11 @@ Run `help` for the full list. Highlights:
|
|||||||
| `pushremote [desc]` | mirror the repo to a public server (gitea/github/gitlab) |
|
| `pushremote [desc]` | mirror the repo to a public server (gitea/github/gitlab) |
|
||||||
| `pull` / `fetch` | pull / fetch from the server |
|
| `pull` / `fetch` | pull / fetch from the server |
|
||||||
| `status [-a]` / `diff` | short git status (`-a`: overview of all projects) |
|
| `status [-a]` / `diff` | short git status (`-a`: overview of all projects) |
|
||||||
| `overview` | dirty / ahead-behind summary of all projects |
|
| `overview` | inventory of all projects, local and on the server |
|
||||||
| `log` | show the project log |
|
| `log` | show the project log |
|
||||||
| `edit [n]` | interactive rebase of the last n commits |
|
| `edit [n]` | interactive rebase of the last n commits |
|
||||||
| `clone [-a] <repo>` | clone a repository (or archive) from the server |
|
| `clone [-a] <repo>` | clone a repository (or archive) from the server |
|
||||||
| `list [-a] [pattern]` | list repositories on the server |
|
| `list [-a] [pattern]` | list repositories on the server (`-a`: archives, with sizes) |
|
||||||
| `show <repo>` | show a repository log directly on the server |
|
| `show <repo>` | show a repository log directly on the server |
|
||||||
| `archive [comment]` | snapshot the server-side repo into `./archive` |
|
| `archive [comment]` | snapshot the server-side repo into `./archive` |
|
||||||
| `init` | make a new repository from the current directory |
|
| `init` | make a new repository from the current directory |
|
||||||
@@ -125,6 +126,111 @@ pushremote targets (in push order):
|
|||||||
Tokens are masked, so the output is safe to paste into a bug report.
|
Tokens are masked, so the output is safe to paste into a bug report.
|
||||||
`config -k` prints just the setting names, one per line.
|
`config -k` prints just the setting names, one per line.
|
||||||
|
|
||||||
|
### Listing the server
|
||||||
|
|
||||||
|
`list` shows what is on the git server, name first and aligned, ordered by
|
||||||
|
modification time — `push` touches the bare repository, so the most recently
|
||||||
|
worked-on project sits closest to the prompt:
|
||||||
|
|
||||||
|
```
|
||||||
|
< src > list
|
||||||
|
Betaflight3.0.0 Sep 28 2016 181M
|
||||||
|
website Mar 3 2024 2.1M
|
||||||
|
notes Jan 3 14:32 876K
|
||||||
|
3 repositories · 184M
|
||||||
|
```
|
||||||
|
|
||||||
|
The size is the repository's real disk usage on the server, asked of `du` in
|
||||||
|
the same round trip as the listing — a long listing reports the inode size for
|
||||||
|
a directory, which is the same number for every repository and says nothing. If
|
||||||
|
the server produces no usable sizes the column is left out rather than filled
|
||||||
|
with zeroes.
|
||||||
|
|
||||||
|
`list -a` lists the archives instead, whose sizes come from the listing itself;
|
||||||
|
a pattern filters by name (`list note`).
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
`overview` (or `status -a`) is the one view that needs mgsh: it is the only
|
||||||
|
thing that sees the local base directory *and* the git server at once.
|
||||||
|
|
||||||
|
```
|
||||||
|
< src > overview
|
||||||
|
notes * ↑2 laptop 3h
|
||||||
|
website * ✓ desktop 2d → hub
|
||||||
|
Betaflight3.0.0 ✓ workstation 20d → gitea hub
|
||||||
|
experiments init
|
||||||
|
sandbox (wip) – laptop 1h init
|
||||||
|
4 projects · 2 dirty · 2 in sync · 2 to init
|
||||||
|
```
|
||||||
|
|
||||||
|
Every field sits in its own column, so the eye can go down one instead of
|
||||||
|
hunting along each line. The projects that need something done come first, the
|
||||||
|
settled ones next, and the directories the git server does not have yet come
|
||||||
|
last — those are a different kind of task. Within each group the order stays
|
||||||
|
alphabetical, so positions do not jump around.
|
||||||
|
|
||||||
|
| column | meaning |
|
||||||
|
|---|---|
|
||||||
|
| name | the project, with its branch appended when it is not `master`/`main` |
|
||||||
|
| `*` | uncommitted changes |
|
||||||
|
| `↑n` `↓n` | commits ahead of / behind the upstream (`↑2↓1` when both) |
|
||||||
|
| `✓` | in sync with the upstream |
|
||||||
|
| `–` | the branch tracks nothing — never pushed |
|
||||||
|
| `init` | the git server does not have this one; run `init` |
|
||||||
|
| host, age | who last committed and when, from the `[user@host]` stamp |
|
||||||
|
| `→` | mirror targets this repository has a remote for |
|
||||||
|
|
||||||
|
Every directory under the base gets a row, including those that are not
|
||||||
|
repositories at all — `init` is exactly what turns one into a project, so it
|
||||||
|
belongs in the table rather than in a list underneath it. Such a row simply has
|
||||||
|
no git state to show.
|
||||||
|
|
||||||
|
The host and age come from the commit itself — `push` writes `[user@host]` into
|
||||||
|
every message, so `overview` can say where a project was last worked on without
|
||||||
|
storing anything. On a setup spanning a laptop and a workstation that is usually
|
||||||
|
the piece of information you actually wanted.
|
||||||
|
|
||||||
|
The `init` column is the join no git command can do. If the server cannot be
|
||||||
|
reached, no row is marked and mgsh says so — not knowing is not the same as
|
||||||
|
knowing they are missing. The other direction, repositories on the server that
|
||||||
|
are not here, is what `list` shows.
|
||||||
|
|
||||||
|
### Credential check
|
||||||
|
|
||||||
|
`push` runs `git add --all .`, so anything lying in the project gets committed —
|
||||||
|
and with `mirror = true` it reaches a public server in the same breath. That is
|
||||||
|
the only action in mgsh that cannot be undone: a deleted server repository comes
|
||||||
|
back from an archive, a published credential does not.
|
||||||
|
|
||||||
|
So before anything is committed, the staged diff is checked for private keys,
|
||||||
|
GitHub/GitLab/Slack/AWS/PyPI tokens and credential-shaped assignments:
|
||||||
|
|
||||||
|
```
|
||||||
|
< src/notes > push new notes
|
||||||
|
2 possible credential(s) in what is about to be committed:
|
||||||
|
.env:3 credential assignment
|
||||||
|
API_KEY="<the offending line is shown in full here>"
|
||||||
|
deploy_key:1 private key
|
||||||
|
<the BEGIN … PRIVATE KEY header is shown here>
|
||||||
|
(set 'secretscan = off' to skip this check)
|
||||||
|
push anyway? y/N ?
|
||||||
|
```
|
||||||
|
|
||||||
|
Declining stops the push with nothing committed; the changes stay staged, so
|
||||||
|
`git restore --staged <file>` and a `.gitignore` entry are all it takes.
|
||||||
|
|
||||||
|
For a line that only *looks* like a credential and is meant to stay, put
|
||||||
|
`mgsh:allow` in it — a comment on that line is enough. That is better than
|
||||||
|
turning the whole check off for one false positive.
|
||||||
|
|
||||||
|
This is not a complete secret scanner and does not try to be one. It aims for a
|
||||||
|
high hit rate on what actually leaks, with few enough false alarms that the
|
||||||
|
prompt still means something: values that are plainly environment references,
|
||||||
|
constants, template slots (`<token>`, `${VAR}`) or masked stand-ins are ignored —
|
||||||
|
a test checks that mgsh's own README and `mgshrc.example`, both full of
|
||||||
|
credential-shaped text, stay quiet. Switch it off with `secretscan = off`.
|
||||||
|
|
||||||
### Aliases
|
### Aliases
|
||||||
|
|
||||||
`alias <name> '<command>'` defines a reusable shortcut, persisted to
|
`alias <name> '<command>'` defines a reusable shortcut, persisted to
|
||||||
@@ -153,38 +259,37 @@ alias ec '!echo $1' # ec hello -> echo hello (shell)
|
|||||||
|
|
||||||
Besides the internal ssh git server, `pushremote` mirrors the active project to
|
Besides the internal ssh git server, `pushremote` mirrors the active project to
|
||||||
one or more public hosting servers (Gitea, GitHub or GitLab) over their REST
|
one or more public hosting servers (Gitea, GitHub or GitLab) over their REST
|
||||||
API. A single server is configured flat:
|
API. Each server is one `remote.<name>.<field>` block:
|
||||||
|
|
||||||
```ini
|
|
||||||
remoteurl = https://git.example.com # base URL of the server
|
|
||||||
remotekey = <personal-access-token> # API token
|
|
||||||
# remotetype = gitea # optional; auto-detected from remoteurl
|
|
||||||
# remotevisibility = private # visibility of created repos (default private)
|
|
||||||
# mirror = true # `push` also mirrors via pushremote
|
|
||||||
```
|
|
||||||
|
|
||||||
Several servers get one `remote.<name>.*` block each:
|
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
remote.gitea.url = https://git.example.com
|
remote.gitea.url = https://git.example.com
|
||||||
remote.gitea.key = <personal-access-token>
|
remote.gitea.key = <personal-access-token>
|
||||||
|
remote.gitea.type = gitea # optional; auto-detected from the url
|
||||||
|
remote.gitea.visibility = private # or public (default private)
|
||||||
|
|
||||||
remote.hub.url = https://github.com
|
remote.gitlab.url = https://gitlab.example.com
|
||||||
remote.hub.key = <personal-access-token>
|
remote.gitlab.key = <personal-access-token>
|
||||||
remote.hub.visibility = public
|
remote.gitlab.type = gitlab
|
||||||
|
remote.gitlab.visibility = public
|
||||||
|
|
||||||
# remotes = gitea, hub # optional: restrict and order the set
|
# remotes = gitea, gitlab # optional: restrict and order the set
|
||||||
|
# mirror = true # `push` also mirrors via pushremote
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`<name>` is yours to pick; there is no other spelling. Older versions had a flat
|
||||||
|
`remoteurl`/`remotekey` pair for a single server — mgsh converts those to
|
||||||
|
`remote.public.*` in place on the next start and says so, keeping the git remote
|
||||||
|
name those versions used.
|
||||||
|
|
||||||
| command | pushes to |
|
| command | pushes to |
|
||||||
|------------------------|-----------------------------------------------|
|
|------------------------|-----------------------------------------------|
|
||||||
| `pushremote` | every configured target, in order |
|
| `pushremote` | every configured target, in order |
|
||||||
| `pushremote @hub` | only `hub` |
|
| `pushremote @gitea` | only `gitea` |
|
||||||
| `pushremote @hub @gitea` | those two |
|
| `pushremote @gitea @gitlab` | those two |
|
||||||
| `pushremote a fix` | every target, description "a fix" |
|
| `pushremote a fix` | every target, description "a fix" |
|
||||||
|
|
||||||
Each target owns a git remote of the same name in the repository (the flat form
|
Each target owns a git remote of the same name in the repository, so
|
||||||
uses `public`, as before), so `git push hub` keeps working outside mgsh. A
|
`git push gitlab` keeps working outside mgsh. A
|
||||||
target that fails does not stop the others; with more than one target
|
target that fails does not stop the others; with more than one target
|
||||||
`pushremote` prints an `n/m remotes updated` summary. `remotes = …` restricts
|
`pushremote` prints an `n/m remotes updated` summary. `remotes = …` restricts
|
||||||
and orders the set, which is mostly useful in a project `.mgshrc` — see below.
|
and orders the set, which is mostly useful in a project `.mgshrc` — see below.
|
||||||
@@ -198,9 +303,10 @@ The token is sent as a one-shot HTTP auth header: it is never written into the
|
|||||||
repo's git config, and it reaches git through the environment rather than the
|
repo's git config, and it reaches git through the environment rather than the
|
||||||
command line, so it does not show up in the process table. Because `~/.mgshrc`
|
command line, so it does not show up in the process table. Because `~/.mgshrc`
|
||||||
then holds a credential, mgsh creates it mode `600` and warns at startup if an
|
then holds a credential, mgsh creates it mode `600` and warns at startup if an
|
||||||
existing file is readable by others. The provider is auto-detected from `remoteurl` (`github.com` →
|
existing file is readable by others. The provider is auto-detected from the url
|
||||||
GitHub, `gitlab*` → GitLab, otherwise Gitea) and can be forced with
|
(`github.com` → GitHub, `gitlab*` → GitLab, otherwise Gitea) and can be forced
|
||||||
`remotetype`. Set `mirror = true` to have every `push` mirror automatically.
|
with `remote.<name>.type`. Set `mirror = true` to have every `push` mirror
|
||||||
|
automatically.
|
||||||
|
|
||||||
### Releases
|
### Releases
|
||||||
|
|
||||||
@@ -284,8 +390,10 @@ alias co 'checkout $1'
|
|||||||
### Settings reference
|
### Settings reference
|
||||||
|
|
||||||
Every setting can also be given as an environment variable named `MGSH_<KEY>`
|
Every setting can also be given as an environment variable named `MGSH_<KEY>`
|
||||||
(e.g. `MGSH_GITHOST`), which wins over both files. "Scope" says whether a
|
(e.g. `MGSH_GITHOST`), which wins over both files; a mirror field is
|
||||||
project `.mgshrc` may override the setting.
|
`MGSH_REMOTE_<NAME>_<FIELD>`, so `MGSH_REMOTE_GITLAB_KEY` sets
|
||||||
|
`remote.gitlab.key`. "Scope" says whether a project `.mgshrc` may override the
|
||||||
|
setting.
|
||||||
|
|
||||||
| setting | scope | meaning |
|
| setting | scope | meaning |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
@@ -299,16 +407,13 @@ project `.mgshrc` may override the setting.
|
|||||||
| `gitemail` | global | `user.email` written to the global git config |
|
| `gitemail` | global | `user.email` written to the global git config |
|
||||||
| `pushdefault` | global | `push.default` written to the global git config |
|
| `pushdefault` | global | `push.default` written to the global git config |
|
||||||
| `editor` | project | opener used by `open`/`view` when the project has no Xcode workspace (default `coda`) |
|
| `editor` | project | opener used by `open`/`view` when the project has no Xcode workspace (default `coda`) |
|
||||||
| `remoteurl` | project | base URL of a single mirror server (target name `public`) |
|
| `remote.<name>.url` | project | base URL of the mirror target `<name>` |
|
||||||
| `remotekey` | project | API token for `remoteurl` |
|
|
||||||
| `remotetype` | project | `gitea`\|`github`\|`gitlab`; auto-detected from the URL when unset |
|
|
||||||
| `remotevisibility` | project | `private` (default) or `public` for repositories created by `pushremote` |
|
|
||||||
| `remote.<name>.url` | project | base URL of the named mirror target |
|
|
||||||
| `remote.<name>.key` | project | API token for that target |
|
| `remote.<name>.key` | project | API token for that target |
|
||||||
| `remote.<name>.type` | project | provider override for that target |
|
| `remote.<name>.type` | project | `gitea`\|`github`\|`gitlab`; auto-detected from the url when unset |
|
||||||
| `remote.<name>.visibility` | project | visibility for that target |
|
| `remote.<name>.visibility` | project | `private` (default) or `public` for repositories `pushremote` creates |
|
||||||
| `remotes` | project | comma- or space-separated list restricting and ordering the mirror targets |
|
| `remotes` | project | comma- or space-separated list restricting and ordering the mirror targets |
|
||||||
| `mirror` | project | truthy (`1`/`true`/`yes`/`on`) → every `push` also mirrors |
|
| `mirror` | project | truthy (`1`/`true`/`yes`/`on`) → every `push` also mirrors |
|
||||||
|
| `secretscan` | project | `off` disables the credential check `push` runs before committing (on by default; only an explicit `off` disables it) |
|
||||||
|
|
||||||
The three settings written to the global git config are applied at startup, and
|
The three settings written to the global git config are applied at startup, and
|
||||||
only when they actually differ, so a plain `mgsh status` does not rewrite
|
only when they actually differ, so a plain `mgsh status` does not rewrite
|
||||||
|
|||||||
@@ -2,7 +2,9 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Colors for the prompt, banner and output, using the Catppuccin Mocha palette
|
// Colors for the prompt, banner and output, using the Catppuccin Mocha palette
|
||||||
@@ -35,25 +37,57 @@ func errorln(msg string) {
|
|||||||
fmt.Println(col(cRed, msg))
|
fmt.Println(col(cRed, msg))
|
||||||
}
|
}
|
||||||
|
|
||||||
// padRight pads an ASCII string with trailing spaces to width n.
|
// padRight pads s with trailing spaces to a width of n columns. It counts
|
||||||
|
// runes, not bytes: the overview pads fields holding ↑ ↓ ✓, each of which is
|
||||||
|
// one column wide but three bytes long.
|
||||||
func padRight(s string, n int) string {
|
func padRight(s string, n int) string {
|
||||||
if len(s) < n {
|
if l := utf8.RuneCountInString(s); l < n {
|
||||||
return s + strings.Repeat(" ", n-len(s))
|
return s + strings.Repeat(" ", n-l)
|
||||||
}
|
}
|
||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|
||||||
// colorRepoLine colors a `list` entry: the leading `ls -ltr` date (3 fields) in
|
// formatRepoList renders the server listing for `list`: the name first, in a
|
||||||
// yellow and the repository name in green.
|
// column wide enough for the longest one, then the date, and for archives the
|
||||||
func colorRepoLine(s string) string {
|
// size. Names come first because that is what the eye scans for; putting the
|
||||||
if !useColor {
|
// ragged date there instead is what made the old output hard to read.
|
||||||
return s
|
//
|
||||||
|
// The order is left as it arrives: `ls -ltr` sorts by modification time, and
|
||||||
|
// `push` touches the bare repository, so the most recently worked-on project
|
||||||
|
// ends up closest to the prompt.
|
||||||
|
func formatRepoList(entries []lsEntry, withSize bool) string {
|
||||||
|
width := 0
|
||||||
|
for _, e := range entries {
|
||||||
|
if len(e.name) > width {
|
||||||
|
width = len(e.name)
|
||||||
}
|
}
|
||||||
parts := strings.Fields(s)
|
|
||||||
if len(parts) >= 4 {
|
|
||||||
date := strings.Join(parts[:3], " ")
|
|
||||||
name := strings.Join(parts[3:], " ")
|
|
||||||
return col(cYellow, date) + " " + col(cGreen, name)
|
|
||||||
}
|
}
|
||||||
return col(cGreen, s)
|
var b strings.Builder
|
||||||
|
for _, e := range entries {
|
||||||
|
fmt.Fprintf(&b, " %s %s", col(cGreen, padRight(e.name, width)), col(cYellow, e.date))
|
||||||
|
if withSize {
|
||||||
|
fmt.Fprintf(&b, " %s", col(cGray, fmt.Sprintf("%7s", humanSize(e.size))))
|
||||||
|
}
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// humanSize renders a byte count compactly, the way `ls -h` does: a decimal
|
||||||
|
// only while it still carries information, so "3.2M" but "512K".
|
||||||
|
func humanSize(n int64) string {
|
||||||
|
const unit = 1024
|
||||||
|
if n < unit {
|
||||||
|
return strconv.FormatInt(n, 10) + "B"
|
||||||
|
}
|
||||||
|
div, exp := int64(unit), 0
|
||||||
|
for v := n / unit; v >= unit && exp < 4; v /= unit {
|
||||||
|
div *= unit
|
||||||
|
exp++
|
||||||
|
}
|
||||||
|
v := float64(n) / float64(div)
|
||||||
|
if v < 10 {
|
||||||
|
return fmt.Sprintf("%.1f%c", v, "KMGTP"[exp])
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%.0f%c", v, "KMGTP"[exp])
|
||||||
}
|
}
|
||||||
|
|||||||
+127
-21
@@ -14,32 +14,85 @@ import (
|
|||||||
var (
|
var (
|
||||||
optRe = regexp.MustCompile(`^-(\w)$`)
|
optRe = regexp.MustCompile(`^-(\w)$`)
|
||||||
numRe = regexp.MustCompile(`^\d+$`)
|
numRe = regexp.MustCompile(`^\d+$`)
|
||||||
// a `ls -ltr` long-listing line: mode, link count, owner, group, size, then
|
// a `ls -ltr` long-listing line: mode, link count, owner, group, size, the
|
||||||
// the date columns and the name. Owner and group are matched as opaque
|
// three date columns, then the name. Owner and group are matched as opaque
|
||||||
// fields — the bare repositories need not belong to a user or group
|
// fields — the bare repositories need not belong to a user or group
|
||||||
// literally named "git".
|
// literally named "git".
|
||||||
lsEntryRe = regexp.MustCompile(`^\S+\s+\d+\s+\S+\s+\S+\s+\d+\s+(.*)$`)
|
lsEntryRe = regexp.MustCompile(`^\S+\s+\d+\s+\S+\s+\S+\s+(\d+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(.*)$`)
|
||||||
gitDirRe = regexp.MustCompile(`^(.*)\.git$`)
|
gitDirRe = regexp.MustCompile(`^(.*)\.git$`)
|
||||||
sanRe = regexp.MustCompile(`[,;:\\/='"|?><-]+`)
|
sanRe = regexp.MustCompile(`[,;:\\/='"|?><-]+`)
|
||||||
wsRe = regexp.MustCompile(`\s+`)
|
wsRe = regexp.MustCompile(`\s+`)
|
||||||
)
|
)
|
||||||
|
|
||||||
// lsEntry extracts the "<date columns> <name>" tail of a `ls -ltr` line whose
|
// listMarker separates the two sections of the combined listing command, so
|
||||||
// entry name ends in suffix, with the suffix removed. It returns "" for any
|
// `list` gets both the long listing and the disk usage in one round trip.
|
||||||
// other line (the leading "total" line, entries of a different kind).
|
const listMarker = "---mgsh---"
|
||||||
func lsEntry(line, suffix string) string {
|
|
||||||
|
// duRe matches one `du -sk` line: kilobytes, then the path.
|
||||||
|
var duRe = regexp.MustCompile(`^(\d+)\s+(.*)$`)
|
||||||
|
|
||||||
|
// splitAtMarker divides the remote output into the part before and after the
|
||||||
|
// marker line. Everything is in the first section when the marker is absent —
|
||||||
|
// which is what happens when only a plain listing was asked for.
|
||||||
|
func splitAtMarker(lines []string, marker string) (before, after []string) {
|
||||||
|
for i, ln := range lines {
|
||||||
|
if strings.TrimSpace(ln) == marker {
|
||||||
|
return lines[:i], lines[i+1:]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lines, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseDuSizes turns `du -sk` output into a name -> bytes map. A long listing
|
||||||
|
// reports the inode size for a directory — the same number for every bare
|
||||||
|
// repository — so this is the only way to say how large one actually is.
|
||||||
|
// A symlinked repository reports the size of the link, not of its target.
|
||||||
|
func parseDuSizes(lines []string) map[string]int64 {
|
||||||
|
out := map[string]int64{}
|
||||||
|
for _, ln := range lines {
|
||||||
|
m := duRe.FindStringSubmatch(strings.TrimRight(ln, "\r"))
|
||||||
|
if m == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
kb, err := strconv.ParseInt(m[1], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out[strings.TrimPrefix(strings.TrimSpace(m[2]), "./")] = kb * 1024
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// lsEntry is one parsed entry of the server's listing.
|
||||||
|
type lsEntry struct {
|
||||||
|
name string // with the ".git" / ".git.tar.gz" suffix removed
|
||||||
|
date string // the ls date columns, normalised to a fixed 12 columns
|
||||||
|
size int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseLsEntry reads one `ls -ltr` line whose entry name ends in suffix. It
|
||||||
|
// returns false for anything else: the leading "total" line, entries of another
|
||||||
|
// kind, or output that does not look like a long listing at all.
|
||||||
|
func parseLsEntry(line, suffix string) (lsEntry, bool) {
|
||||||
m := lsEntryRe.FindStringSubmatch(strings.TrimSpace(line))
|
m := lsEntryRe.FindStringSubmatch(strings.TrimSpace(line))
|
||||||
if m == nil {
|
if m == nil {
|
||||||
return ""
|
return lsEntry{}, false
|
||||||
}
|
}
|
||||||
name := m[1]
|
name := m[5]
|
||||||
if i := strings.Index(name, " -> "); i >= 0 {
|
if i := strings.Index(name, " -> "); i >= 0 {
|
||||||
name = name[:i] // a symlinked bare repo lists as "link.git -> target.git"
|
name = name[:i] // a symlinked bare repo lists as "link.git -> target.git"
|
||||||
}
|
}
|
||||||
if !strings.HasSuffix(name, suffix) {
|
if !strings.HasSuffix(name, suffix) {
|
||||||
return ""
|
return lsEntry{}, false
|
||||||
}
|
}
|
||||||
return strings.TrimSuffix(name, suffix)
|
size, _ := strconv.ParseInt(m[1], 10, 64)
|
||||||
|
return lsEntry{
|
||||||
|
name: strings.TrimSuffix(name, suffix),
|
||||||
|
// ls pads these itself, but only in its own column widths; re-pad so
|
||||||
|
// "Sep 28 2016" and "Jan 3 14:32" line up at 12 either way
|
||||||
|
date: fmt.Sprintf("%s %2s %5s", m[2], m[3], m[4]),
|
||||||
|
size: size,
|
||||||
|
}, true
|
||||||
}
|
}
|
||||||
|
|
||||||
// validProject reports whether name is usable as a project name: a single path
|
// validProject reports whether name is usable as a project name: a single path
|
||||||
@@ -184,20 +237,66 @@ func runCommandDepth(line string, depth int) bool {
|
|||||||
if opt["a"] {
|
if opt["a"] {
|
||||||
path, suffix = "./archive", ".git.tar.gz"
|
path, suffix = "./archive", ".git.tar.gz"
|
||||||
}
|
}
|
||||||
pat := word(words, 1)
|
one, many := "repository", "repositories"
|
||||||
lines, err := sshOut("/bin/ls -ltr " + shq(path))
|
if opt["a"] {
|
||||||
if err != nil {
|
one, many = "archive", "archives"
|
||||||
errorln("could not list repositories on the git server")
|
|
||||||
break
|
|
||||||
}
|
}
|
||||||
for _, ln := range lines {
|
pat := strings.ToLower(word(words, 1))
|
||||||
if pat != "" && !strings.Contains(strings.ToLower(ln), strings.ToLower(pat)) {
|
remote := "/bin/ls -ltr " + shq(path)
|
||||||
|
if !opt["a"] {
|
||||||
|
// archives are files and carry a real size; repositories are
|
||||||
|
// directories, whose listed size is the inode's, so ask du in the
|
||||||
|
// same round trip. Nothing shell-specific here on purpose: the
|
||||||
|
// login shell may be csh, where "2>/dev/null" is not a redirection
|
||||||
|
// but an argument followed by one.
|
||||||
|
remote += "; echo " + shq(listMarker) + "; du -sk *.git"
|
||||||
|
}
|
||||||
|
lines, err := sshOut(remote)
|
||||||
|
lsLines, duLines := splitAtMarker(lines, listMarker)
|
||||||
|
sizes := parseDuSizes(duLines)
|
||||||
|
|
||||||
|
var entries []lsEntry
|
||||||
|
var total int64
|
||||||
|
for _, ln := range lsLines {
|
||||||
|
e, ok := parseLsEntry(ln, suffix)
|
||||||
|
// the pattern filters the name, not the whole listing line — an
|
||||||
|
// accidental match on the date or the owner helps nobody
|
||||||
|
if !ok || (pat != "" && !strings.Contains(strings.ToLower(e.name), pat)) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if name := lsEntry(ln, suffix); name != "" {
|
if !opt["a"] {
|
||||||
fmt.Println(colorRepoLine(name))
|
e.size = sizes[e.name+suffix] // 0 when du said nothing
|
||||||
}
|
}
|
||||||
|
total += e.size
|
||||||
|
entries = append(entries, e)
|
||||||
}
|
}
|
||||||
|
// The exit status belongs to the last command in the chain, so a `du`
|
||||||
|
// that fails must not discard a listing that arrived intact. Only
|
||||||
|
// complain when nothing usable came back at all.
|
||||||
|
if len(entries) == 0 {
|
||||||
|
if err != nil {
|
||||||
|
errorln("could not list " + many + " on the git server")
|
||||||
|
break
|
||||||
|
}
|
||||||
|
what := "no " + many + " on the git server"
|
||||||
|
if pat != "" {
|
||||||
|
what = "no " + many + " matching '" + word(words, 1) + "'"
|
||||||
|
}
|
||||||
|
fmt.Println(col(cGray, what))
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// no size column when the server gave no usable sizes, rather than a
|
||||||
|
// column of zeroes
|
||||||
|
fmt.Print(formatRepoList(entries, total > 0))
|
||||||
|
label := many
|
||||||
|
if len(entries) == 1 {
|
||||||
|
label = one
|
||||||
|
}
|
||||||
|
summary := fmt.Sprintf("%d %s", len(entries), label)
|
||||||
|
if total > 0 {
|
||||||
|
summary += " · " + humanSize(total)
|
||||||
|
}
|
||||||
|
fmt.Println(col(cGray, summary))
|
||||||
|
|
||||||
case "show": // show a repository's log directly on the server
|
case "show": // show a repository's log directly on the server
|
||||||
prj := PRJ
|
prj := PRJ
|
||||||
@@ -272,6 +371,13 @@ func runCommandDepth(line string, depth int) bool {
|
|||||||
}
|
}
|
||||||
comment := strings.Join(fields[1:], " ")
|
comment := strings.Join(fields[1:], " ")
|
||||||
git(DIR, "add", "--all", ".")
|
git(DIR, "add", "--all", ".")
|
||||||
|
// last look before anything is committed: `add --all` sweeps up whatever
|
||||||
|
// is lying around, and with mirroring on it goes straight to a public
|
||||||
|
// server. Nothing has been committed yet, so declining costs nothing.
|
||||||
|
if !secretsApproved(DIR) {
|
||||||
|
errorln("push cancelled — your changes are staged but not committed")
|
||||||
|
break
|
||||||
|
}
|
||||||
msg := strings.TrimSpace(fmt.Sprintf("[%s@%s] %s", USER, HOST, comment))
|
msg := strings.TrimSpace(fmt.Sprintf("[%s@%s] %s", USER, HOST, comment))
|
||||||
git(DIR, "commit", "-m", msg) // may be "nothing to commit"; continue anyway
|
git(DIR, "commit", "-m", msg) // may be "nothing to commit"; continue anyway
|
||||||
if !gitOK(DIR, "push") {
|
if !gitOK(DIR, "push") {
|
||||||
@@ -711,7 +817,7 @@ var helpItems = []struct{ cmd, desc string }{
|
|||||||
{"pull", "pull changes from git server"},
|
{"pull", "pull changes from git server"},
|
||||||
{"fetch", "fetch changes from git server"},
|
{"fetch", "fetch changes from git server"},
|
||||||
{"status [-a]", "short git status (-a: overview of all projects)"},
|
{"status [-a]", "short git status (-a: overview of all projects)"},
|
||||||
{"overview", "status of all projects (dirty, ahead/behind)"},
|
{"overview", "inventory of all projects, local and on the server"},
|
||||||
{"diff [args]", "show git diff"},
|
{"diff [args]", "show git diff"},
|
||||||
{"edit [number]", "edit last [number] commits (default is 10)"},
|
{"edit [number]", "edit last [number] commits (default is 10)"},
|
||||||
{"clone [-a] <repository>", "clone repository from git server (-a for archive)"},
|
{"clone [-a] <repository>", "clone repository from git server (-a for archive)"},
|
||||||
|
|||||||
+1
-7
@@ -82,18 +82,12 @@ func fetchServerRepos() {
|
|||||||
if serverFetched {
|
if serverFetched {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
lines, err := sshOut("/bin/ls .")
|
repos, err := serverRepoNames()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// a transient failure (server down, no network) must not cache an
|
// a transient failure (server down, no network) must not cache an
|
||||||
// empty list for the rest of the session — the next Tab tries again
|
// empty list for the rest of the session — the next Tab tries again
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
var repos []string
|
|
||||||
for _, ln := range lines {
|
|
||||||
if m := gitDirRe.FindStringSubmatch(strings.TrimSpace(ln)); m != nil {
|
|
||||||
repos = append(repos, m[1])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// a missing ./archive is a permanent, unremarkable state: still cache
|
// a missing ./archive is a permanent, unremarkable state: still cache
|
||||||
var archives []string
|
var archives []string
|
||||||
if lines, err := sshOut("/bin/ls archive"); err == nil {
|
if lines, err := sshOut("/bin/ls archive"); err == nil {
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"regexp"
|
"regexp"
|
||||||
|
"slices"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
@@ -33,11 +34,8 @@ type Config struct {
|
|||||||
GitEmail string // git user.email to set globally ("" = leave alone)
|
GitEmail string // git user.email to set globally ("" = leave alone)
|
||||||
PushDefault string // git push.default to set globally ("" = leave alone)
|
PushDefault string // git push.default to set globally ("" = leave alone)
|
||||||
Editor string // editor/opener used as fallback by `open` ("" = coda)
|
Editor string // editor/opener used as fallback by `open` ("" = coda)
|
||||||
RemoteURL string // public mirror server base URL (Gitea/GitHub/GitLab)
|
|
||||||
RemoteKey string // API token for the mirror server (used by `pushremote`)
|
|
||||||
RemoteType string // "gitea"|"github"|"gitlab" (auto-detected when empty)
|
|
||||||
RemoteVis string // visibility of created repos: "private" (default)|"public"
|
|
||||||
Mirror string // truthy -> `push` also mirrors via `pushremote`
|
Mirror string // truthy -> `push` also mirrors via `pushremote`
|
||||||
|
SecretScan string // falsy -> `push` skips the credential scan
|
||||||
Remotes []RemoteTarget
|
Remotes []RemoteTarget
|
||||||
RemoteNames string // "remotes": explicit, ordered subset of targets to use
|
RemoteNames string // "remotes": explicit, ordered subset of targets to use
|
||||||
}
|
}
|
||||||
@@ -53,22 +51,25 @@ type RemoteTarget struct {
|
|||||||
Vis string // "private" (default) | "public"
|
Vis string // "private" (default) | "public"
|
||||||
}
|
}
|
||||||
|
|
||||||
// legacyRemoteName is the target name for the flat remoteurl/remotekey pair,
|
// legacyRemoteName is the target the pre-4.1 flat remoteurl/remotekey settings
|
||||||
// matching the git remote that earlier versions created.
|
// are migrated to. It matches the git remote those versions created, so a
|
||||||
|
// converted configuration keeps pushing to the same place.
|
||||||
const legacyRemoteName = "public"
|
const legacyRemoteName = "public"
|
||||||
|
|
||||||
|
// legacyRemoteKeys maps the old flat spelling onto the named-target form. A
|
||||||
|
// mirror target is defined one way now, not two.
|
||||||
|
var legacyRemoteKeys = map[string]string{
|
||||||
|
"remoteurl": "remote." + legacyRemoteName + ".url",
|
||||||
|
"remotekey": "remote." + legacyRemoteName + ".key",
|
||||||
|
"remotetype": "remote." + legacyRemoteName + ".type",
|
||||||
|
"remotevisibility": "remote." + legacyRemoteName + ".visibility",
|
||||||
|
}
|
||||||
|
|
||||||
// mirrorTargets returns the usable mirror targets in configured order, plus the
|
// mirrorTargets returns the usable mirror targets in configured order, plus the
|
||||||
// names of targets that are defined but unusable (missing url or key) so the
|
// names of targets that are defined but unusable (missing url or key) so the
|
||||||
// caller can complain about them instead of silently skipping.
|
// caller can complain about them instead of silently skipping.
|
||||||
func (c Config) mirrorTargets() (usable []RemoteTarget, incomplete []string) {
|
func (c Config) mirrorTargets() (usable []RemoteTarget, incomplete []string) {
|
||||||
var all []RemoteTarget
|
all := c.Remotes
|
||||||
if c.RemoteURL != "" || c.RemoteKey != "" {
|
|
||||||
all = append(all, RemoteTarget{
|
|
||||||
Name: legacyRemoteName, URL: c.RemoteURL, Key: c.RemoteKey,
|
|
||||||
Type: c.RemoteType, Vis: c.RemoteVis,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
all = append(all, c.Remotes...)
|
|
||||||
|
|
||||||
// `remotes = a, b` narrows and orders the set — a project .mgshrc uses it
|
// `remotes = a, b` narrows and orders the set — a project .mgshrc uses it
|
||||||
// to mirror to only some of the globally configured servers.
|
// to mirror to only some of the globally configured servers.
|
||||||
@@ -146,11 +147,54 @@ func loadConfig() Config {
|
|||||||
m := parseConfig(string(data))
|
m := parseConfig(string(data))
|
||||||
applyConfig(&c, m)
|
applyConfig(&c, m)
|
||||||
warnConfigPerms(path, m)
|
warnConfigPerms(path, m)
|
||||||
|
migrateRemoteKeys(path, string(data))
|
||||||
}
|
}
|
||||||
applyEnv(&c)
|
applyEnv(&c)
|
||||||
return c
|
return c
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// migrateRemoteKeys converts the pre-4.1 flat remote settings in a config file
|
||||||
|
// to the remote.<name>.<field> spelling, so a mirror target is defined one way
|
||||||
|
// and not two. Only the key is rewritten: values, comments, blank lines and the
|
||||||
|
// file's permissions stay exactly as they are, and commented-out lines are left
|
||||||
|
// alone. Reports what it changed rather than doing it silently.
|
||||||
|
func migrateRemoteKeys(path, data string) {
|
||||||
|
lines := strings.Split(data, "\n")
|
||||||
|
var renamed []string
|
||||||
|
|
||||||
|
for i, ln := range lines {
|
||||||
|
trimmed := strings.TrimLeft(ln, " \t")
|
||||||
|
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
sep := strings.IndexAny(trimmed, "=:")
|
||||||
|
if sep < 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key := strings.TrimRight(trimmed[:sep], " \t")
|
||||||
|
dotted, ok := legacyRemoteKeys[strings.ToLower(key)]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
indent := ln[:len(ln)-len(trimmed)]
|
||||||
|
gap := trimmed[len(key):sep] // whatever alignment was there
|
||||||
|
lines[i] = indent + dotted + gap + trimmed[sep:]
|
||||||
|
renamed = append(renamed, key+" → "+dotted)
|
||||||
|
}
|
||||||
|
if len(renamed) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.WriteFile(path, []byte(strings.Join(lines, "\n")), configMode); err != nil {
|
||||||
|
errorln("could not update " + path + ": " + err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Println(col(cGray, path+": mirror settings renamed to the remote.<name>.* form"))
|
||||||
|
for _, r := range renamed {
|
||||||
|
fmt.Println(col(cGray, " "+r))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// projectGlobalOnly lists settings a project-level .mgshrc must not change:
|
// projectGlobalOnly lists settings a project-level .mgshrc must not change:
|
||||||
// `base` decides where projects live in the first place, and the git identity
|
// `base` decides where projects live in the first place, and the git identity
|
||||||
// keys are written to the user's *global* git config at startup — applying
|
// keys are written to the user's *global* git config at startup — applying
|
||||||
@@ -278,12 +322,14 @@ func writeConfigTemplate(path string) {
|
|||||||
b.WriteString("# pushdefault = matching\n")
|
b.WriteString("# pushdefault = matching\n")
|
||||||
b.WriteString("# editor = code\n\n")
|
b.WriteString("# editor = code\n\n")
|
||||||
b.WriteString("# --- public mirrors for `pushremote` ---\n")
|
b.WriteString("# --- public mirrors for `pushremote` ---\n")
|
||||||
b.WriteString("# One block per server; `pushremote` pushes to all of them,\n")
|
b.WriteString("# One 'remote.<name>.*' block per server. `pushremote` pushes to all\n")
|
||||||
b.WriteString("# `pushremote @hub` to a single one.\n")
|
b.WriteString("# of them, `pushremote @gitlab` to a single one. <name> is also the\n")
|
||||||
b.WriteString("# remote.hub.url = https://github.com\n")
|
b.WriteString("# git remote created in the repository.\n")
|
||||||
b.WriteString("# remote.hub.key = <personal-access-token>\n")
|
b.WriteString("# remote.gitlab.url = https://gitlab.example.com\n")
|
||||||
b.WriteString("# remote.hub.visibility = public\n")
|
b.WriteString("# remote.gitlab.key = <personal-access-token>\n")
|
||||||
b.WriteString("# remotes = hub # optional: restrict/order the set\n")
|
b.WriteString("# remote.gitlab.type = gitlab # optional; detected from the url\n")
|
||||||
|
b.WriteString("# remote.gitlab.visibility = private # or public (default private)\n")
|
||||||
|
b.WriteString("# remotes = gitlab # optional: restrict/order the set\n")
|
||||||
b.WriteString("# mirror = true # `push` also mirrors\n\n")
|
b.WriteString("# mirror = true # `push` also mirrors\n\n")
|
||||||
b.WriteString("# A project may override any of these (except base and the git\n")
|
b.WriteString("# A project may override any of these (except base and the git\n")
|
||||||
b.WriteString("# identity) in its own <project>/.mgshrc.\n")
|
b.WriteString("# identity) in its own <project>/.mgshrc.\n")
|
||||||
@@ -364,13 +410,36 @@ func applyConfig(c *Config, m map[string]string) {
|
|||||||
set("gitemail", &c.GitEmail)
|
set("gitemail", &c.GitEmail)
|
||||||
set("pushdefault", &c.PushDefault)
|
set("pushdefault", &c.PushDefault)
|
||||||
set("editor", &c.Editor)
|
set("editor", &c.Editor)
|
||||||
set("remoteurl", &c.RemoteURL)
|
|
||||||
set("remotekey", &c.RemoteKey)
|
|
||||||
set("remotetype", &c.RemoteType)
|
|
||||||
set("remotevisibility", &c.RemoteVis)
|
|
||||||
set("remotes", &c.RemoteNames)
|
set("remotes", &c.RemoteNames)
|
||||||
set("mirror", &c.Mirror)
|
set("mirror", &c.Mirror)
|
||||||
applyRemoteTargets(c, m)
|
set("secretscan", &c.SecretScan)
|
||||||
|
applyRemoteTargets(c, foldLegacyRemoteKeys(m))
|
||||||
|
}
|
||||||
|
|
||||||
|
// foldLegacyRemoteKeys rewrites the pre-4.1 flat remote settings into the
|
||||||
|
// named-target form, so a configuration that has not been converted yet still
|
||||||
|
// works while it is being read. The file itself is converted by
|
||||||
|
// migrateRemoteKeys; this only makes the current run behave.
|
||||||
|
func foldLegacyRemoteKeys(m map[string]string) map[string]string {
|
||||||
|
folded, copied := m, false
|
||||||
|
for old, dotted := range legacyRemoteKeys {
|
||||||
|
v, ok := m[old]
|
||||||
|
if !ok || v == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, taken := m[dotted]; taken {
|
||||||
|
continue // an explicit new-style setting always wins
|
||||||
|
}
|
||||||
|
if !copied { // copy on first write, never touch the caller's map
|
||||||
|
folded = make(map[string]string, len(m))
|
||||||
|
for k, val := range m {
|
||||||
|
folded[k] = val
|
||||||
|
}
|
||||||
|
copied = true
|
||||||
|
}
|
||||||
|
folded[dotted] = v
|
||||||
|
}
|
||||||
|
return folded
|
||||||
}
|
}
|
||||||
|
|
||||||
// remoteFieldRe matches a named mirror target setting: remote.<name>.<field>.
|
// remoteFieldRe matches a named mirror target setting: remote.<name>.<field>.
|
||||||
@@ -437,10 +506,55 @@ func applyEnv(c *Config) {
|
|||||||
env("MGSH_GITEMAIL", &c.GitEmail)
|
env("MGSH_GITEMAIL", &c.GitEmail)
|
||||||
env("MGSH_PUSHDEFAULT", &c.PushDefault)
|
env("MGSH_PUSHDEFAULT", &c.PushDefault)
|
||||||
env("MGSH_EDITOR", &c.Editor)
|
env("MGSH_EDITOR", &c.Editor)
|
||||||
env("MGSH_REMOTEURL", &c.RemoteURL)
|
|
||||||
env("MGSH_REMOTEKEY", &c.RemoteKey)
|
|
||||||
env("MGSH_REMOTETYPE", &c.RemoteType)
|
|
||||||
env("MGSH_REMOTEVISIBILITY", &c.RemoteVis)
|
|
||||||
env("MGSH_REMOTES", &c.RemoteNames)
|
env("MGSH_REMOTES", &c.RemoteNames)
|
||||||
env("MGSH_MIRROR", &c.Mirror)
|
env("MGSH_MIRROR", &c.Mirror)
|
||||||
|
env("MGSH_SECRETSCAN", &c.SecretScan)
|
||||||
|
applyRemoteEnv(c)
|
||||||
|
}
|
||||||
|
|
||||||
|
// remoteFields are the settings a mirror target is made of.
|
||||||
|
var remoteFields = []string{"url", "key", "type", "visibility"}
|
||||||
|
|
||||||
|
// applyRemoteEnv reads MGSH_REMOTE_<NAME>_<FIELD>, the environment spelling of
|
||||||
|
// a remote.<name>.<field> setting — MGSH_REMOTE_GITLAB_KEY for
|
||||||
|
// remote.gitlab.key. The field is taken from the end, so a target name may
|
||||||
|
// contain underscores itself.
|
||||||
|
func applyRemoteEnv(c *Config) {
|
||||||
|
const prefix = "MGSH_REMOTE_"
|
||||||
|
|
||||||
|
// sorted, so a target these variables introduce lands in the push order the
|
||||||
|
// same way on every run
|
||||||
|
envs := os.Environ()
|
||||||
|
sort.Strings(envs)
|
||||||
|
|
||||||
|
for _, kv := range envs {
|
||||||
|
eq := strings.IndexByte(kv, '=')
|
||||||
|
if eq < 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name, value := kv[:eq], kv[eq+1:]
|
||||||
|
if value == "" || !strings.HasPrefix(name, prefix) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rest := name[len(prefix):]
|
||||||
|
us := strings.LastIndexByte(rest, '_')
|
||||||
|
if us <= 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
target, field := strings.ToLower(rest[:us]), strings.ToLower(rest[us+1:])
|
||||||
|
if !slices.Contains(remoteFields, field) {
|
||||||
|
continue // MGSH_REMOTES and anything else that merely starts alike
|
||||||
|
}
|
||||||
|
t := &c.Remotes[c.remoteIndex(target)]
|
||||||
|
switch field {
|
||||||
|
case "url":
|
||||||
|
t.URL = value
|
||||||
|
case "key":
|
||||||
|
t.Key = value
|
||||||
|
case "type":
|
||||||
|
t.Type = value
|
||||||
|
case "visibility":
|
||||||
|
t.Vis = value
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -152,6 +152,22 @@ func sshOut(remote string) ([]string, error) {
|
|||||||
return lines, err
|
return lines, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// serverRepoNames lists the bare repositories on the git server, without the
|
||||||
|
// ".git" suffix.
|
||||||
|
func serverRepoNames() ([]string, error) {
|
||||||
|
lines, err := sshOut("/bin/ls .")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, ln := range lines {
|
||||||
|
if m := gitDirRe.FindStringSubmatch(strings.TrimSpace(ln)); m != nil {
|
||||||
|
out = append(out, m[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
// serverEntryExists reports whether entry is present in the remote directory
|
// serverEntryExists reports whether entry is present in the remote directory
|
||||||
// path (relative to the git user's home). The error is returned rather than
|
// path (relative to the git user's home). The error is returned rather than
|
||||||
// folded into the bool so a failed lookup is never mistaken for "not there".
|
// folded into the bool so a failed lookup is never mistaken for "not there".
|
||||||
|
|||||||
+297
-36
@@ -9,6 +9,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestSanitizeComment(t *testing.T) {
|
func TestSanitizeComment(t *testing.T) {
|
||||||
@@ -74,19 +75,65 @@ func TestFormatLogRecentCompact(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestColorRepoLine(t *testing.T) {
|
func TestFormatRepoList(t *testing.T) {
|
||||||
useColor = false
|
useColor = false
|
||||||
in := "Sep 28 2016 Betaflight3.0.0"
|
entries := []lsEntry{
|
||||||
if got := colorRepoLine(in); got != in {
|
{name: "short", date: "Sep 28 2016", size: 4096},
|
||||||
t.Errorf("colorRepoLine with color off changed input: %q", got)
|
{name: "a-much-longer-name", date: "Jan 3 14:32", size: 1536},
|
||||||
}
|
}
|
||||||
|
|
||||||
useColor = true
|
out := formatRepoList(entries, false)
|
||||||
got := colorRepoLine(in)
|
lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
|
||||||
if !strings.Contains(got, "Betaflight3.0.0") || !strings.Contains(got, cGreen) || !strings.Contains(got, cYellow) {
|
if len(lines) != 2 {
|
||||||
t.Errorf("colorRepoLine did not color parts: %q", got)
|
t.Fatalf("expected 2 lines, got %d: %q", len(lines), out)
|
||||||
}
|
}
|
||||||
|
// order is preserved: `ls -ltr` already sorted by modification time
|
||||||
|
if !strings.Contains(lines[0], "short") || !strings.Contains(lines[1], "a-much-longer-name") {
|
||||||
|
t.Errorf("order not preserved: %q", out)
|
||||||
|
}
|
||||||
|
// the date starts at the same column on every line
|
||||||
|
if strings.Index(lines[0], "Sep") != strings.Index(lines[1], "Jan") {
|
||||||
|
t.Errorf("date column not aligned:\n%s", out)
|
||||||
|
}
|
||||||
|
if strings.Contains(out, "4.0K") {
|
||||||
|
t.Errorf("size shown for repositories: %q", out)
|
||||||
|
}
|
||||||
|
|
||||||
|
if withSize := formatRepoList(entries, true); !strings.Contains(withSize, "4.0K") ||
|
||||||
|
!strings.Contains(withSize, "1.5K") {
|
||||||
|
t.Errorf("archive sizes missing: %q", withSize)
|
||||||
|
}
|
||||||
|
|
||||||
|
// colour must decorate the layout, never change it
|
||||||
|
useColor = true
|
||||||
|
colored := formatRepoList(entries, false)
|
||||||
useColor = false
|
useColor = false
|
||||||
|
strip := func(s string) string {
|
||||||
|
for _, c := range []string{cReset, cGreen, cYellow, cGray} {
|
||||||
|
s = strings.ReplaceAll(s, c, "")
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
if strip(colored) != out {
|
||||||
|
t.Errorf("colour changed the layout:\n%q\n%q", strip(colored), out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHumanSize(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
n int64
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{0, "0B"}, {512, "512B"}, {1024, "1.0K"}, {1536, "1.5K"},
|
||||||
|
{1024 * 1024, "1.0M"}, {3 * 1024 * 1024 * 1024, "3.0G"},
|
||||||
|
// past 10 the decimal carries nothing, as with `ls -h`
|
||||||
|
{512 * 1024, "512K"}, {99 * 1024 * 1024, "99M"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := humanSize(c.n); got != c.want {
|
||||||
|
t.Errorf("humanSize(%d) = %q, want %q", c.n, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseConfig(t *testing.T) {
|
func TestParseConfig(t *testing.T) {
|
||||||
@@ -147,38 +194,49 @@ gitemail = # value is only a comment
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLsEntry(t *testing.T) {
|
func TestParseLsEntry(t *testing.T) {
|
||||||
cases := []struct{ line, suffix, want string }{
|
cases := []struct {
|
||||||
// ownership is not assumed: any user/group must list
|
line, suffix string
|
||||||
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"},
|
name, date string
|
||||||
{"drwxr-xr-x 7 deploy deploy 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"},
|
size int64
|
||||||
{"drwxr-xr-x 7 mike staff 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"},
|
ok bool
|
||||||
{"drwxr-xr-x. 7 git users 4096 Sep 28 2016 myproj.git", ".git", "Sep 28 2016 myproj"},
|
}{
|
||||||
// archives only match the archive suffix, and vice versa
|
// ownership is not assumed: any user/group must parse
|
||||||
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git.tar.gz", "Sep 28 2016 myproj"},
|
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
|
||||||
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git", ""},
|
{"drwxr-xr-x 7 deploy deploy 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
|
||||||
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git.tar.gz", ""},
|
{"drwxr-xr-x. 7 git users 4096 Sep 28 2016 myproj.git", ".git", "myproj", "Sep 28 2016", 4096, true},
|
||||||
|
// a recent entry carries a time instead of a year, and still lines up
|
||||||
|
{"drwxr-xr-x 7 mike staff 224 Jan 3 14:32 myproj.git", ".git", "myproj", "Jan 3 14:32", 224, true},
|
||||||
|
// a symlinked bare repo lists its target too — only the link name counts
|
||||||
|
{"lrwxrwxrwx 1 git git 14 Sep 28 2016 myproj.git -> /srv/other.git", ".git", "myproj", "Sep 28 2016", 14, true},
|
||||||
|
// archives carry a size worth showing
|
||||||
|
{"-rw-r--r-- 1 git git 524288 Sep 28 2016 myproj.git.tar.gz", ".git.tar.gz", "myproj", "Sep 28 2016", 524288, true},
|
||||||
|
// suffixes must not cross over
|
||||||
|
{"-rw-r--r-- 1 git git 512 Sep 28 2016 myproj.git.tar.gz", ".git", "", "", 0, false},
|
||||||
|
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 myproj.git", ".git.tar.gz", "", "", 0, false},
|
||||||
|
{"lrwxrwxrwx 1 git git 5 Sep 28 2016 notes -> x.git", ".git", "", "", 0, false},
|
||||||
// non-entries
|
// non-entries
|
||||||
{"total 48", ".git", ""},
|
{"total 48", ".git", "", "", 0, false},
|
||||||
{"", ".git", ""},
|
{"", ".git", "", "", 0, false},
|
||||||
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 notes", ".git", ""},
|
{"drwxr-xr-x 7 git git 4096 Sep 28 2016 notes", ".git", "", "", 0, false},
|
||||||
}
|
}
|
||||||
for _, c := range cases {
|
for _, c := range cases {
|
||||||
if got := lsEntry(c.line, c.suffix); got != c.want {
|
e, ok := parseLsEntry(c.line, c.suffix)
|
||||||
t.Errorf("lsEntry(%q, %q) = %q, want %q", c.line, c.suffix, got, c.want)
|
if ok != c.ok {
|
||||||
|
t.Errorf("parseLsEntry(%q, %q) ok = %v, want %v", c.line, c.suffix, ok, c.ok)
|
||||||
|
continue
|
||||||
}
|
}
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
}
|
}
|
||||||
|
if e.name != c.name || e.size != c.size {
|
||||||
|
t.Errorf("parseLsEntry(%q) = %+v, want name %q size %d", c.line, e, c.name, c.size)
|
||||||
}
|
}
|
||||||
|
// every date renders to the same width, whichever form ls used
|
||||||
func TestLsEntrySymlink(t *testing.T) {
|
if e.date != c.date || len(e.date) != 12 {
|
||||||
// a symlinked bare repo lists its target too — only the link name counts
|
t.Errorf("parseLsEntry(%q) date = %q (len %d), want %q at 12",
|
||||||
in := "lrwxrwxrwx 1 git git 14 Sep 28 2016 myproj.git -> /srv/other.git"
|
c.line, e.date, len(e.date), c.date)
|
||||||
if got := lsEntry(in, ".git"); got != "Sep 28 2016 myproj" {
|
|
||||||
t.Errorf("lsEntry(symlink) = %q, want %q", got, "Sep 28 2016 myproj")
|
|
||||||
}
|
}
|
||||||
// and a symlink to something that is not a repo must not match
|
|
||||||
if got := lsEntry("lrwxrwxrwx 1 git git 5 Sep 28 2016 notes -> x.git", ".git"); got != "" {
|
|
||||||
t.Errorf("lsEntry(non-repo symlink) = %q, want empty", got)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -410,7 +468,7 @@ remote.broken.url = https://nowhere.example # no key -> unusable
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestMirrorTargetsLegacyAndSelection(t *testing.T) {
|
func TestMirrorTargetsLegacyAndSelection(t *testing.T) {
|
||||||
// the flat remoteurl/remotekey pair stays supported, as target "public"
|
// the pre-4.1 flat pair still loads, folded onto the target "public"
|
||||||
var c Config
|
var c Config
|
||||||
applyConfig(&c, parseConfig("remoteurl = https://git.example.com\nremotekey = tok\n"))
|
applyConfig(&c, parseConfig("remoteurl = https://git.example.com\nremotekey = tok\n"))
|
||||||
usable, _ := c.mirrorTargets()
|
usable, _ := c.mirrorTargets()
|
||||||
@@ -749,6 +807,7 @@ func TestTruthy(t *testing.T) {
|
|||||||
func TestFormatProjStatus(t *testing.T) {
|
func TestFormatProjStatus(t *testing.T) {
|
||||||
useColor = false
|
useColor = false
|
||||||
defer func() { useColor = false }()
|
defer func() { useColor = false }()
|
||||||
|
w := overviewWidths{label: 12, sync: 5, host: 7}
|
||||||
cases := []struct {
|
cases := []struct {
|
||||||
s projStatus
|
s projStatus
|
||||||
contains []string
|
contains []string
|
||||||
@@ -763,10 +822,16 @@ func TestFormatProjStatus(t *testing.T) {
|
|||||||
{projStatus{name: "d", branch: "feature", dirty: true},
|
{projStatus{name: "d", branch: "feature", dirty: true},
|
||||||
[]string{"d", "*", "(feature)"}, nil},
|
[]string{"d", "*", "(feature)"}, nil},
|
||||||
{projStatus{name: "e", branch: "master"}, // clean, no upstream
|
{projStatus{name: "e", branch: "master"}, // clean, no upstream
|
||||||
[]string{"e", "no upstream"}, []string{"*"}},
|
[]string{"e", "–"}, []string{"*", "✓"}},
|
||||||
|
{projStatus{name: "f", branch: "master", hasUpstream: true, ahead: 1, behind: 2},
|
||||||
|
[]string{"f", "↑1↓2"}, []string{"✓"}}, // diverged shows both
|
||||||
|
{projStatus{name: "g", branch: "master", hasUpstream: true, lastHost: "laptop",
|
||||||
|
mirrors: []string{"hub", "gitea"}},
|
||||||
|
[]string{"g", "laptop", "→ hub gitea"}, nil},
|
||||||
}
|
}
|
||||||
for _, c := range cases {
|
for _, c := range cases {
|
||||||
got := formatProjStatus(c.s, 8)
|
c.s.isRepo = true // these all describe real repositories
|
||||||
|
got := formatProjStatus(c.s, w)
|
||||||
for _, sub := range c.contains {
|
for _, sub := range c.contains {
|
||||||
if !strings.Contains(got, sub) {
|
if !strings.Contains(got, sub) {
|
||||||
t.Errorf("formatProjStatus(%+v) = %q, missing %q", c.s, got, sub)
|
t.Errorf("formatProjStatus(%+v) = %q, missing %q", c.s, got, sub)
|
||||||
@@ -780,6 +845,62 @@ func TestFormatProjStatus(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestOverviewColumnsAlign is the point of the table: every field has to start
|
||||||
|
// at the same column on every row, whatever the name lengths or the multi-byte
|
||||||
|
// status glyphs do.
|
||||||
|
func TestOverviewColumnsAlign(t *testing.T) {
|
||||||
|
useColor = false
|
||||||
|
// host names must not occur anywhere else in a row, or the index search
|
||||||
|
// below would find them inside a project or branch name instead
|
||||||
|
rows := []projStatus{
|
||||||
|
{name: "a", branch: "master", hasUpstream: true, ahead: 12, behind: 3, lastHost: "workstation"},
|
||||||
|
{name: "a-very-long-project-name", branch: "wip", dirty: true, lastHost: "buildbox"},
|
||||||
|
{name: "mid", branch: "main", hasUpstream: true, lastHost: "laptop"},
|
||||||
|
}
|
||||||
|
w := measureOverview(rows)
|
||||||
|
|
||||||
|
var widths []int
|
||||||
|
for _, r := range rows {
|
||||||
|
line := formatProjStatus(r, w)
|
||||||
|
// the host column starts right after the padded sync field
|
||||||
|
idx := strings.Index(line, r.lastHost)
|
||||||
|
if idx < 0 {
|
||||||
|
t.Fatalf("host %q missing from %q", r.lastHost, line)
|
||||||
|
}
|
||||||
|
widths = append(widths, utf8.RuneCountInString(line[:idx]))
|
||||||
|
}
|
||||||
|
for i := 1; i < len(widths); i++ {
|
||||||
|
if widths[i] != widths[0] {
|
||||||
|
t.Errorf("host column starts at %d on row %d, %d on row 0:\n%s",
|
||||||
|
widths[i], i, widths[0], strings.Join([]string{
|
||||||
|
formatProjStatus(rows[0], w), formatProjStatus(rows[i], w)}, "\n"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAttentionRank(t *testing.T) {
|
||||||
|
ranks := []struct {
|
||||||
|
s projStatus
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{projStatus{isRepo: true, dirty: true}, 0},
|
||||||
|
{projStatus{isRepo: true, ahead: 1}, 0},
|
||||||
|
{projStatus{isRepo: true, behind: 1}, 0},
|
||||||
|
{projStatus{isRepo: true, hasUpstream: true}, 1},
|
||||||
|
{projStatus{isRepo: true}, 1}, // clean, no upstream
|
||||||
|
// not on the server is a different kind of task and goes last, even
|
||||||
|
// when the working tree is dirty — it cannot be pushed anyway
|
||||||
|
{projStatus{isRepo: true, notOnServer: true}, 2},
|
||||||
|
{projStatus{isRepo: true, dirty: true, notOnServer: true}, 2},
|
||||||
|
{projStatus{notOnServer: true}, 2},
|
||||||
|
}
|
||||||
|
for _, c := range ranks {
|
||||||
|
if got := attentionRank(c.s); got != c.want {
|
||||||
|
t.Errorf("attentionRank(%+v) = %d, want %d", c.s, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestDetectRemoteKind(t *testing.T) {
|
func TestDetectRemoteKind(t *testing.T) {
|
||||||
cases := []struct {
|
cases := []struct {
|
||||||
url, override string
|
url, override string
|
||||||
@@ -886,3 +1007,143 @@ func TestExpandAlias(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestSplitAtMarker(t *testing.T) {
|
||||||
|
lines := []string{"a", "b", "---mgsh---", "c", "d"}
|
||||||
|
before, after := splitAtMarker(lines, "---mgsh---")
|
||||||
|
if strings.Join(before, ",") != "a,b" || strings.Join(after, ",") != "c,d" {
|
||||||
|
t.Errorf("split = %v / %v", before, after)
|
||||||
|
}
|
||||||
|
// no marker: everything is the first section, so a server that produced no
|
||||||
|
// du output simply yields no sizes
|
||||||
|
before, after = splitAtMarker([]string{"a", "b"}, "---mgsh---")
|
||||||
|
if strings.Join(before, ",") != "a,b" || after != nil {
|
||||||
|
t.Errorf("split without marker = %v / %v", before, after)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDuSizes(t *testing.T) {
|
||||||
|
lines := []string{
|
||||||
|
"185432\tBetaflight3.0.0.git",
|
||||||
|
"2144\twebsite.git",
|
||||||
|
"876 spaced-with-blanks.git", // some du implementations use spaces
|
||||||
|
"1024\t./with-dot-slash.git",
|
||||||
|
"1500\tmy project.git", // a name with a space survives
|
||||||
|
"garbage",
|
||||||
|
"",
|
||||||
|
}
|
||||||
|
got := parseDuSizes(lines)
|
||||||
|
want := map[string]int64{
|
||||||
|
"Betaflight3.0.0.git": 185432 * 1024,
|
||||||
|
"website.git": 2144 * 1024,
|
||||||
|
"spaced-with-blanks.git": 876 * 1024,
|
||||||
|
"with-dot-slash.git": 1024 * 1024,
|
||||||
|
"my project.git": 1500 * 1024,
|
||||||
|
}
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Fatalf("parseDuSizes = %v, want %d entries", got, len(want))
|
||||||
|
}
|
||||||
|
for k, v := range want {
|
||||||
|
if got[k] != v {
|
||||||
|
t.Errorf("parseDuSizes[%q] = %d, want %d", k, got[k], v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMigrateRemoteKeys rewrites the pre-4.1 flat spelling in place. Only the
|
||||||
|
// key changes: values, comments and everything else stay byte for byte.
|
||||||
|
func TestMigrateRemoteKeys(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
path := filepath.Join(dir, ".mgshrc")
|
||||||
|
before := `# my config
|
||||||
|
base = /home/me/src
|
||||||
|
remoteurl = https://git.example.com # the mirror
|
||||||
|
remotekey = s3cr3t-token
|
||||||
|
remotevisibility: public
|
||||||
|
# remotetype = gitea (commented out, must stay put)
|
||||||
|
alias co 'checkout $1'
|
||||||
|
`
|
||||||
|
if err := os.WriteFile(path, []byte(before), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
out := captureStdout(t, func() { migrateRemoteKeys(path, before) })
|
||||||
|
|
||||||
|
got := readFile(t, path)
|
||||||
|
for _, want := range []string{
|
||||||
|
"remote.public.url = https://git.example.com # the mirror",
|
||||||
|
"remote.public.key = s3cr3t-token",
|
||||||
|
" remote.public.visibility: public",
|
||||||
|
"# remotetype = gitea (commented out, must stay put)",
|
||||||
|
"base = /home/me/src",
|
||||||
|
"alias co 'checkout $1'",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(got, want) {
|
||||||
|
t.Errorf("migrated file missing %q:\n%s", want, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(got, "\nremoteurl") || strings.Contains(got, "\nremotekey") {
|
||||||
|
t.Errorf("old spelling left behind:\n%s", got)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "remoteurl → remote.public.url") {
|
||||||
|
t.Errorf("migration was not reported: %q", out)
|
||||||
|
}
|
||||||
|
// the file keeps its private mode
|
||||||
|
if fi, err := os.Stat(path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
} else if fi.Mode().Perm() != 0o600 {
|
||||||
|
t.Errorf("mode after migration = %04o, want 0600", fi.Mode().Perm())
|
||||||
|
}
|
||||||
|
|
||||||
|
// running again changes nothing and says nothing
|
||||||
|
second := captureStdout(t, func() { migrateRemoteKeys(path, readFile(t, path)) })
|
||||||
|
if strings.TrimSpace(second) != "" {
|
||||||
|
t.Errorf("a converted file was migrated again: %q", second)
|
||||||
|
}
|
||||||
|
if readFile(t, path) != got {
|
||||||
|
t.Error("a second migration changed the file")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLegacyKeysDoNotOverrideExplicitOnes: a config carrying both spellings must
|
||||||
|
// keep what the new one says.
|
||||||
|
func TestLegacyKeysDoNotOverrideExplicitOnes(t *testing.T) {
|
||||||
|
var c Config
|
||||||
|
applyConfig(&c, parseConfig(
|
||||||
|
"remoteurl = https://old.example\nremote.public.url = https://new.example\n"+
|
||||||
|
"remote.public.key = tok\n"))
|
||||||
|
targets, _ := c.mirrorTargets()
|
||||||
|
if len(targets) != 1 || targets[0].URL != "https://new.example" {
|
||||||
|
t.Errorf("targets = %+v, want the remote.public.url value", targets)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRemoteEnvOverrides: MGSH_REMOTE_<NAME>_<FIELD> is the environment
|
||||||
|
// spelling of remote.<name>.<field>.
|
||||||
|
func TestRemoteEnvOverrides(t *testing.T) {
|
||||||
|
t.Setenv("MGSH_REMOTE_GITLAB_URL", "https://gitlab.example")
|
||||||
|
t.Setenv("MGSH_REMOTE_GITLAB_KEY", "env-token")
|
||||||
|
t.Setenv("MGSH_REMOTE_GITLAB_VISIBILITY", "public")
|
||||||
|
t.Setenv("MGSH_REMOTE_MY_HUB_URL", "https://hub.example") // name with an underscore
|
||||||
|
t.Setenv("MGSH_REMOTE_MY_HUB_KEY", "hub-token")
|
||||||
|
t.Setenv("MGSH_REMOTES", "") // must not be mistaken for a target field
|
||||||
|
|
||||||
|
var c Config
|
||||||
|
applyConfig(&c, parseConfig("remote.gitlab.url = https://from-file.example\nremote.gitlab.key = file-token\n"))
|
||||||
|
applyEnv(&c)
|
||||||
|
|
||||||
|
targets, incomplete := c.mirrorTargets()
|
||||||
|
if len(incomplete) != 0 {
|
||||||
|
t.Fatalf("incomplete targets: %v", incomplete)
|
||||||
|
}
|
||||||
|
byName := map[string]RemoteTarget{}
|
||||||
|
for _, tg := range targets {
|
||||||
|
byName[tg.Name] = tg
|
||||||
|
}
|
||||||
|
if g := byName["gitlab"]; g.URL != "https://gitlab.example" || g.Key != "env-token" || g.Vis != "public" {
|
||||||
|
t.Errorf("env did not override the file: %+v", g)
|
||||||
|
}
|
||||||
|
// the field is taken from the end, so the name may contain underscores
|
||||||
|
if h := byName["my_hub"]; h.URL != "https://hub.example" || h.Key != "hub-token" {
|
||||||
|
t.Errorf("MGSH_REMOTE_MY_HUB_* = %+v, want target my_hub", h)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+19
-13
@@ -23,25 +23,31 @@ gitpath = /home/git
|
|||||||
# editor = code
|
# editor = code
|
||||||
|
|
||||||
# --- pushremote: mirror to public servers (gitea/github/gitlab) via their API ---
|
# --- pushremote: mirror to public servers (gitea/github/gitlab) via their API ---
|
||||||
# A single server, the flat form (this target is named "public"):
|
# One "remote.<name>.<field>" block per server, with the fields url, key, type
|
||||||
# remoteurl = https://git.example.com
|
# and visibility. <name> is yours to pick and becomes the git remote created in
|
||||||
# remotekey = <personal-access-token>
|
# the repository, so `git push gitlab` keeps working outside mgsh.
|
||||||
# remotetype = gitea # optional; auto-detected from remoteurl
|
#
|
||||||
# remotevisibility = private # visibility of created repos (default private)
|
# `pushremote` pushes to every configured server in the order given,
|
||||||
|
# `pushremote @gitlab` to a single one.
|
||||||
#
|
#
|
||||||
# Or any number of named servers. `pushremote` pushes to all of them in the
|
|
||||||
# order given, `pushremote @hub` to a single one. Each target gets a git remote
|
|
||||||
# of the same name in the repository.
|
|
||||||
# remote.gitea.url = https://git.example.com
|
# remote.gitea.url = https://git.example.com
|
||||||
# remote.gitea.key = <personal-access-token>
|
# remote.gitea.key = <personal-access-token>
|
||||||
# remote.hub.url = https://github.com
|
# remote.gitea.type = gitea # optional; auto-detected from the url
|
||||||
# remote.hub.key = <personal-access-token>
|
# remote.gitea.visibility = private # or public (default private)
|
||||||
# remote.hub.type = github # optional; auto-detected from the url
|
|
||||||
# remote.hub.visibility = public # default private
|
|
||||||
# remotes = gitea, hub # optional: restrict and order the set
|
|
||||||
#
|
#
|
||||||
|
# remote.gitlab.url = https://gitlab.example.com
|
||||||
|
# remote.gitlab.key = <personal-access-token>
|
||||||
|
# remote.gitlab.type = gitlab
|
||||||
|
# remote.gitlab.visibility = public
|
||||||
|
#
|
||||||
|
# remotes = gitea, gitlab # optional: restrict and order the set
|
||||||
# mirror = true # `push` also mirrors via pushremote
|
# mirror = true # `push` also mirrors via pushremote
|
||||||
|
|
||||||
|
# --- safety ---
|
||||||
|
# `push` checks the staged diff for private keys and API tokens before it
|
||||||
|
# commits, and asks before continuing. Only an explicit "off" disables it.
|
||||||
|
# secretscan = off
|
||||||
|
|
||||||
# --- per-project overrides ---
|
# --- per-project overrides ---
|
||||||
# A <project>/.mgshrc overrides all of the above for that project only, except
|
# A <project>/.mgshrc overrides all of the above for that project only, except
|
||||||
# base, gitname, gitemail and pushdefault, which stay global. Typical use:
|
# base, gitname, gitemail and pushdefault, which stay global. Typical use:
|
||||||
|
|||||||
+313
-51
@@ -1,25 +1,49 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
// overview.go — the `overview` command (also reachable as `status -a`): a
|
// overview.go — the `overview` command (also reachable as `status -a`).
|
||||||
// one-line-per-project summary of every git project under BASE, showing the
|
//
|
||||||
// dirty state and how far each branch is ahead/behind its upstream.
|
// mgsh is the only thing that sees all three places a project can live: the
|
||||||
|
// local base directory, the internal ssh server, and the public mirrors. Joining
|
||||||
|
// those answers the questions plain git cannot — which projects were never
|
||||||
|
// pushed to the server, and which machine last touched each one (every `push`
|
||||||
|
// stamps "[user@host]" into the commit message, so that comes for free).
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
|
|
||||||
// projStatus is the collected state of one project for the overview.
|
// projStatus is the collected state of one project for the overview.
|
||||||
type projStatus struct {
|
type projStatus struct {
|
||||||
name string
|
name string
|
||||||
branch string
|
branch string
|
||||||
|
isRepo bool // has a .git of its own
|
||||||
dirty bool
|
dirty bool
|
||||||
ahead, behind int
|
ahead, behind int
|
||||||
hasUpstream bool
|
hasUpstream bool
|
||||||
|
notOnServer bool // known to be missing from the git server
|
||||||
|
lastHost string // machine that made the last commit, from "[user@host]"
|
||||||
|
lastWhen time.Time // when that was
|
||||||
|
mirrors []string // configured mirror remotes present in this repo
|
||||||
}
|
}
|
||||||
|
|
||||||
// overviewAll prints a status summary for all git projects under BASE.
|
// commitHostRe pulls the host out of the "[user@host] subject" line that `push`
|
||||||
|
// writes, so the overview can say where a project was last worked on.
|
||||||
|
var commitHostRe = regexp.MustCompile(`^\[[^@\]]*@([^\]]+)\]`)
|
||||||
|
|
||||||
|
// overviewScanLimit bounds how many projects are inspected at once. The work is
|
||||||
|
// all subprocess latency, so some concurrency helps a lot and more does not.
|
||||||
|
const overviewScanLimit = 8
|
||||||
|
|
||||||
|
// overviewAll prints a status summary for all git projects under BASE, plus the
|
||||||
|
// projects that exist on only one side of the local/server divide.
|
||||||
func overviewAll() {
|
func overviewAll() {
|
||||||
entries, err := os.ReadDir(BASE)
|
entries, err := os.ReadDir(BASE)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -27,30 +51,52 @@ func overviewAll() {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
var rows []projStatus
|
// ask the server while the local tree is being walked
|
||||||
width := 0
|
type serverList struct {
|
||||||
|
names []string
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
srvCh := make(chan serverList, 1)
|
||||||
|
go func() {
|
||||||
|
names, err := serverRepoNames()
|
||||||
|
srvCh <- serverList{names, err}
|
||||||
|
}()
|
||||||
|
|
||||||
|
// every directory gets a row, repository or not: one that is not a
|
||||||
|
// repository yet is exactly what `init` is for, and putting it in the table
|
||||||
|
// beats a separate list underneath
|
||||||
|
var local []string
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
if !e.IsDir() || strings.HasPrefix(e.Name(), ".") {
|
if e.IsDir() && !strings.HasPrefix(e.Name(), ".") {
|
||||||
continue
|
local = append(local, e.Name())
|
||||||
}
|
|
||||||
dir := BASE + "/" + e.Name()
|
|
||||||
if !isDir(dir + "/.git") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
rows = append(rows, projectStatus(e.Name(), dir))
|
|
||||||
if len(e.Name()) > width {
|
|
||||||
width = len(e.Name())
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
rows := scanProjects(local)
|
||||||
|
srv := <-srvCh
|
||||||
|
markUnpublished(rows, srv.names, srv.err)
|
||||||
|
|
||||||
|
// what needs doing first, alphabetical within each group
|
||||||
|
sort.SliceStable(rows, func(i, j int) bool {
|
||||||
|
return attentionRank(rows[i]) < attentionRank(rows[j])
|
||||||
|
})
|
||||||
|
|
||||||
if len(rows) == 0 {
|
if len(rows) == 0 {
|
||||||
fmt.Println(col(cGray, "no git projects under "+BASE))
|
fmt.Println(col(cGray, "nothing under "+BASE))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
dirtyN, syncN := 0, 0
|
w := measureOverview(rows)
|
||||||
|
repoN, dirtyN, syncN, initN := 0, 0, 0, 0
|
||||||
for _, r := range rows {
|
for _, r := range rows {
|
||||||
fmt.Println(formatProjStatus(r, width))
|
fmt.Println(formatProjStatus(r, w))
|
||||||
|
if r.notOnServer {
|
||||||
|
initN++
|
||||||
|
}
|
||||||
|
if !r.isRepo {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
repoN++
|
||||||
if r.dirty {
|
if r.dirty {
|
||||||
dirtyN++
|
dirtyN++
|
||||||
}
|
}
|
||||||
@@ -58,51 +104,267 @@ func overviewAll() {
|
|||||||
syncN++
|
syncN++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
fmt.Printf("%s\n", col(cGray, fmt.Sprintf("%d projects · %d dirty · %d in sync", len(rows), dirtyN, syncN)))
|
|
||||||
|
summary := fmt.Sprintf("%d projects · %d dirty · %d in sync", repoN, dirtyN, syncN)
|
||||||
|
if initN > 0 {
|
||||||
|
summary += fmt.Sprintf(" · %d to init", initN)
|
||||||
|
}
|
||||||
|
fmt.Println(col(cGray, summary))
|
||||||
|
if srv.err != nil {
|
||||||
|
fmt.Println(col(cGray, " git server not reachable — local view only"))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// projectStatus gathers the git state of a single project directory.
|
// scanProjects collects the state of every project concurrently. Each project
|
||||||
|
// costs two git subprocesses, and serially that is the slowest thing mgsh does.
|
||||||
|
func scanProjects(names []string) []projStatus {
|
||||||
|
rows := make([]projStatus, len(names))
|
||||||
|
sem := make(chan struct{}, overviewScanLimit)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i, n := range names {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(i int, n string) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
rows[i] = projectStatus(n, BASE+"/"+n)
|
||||||
|
}(i, n)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
// markUnpublished flags the rows the git server has never seen — the ones
|
||||||
|
// `init` is for. A listing that failed leaves every row unmarked: not knowing
|
||||||
|
// is not the same as knowing they are missing, and marking all of them would
|
||||||
|
// tell the user to re-init their whole base directory.
|
||||||
|
func markUnpublished(rows []projStatus, server []string, err error) {
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
onServer := map[string]bool{}
|
||||||
|
for _, n := range server {
|
||||||
|
onServer[n] = true
|
||||||
|
}
|
||||||
|
for i := range rows {
|
||||||
|
rows[i].notOnServer = !onServer[rows[i].name]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// projectStatus gathers the git state of a single project directory. A
|
||||||
|
// directory without a repository is reported as it is, and costs no
|
||||||
|
// subprocesses at all.
|
||||||
func projectStatus(name, dir string) projStatus {
|
func projectStatus(name, dir string) projStatus {
|
||||||
s := projStatus{name: name, branch: "-"}
|
s := projStatus{name: name, branch: "-"}
|
||||||
if out, err := gitCapture(dir, "rev-parse", "--abbrev-ref", "HEAD"); err == nil {
|
if s.isRepo = isDir(dir + "/.git"); !s.isRepo {
|
||||||
s.branch = strings.TrimSpace(out)
|
return s
|
||||||
}
|
|
||||||
if out, err := gitCapture(dir, "status", "--porcelain"); err == nil && strings.TrimSpace(out) != "" {
|
|
||||||
s.dirty = true
|
|
||||||
}
|
|
||||||
// left/right counts against the upstream: "<behind>\t<ahead>"
|
|
||||||
if out, err := gitCapture(dir, "rev-list", "--left-right", "--count", "@{upstream}...HEAD"); err == nil {
|
|
||||||
if _, e := fmt.Sscanf(strings.TrimSpace(out), "%d\t%d", &s.behind, &s.ahead); e == nil {
|
|
||||||
s.hasUpstream = true
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
readStatus(&s, dir)
|
||||||
|
readLastCommit(&s, dir)
|
||||||
|
s.mirrors = configuredMirrors(dir)
|
||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|
||||||
// formatProjStatus renders one aligned overview row.
|
// readStatus fills in branch, upstream, ahead/behind and dirty from a single
|
||||||
func formatProjStatus(s projStatus, width int) string {
|
// `git status` — the porcelain v2 header carries all four.
|
||||||
var marks []string
|
func readStatus(s *projStatus, dir string) {
|
||||||
if s.dirty {
|
out, err := gitCapture(dir, "status", "--porcelain=v2", "--branch")
|
||||||
marks = append(marks, col(cYellow, "*"))
|
if err != nil {
|
||||||
|
return
|
||||||
}
|
}
|
||||||
if s.ahead > 0 {
|
for _, ln := range splitLines(out) {
|
||||||
marks = append(marks, col(cGreen, fmt.Sprintf("↑%d", s.ahead)))
|
if !strings.HasPrefix(ln, "# ") {
|
||||||
|
s.dirty = true // any entry line means the tree is not clean
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
f := strings.Fields(ln)
|
||||||
|
if len(f) < 3 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch f[1] {
|
||||||
|
case "branch.head":
|
||||||
|
s.branch = f[2]
|
||||||
|
case "branch.upstream":
|
||||||
|
s.hasUpstream = true
|
||||||
|
case "branch.ab":
|
||||||
|
if len(f) >= 4 {
|
||||||
|
s.ahead, _ = strconv.Atoi(strings.TrimPrefix(f[2], "+"))
|
||||||
|
s.behind, _ = strconv.Atoi(strings.TrimPrefix(f[3], "-"))
|
||||||
}
|
}
|
||||||
if s.behind > 0 {
|
|
||||||
marks = append(marks, col(cRed, fmt.Sprintf("↓%d", s.behind)))
|
|
||||||
}
|
}
|
||||||
state := strings.Join(marks, " ")
|
|
||||||
if state == "" {
|
|
||||||
if s.hasUpstream {
|
|
||||||
state = col(cGreen, "✓")
|
|
||||||
} else {
|
|
||||||
state = col(cGray, "✓ (no upstream)")
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
line := " " + col(cGreen, padRight(s.name, width+2)) + state
|
// readLastCommit records when the project was last committed to and from which
|
||||||
if s.branch != "master" && s.branch != "main" && s.branch != "-" {
|
// machine, taken from the "[user@host]" prefix `push` writes.
|
||||||
line += col(cGray, " ("+s.branch+")")
|
func readLastCommit(s *projStatus, dir string) {
|
||||||
|
out, err := gitCapture(dir, "log", "-1", "--format=%ct%x00%s")
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
parts := strings.SplitN(strings.TrimSpace(out), "\x00", 2)
|
||||||
|
if len(parts) != 2 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if epoch, err := strconv.ParseInt(parts[0], 10, 64); err == nil {
|
||||||
|
s.lastWhen = time.Unix(epoch, 0)
|
||||||
|
}
|
||||||
|
if m := commitHostRe.FindStringSubmatch(parts[1]); m != nil {
|
||||||
|
s.lastHost = m[1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// configuredMirrors returns the mirror targets this repository actually has a
|
||||||
|
// remote for — free to determine, since it is only local git config.
|
||||||
|
func configuredMirrors(dir string) []string {
|
||||||
|
targets, _ := cfg.mirrorTargets()
|
||||||
|
if len(targets) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out, err := gitCapture(dir, "remote")
|
||||||
|
if err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
have := map[string]bool{}
|
||||||
|
for _, r := range splitLines(out) {
|
||||||
|
have[strings.TrimSpace(r)] = true
|
||||||
|
}
|
||||||
|
var found []string
|
||||||
|
for _, t := range targets {
|
||||||
|
if have[t.Name] {
|
||||||
|
found = append(found, t.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return found
|
||||||
|
}
|
||||||
|
|
||||||
|
// overviewWidths are the column widths of the overview table, measured from the
|
||||||
|
// rows so every field starts at the same place. Ragged columns were what made
|
||||||
|
// the old one-line-per-project output hard to read.
|
||||||
|
type overviewWidths struct {
|
||||||
|
label, sync, host int
|
||||||
|
hint bool // any row carries an action hint
|
||||||
|
}
|
||||||
|
|
||||||
|
// hintWidth is the width of the action column, sized for its only word.
|
||||||
|
const hintWidth = 4
|
||||||
|
|
||||||
|
// measureOverview sizes the columns for a set of rows.
|
||||||
|
func measureOverview(rows []projStatus) overviewWidths {
|
||||||
|
var w overviewWidths
|
||||||
|
for _, r := range rows {
|
||||||
|
w.label = max(w.label, utf8.RuneCountInString(projLabel(r)))
|
||||||
|
w.sync = max(w.sync, utf8.RuneCountInString(syncState(r)))
|
||||||
|
w.host = max(w.host, utf8.RuneCountInString(r.lastHost))
|
||||||
|
w.hint = w.hint || r.notOnServer
|
||||||
|
}
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// projLabel is the first column: the project, with its branch appended when it
|
||||||
|
// is not the usual one. Keeping the branch attached to the name costs no extra
|
||||||
|
// column and keeps the table narrow.
|
||||||
|
func projLabel(s projStatus) string {
|
||||||
|
if s.branch != "" && s.branch != "-" && s.branch != "master" && s.branch != "main" {
|
||||||
|
return s.name + " (" + s.branch + ")"
|
||||||
|
}
|
||||||
|
return s.name
|
||||||
|
}
|
||||||
|
|
||||||
|
// syncState renders the relation to the upstream as one short field: ahead,
|
||||||
|
// behind, both, in sync, or "–" for a branch that tracks nothing. The old
|
||||||
|
// spelled-out "(no upstream)" was fifteen columns wide and pushed every
|
||||||
|
// following field out of line.
|
||||||
|
func syncState(s projStatus) string {
|
||||||
|
switch {
|
||||||
|
case !s.isRepo:
|
||||||
|
return "" // nothing to compare: there is no repository here yet
|
||||||
|
case s.ahead > 0 && s.behind > 0:
|
||||||
|
return fmt.Sprintf("↑%d↓%d", s.ahead, s.behind)
|
||||||
|
case s.ahead > 0:
|
||||||
|
return fmt.Sprintf("↑%d", s.ahead)
|
||||||
|
case s.behind > 0:
|
||||||
|
return fmt.Sprintf("↓%d", s.behind)
|
||||||
|
case s.hasUpstream:
|
||||||
|
return "✓"
|
||||||
|
default:
|
||||||
|
return "–"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// syncColor weights a row visually: anything needing action is coloured, a
|
||||||
|
// project that is clean and in sync recedes into grey.
|
||||||
|
func syncColor(s projStatus) string {
|
||||||
|
switch {
|
||||||
|
case s.behind > 0:
|
||||||
|
return cRed
|
||||||
|
case s.ahead > 0:
|
||||||
|
return cGreen
|
||||||
|
default:
|
||||||
|
return cGray
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// attentionRank groups the rows: work in progress at the top, then everything
|
||||||
|
// that is settled, and last the directories the server does not have yet. With
|
||||||
|
// many projects, scanning the whole list for the two dirty ones is the actual
|
||||||
|
// work — and an un-inited directory is a different kind of task, not something
|
||||||
|
// to push past the daily ones.
|
||||||
|
func attentionRank(s projStatus) int {
|
||||||
|
switch {
|
||||||
|
case s.notOnServer:
|
||||||
|
return 2
|
||||||
|
case s.dirty || s.ahead > 0 || s.behind > 0:
|
||||||
|
return 0
|
||||||
|
default:
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// formatProjStatus renders one row of the overview table.
|
||||||
|
func formatProjStatus(s projStatus, w overviewWidths) string {
|
||||||
|
dirty := " "
|
||||||
|
if s.dirty {
|
||||||
|
dirty = "*"
|
||||||
|
}
|
||||||
|
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString(" ")
|
||||||
|
b.WriteString(col(cGreen, padRight(projLabel(s), w.label)))
|
||||||
|
b.WriteString(" " + col(cYellow, dirty) + " ")
|
||||||
|
b.WriteString(col(syncColor(s), padRight(syncState(s), w.sync)))
|
||||||
|
|
||||||
|
if w.host > 0 {
|
||||||
|
age := ""
|
||||||
|
if !s.lastWhen.IsZero() {
|
||||||
|
age = shortAge(time.Since(s.lastWhen))
|
||||||
|
}
|
||||||
|
b.WriteString(" " + col(cGray, padRight(s.lastHost, w.host)))
|
||||||
|
b.WriteString(" " + col(cGray, fmt.Sprintf("%4s", age)))
|
||||||
|
}
|
||||||
|
if w.hint {
|
||||||
|
hint := ""
|
||||||
|
if s.notOnServer {
|
||||||
|
hint = "init"
|
||||||
|
}
|
||||||
|
b.WriteString(" " + col(cYellow, padRight(hint, hintWidth)))
|
||||||
|
}
|
||||||
|
if len(s.mirrors) > 0 {
|
||||||
|
b.WriteString(col(cGray, " → "+strings.Join(s.mirrors, " ")))
|
||||||
|
}
|
||||||
|
return strings.TrimRight(b.String(), " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// shortAge renders a duration compactly: 90s -> "1m", 36h -> "1d".
|
||||||
|
func shortAge(d time.Duration) string {
|
||||||
|
switch {
|
||||||
|
case d < time.Minute:
|
||||||
|
return "now"
|
||||||
|
case d < time.Hour:
|
||||||
|
return fmt.Sprintf("%dm", int(d.Minutes()))
|
||||||
|
case d < 24*time.Hour:
|
||||||
|
return fmt.Sprintf("%dh", int(d.Hours()))
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("%dd", int(d.Hours()/24))
|
||||||
}
|
}
|
||||||
return line
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,182 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// overview_test.go — the inventory view: what mgsh knows that plain git cannot.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestReadStatusParsesPorcelainV2 covers the single `git status` call that
|
||||||
|
// replaced three separate ones: branch, upstream, ahead/behind and dirty all
|
||||||
|
// come out of its header.
|
||||||
|
func TestReadStatusParsesPorcelainV2(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
bare := filepath.Join(t.TempDir(), "o.git")
|
||||||
|
mustGit(t, "", "init", "--bare", "-q", bare)
|
||||||
|
mustGit(t, dir, "init", "-q")
|
||||||
|
mustGit(t, dir, "config", "user.name", "t")
|
||||||
|
mustGit(t, dir, "config", "user.email", "t@e")
|
||||||
|
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "[mike@laptop] work")
|
||||||
|
mustGit(t, dir, "remote", "add", "origin", bare)
|
||||||
|
mustGit(t, dir, "push", "-q", "-u", "origin", "HEAD")
|
||||||
|
|
||||||
|
var s projStatus
|
||||||
|
readStatus(&s, dir)
|
||||||
|
if !s.hasUpstream || s.ahead != 0 || s.behind != 0 || s.dirty {
|
||||||
|
t.Errorf("clean synced repo = %+v", s)
|
||||||
|
}
|
||||||
|
if s.branch == "" || s.branch == "-" {
|
||||||
|
t.Errorf("branch not read: %q", s.branch)
|
||||||
|
}
|
||||||
|
|
||||||
|
// one unstaged file and one unpushed commit
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "x"), []byte("x"), 0644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "[mike@desktop] more")
|
||||||
|
|
||||||
|
s = projStatus{}
|
||||||
|
readStatus(&s, dir)
|
||||||
|
if !s.dirty {
|
||||||
|
t.Error("untracked file did not register as dirty")
|
||||||
|
}
|
||||||
|
if s.ahead != 1 {
|
||||||
|
t.Errorf("ahead = %d, want 1", s.ahead)
|
||||||
|
}
|
||||||
|
|
||||||
|
// and the host stamp `push` writes is picked up
|
||||||
|
var l projStatus
|
||||||
|
readLastCommit(&l, dir)
|
||||||
|
if l.lastHost != "desktop" {
|
||||||
|
t.Errorf("lastHost = %q, want desktop", l.lastHost)
|
||||||
|
}
|
||||||
|
if l.lastWhen.IsZero() {
|
||||||
|
t.Error("lastWhen not read")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCommitHostRe: only mgsh's own "[user@host]" stamp counts.
|
||||||
|
func TestCommitHostRe(t *testing.T) {
|
||||||
|
cases := map[string]string{
|
||||||
|
"[mike@laptop] fixed a thing": "laptop",
|
||||||
|
"[mike@build-01] ": "build-01",
|
||||||
|
"[@host] no user": "host",
|
||||||
|
"fixed a thing": "",
|
||||||
|
"[not a stamp] text": "",
|
||||||
|
"see [a@b] mid-line": "",
|
||||||
|
}
|
||||||
|
for subj, want := range cases {
|
||||||
|
got := ""
|
||||||
|
if m := commitHostRe.FindStringSubmatch(subj); m != nil {
|
||||||
|
got = m[1]
|
||||||
|
}
|
||||||
|
if got != want {
|
||||||
|
t.Errorf("host of %q = %q, want %q", subj, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMarkUnpublishedUnreachableServer: a listing that failed must leave every
|
||||||
|
// row unmarked. Not knowing is not the same as knowing they are missing —
|
||||||
|
// marking all of them would tell the user to re-init their whole base.
|
||||||
|
func TestMarkUnpublishedUnreachableServer(t *testing.T) {
|
||||||
|
rows := []projStatus{{name: "a"}, {name: "b"}}
|
||||||
|
markUnpublished(rows, nil, errors.New("network is unreachable"))
|
||||||
|
for _, r := range rows {
|
||||||
|
if r.notOnServer {
|
||||||
|
t.Errorf("%s marked as missing although the server could not be listed", r.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMarkUnpublished flags only what the server really does not have.
|
||||||
|
func TestMarkUnpublished(t *testing.T) {
|
||||||
|
rows := []projStatus{{name: "both"}, {name: "onlyhere"}}
|
||||||
|
markUnpublished(rows, []string{"both", "onlythere"}, nil)
|
||||||
|
if rows[0].notOnServer {
|
||||||
|
t.Error("a project present on both sides was marked")
|
||||||
|
}
|
||||||
|
if !rows[1].notOnServer {
|
||||||
|
t.Error("a local-only project was not marked")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUnpublishedRowsCarryTheHint: the entries live in the table now, with the
|
||||||
|
// action in their own column, rather than in a list underneath it.
|
||||||
|
func TestUnpublishedRowsCarryTheHint(t *testing.T) {
|
||||||
|
useColor = false
|
||||||
|
rows := []projStatus{
|
||||||
|
{name: "published", isRepo: true, hasUpstream: true},
|
||||||
|
{name: "fresh", isRepo: true, notOnServer: true},
|
||||||
|
{name: "notarepo", notOnServer: true},
|
||||||
|
}
|
||||||
|
w := measureOverview(rows)
|
||||||
|
if !w.hint {
|
||||||
|
t.Fatal("hint column not reserved although rows need it")
|
||||||
|
}
|
||||||
|
got := []string{}
|
||||||
|
for _, r := range rows {
|
||||||
|
got = append(got, formatProjStatus(r, w))
|
||||||
|
}
|
||||||
|
if strings.Contains(got[0], "init") {
|
||||||
|
t.Errorf("a published project was hinted: %q", got[0])
|
||||||
|
}
|
||||||
|
for _, i := range []int{1, 2} {
|
||||||
|
if !strings.Contains(got[i], "init") {
|
||||||
|
t.Errorf("row %d missing the init hint: %q", i, got[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// a directory that is not a repository has no sync state to report
|
||||||
|
if strings.ContainsAny(got[2], "✓–↑↓") {
|
||||||
|
t.Errorf("non-repository row claims a git state: %q", got[2])
|
||||||
|
}
|
||||||
|
// with nothing to hint the column disappears entirely
|
||||||
|
if measureOverview(rows[:1]).hint {
|
||||||
|
t.Error("hint column reserved although no row needs it")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestShortAge(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
d time.Duration
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{30 * time.Second, "now"},
|
||||||
|
{90 * time.Second, "1m"},
|
||||||
|
{2 * time.Hour, "2h"},
|
||||||
|
{36 * time.Hour, "1d"},
|
||||||
|
{20 * 24 * time.Hour, "20d"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := shortAge(c.d); got != c.want {
|
||||||
|
t.Errorf("shortAge(%v) = %q, want %q", c.d, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// captureStdout collects everything a function prints.
|
||||||
|
func captureStdout(t *testing.T, fn func()) string {
|
||||||
|
t.Helper()
|
||||||
|
old := os.Stdout
|
||||||
|
r, w, err := os.Pipe()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
os.Stdout = w
|
||||||
|
done := make(chan string)
|
||||||
|
go func() {
|
||||||
|
var b strings.Builder
|
||||||
|
io.Copy(&b, r)
|
||||||
|
done <- b.String()
|
||||||
|
}()
|
||||||
|
fn()
|
||||||
|
w.Close()
|
||||||
|
os.Stdout = old
|
||||||
|
return <-done
|
||||||
|
}
|
||||||
@@ -4,23 +4,18 @@ package main
|
|||||||
// git hosting server (Gitea, GitHub or GitLab), creating the repository via the
|
// git hosting server (Gitea, GitHub or GitLab), creating the repository via the
|
||||||
// server's REST API when it does not exist yet.
|
// server's REST API when it does not exist yet.
|
||||||
//
|
//
|
||||||
// Configuration (in ~/.mgshrc, a project .mgshrc, or MGSH_* env) — either a
|
// Configuration (in ~/.mgshrc, a project .mgshrc, or MGSH_* env): one
|
||||||
// single flat target:
|
// remote.<name>.<field> block per server, which `pushremote` mirrors to in turn.
|
||||||
//
|
//
|
||||||
// remoteurl = https://git.example.com base URL of the server
|
// remote.gitlab.url = https://gitlab.example.com
|
||||||
// remotekey = <api-token> personal access token
|
// remote.gitlab.key = <api-token>
|
||||||
// remotetype = gitea|github|gitlab optional; auto-detected from the URL
|
// remote.gitlab.type = gitlab optional; detected from the url
|
||||||
|
// remote.gitlab.visibility = public or private (the default)
|
||||||
|
// remotes = gitlab optional: restrict/order the set
|
||||||
//
|
//
|
||||||
// or any number of named ones, which `pushremote` mirrors to in turn:
|
// Each target owns a git remote of its own name in the repository. There is no
|
||||||
//
|
// second spelling: the pre-4.1 flat remoteurl/remotekey pair is migrated to
|
||||||
// remote.gitea.url = https://git.example.com
|
// remote.public.* on load.
|
||||||
// remote.gitea.key = <api-token>
|
|
||||||
// remote.hub.url = https://github.com
|
|
||||||
// remote.hub.key = <api-token>
|
|
||||||
// remote.hub.visibility = public
|
|
||||||
// remotes = gitea, hub optional: restrict/order the set
|
|
||||||
//
|
|
||||||
// Each target owns a git remote of the same name in the repository.
|
|
||||||
//
|
//
|
||||||
// The token is used for the API calls and, via an HTTP Basic auth header, for
|
// The token is used for the API calls and, via an HTTP Basic auth header, for
|
||||||
// the git push. It is never written into the repository's git config, and it is
|
// the git push. It is never written into the repository's git config, and it is
|
||||||
@@ -287,7 +282,7 @@ func handlePushRemote(args string) {
|
|||||||
targets = pickRemotes(targets, names)
|
targets = pickRemotes(targets, names)
|
||||||
if len(targets) == 0 {
|
if len(targets) == 0 {
|
||||||
if len(names) == 0 { // an unknown @name already reported itself
|
if len(names) == 0 { // an unknown @name already reported itself
|
||||||
errorln("pushremote needs 'remoteurl'/'remotekey' or a 'remote.<name>.*' block in " + configFile())
|
errorln("pushremote needs a 'remote.<name>.url' and 'remote.<name>.key' in " + configFile())
|
||||||
}
|
}
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -329,3 +329,65 @@ func mustGit(t *testing.T, dir string, args ...string) {
|
|||||||
t.Fatalf("git %v: %v\n%s", args, err, out)
|
t.Fatalf("git %v: %v\n%s", args, err, out)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestListSurvivesFailingDu: `list` chains the listing and `du` into one remote
|
||||||
|
// command, and the exit status is the *last* command's. A server whose du fails
|
||||||
|
// — a shell that mis-parses the arguments, a du that is not there, a permission
|
||||||
|
// problem — must still get its repositories listed.
|
||||||
|
func TestListSurvivesFailingDu(t *testing.T) {
|
||||||
|
useProject(t, "x")
|
||||||
|
fakeServer(t, func(cmd string) (string, error) {
|
||||||
|
return "total 4\n" +
|
||||||
|
"drwxr-xr-x 7 git git 4096 Jan 3 14:32 notes.git\n" +
|
||||||
|
"drwxr-xr-x 7 git git 4096 Sep 28 2016 website.git\n" +
|
||||||
|
listMarker + "\n",
|
||||||
|
errors.New("exit status 1") // du blew up, ls did not
|
||||||
|
})
|
||||||
|
|
||||||
|
out := captureStdout(t, func() { runCommand("list") })
|
||||||
|
|
||||||
|
if strings.Contains(out, "could not list") {
|
||||||
|
t.Errorf("a failing du discarded a good listing:\n%s", out)
|
||||||
|
}
|
||||||
|
for _, want := range []string{"notes", "website", "2 repositories"} {
|
||||||
|
if !strings.Contains(out, want) {
|
||||||
|
t.Errorf("listing missing %q:\n%s", want, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// without sizes there must be no size column, not a column of zeroes
|
||||||
|
if strings.Contains(out, "0B") {
|
||||||
|
t.Errorf("zero sizes shown when du produced none:\n%s", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestListReportsATrulyFailedListing: when nothing usable came back, the error
|
||||||
|
// still has to surface.
|
||||||
|
func TestListReportsATrulyFailedListing(t *testing.T) {
|
||||||
|
useProject(t, "x")
|
||||||
|
fakeServer(t, func(cmd string) (string, error) {
|
||||||
|
return "", errors.New("ssh: connect failed")
|
||||||
|
})
|
||||||
|
out := captureStdout(t, func() { runCommand("list") })
|
||||||
|
if !strings.Contains(out, "could not list") {
|
||||||
|
t.Errorf("a failed listing was not reported:\n%s", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestListSendsNoShellSpecificSyntax guards the bug this replaced: the remote
|
||||||
|
// command is run by the git user's login shell, which may be csh, where
|
||||||
|
// "2>/dev/null" is an argument followed by a redirection rather than a
|
||||||
|
// redirection of stderr.
|
||||||
|
func TestListSendsNoShellSpecificSyntax(t *testing.T) {
|
||||||
|
useProject(t, "x")
|
||||||
|
sent := fakeServer(t, func(cmd string) (string, error) { return "", nil })
|
||||||
|
captureStdout(t, func() { runCommand("list") })
|
||||||
|
|
||||||
|
if len(*sent) == 0 {
|
||||||
|
t.Fatal("list sent nothing")
|
||||||
|
}
|
||||||
|
for _, c := range *sent {
|
||||||
|
if strings.Contains(c, "2>") || strings.Contains(c, "&>") {
|
||||||
|
t.Errorf("remote command uses sh-only redirection: %q", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+206
@@ -0,0 +1,206 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// secrets.go — a last look at what `push` is about to commit.
|
||||||
|
//
|
||||||
|
// `push` runs `git add --all .`, so anything lying in the project — an .env, a
|
||||||
|
// stray key file, a token pasted into a config — is committed and pushed, and
|
||||||
|
// with `mirror = true` it reaches a *public* server in the same breath. That is
|
||||||
|
// the one action in mgsh that cannot be undone: a deleted server repository can
|
||||||
|
// come back from an archive, a published credential is burnt.
|
||||||
|
//
|
||||||
|
// So the staged diff is scanned for a small set of high-signal patterns before
|
||||||
|
// the commit is made. This is not a complete secret scanner and does not try to
|
||||||
|
// be one; it aims for a high hit rate on the things that actually leak, with
|
||||||
|
// few enough false alarms that the prompt still means something. Turn it off
|
||||||
|
// with `secretscan = off`.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// secretHit is one suspicious added line.
|
||||||
|
type secretHit struct {
|
||||||
|
file string
|
||||||
|
lineNo int
|
||||||
|
kind string
|
||||||
|
text string
|
||||||
|
}
|
||||||
|
|
||||||
|
// secretPattern matches one kind of credential. `certain` patterns are
|
||||||
|
// unmistakable and are reported as they are; the others match a shape that
|
||||||
|
// merely looks like a secret and are filtered through looksLikePlaceholder.
|
||||||
|
type secretPattern struct {
|
||||||
|
kind string
|
||||||
|
re *regexp.Regexp
|
||||||
|
certain bool
|
||||||
|
}
|
||||||
|
|
||||||
|
var secretPatterns = []secretPattern{
|
||||||
|
{"private key", regexp.MustCompile(`-----BEGIN (?:[A-Z]+ )?PRIVATE KEY-----`), true},
|
||||||
|
{"GitHub token", regexp.MustCompile(`\bgh[pousr]_[A-Za-z0-9]{20,}`), true},
|
||||||
|
{"GitLab token", regexp.MustCompile(`\bglpat-[A-Za-z0-9_-]{16,}`), true},
|
||||||
|
{"AWS access key", regexp.MustCompile(`\b(?:AKIA|ASIA)[0-9A-Z]{16}\b`), true},
|
||||||
|
{"Slack token", regexp.MustCompile(`\bxox[baprs]-[A-Za-z0-9-]{10,}`), true},
|
||||||
|
{"PyPI token", regexp.MustCompile(`\bpypi-AgEIcHlwaS5vcmc[A-Za-z0-9_-]{10,}`), true},
|
||||||
|
{"credential assignment", regexp.MustCompile(
|
||||||
|
`(?i)\b(?:password|passwd|secret|api[_-]?key|apikey|access[_-]?key|auth[_-]?token|token)\b` +
|
||||||
|
`\s*[:=]\s*(?:"([^"\s]{12,})"|'([^'\s]{12,})'|([^\s"';,]{20,}))\s*;?\s*$`), false},
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
diffFileRe = regexp.MustCompile(`^\+\+\+ b/(.*)$`)
|
||||||
|
diffHunkRe = regexp.MustCompile(`^@@ -\d+(?:,\d+)? \+(\d+)`)
|
||||||
|
// a value that is plainly a reference or a stand-in, not a credential
|
||||||
|
constRefRe = regexp.MustCompile(`^[A-Z][A-Z0-9_]*$`)
|
||||||
|
dottedRefRe = regexp.MustCompile(`^[\w-]+(?:\.[\w-]+)+$`)
|
||||||
|
maskedRe = regexp.MustCompile(`^[*x•.]+$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
// placeholderWords are the values people write when they mean "fill this in".
|
||||||
|
var placeholderWords = map[string]bool{
|
||||||
|
"changeme": true, "change_me": true, "password": true, "secret": true,
|
||||||
|
"token": true, "your_token": true, "your-token": true, "yourtoken": true,
|
||||||
|
"todo": true, "none": true, "null": true, "example": true, "redacted": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// looksLikePlaceholder reports whether a matched value is obviously not a real
|
||||||
|
// credential: a template slot, an environment reference, a constant name, or a
|
||||||
|
// masked stand-in. Documentation and example files are full of these, and every
|
||||||
|
// one of them that reaches the prompt makes the prompt worth less.
|
||||||
|
func looksLikePlaceholder(v string) bool {
|
||||||
|
v = strings.Trim(v, `"'`)
|
||||||
|
if v == "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(v, "<>${}()") { // <token>, ${VAR}, $(cmd), {{ tpl }}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if maskedRe.MatchString(v) || constRefRe.MatchString(v) || dottedRefRe.MatchString(v) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if placeholderWords[strings.ToLower(v)] {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
// a value made of one repeated character carries no information
|
||||||
|
if strings.Count(v, string(v[0])) == len(v) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanDiff finds suspicious added lines in a unified diff. Only added lines are
|
||||||
|
// examined: removing a secret is what we want people to do.
|
||||||
|
func scanDiff(diff string) []secretHit {
|
||||||
|
var hits []secretHit
|
||||||
|
file := ""
|
||||||
|
lineNo := 0
|
||||||
|
|
||||||
|
for _, ln := range strings.Split(diff, "\n") {
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(ln, "+++ "):
|
||||||
|
file = ""
|
||||||
|
if m := diffFileRe.FindStringSubmatch(ln); m != nil {
|
||||||
|
file = m[1]
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
case strings.HasPrefix(ln, "@@"):
|
||||||
|
if m := diffHunkRe.FindStringSubmatch(ln); m != nil {
|
||||||
|
lineNo, _ = strconv.Atoi(m[1])
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
case strings.HasPrefix(ln, "---") || strings.HasPrefix(ln, "diff ") ||
|
||||||
|
strings.HasPrefix(ln, "index ") || strings.HasPrefix(ln, "new file") ||
|
||||||
|
strings.HasPrefix(ln, "deleted file") || strings.HasPrefix(ln, "similarity "):
|
||||||
|
continue
|
||||||
|
case strings.HasPrefix(ln, "-"):
|
||||||
|
continue // removed line: not our problem
|
||||||
|
case !strings.HasPrefix(ln, "+"):
|
||||||
|
lineNo++ // context line
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
text := ln[1:]
|
||||||
|
if kind := matchSecret(text); kind != "" {
|
||||||
|
hits = append(hits, secretHit{file: file, lineNo: lineNo, kind: kind, text: text})
|
||||||
|
}
|
||||||
|
lineNo++
|
||||||
|
}
|
||||||
|
return hits
|
||||||
|
}
|
||||||
|
|
||||||
|
// allowMarker suppresses the check for one line. Any scanner needs a per-line
|
||||||
|
// escape: a project will eventually hold something credential-shaped on
|
||||||
|
// purpose, and switching the whole check off for that is far too blunt.
|
||||||
|
const allowMarker = "mgsh:allow"
|
||||||
|
|
||||||
|
// matchSecret returns the kind of credential a line appears to contain, or "".
|
||||||
|
func matchSecret(text string) string {
|
||||||
|
if strings.Contains(text, allowMarker) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
for _, p := range secretPatterns {
|
||||||
|
m := p.re.FindStringSubmatch(text)
|
||||||
|
if m == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if p.certain {
|
||||||
|
return p.kind
|
||||||
|
}
|
||||||
|
// the first non-empty capture group is the value that was assigned
|
||||||
|
value := ""
|
||||||
|
for _, g := range m[1:] {
|
||||||
|
if g != "" {
|
||||||
|
value = g
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !looksLikePlaceholder(value) {
|
||||||
|
return p.kind
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// secretScanEnabled reports whether the scan runs. It is on unless explicitly
|
||||||
|
// switched off, so a typo in the setting leaves the safety net in place.
|
||||||
|
func secretScanEnabled() bool { return !falsy(cfg.SecretScan) }
|
||||||
|
|
||||||
|
// secretsApproved scans what `push` has staged. With nothing suspicious found
|
||||||
|
// it returns true silently; otherwise it shows the findings and asks. Returns
|
||||||
|
// false when the push should stop.
|
||||||
|
func secretsApproved(dir string) bool {
|
||||||
|
if !secretScanEnabled() {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
diff, err := gitCapture(dir, "diff", "--cached", "-U0", "--no-color")
|
||||||
|
if err != nil {
|
||||||
|
return true // nothing staged, or no HEAD yet: not our call to block
|
||||||
|
}
|
||||||
|
hits := scanDiff(diff)
|
||||||
|
if len(hits) == 0 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println(col(cRed, fmt.Sprintf("%d possible credential(s) in what is about to be committed:", len(hits))))
|
||||||
|
for _, h := range hits {
|
||||||
|
where := h.file
|
||||||
|
if h.lineNo > 0 {
|
||||||
|
where += ":" + strconv.Itoa(h.lineNo)
|
||||||
|
}
|
||||||
|
fmt.Printf(" %s %s\n %s\n",
|
||||||
|
col(cYellow, where), col(cGray, h.kind), col(cRed, ellipsis(strings.TrimSpace(h.text), 100)))
|
||||||
|
}
|
||||||
|
fmt.Println(col(cGray, " (set 'secretscan = off' to skip this check)"))
|
||||||
|
return yesno("push anyway?", false)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ellipsis shortens s to at most n characters.
|
||||||
|
func ellipsis(s string, n int) string {
|
||||||
|
if len(s) <= n {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return s[:n-1] + "…"
|
||||||
|
}
|
||||||
+212
@@ -0,0 +1,212 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestMatchSecretCatchesRealCredentials: the shapes that actually leak.
|
||||||
|
func TestMatchSecretCatchesRealCredentials(t *testing.T) {
|
||||||
|
lines := []string{
|
||||||
|
"-----BEGIN OPENSSH PRIVATE KEY-----",
|
||||||
|
"-----BEGIN RSA PRIVATE KEY-----",
|
||||||
|
" ghp_aB3dEfGh1jKlMn0pQrStUvWxYz012345678",
|
||||||
|
"GITLAB=glpat-aB3dEfGh1jKlMn0pQrSt",
|
||||||
|
`aws_access_key_id = AKIAIOSFODNN7EXAMPLE`,
|
||||||
|
"slack: xoxb-1234567890-abcdefghij",
|
||||||
|
`API_KEY="s3cr3tV4lu3W1thStuff"`,
|
||||||
|
"password = hunter2hunter2hunter2",
|
||||||
|
"token: 'aB3dEfGh1jKlMn0pQrSt'",
|
||||||
|
"auth-token=9f8e7d6c5b4a39281706abcdef123456",
|
||||||
|
}
|
||||||
|
for _, ln := range lines {
|
||||||
|
if matchSecret(ln) == "" {
|
||||||
|
t.Errorf("missed a credential in %q", ln)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMatchSecretIgnoresNoise: everyday code and documentation must not trip
|
||||||
|
// the prompt, or people learn to answer "yes" without reading it.
|
||||||
|
func TestMatchSecretIgnoresNoise(t *testing.T) {
|
||||||
|
lines := []string{
|
||||||
|
"remotekey = <personal-access-token>", // our own README
|
||||||
|
"# remotekey = <personal-access-token>", // and mgshrc.example
|
||||||
|
"token = process.env.GITHUB_TOKEN", // reference, not a value
|
||||||
|
"const token = getToken()", // call
|
||||||
|
"password = ${DB_PASSWORD}", // template
|
||||||
|
`api_key = "changeme"`, // placeholder
|
||||||
|
"secret: TODO", //
|
||||||
|
"key gh***************xk", // masked, from `config`
|
||||||
|
"password = xxxxxxxxxxxxxxxxxxxxxxx", // masked
|
||||||
|
"// the token is never persisted in the repo", // prose
|
||||||
|
"apiKey := os.Getenv(\"MGSH_REMOTEKEY\")", // lookup
|
||||||
|
"token = SOME_CONSTANT_NAME", // constant
|
||||||
|
"secret = my.config.value", // dotted reference
|
||||||
|
"+++ b/token.go", // diff furniture
|
||||||
|
"password = short", // too short to be one
|
||||||
|
"Authorization: Basic <base64(owner:token)>", // documentation
|
||||||
|
"remote.hub.key = <personal-access-token>",
|
||||||
|
}
|
||||||
|
for _, ln := range lines {
|
||||||
|
if kind := matchSecret(ln); kind != "" {
|
||||||
|
t.Errorf("false positive (%s) on %q", kind, ln)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestScanDiffReportsFileAndLine: the report has to point at the right place,
|
||||||
|
// and must ignore removed lines — deleting a secret is the desired action.
|
||||||
|
func TestScanDiffReportsFileAndLine(t *testing.T) {
|
||||||
|
diff := `diff --git a/.env b/.env
|
||||||
|
new file mode 100644
|
||||||
|
--- /dev/null
|
||||||
|
+++ b/.env
|
||||||
|
@@ -0,0 +1,3 @@
|
||||||
|
+HOME=/tmp
|
||||||
|
+API_KEY="s3cr3tV4lu3W1thStuff"
|
||||||
|
+DEBUG=1
|
||||||
|
diff --git a/old.txt b/old.txt
|
||||||
|
--- a/old.txt
|
||||||
|
+++ b/old.txt
|
||||||
|
@@ -7,1 +7,0 @@
|
||||||
|
-password = hunter2hunter2hunter2
|
||||||
|
`
|
||||||
|
hits := scanDiff(diff)
|
||||||
|
if len(hits) != 1 {
|
||||||
|
t.Fatalf("expected exactly one hit, got %d: %+v", len(hits), hits)
|
||||||
|
}
|
||||||
|
h := hits[0]
|
||||||
|
if h.file != ".env" {
|
||||||
|
t.Errorf("file = %q, want .env", h.file)
|
||||||
|
}
|
||||||
|
if h.lineNo != 2 {
|
||||||
|
t.Errorf("lineNo = %d, want 2", h.lineNo)
|
||||||
|
}
|
||||||
|
if !strings.Contains(h.text, "API_KEY") {
|
||||||
|
t.Errorf("text = %q", h.text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestScanDiffCountsLinesAcrossHunks keeps the line numbers honest when a file
|
||||||
|
// is edited in several places.
|
||||||
|
func TestScanDiffCountsLinesAcrossHunks(t *testing.T) {
|
||||||
|
diff := `+++ b/config.yml
|
||||||
|
@@ -1,0 +1,1 @@
|
||||||
|
+harmless: yes
|
||||||
|
@@ -40,0 +41,2 @@
|
||||||
|
+also fine
|
||||||
|
+aws_key = AKIAIOSFODNN7EXAMPLE
|
||||||
|
`
|
||||||
|
hits := scanDiff(diff)
|
||||||
|
if len(hits) != 1 || hits[0].lineNo != 42 {
|
||||||
|
t.Fatalf("hits = %+v, want one at line 42", hits)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSecretsApprovedBlocksThePush drives the real thing: a staged .env, the
|
||||||
|
// scan, and the answer deciding whether push continues.
|
||||||
|
func TestSecretsApprovedBlocksThePush(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
mustGit(t, dir, "init", "-q")
|
||||||
|
mustGit(t, dir, "config", "user.name", "t")
|
||||||
|
mustGit(t, dir, "config", "user.email", "t@e")
|
||||||
|
mustGit(t, dir, "commit", "-q", "--allow-empty", "-m", "base")
|
||||||
|
|
||||||
|
old := cfg
|
||||||
|
defer func() { cfg = old }()
|
||||||
|
cfg = Config{}
|
||||||
|
|
||||||
|
// clean tree: no prompt, no interference
|
||||||
|
asked := fakeAnswers(t, false)
|
||||||
|
if !secretsApproved(dir) {
|
||||||
|
t.Fatal("a clean tree must not block the push")
|
||||||
|
}
|
||||||
|
if len(*asked) != 0 {
|
||||||
|
t.Fatalf("asked about a clean tree: %v", *asked)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, ".env"),
|
||||||
|
[]byte("API_KEY=\"s3cr3tV4lu3W1thStuff\"\n"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
mustGit(t, dir, "add", "--all", ".")
|
||||||
|
|
||||||
|
declined := fakeAnswers(t, false)
|
||||||
|
if secretsApproved(dir) {
|
||||||
|
t.Error("a staged credential must stop the push when declined")
|
||||||
|
}
|
||||||
|
if len(*declined) == 0 {
|
||||||
|
t.Error("the user was never asked")
|
||||||
|
}
|
||||||
|
|
||||||
|
accepted := fakeAnswers(t, true)
|
||||||
|
if !secretsApproved(dir) {
|
||||||
|
t.Error("an explicit yes must let the push through")
|
||||||
|
}
|
||||||
|
if len(*accepted) == 0 {
|
||||||
|
t.Error("the user was never asked")
|
||||||
|
}
|
||||||
|
|
||||||
|
// and the escape hatch really switches it off
|
||||||
|
cfg.SecretScan = "off"
|
||||||
|
never := fakeAnswers(t, false)
|
||||||
|
if !secretsApproved(dir) {
|
||||||
|
t.Error("secretscan = off must not block")
|
||||||
|
}
|
||||||
|
if len(*never) != 0 {
|
||||||
|
t.Errorf("secretscan = off still asked: %v", *never)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSecretScanDefaultsToOn: only a deliberate "off" disables it, so a typo
|
||||||
|
// leaves the safety net in place.
|
||||||
|
func TestSecretScanDefaultsToOn(t *testing.T) {
|
||||||
|
old := cfg
|
||||||
|
defer func() { cfg = old }()
|
||||||
|
for _, v := range []string{"", "true", "on", "yes", "wharrgarbl", "1"} {
|
||||||
|
cfg = Config{SecretScan: v}
|
||||||
|
if !secretScanEnabled() {
|
||||||
|
t.Errorf("secretscan = %q disabled the scan", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, v := range []string{"off", "0", "false", "no", " OFF "} {
|
||||||
|
cfg = Config{SecretScan: v}
|
||||||
|
if secretScanEnabled() {
|
||||||
|
t.Errorf("secretscan = %q did not disable the scan", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOwnDocsDoNotTripTheScanner: mgsh's own README and example config are full
|
||||||
|
// of credential-shaped text; committing mgsh itself must stay quiet.
|
||||||
|
func TestOwnDocsDoNotTripTheScanner(t *testing.T) {
|
||||||
|
for _, f := range []string{"README.md", "mgshrc.example"} {
|
||||||
|
data, err := os.ReadFile(f)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for i, ln := range strings.Split(string(data), "\n") {
|
||||||
|
if kind := matchSecret(ln); kind != "" {
|
||||||
|
t.Errorf("%s:%d would trip the scanner (%s): %q", f, i+1, kind, ln)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestAllowMarkerSuppressesOneLine: the per-line escape for something that only
|
||||||
|
// looks like a credential and is meant to stay.
|
||||||
|
func TestAllowMarkerSuppressesOneLine(t *testing.T) {
|
||||||
|
line := `API_KEY="s3cr3tV4lu3W1thStuff"`
|
||||||
|
if matchSecret(line) == "" {
|
||||||
|
t.Fatal("test line is not detected at all")
|
||||||
|
}
|
||||||
|
if kind := matchSecret(line + " # mgsh:allow — sample value"); kind != "" {
|
||||||
|
t.Errorf("mgsh:allow did not suppress the hit (%s)", kind)
|
||||||
|
}
|
||||||
|
if kind := matchSecret("-----BEGIN OPENSSH PRIVATE KEY----- mgsh:allow"); kind != "" {
|
||||||
|
t.Errorf("mgsh:allow did not suppress a certain pattern (%s)", kind)
|
||||||
|
}
|
||||||
|
}
|
||||||
+2
-2
@@ -44,6 +44,7 @@ func showConfig() {
|
|||||||
{"pushdefault", cfg.PushDefault},
|
{"pushdefault", cfg.PushDefault},
|
||||||
{"editor", cfg.Editor},
|
{"editor", cfg.Editor},
|
||||||
{"mirror", cfg.Mirror},
|
{"mirror", cfg.Mirror},
|
||||||
|
{"secretscan", cfg.SecretScan},
|
||||||
{"remotes", cfg.RemoteNames},
|
{"remotes", cfg.RemoteNames},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -146,8 +147,7 @@ func configKeys() []string {
|
|||||||
keys := []string{
|
keys := []string{
|
||||||
"base", "githost", "gitport", "gituser", "gitpath", "gitkey",
|
"base", "githost", "gitport", "gituser", "gitpath", "gitkey",
|
||||||
"gitname", "gitemail", "pushdefault", "editor",
|
"gitname", "gitemail", "pushdefault", "editor",
|
||||||
"remoteurl", "remotekey", "remotetype", "remotevisibility",
|
"remotes", "mirror", "secretscan",
|
||||||
"remotes", "mirror",
|
|
||||||
}
|
}
|
||||||
sort.Strings(keys)
|
sort.Strings(keys)
|
||||||
return keys
|
return keys
|
||||||
|
|||||||
@@ -39,6 +39,17 @@ func truthy(s string) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// falsy reports whether a config string explicitly means "off". It is not the
|
||||||
|
// negation of truthy: for a setting that defaults to on, an unset value or a
|
||||||
|
// typo must leave it on, and only a deliberate "off" may switch it off.
|
||||||
|
func falsy(s string) bool {
|
||||||
|
switch strings.ToLower(strings.TrimSpace(s)) {
|
||||||
|
case "0", "false", "no", "off":
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
func fileExists(p string) bool {
|
func fileExists(p string) bool {
|
||||||
fi, err := os.Stat(p)
|
fi, err := os.Stat(p)
|
||||||
return err == nil && !fi.IsDir()
|
return err == nil && !fi.IsDir()
|
||||||
|
|||||||
+1
-1
@@ -1 +1 @@
|
|||||||
4.0.24
|
4.0.42
|
||||||
|
|||||||
Reference in New Issue
Block a user