2 Commits
Author SHA1 Message Date
mikeandClaude Opus 5 8c4d8da4e9 Attach ./bin and ./assets to a release
`release` now uploads every file in the project's ./bin and ./assets when
those directories exist. Nothing to configure, and nothing happens for a
project that has neither.

This is the part deliberately left out when `release` was written,
because it is where the three providers stop resembling each other:

  Gitea   multipart POST to .../releases/<id>/assets?name=<name>
  GitHub  raw POST to the separate upload host the release object names
          in upload_url, whose RFC 6570 template suffix has to go first
  GitLab  a release stores links, not files: the file goes into the
          project's generic package registry and the release gets a
          package link pointing at it

So findRelease and createRelease now return a releaseRef carrying the id
and, for GitHub, that upload host -- the id alone cannot address an
upload. Uploads stream from disk rather than buffering: these are whole
binaries, and the Gitea multipart body is assembled through a pipe.

Only regular files directly in those directories are taken. Symlinks are
skipped, which matters here: build.sh leaves bin/mgsh pointing at one of
its siblings, and uploading the same 9M twice under two names helps
nobody. A name present in both directories is used from bin and reported
for assets. Re-releasing a tag replaces same-named assets rather than
failing on them, since rebuilding and publishing again is the normal
reason to do it, and a file that fails does not stop the rest.

Each provider's request shape is pinned down against the recording
stand-in, and the whole chain was run once end to end -- real repository,
real binaries, a fake Gitea that also serves git-http-backend so the tag
push is real too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 20:13:09 +02:00
mikeandClaude Opus 5 f3d8cbe281 Cross-compile into ./bin for the usual platforms
build.sh produced one binary for the machine it ran on. It now builds
darwin/arm64, darwin/amd64, linux/amd64 and linux/arm64 into ./bin, all
from the single version that run bumped, so the four never disagree about
what they are. PLATFORMS="linux/amd64" ./build.sh narrows it.

bin/mgsh is a symlink to the host's build, so there is still one stable
path to "the binary for this machine". The old ./mgsh in the repo root is
no longer written, and the script deletes it once if it finds it -- a
stale binary on a path people have in their fingers is worse than a
missing one.

CGO_ENABLED=0 throughout: the cross builds then need no toolchain per
target and the binaries are static. That is only safe because os/user
still resolves the current user without cgo, which mgsh needs for the
"[user@host]" stamp on every commit -- checked on darwin before relying
on it.

Windows is left out on purpose: mgsh shells out to stty and /bin/sh, so
it would compile there and then not work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 19:53:29 +02:00
8 changed files with 795 additions and 52 deletions
+1
View File
@@ -8,4 +8,5 @@
.TemporaryItems
.Trashes
mgsh
bin/
.mgshrc
+65 -11
View File
@@ -17,14 +17,36 @@ directory. Go port of the original Perl `mgsh` (`mgsh.perl`).
## Build
```sh
./build.sh # builds ./mgsh and bumps the patch version by 0.0.1
go build -o mgsh . # plain build, keeps the default version
./build.sh # all platforms into ./bin, bumps the patch version
PLATFORMS="linux/amd64" ./build.sh # just one
go build -o mgsh . # plain build, keeps the default version
```
`build.sh` reads `version.txt`, increments the patch component, injects it via
`-ldflags -X main.VERSION`, and writes it back — so `version.txt` always holds
the version of the binary just built. Dependencies are fetched via Go modules
(`go.mod` / `go.sum`) on first build.
`build.sh` cross-compiles for `darwin/arm64`, `darwin/amd64`, `linux/amd64` and
`linux/arm64` into `./bin`:
```
bin/mgsh -> mgsh-darwin-arm64 (this machine)
bin/mgsh-darwin-amd64
bin/mgsh-darwin-arm64
bin/mgsh-linux-amd64
bin/mgsh-linux-arm64
```
`bin/mgsh` is a symlink to the build for the host, so there is one stable path
to "the binary for this machine". `bin/` is git-ignored. Windows is deliberately
absent: mgsh shells out to `stty` and `/bin/sh`, so it would compile there and
then not work.
Everything is built with `CGO_ENABLED=0`, which makes the cross builds need no
toolchain per target and the binaries static; `os/user` resolves the current
user without cgo on both darwin and linux.
It reads `version.txt`, increments the patch component, injects it via
`-ldflags -X main.VERSION` into **all** platforms of that run, and writes it
back — so `version.txt` always holds the version of the binaries just built, and
they all carry the same one. Dependencies are fetched via Go modules (`go.mod` /
`go.sum`) on first build.
Run the tests with `go test ./...`.
@@ -370,11 +392,43 @@ on different servers. `release` also refuses when the repository is not on the
mirror yet and tells you to run `pushremote` first, rather than creating it as a
side effect.
**No binary assets.** The three providers handle uploads in three incompatible
ways — Gitea attaches them to the release, GitHub uses a separate upload host,
and GitLab does not host them at all but expects a link into its package
registry. mgsh publishes source releases with notes; if you need binaries,
upload them with the provider's own tooling.
#### Binaries and assets
If the project has a `./bin` or `./assets` directory, every file in it is
attached to the release — nothing to configure, and nothing happens for a
project that has neither:
```
< src/mgsh > release v4.1.0
attaching 5 assets, 38M from ./bin and ./assets
remote gitea released https://git.example.com/mike/mgsh.git
uploading logo.png 2.0K
uploading mgsh-darwin-amd64 9.8M
uploading mgsh-darwin-arm64 9.2M
uploading mgsh-linux-amd64 9.7M
uploading mgsh-linux-arm64 8.9M
```
Only regular files directly in those directories are taken: subdirectories are
not descended into, and symlinks are skipped — `bin/mgsh` points at one of its
own siblings, and uploading the same binary twice under two names helps nobody.
A name present in both directories is used from `bin` and reported for
`assets`, since one asset name can only mean one file.
Re-releasing the same tag replaces same-named assets instead of failing or
piling up duplicates, because rebuilding and publishing again is the normal
reason to do it. A file that fails to upload does not stop the rest.
This is where the providers stop resembling each other, and mgsh papers over it:
| | how the bytes get there |
|---|---|
| Gitea | multipart `POST` to `…/releases/<id>/assets?name=<name>` |
| GitHub | raw `POST` to the separate upload host named by the release's `upload_url` |
| GitLab | a release stores links, not files: the file goes into the project's generic **package registry** and the release gets a `package` link pointing at it |
The GitLab route needs the package registry enabled on the project — it is on by
default, but a self-hosted instance can turn it off.
## Configuration
+323
View File
@@ -0,0 +1,323 @@
package main
// assets.go — attaching files to a release.
//
// `release` uploads everything in the project's ./bin and ./assets when those
// directories exist. This is where the three providers stop resembling each
// other, so each gets its own path:
//
// Gitea multipart POST to .../releases/<id>/assets?name=<name>
// GitHub raw POST to the upload host named by the release object's upload_url
// GitLab no asset hosting on a release at all: the file goes into the generic
// package registry, and the release gets a link pointing at it
//
// Re-releasing the same tag replaces same-named assets rather than failing or
// piling up duplicates, because rebuilding and publishing again is the normal
// reason to do it.
import (
"encoding/json"
"fmt"
"io"
"mime"
"mime/multipart"
"net/http"
"net/url"
"os"
"path/filepath"
"sort"
"strconv"
)
// assetDirs are the project directories whose files a release carries.
var assetDirs = []string{"bin", "assets"}
// releaseAsset is one file to attach to a release.
type releaseAsset struct {
name string
path string
size int64
}
// collectAssets gathers the files to attach: every regular file directly in
// the project's ./bin and ./assets. Subdirectories are not descended into, and
// symlinks are skipped — bin/mgsh points at one of its own siblings, and
// uploading the same binary twice under two names helps nobody.
func collectAssets(dir string) (assets []releaseAsset, skipped []string) {
seen := map[string]bool{}
for _, sub := range assetDirs {
path := filepath.Join(dir, sub)
entries, err := os.ReadDir(path)
if err != nil {
continue // the directory simply is not there
}
for _, e := range entries {
if e.IsDir() || e.Type()&os.ModeSymlink != 0 {
continue
}
fi, err := e.Info()
if err != nil || !fi.Mode().IsRegular() {
continue
}
if seen[e.Name()] {
// bin/x and assets/x would fight over one asset name
skipped = append(skipped, filepath.Join(sub, e.Name()))
continue
}
seen[e.Name()] = true
assets = append(assets, releaseAsset{
name: e.Name(),
path: filepath.Join(path, e.Name()),
size: fi.Size(),
})
}
}
sort.Slice(assets, func(i, j int) bool { return assets[i].name < assets[j].name })
return assets, skipped
}
// contentType guesses a type from the file name, falling back to the one for
// "some bytes" — which is what a compiled binary is.
func contentType(name string) string {
if t := mime.TypeByExtension(filepath.Ext(name)); t != "" {
return t
}
return "application/octet-stream"
}
// doUpload sends a request with a raw body of a known size. The JSON helper
// cannot express these: an asset upload is bytes, not an object, and the
// providers insist on a Content-Length rather than a chunked body.
func (r *remoteAPI) doUpload(method, endpoint, ctype string, body io.Reader, size int64) (int, []byte, error) {
req, err := http.NewRequest(method, endpoint, body)
if err != nil {
return 0, nil, err
}
req.ContentLength = size
req.Header.Set("Content-Type", ctype)
hk, hv := r.authHeader()
req.Header.Set(hk, hv)
req.Header.Set("Accept", "application/json")
resp, err := r.http.Do(req)
if err != nil {
return 0, nil, err
}
defer resp.Body.Close()
data, _ := io.ReadAll(resp.Body)
return resp.StatusCode, data, nil
}
// uploadAssets attaches every file to the release, reporting each one as it
// goes: these are megabytes over a network, and silence for a minute looks like
// a hang. A file that fails does not stop the rest.
func (r *remoteAPI) uploadAssets(owner, repo string, ref releaseRef, assets []releaseAsset) error {
failed := 0
for _, a := range assets {
fmt.Printf(" %s %s %s\n", col(cGray, "uploading"),
col(cGreen, padRight(a.name, 28)), col(cGray, humanSize(a.size)))
if err := r.uploadAsset(owner, repo, ref, a); err != nil {
errorln(" " + a.name + ": " + err.Error())
failed++
}
}
if failed > 0 {
return fmt.Errorf("%d of %d assets failed", failed, len(assets))
}
return nil
}
// uploadAsset attaches one file, replacing an asset of the same name that is
// already on the release.
func (r *remoteAPI) uploadAsset(owner, repo string, ref releaseRef, a releaseAsset) error {
if err := r.removeAsset(owner, repo, ref, a.name); err != nil {
return err
}
switch r.kind {
case kindGitLab:
return r.uploadAssetGitLab(owner, repo, ref, a)
case kindGitHub:
return r.uploadAssetGitHub(ref, a)
default:
return r.uploadAssetGitea(owner, repo, ref, a)
}
}
// uploadAssetGitea posts the file as a multipart form, streamed from disk
// rather than buffered: these are whole binaries.
func (r *remoteAPI) uploadAssetGitea(owner, repo string, ref releaseRef, a releaseAsset) error {
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
pr, pw := io.Pipe()
mw := multipart.NewWriter(pw)
go func() {
part, err := mw.CreateFormFile("attachment", a.name)
if err == nil {
_, err = io.Copy(part, f)
}
if err != nil {
pw.CloseWithError(err)
return
}
pw.CloseWithError(mw.Close())
}()
ep := r.releasePath(owner, repo) + "/" + ref.id + "/assets?name=" + url.QueryEscape(a.name)
// the multipart length is not known up front, so this one is chunked
code, data, err := r.doUpload("POST", ep, mw.FormDataContentType(), pr, -1)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
}
return nil
}
// uploadAssetGitHub posts the raw bytes to the host the release object named.
func (r *remoteAPI) uploadAssetGitHub(ref releaseRef, a releaseAsset) error {
if ref.uploadURL == "" {
return fmt.Errorf("the release carries no upload_url")
}
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
ep := ref.uploadURL + "?name=" + url.QueryEscape(a.name)
code, data, err := r.doUpload("POST", ep, contentType(a.name), f, a.size)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
}
return nil
}
// uploadAssetGitLab puts the file into the project's generic package registry
// and links the release to it, since a GitLab release stores links, not files.
func (r *remoteAPI) uploadAssetGitLab(owner, repo string, ref releaseRef, a releaseAsset) error {
f, err := os.Open(a.path)
if err != nil {
return err
}
defer f.Close()
pkg := r.packagePath(owner, repo, ref.tag, a.name)
code, data, err := r.doUpload("PUT", pkg, contentType(a.name), f, a.size)
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("package upload failed (HTTP %d): %s", code, firstLine(data))
}
code, data, err = r.do("POST", r.releaseByTagPath(owner, repo, ref.tag)+"/assets/links",
map[string]any{"name": a.name, "url": pkg, "link_type": "package"})
if err != nil {
return err
}
if code != 200 && code != 201 {
return fmt.Errorf("linking the package failed (HTTP %d): %s", code, firstLine(data))
}
return nil
}
// packagePath is where a release asset lives in GitLab's generic registry: one
// package named after the repository, one version per tag.
func (r *remoteAPI) packagePath(owner, repo, tag, name string) string {
return r.repoPath(owner, repo) + "/packages/generic/" +
url.PathEscape(repo) + "/" + url.PathEscape(tag) + "/" + url.PathEscape(name)
}
// removeAsset deletes an asset of the given name from the release when one is
// there, so re-releasing a tag after a rebuild replaces the files instead of
// failing or leaving two of each.
func (r *remoteAPI) removeAsset(owner, repo string, ref releaseRef, name string) error {
listEP, delEP := r.assetEndpoints(owner, repo, ref)
code, data, err := r.do("GET", listEP, nil)
if err != nil {
return err
}
if code == 404 {
return nil // nothing uploaded yet
}
if code != 200 {
return fmt.Errorf("listing assets failed (HTTP %d): %s", code, firstLine(data))
}
for _, id := range assetIDsNamed(data, name) {
if code, data, err := r.do("DELETE", delEP+"/"+id, nil); err != nil {
return err
} else if code != 200 && code != 202 && code != 204 {
return fmt.Errorf("deleting the previous %s failed (HTTP %d): %s", name, code, firstLine(data))
}
}
return nil
}
// assetEndpoints returns where a release's assets are listed and deleted.
// GitHub deletes an asset through the repository rather than the release.
func (r *remoteAPI) assetEndpoints(owner, repo string, ref releaseRef) (list, del string) {
switch r.kind {
case kindGitLab:
base := r.releaseByTagPath(owner, repo, ref.tag) + "/assets/links"
return base, base
case kindGitHub:
return r.releasePath(owner, repo) + "/" + ref.id + "/assets",
r.releasePath(owner, repo) + "/assets"
default: // Gitea
base := r.releasePath(owner, repo) + "/" + ref.id + "/assets"
return base, base
}
}
// assetIDsNamed picks the ids of the listed assets called name. All three
// providers answer with an array of objects carrying "id" and "name".
func assetIDsNamed(data []byte, name string) []string {
var list []struct {
ID json.Number `json:"id"`
Name string `json:"name"`
}
if err := json.Unmarshal(data, &list); err != nil {
return nil
}
var ids []string
for _, a := range list {
if a.Name == name && a.ID.String() != "" {
ids = append(ids, a.ID.String())
}
}
return ids
}
// assetSummary describes what will be uploaded, for the line `release` prints
// before it starts.
func assetSummary(assets []releaseAsset) string {
var total int64
for _, a := range assets {
total += a.size
}
unit := "assets"
if len(assets) == 1 {
unit = "asset"
}
return strconv.Itoa(len(assets)) + " " + unit + ", " + humanSize(total)
}
// assetDirList names the directories that were actually found, for the same line.
func assetDirList(dir string) []string {
var found []string
for _, sub := range assetDirs {
if isDir(filepath.Join(dir, sub)) {
found = append(found, "./"+sub)
}
}
return found
}
+284
View File
@@ -0,0 +1,284 @@
package main
// assets_test.go — attaching files to a release.
//
// The three providers upload assets in three unrelated ways, and the difference
// is invisible until a real server rejects the request. The recording stand-in
// lets each one be pinned down: where the bytes go, how they are wrapped, and
// what else has to happen around them.
import (
"os"
"path/filepath"
"strings"
"testing"
)
// assetTree builds a project with ./bin and ./assets and returns its path.
func assetTree(t *testing.T, files map[string]string) string {
t.Helper()
dir := t.TempDir()
for rel, content := range files {
p := filepath.Join(dir, rel)
if err := os.MkdirAll(filepath.Dir(p), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(content), 0755); err != nil {
t.Fatal(err)
}
}
return dir
}
func TestCollectAssets(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/mgsh-linux-amd64": "ELF...",
"bin/mgsh-darwin-arm64": "MACH...",
"assets/logo.png": "PNG",
"assets/notes.txt": "text",
"bin/sub/nested": "not descended into",
"src/main.go": "not an asset directory",
})
// bin/mgsh is a symlink to one of its siblings, as build.sh leaves it
if err := os.Symlink("mgsh-linux-amd64", filepath.Join(dir, "bin", "mgsh")); err != nil {
t.Fatal(err)
}
assets, skipped := collectAssets(dir)
var names []string
for _, a := range assets {
names = append(names, a.name)
}
want := "logo.png,mgsh-darwin-arm64,mgsh-linux-amd64,notes.txt"
if strings.Join(names, ",") != want {
t.Errorf("collected %v, want %s", names, want)
}
if len(skipped) != 0 {
t.Errorf("skipped = %v, want none", skipped)
}
for _, a := range assets {
if a.size == 0 {
t.Errorf("%s has no size", a.name)
}
if _, err := os.Stat(a.path); err != nil {
t.Errorf("%s: %v", a.name, err)
}
}
// a project with neither directory contributes nothing at all
if got, _ := collectAssets(t.TempDir()); len(got) != 0 {
t.Errorf("a project without bin/ or assets/ produced %v", got)
}
}
// TestCollectAssetsNameCollision: one asset name can only mean one file, so the
// second directory's copy is reported rather than silently overwriting.
func TestCollectAssetsNameCollision(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/tool": "the binary",
"assets/tool": "something else with the same name",
})
assets, skipped := collectAssets(dir)
if len(assets) != 1 || assets[0].name != "tool" ||
!strings.HasSuffix(assets[0].path, "bin/tool") {
t.Errorf("assets = %+v, want only bin/tool", assets)
}
if len(skipped) != 1 || !strings.Contains(skipped[0], "tool") {
t.Errorf("skipped = %v, want the assets/ copy", skipped)
}
}
// TestUploadAssetGitea: a multipart POST under the field name Gitea expects,
// with the asset name in the query.
func TestUploadAssetGitea(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/mgsh-linux-amd64": "binary-bytes-here"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "gitea")
ref := releaseRef{tag: "v1.0", id: "7"}
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
req := f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets")
if req == nil {
t.Fatalf("no upload, requests: %v", f.paths())
}
if !strings.Contains(req.query, "name=mgsh-linux-amd64") {
t.Errorf("query = %q, want the asset name", req.query)
}
if !strings.HasPrefix(req.ctype, "multipart/form-data") {
t.Errorf("content type = %q, want multipart", req.ctype)
}
body := string(req.raw)
if !strings.Contains(body, `name="attachment"`) {
t.Errorf("form field is not 'attachment': %q", body)
}
if !strings.Contains(body, "binary-bytes-here") {
t.Errorf("the file contents did not make it: %q", body)
}
}
// TestUploadAssetGitHub: the bytes go raw to the host the release object named,
// not to the API host.
func TestUploadAssetGitHub(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "raw-bytes"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
f.route("POST /uploads/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "github")
// the template suffix from the release object has to be stripped
ref := api.parseReleaseRef("v1.0", []byte(`{"id":7,"upload_url":"`+
f.URL+`/uploads/repos/mike/mgsh/releases/7/assets{?name,label}"}`))
if ref.id != "7" || strings.Contains(ref.uploadURL, "{") {
t.Fatalf("release ref = %+v, want id 7 and a bare upload url", ref)
}
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
req := f.find("POST /uploads/repos/mike/mgsh/releases/7/assets")
if req == nil {
t.Fatalf("no upload to the upload host, requests: %v", f.paths())
}
if string(req.raw) != "raw-bytes" {
t.Errorf("body = %q, want the file verbatim", req.raw)
}
if !strings.Contains(req.query, "name=tool") {
t.Errorf("query = %q, want the asset name", req.query)
}
}
// TestUploadAssetGitHubWithoutUploadURL: without one there is nowhere to put
// the bytes, and that has to be said rather than guessed at.
func TestUploadAssetGitHubWithoutUploadURL(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "x"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
api := newRemoteAPI(f.URL, "tok", "github")
err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0])
if err == nil || !strings.Contains(err.Error(), "upload_url") {
t.Errorf("error = %v, want it to name the missing upload_url", err)
}
}
// TestUploadAssetGitLab: a GitLab release stores links, not files, so the file
// goes into the generic package registry first and the release then points at
// it.
func TestUploadAssetGitLab(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "package-bytes"})
assets, _ := collectAssets(dir)
const pkg = "/api/v4/projects/mike%2Fmgsh/packages/generic/mgsh/v1.0/tool"
const links = "/api/v4/projects/mike%2Fmgsh/releases/v1.0/assets/links"
f := newFakeProvider(t)
f.route("GET "+links, 200, `[]`)
f.route("PUT "+pkg, 201, `{"message":"201 Created"}`)
f.route("POST "+links, 201, `{"id":1}`)
api := newRemoteAPI(f.URL, "tok", "gitlab")
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1.0", id: "v1.0"}, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
up := f.find("PUT " + pkg)
if up == nil {
t.Fatalf("no package upload, requests: %v", f.paths())
}
if string(up.raw) != "package-bytes" {
t.Errorf("package body = %q, want the file verbatim", up.raw)
}
link := f.find("POST " + links)
if link == nil {
t.Fatalf("the release was not linked to the package, requests: %v", f.paths())
}
if link.body["name"] != "tool" || link.body["link_type"] != "package" {
t.Errorf("link body = %v", link.body)
}
if u, _ := link.body["url"].(string); !strings.HasSuffix(u, pkg) {
t.Errorf("link url = %q, want it to point at the package", u)
}
}
// TestUploadAssetReplacesPrevious: re-releasing after a rebuild has to replace
// the old file, not fail on it or leave two of each.
func TestUploadAssetReplacesPrevious(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/tool": "v2"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200,
`[{"id":41,"name":"other"},{"id":42,"name":"tool"}]`)
f.route("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42", 204, ``)
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":43}`)
api := newRemoteAPI(f.URL, "tok", "gitea")
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0]); err != nil {
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
}
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42") == nil {
t.Errorf("the previous asset was not removed: %v", f.paths())
}
// only the one with the matching name
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/41") != nil {
t.Errorf("an unrelated asset was deleted: %v", f.paths())
}
if f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets") == nil {
t.Errorf("the replacement was not uploaded: %v", f.paths())
}
}
// TestUploadAssetsReportsFailures: one bad file must not stop the others, and
// the caller has to hear about it.
func TestUploadAssetsReportsFailures(t *testing.T) {
dir := assetTree(t, map[string]string{"bin/good": "ok", "bin/bad": "boom"})
assets, _ := collectAssets(dir)
f := newFakeProvider(t)
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
// the fake answers 404 for anything unrouted, so the POST fails
api := newRemoteAPI(f.URL, "tok", "gitea")
out := captureStdout(t, func() {
if err := api.uploadAssets("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets); err == nil {
t.Error("failed uploads were not reported")
}
})
// both were attempted, and each was announced before it started
for _, name := range []string{"good", "bad"} {
if !strings.Contains(out, name) {
t.Errorf("%s was not attempted or not announced:\n%s", name, out)
}
}
}
func TestAssetSummaryAndDirList(t *testing.T) {
dir := assetTree(t, map[string]string{
"bin/a": strings.Repeat("x", 1024),
"assets/b": strings.Repeat("y", 512),
"src/main.go": "not counted",
})
assets, _ := collectAssets(dir)
if got := assetSummary(assets); got != "2 assets, 1.5K" {
t.Errorf("assetSummary = %q, want \"2 assets, 1.5K\"", got)
}
if got := strings.Join(assetDirList(dir), ","); got != "./bin,./assets" {
t.Errorf("assetDirList = %q", got)
}
// singular reads properly, and a project without the directories lists none
if got := assetSummary(assets[:1]); !strings.HasPrefix(got, "1 asset,") {
t.Errorf("assetSummary(one) = %q", got)
}
if got := assetDirList(t.TempDir()); len(got) != 0 {
t.Errorf("assetDirList of a bare project = %v", got)
}
}
+41 -4
View File
@@ -1,12 +1,22 @@
#!/bin/sh
# Build mgsh, auto-incrementing the patch version by 0.0.1 on every build.
# Build mgsh for the usual platforms into ./bin, auto-incrementing the patch
# version by 0.0.1 on every build.
#
# version.txt holds the currently built version. Each run increments the patch
# component, then builds with that version injected via -ldflags, and writes it
# back. So version.txt always reflects the version of the binary just built.
# component, then builds every platform with that one version injected via
# -ldflags, and writes it back. So version.txt always reflects the version of
# the binaries just built, and all of them carry the same one.
#
# Override the platform list to build just one:
# PLATFORMS="linux/amd64" ./build.sh
#
# Windows is deliberately absent: mgsh shells out to stty and /bin/sh, so it
# would compile there and then not work.
set -e
cd "$(dirname "$0")"
PLATFORMS=${PLATFORMS:-"darwin/arm64 darwin/amd64 linux/amd64 linux/arm64"}
V=$(cat version.txt 2>/dev/null || echo 4.0.0)
# split MAJOR.MINOR.PATCH and increment PATCH (no carry: 4.0.9 -> 4.0.10)
@@ -17,7 +27,34 @@ PATCH=${REST#*.}
PATCH=$((PATCH + 1))
NV="$MAJOR.$MINOR.$PATCH"
go build -ldflags "-X main.VERSION=$NV" -o mgsh .
# earlier versions built ./mgsh in the repo root; drop it so nothing keeps
# running a stale binary from a path that is no longer written
if [ -f mgsh ]; then
rm -f mgsh
echo "removed stale ./mgsh (the build now writes ./bin)"
fi
mkdir -p bin
HOST="$(go env GOOS)/$(go env GOARCH)"
for p in $PLATFORMS; do
os=${p%/*}
arch=${p#*/}
out="bin/mgsh-$os-$arch"
# CGO_ENABLED=0 throughout: it makes the cross builds work without a
# toolchain per target and the binaries static, and os/user still resolves
# the current user without cgo on both darwin and linux.
CGO_ENABLED=0 GOOS="$os" GOARCH="$arch" \
go build -ldflags "-X main.VERSION=$NV" -o "$out" .
if [ "$p" = "$HOST" ]; then
ln -sf "mgsh-$os-$arch" bin/mgsh # the one for this machine
echo " $out -> bin/mgsh"
else
echo " $out"
fi
done
echo "$NV" > version.txt
echo "built mgsh v$NV"
+65 -25
View File
@@ -56,46 +56,68 @@ func (r *remoteAPI) releaseByTagPath(owner, repo, tag string) string {
return r.releasePath(owner, repo) + "/tags/" + url.PathEscape(tag)
}
// findRelease returns the provider's id for the release of tag and whether it
// exists at all. GitLab addresses releases by tag, so there the tag is the id.
func (r *remoteAPI) findRelease(owner, repo, tag string) (id string, found bool, err error) {
// releaseRef identifies a release for the calls that follow creating it.
type releaseRef struct {
tag string
id string // provider id; the tag itself on GitLab
// GitHub uploads assets to a different host than its API, and names it in
// the release object as an RFC 6570 template
uploadURL string
}
// parseReleaseRef reads the identifying fields out of a release object.
func (r *remoteAPI) parseReleaseRef(tag string, data []byte) releaseRef {
ref := releaseRef{tag: tag, id: tag}
if r.kind == kindGitLab {
return ref // GitLab addresses a release by its tag throughout
}
var res struct {
ID int64 `json:"id"`
UploadURL string `json:"upload_url"`
}
json.Unmarshal(data, &res)
ref.id = strconv.FormatInt(res.ID, 10)
// ".../assets{?name,label}" -> ".../assets"
if i := strings.IndexByte(res.UploadURL, '{'); i >= 0 {
ref.uploadURL = res.UploadURL[:i]
} else {
ref.uploadURL = res.UploadURL
}
return ref
}
// findRelease returns the release of tag and whether it exists at all.
func (r *remoteAPI) findRelease(owner, repo, tag string) (ref releaseRef, found bool, err error) {
code, data, err := r.do("GET", r.releaseByTagPath(owner, repo, tag), nil)
if err != nil {
return "", false, err
return releaseRef{}, false, err
}
switch code {
case 404:
return "", false, nil
return releaseRef{}, false, nil
case 200:
if r.kind == kindGitLab {
return tag, true, nil
}
var res struct {
ID int64 `json:"id"`
}
json.Unmarshal(data, &res)
return strconv.FormatInt(res.ID, 10), true, nil
return r.parseReleaseRef(tag, data), true, nil
default:
return "", false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data))
return releaseRef{}, false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data))
}
}
// createRelease publishes a new release for an already pushed tag.
func (r *remoteAPI) createRelease(owner, repo string, rel release) error {
func (r *remoteAPI) createRelease(owner, repo string, rel release) (releaseRef, error) {
code, data, err := r.do("POST", r.releasePath(owner, repo), r.releaseBody(rel, true))
if err != nil {
return err
return releaseRef{}, err
}
if code != 200 && code != 201 {
return fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data))
return releaseRef{}, fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data))
}
return nil
return r.parseReleaseRef(rel.Tag, data), nil
}
// updateRelease rewrites the name and notes of an existing release. Gitea and
// GitHub patch it by numeric id, GitLab puts it by tag.
func (r *remoteAPI) updateRelease(owner, repo, id string, rel release) error {
method, ep := "PATCH", r.releasePath(owner, repo)+"/"+id
func (r *remoteAPI) updateRelease(owner, repo string, ref releaseRef, rel release) error {
method, ep := "PATCH", r.releasePath(owner, repo)+"/"+ref.id
if r.kind == kindGitLab {
method, ep = "PUT", r.releaseByTagPath(owner, repo, rel.Tag)
}
@@ -164,9 +186,20 @@ func handleRelease(args string) {
body = releaseNotes(DIR, tag)
}
// ./bin and ./assets ride along when they are there; nothing to configure,
// and nothing happens for a project that has neither
assets, skipped := collectAssets(DIR)
for _, s := range skipped {
errorln("skipping " + s + ": another directory already contributes that name")
}
if len(assets) > 0 {
fmt.Printf("%s %s %s\n", col(cGray, "attaching"), col(cYellow, assetSummary(assets)),
col(cGray, "from "+strings.Join(assetDirList(DIR), " and ")))
}
done := 0
for _, t := range targets {
if publishRelease(t, PRJ, tag, body) {
if publishRelease(t, PRJ, tag, body, assets) {
done++
}
}
@@ -223,7 +256,7 @@ func releaseNotes(dir, tag string) string {
// publishRelease pushes the tag to one mirror target and turns it into a
// release there. It returns success, so one unreachable server does not stop
// the remaining ones.
func publishRelease(t RemoteTarget, repo, tag, body string) bool {
func publishRelease(t RemoteTarget, repo, tag, body string, assets []releaseAsset) bool {
api := newRemoteAPI(t.URL, t.Key, t.Type)
owner, err := api.authUser()
@@ -248,17 +281,17 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
}
rel := release{Tag: tag, Name: tag, Body: body, Prerelease: preReleaseRe.MatchString(tag)}
id, found, err := api.findRelease(owner, repo, tag)
ref, found, err := api.findRelease(owner, repo, tag)
if err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
if found {
if err := api.updateRelease(owner, repo, id, rel); err != nil {
if err := api.updateRelease(owner, repo, ref, rel); err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
} else if err := api.createRelease(owner, repo, rel); err != nil {
} else if ref, err = api.createRelease(owner, repo, rel); err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
@@ -272,5 +305,12 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
}
fmt.Printf("%s %s %s %s\n", col(cGray, "remote"), col(cYellow, t.Name),
col(cGreen, what), col(cCyan, api.repoWebURL(owner, repo)))
if len(assets) > 0 {
if err := api.uploadAssets(owner, repo, ref, assets); err != nil {
errorln(t.Name + ": " + err.Error())
return false
}
}
return true
}
+15 -11
View File
@@ -9,6 +9,7 @@ package main
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"os"
@@ -20,6 +21,8 @@ import (
// recordedReq is one request the fake provider received.
type recordedReq struct {
method, path, query string
ctype string
raw []byte // the body as sent, for the asset uploads
body map[string]any
}
@@ -42,11 +45,12 @@ func newFakeProvider(t *testing.T) *fakeProvider {
body string
}{}}
f.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery}
rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery,
ctype: r.Header.Get("Content-Type")}
if r.Body != nil {
var m map[string]any
json.NewDecoder(r.Body).Decode(&m)
rec.body = m
raw, _ := io.ReadAll(r.Body)
rec.raw = raw
json.Unmarshal(raw, &rec.body)
}
f.got = append(f.got, rec)
@@ -106,7 +110,7 @@ func TestCreateReleasePerProvider(t *testing.T) {
f.route("POST "+c.wantPath, 201, `{"id":7}`)
api := newRemoteAPI(f.URL, "tok", c.typ)
if err := api.createRelease("mike", "mgsh", rel); err != nil {
if _, err := api.createRelease("mike", "mgsh", rel); err != nil {
t.Fatalf("%s: createRelease: %v (requests: %v)", c.typ, err, f.paths())
}
@@ -145,11 +149,11 @@ func TestFindReleaseAndUpdate(t *testing.T) {
f.route(c.wantMethod+" "+c.wantUpdate, 200, `{}`)
api := newRemoteAPI(f.URL, "tok", c.typ)
id, found, err := api.findRelease("mike", "mgsh", "v1.2")
ref, found, err := api.findRelease("mike", "mgsh", "v1.2")
if err != nil || !found {
t.Fatalf("%s: findRelease = %q,%v,%v", c.typ, id, found, err)
t.Fatalf("%s: findRelease = %+v,%v,%v", c.typ, ref, found, err)
}
if err := api.updateRelease("mike", "mgsh", id, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil {
if err := api.updateRelease("mike", "mgsh", ref, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil {
t.Fatalf("%s: updateRelease: %v (requests: %v)", c.typ, err, f.paths())
}
req := f.find(c.wantMethod + " " + c.wantUpdate)
@@ -168,9 +172,9 @@ func TestFindReleaseAndUpdate(t *testing.T) {
func TestFindReleaseMissing(t *testing.T) {
f := newFakeProvider(t) // everything 404s
api := newRemoteAPI(f.URL, "tok", "gitea")
id, found, err := api.findRelease("mike", "mgsh", "v9")
if err != nil || found || id != "" {
t.Fatalf("findRelease on empty server = %q,%v,%v", id, found, err)
ref, found, err := api.findRelease("mike", "mgsh", "v9")
if err != nil || found || ref.id != "" {
t.Fatalf("findRelease on empty server = %+v,%v,%v", ref, found, err)
}
}
+1 -1
View File
@@ -1 +1 @@
4.0.50
4.0.55