Attach ./bin and ./assets to a release
`release` now uploads every file in the project's ./bin and ./assets when
those directories exist. Nothing to configure, and nothing happens for a
project that has neither.
This is the part deliberately left out when `release` was written,
because it is where the three providers stop resembling each other:
Gitea multipart POST to .../releases/<id>/assets?name=<name>
GitHub raw POST to the separate upload host the release object names
in upload_url, whose RFC 6570 template suffix has to go first
GitLab a release stores links, not files: the file goes into the
project's generic package registry and the release gets a
package link pointing at it
So findRelease and createRelease now return a releaseRef carrying the id
and, for GitHub, that upload host -- the id alone cannot address an
upload. Uploads stream from disk rather than buffering: these are whole
binaries, and the Gitea multipart body is assembled through a pipe.
Only regular files directly in those directories are taken. Symlinks are
skipped, which matters here: build.sh leaves bin/mgsh pointing at one of
its siblings, and uploading the same 9M twice under two names helps
nobody. A name present in both directories is used from bin and reported
for assets. Re-releasing a tag replaces same-named assets rather than
failing on them, since rebuilding and publishing again is the normal
reason to do it, and a file that fails does not stop the rest.
Each provider's request shape is pinned down against the recording
stand-in, and the whole chain was run once end to end -- real repository,
real binaries, a fake Gitea that also serves git-http-backend so the tag
push is real too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -392,11 +392,43 @@ on different servers. `release` also refuses when the repository is not on the
|
||||
mirror yet and tells you to run `pushremote` first, rather than creating it as a
|
||||
side effect.
|
||||
|
||||
**No binary assets.** The three providers handle uploads in three incompatible
|
||||
ways — Gitea attaches them to the release, GitHub uses a separate upload host,
|
||||
and GitLab does not host them at all but expects a link into its package
|
||||
registry. mgsh publishes source releases with notes; if you need binaries,
|
||||
upload them with the provider's own tooling.
|
||||
#### Binaries and assets
|
||||
|
||||
If the project has a `./bin` or `./assets` directory, every file in it is
|
||||
attached to the release — nothing to configure, and nothing happens for a
|
||||
project that has neither:
|
||||
|
||||
```
|
||||
< src/mgsh > release v4.1.0
|
||||
attaching 5 assets, 38M from ./bin and ./assets
|
||||
remote gitea released https://git.example.com/mike/mgsh.git
|
||||
uploading logo.png 2.0K
|
||||
uploading mgsh-darwin-amd64 9.8M
|
||||
uploading mgsh-darwin-arm64 9.2M
|
||||
uploading mgsh-linux-amd64 9.7M
|
||||
uploading mgsh-linux-arm64 8.9M
|
||||
```
|
||||
|
||||
Only regular files directly in those directories are taken: subdirectories are
|
||||
not descended into, and symlinks are skipped — `bin/mgsh` points at one of its
|
||||
own siblings, and uploading the same binary twice under two names helps nobody.
|
||||
A name present in both directories is used from `bin` and reported for
|
||||
`assets`, since one asset name can only mean one file.
|
||||
|
||||
Re-releasing the same tag replaces same-named assets instead of failing or
|
||||
piling up duplicates, because rebuilding and publishing again is the normal
|
||||
reason to do it. A file that fails to upload does not stop the rest.
|
||||
|
||||
This is where the providers stop resembling each other, and mgsh papers over it:
|
||||
|
||||
| | how the bytes get there |
|
||||
|---|---|
|
||||
| Gitea | multipart `POST` to `…/releases/<id>/assets?name=<name>` |
|
||||
| GitHub | raw `POST` to the separate upload host named by the release's `upload_url` |
|
||||
| GitLab | a release stores links, not files: the file goes into the project's generic **package registry** and the release gets a `package` link pointing at it |
|
||||
|
||||
The GitLab route needs the package registry enabled on the project — it is on by
|
||||
default, but a self-hosted instance can turn it off.
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
package main
|
||||
|
||||
// assets.go — attaching files to a release.
|
||||
//
|
||||
// `release` uploads everything in the project's ./bin and ./assets when those
|
||||
// directories exist. This is where the three providers stop resembling each
|
||||
// other, so each gets its own path:
|
||||
//
|
||||
// Gitea multipart POST to .../releases/<id>/assets?name=<name>
|
||||
// GitHub raw POST to the upload host named by the release object's upload_url
|
||||
// GitLab no asset hosting on a release at all: the file goes into the generic
|
||||
// package registry, and the release gets a link pointing at it
|
||||
//
|
||||
// Re-releasing the same tag replaces same-named assets rather than failing or
|
||||
// piling up duplicates, because rebuilding and publishing again is the normal
|
||||
// reason to do it.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// assetDirs are the project directories whose files a release carries.
|
||||
var assetDirs = []string{"bin", "assets"}
|
||||
|
||||
// releaseAsset is one file to attach to a release.
|
||||
type releaseAsset struct {
|
||||
name string
|
||||
path string
|
||||
size int64
|
||||
}
|
||||
|
||||
// collectAssets gathers the files to attach: every regular file directly in
|
||||
// the project's ./bin and ./assets. Subdirectories are not descended into, and
|
||||
// symlinks are skipped — bin/mgsh points at one of its own siblings, and
|
||||
// uploading the same binary twice under two names helps nobody.
|
||||
func collectAssets(dir string) (assets []releaseAsset, skipped []string) {
|
||||
seen := map[string]bool{}
|
||||
for _, sub := range assetDirs {
|
||||
path := filepath.Join(dir, sub)
|
||||
entries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
continue // the directory simply is not there
|
||||
}
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || e.Type()&os.ModeSymlink != 0 {
|
||||
continue
|
||||
}
|
||||
fi, err := e.Info()
|
||||
if err != nil || !fi.Mode().IsRegular() {
|
||||
continue
|
||||
}
|
||||
if seen[e.Name()] {
|
||||
// bin/x and assets/x would fight over one asset name
|
||||
skipped = append(skipped, filepath.Join(sub, e.Name()))
|
||||
continue
|
||||
}
|
||||
seen[e.Name()] = true
|
||||
assets = append(assets, releaseAsset{
|
||||
name: e.Name(),
|
||||
path: filepath.Join(path, e.Name()),
|
||||
size: fi.Size(),
|
||||
})
|
||||
}
|
||||
}
|
||||
sort.Slice(assets, func(i, j int) bool { return assets[i].name < assets[j].name })
|
||||
return assets, skipped
|
||||
}
|
||||
|
||||
// contentType guesses a type from the file name, falling back to the one for
|
||||
// "some bytes" — which is what a compiled binary is.
|
||||
func contentType(name string) string {
|
||||
if t := mime.TypeByExtension(filepath.Ext(name)); t != "" {
|
||||
return t
|
||||
}
|
||||
return "application/octet-stream"
|
||||
}
|
||||
|
||||
// doUpload sends a request with a raw body of a known size. The JSON helper
|
||||
// cannot express these: an asset upload is bytes, not an object, and the
|
||||
// providers insist on a Content-Length rather than a chunked body.
|
||||
func (r *remoteAPI) doUpload(method, endpoint, ctype string, body io.Reader, size int64) (int, []byte, error) {
|
||||
req, err := http.NewRequest(method, endpoint, body)
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
req.ContentLength = size
|
||||
req.Header.Set("Content-Type", ctype)
|
||||
hk, hv := r.authHeader()
|
||||
req.Header.Set(hk, hv)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
|
||||
resp, err := r.http.Do(req)
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return resp.StatusCode, data, nil
|
||||
}
|
||||
|
||||
// uploadAssets attaches every file to the release, reporting each one as it
|
||||
// goes: these are megabytes over a network, and silence for a minute looks like
|
||||
// a hang. A file that fails does not stop the rest.
|
||||
func (r *remoteAPI) uploadAssets(owner, repo string, ref releaseRef, assets []releaseAsset) error {
|
||||
failed := 0
|
||||
for _, a := range assets {
|
||||
fmt.Printf(" %s %s %s\n", col(cGray, "uploading"),
|
||||
col(cGreen, padRight(a.name, 28)), col(cGray, humanSize(a.size)))
|
||||
if err := r.uploadAsset(owner, repo, ref, a); err != nil {
|
||||
errorln(" " + a.name + ": " + err.Error())
|
||||
failed++
|
||||
}
|
||||
}
|
||||
if failed > 0 {
|
||||
return fmt.Errorf("%d of %d assets failed", failed, len(assets))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// uploadAsset attaches one file, replacing an asset of the same name that is
|
||||
// already on the release.
|
||||
func (r *remoteAPI) uploadAsset(owner, repo string, ref releaseRef, a releaseAsset) error {
|
||||
if err := r.removeAsset(owner, repo, ref, a.name); err != nil {
|
||||
return err
|
||||
}
|
||||
switch r.kind {
|
||||
case kindGitLab:
|
||||
return r.uploadAssetGitLab(owner, repo, ref, a)
|
||||
case kindGitHub:
|
||||
return r.uploadAssetGitHub(ref, a)
|
||||
default:
|
||||
return r.uploadAssetGitea(owner, repo, ref, a)
|
||||
}
|
||||
}
|
||||
|
||||
// uploadAssetGitea posts the file as a multipart form, streamed from disk
|
||||
// rather than buffered: these are whole binaries.
|
||||
func (r *remoteAPI) uploadAssetGitea(owner, repo string, ref releaseRef, a releaseAsset) error {
|
||||
f, err := os.Open(a.path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
pr, pw := io.Pipe()
|
||||
mw := multipart.NewWriter(pw)
|
||||
go func() {
|
||||
part, err := mw.CreateFormFile("attachment", a.name)
|
||||
if err == nil {
|
||||
_, err = io.Copy(part, f)
|
||||
}
|
||||
if err != nil {
|
||||
pw.CloseWithError(err)
|
||||
return
|
||||
}
|
||||
pw.CloseWithError(mw.Close())
|
||||
}()
|
||||
|
||||
ep := r.releasePath(owner, repo) + "/" + ref.id + "/assets?name=" + url.QueryEscape(a.name)
|
||||
// the multipart length is not known up front, so this one is chunked
|
||||
code, data, err := r.doUpload("POST", ep, mw.FormDataContentType(), pr, -1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 200 && code != 201 {
|
||||
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// uploadAssetGitHub posts the raw bytes to the host the release object named.
|
||||
func (r *remoteAPI) uploadAssetGitHub(ref releaseRef, a releaseAsset) error {
|
||||
if ref.uploadURL == "" {
|
||||
return fmt.Errorf("the release carries no upload_url")
|
||||
}
|
||||
f, err := os.Open(a.path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
ep := ref.uploadURL + "?name=" + url.QueryEscape(a.name)
|
||||
code, data, err := r.doUpload("POST", ep, contentType(a.name), f, a.size)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 200 && code != 201 {
|
||||
return fmt.Errorf("HTTP %d: %s", code, firstLine(data))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// uploadAssetGitLab puts the file into the project's generic package registry
|
||||
// and links the release to it, since a GitLab release stores links, not files.
|
||||
func (r *remoteAPI) uploadAssetGitLab(owner, repo string, ref releaseRef, a releaseAsset) error {
|
||||
f, err := os.Open(a.path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
pkg := r.packagePath(owner, repo, ref.tag, a.name)
|
||||
code, data, err := r.doUpload("PUT", pkg, contentType(a.name), f, a.size)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 200 && code != 201 {
|
||||
return fmt.Errorf("package upload failed (HTTP %d): %s", code, firstLine(data))
|
||||
}
|
||||
|
||||
code, data, err = r.do("POST", r.releaseByTagPath(owner, repo, ref.tag)+"/assets/links",
|
||||
map[string]any{"name": a.name, "url": pkg, "link_type": "package"})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 200 && code != 201 {
|
||||
return fmt.Errorf("linking the package failed (HTTP %d): %s", code, firstLine(data))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// packagePath is where a release asset lives in GitLab's generic registry: one
|
||||
// package named after the repository, one version per tag.
|
||||
func (r *remoteAPI) packagePath(owner, repo, tag, name string) string {
|
||||
return r.repoPath(owner, repo) + "/packages/generic/" +
|
||||
url.PathEscape(repo) + "/" + url.PathEscape(tag) + "/" + url.PathEscape(name)
|
||||
}
|
||||
|
||||
// removeAsset deletes an asset of the given name from the release when one is
|
||||
// there, so re-releasing a tag after a rebuild replaces the files instead of
|
||||
// failing or leaving two of each.
|
||||
func (r *remoteAPI) removeAsset(owner, repo string, ref releaseRef, name string) error {
|
||||
listEP, delEP := r.assetEndpoints(owner, repo, ref)
|
||||
code, data, err := r.do("GET", listEP, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code == 404 {
|
||||
return nil // nothing uploaded yet
|
||||
}
|
||||
if code != 200 {
|
||||
return fmt.Errorf("listing assets failed (HTTP %d): %s", code, firstLine(data))
|
||||
}
|
||||
|
||||
for _, id := range assetIDsNamed(data, name) {
|
||||
if code, data, err := r.do("DELETE", delEP+"/"+id, nil); err != nil {
|
||||
return err
|
||||
} else if code != 200 && code != 202 && code != 204 {
|
||||
return fmt.Errorf("deleting the previous %s failed (HTTP %d): %s", name, code, firstLine(data))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// assetEndpoints returns where a release's assets are listed and deleted.
|
||||
// GitHub deletes an asset through the repository rather than the release.
|
||||
func (r *remoteAPI) assetEndpoints(owner, repo string, ref releaseRef) (list, del string) {
|
||||
switch r.kind {
|
||||
case kindGitLab:
|
||||
base := r.releaseByTagPath(owner, repo, ref.tag) + "/assets/links"
|
||||
return base, base
|
||||
case kindGitHub:
|
||||
return r.releasePath(owner, repo) + "/" + ref.id + "/assets",
|
||||
r.releasePath(owner, repo) + "/assets"
|
||||
default: // Gitea
|
||||
base := r.releasePath(owner, repo) + "/" + ref.id + "/assets"
|
||||
return base, base
|
||||
}
|
||||
}
|
||||
|
||||
// assetIDsNamed picks the ids of the listed assets called name. All three
|
||||
// providers answer with an array of objects carrying "id" and "name".
|
||||
func assetIDsNamed(data []byte, name string) []string {
|
||||
var list []struct {
|
||||
ID json.Number `json:"id"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &list); err != nil {
|
||||
return nil
|
||||
}
|
||||
var ids []string
|
||||
for _, a := range list {
|
||||
if a.Name == name && a.ID.String() != "" {
|
||||
ids = append(ids, a.ID.String())
|
||||
}
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
// assetSummary describes what will be uploaded, for the line `release` prints
|
||||
// before it starts.
|
||||
func assetSummary(assets []releaseAsset) string {
|
||||
var total int64
|
||||
for _, a := range assets {
|
||||
total += a.size
|
||||
}
|
||||
unit := "assets"
|
||||
if len(assets) == 1 {
|
||||
unit = "asset"
|
||||
}
|
||||
return strconv.Itoa(len(assets)) + " " + unit + ", " + humanSize(total)
|
||||
}
|
||||
|
||||
// assetDirList names the directories that were actually found, for the same line.
|
||||
func assetDirList(dir string) []string {
|
||||
var found []string
|
||||
for _, sub := range assetDirs {
|
||||
if isDir(filepath.Join(dir, sub)) {
|
||||
found = append(found, "./"+sub)
|
||||
}
|
||||
}
|
||||
return found
|
||||
}
|
||||
+284
@@ -0,0 +1,284 @@
|
||||
package main
|
||||
|
||||
// assets_test.go — attaching files to a release.
|
||||
//
|
||||
// The three providers upload assets in three unrelated ways, and the difference
|
||||
// is invisible until a real server rejects the request. The recording stand-in
|
||||
// lets each one be pinned down: where the bytes go, how they are wrapped, and
|
||||
// what else has to happen around them.
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// assetTree builds a project with ./bin and ./assets and returns its path.
|
||||
func assetTree(t *testing.T, files map[string]string) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
for rel, content := range files {
|
||||
p := filepath.Join(dir, rel)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(content), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return dir
|
||||
}
|
||||
|
||||
func TestCollectAssets(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{
|
||||
"bin/mgsh-linux-amd64": "ELF...",
|
||||
"bin/mgsh-darwin-arm64": "MACH...",
|
||||
"assets/logo.png": "PNG",
|
||||
"assets/notes.txt": "text",
|
||||
"bin/sub/nested": "not descended into",
|
||||
"src/main.go": "not an asset directory",
|
||||
})
|
||||
// bin/mgsh is a symlink to one of its siblings, as build.sh leaves it
|
||||
if err := os.Symlink("mgsh-linux-amd64", filepath.Join(dir, "bin", "mgsh")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
assets, skipped := collectAssets(dir)
|
||||
var names []string
|
||||
for _, a := range assets {
|
||||
names = append(names, a.name)
|
||||
}
|
||||
want := "logo.png,mgsh-darwin-arm64,mgsh-linux-amd64,notes.txt"
|
||||
if strings.Join(names, ",") != want {
|
||||
t.Errorf("collected %v, want %s", names, want)
|
||||
}
|
||||
if len(skipped) != 0 {
|
||||
t.Errorf("skipped = %v, want none", skipped)
|
||||
}
|
||||
for _, a := range assets {
|
||||
if a.size == 0 {
|
||||
t.Errorf("%s has no size", a.name)
|
||||
}
|
||||
if _, err := os.Stat(a.path); err != nil {
|
||||
t.Errorf("%s: %v", a.name, err)
|
||||
}
|
||||
}
|
||||
|
||||
// a project with neither directory contributes nothing at all
|
||||
if got, _ := collectAssets(t.TempDir()); len(got) != 0 {
|
||||
t.Errorf("a project without bin/ or assets/ produced %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCollectAssetsNameCollision: one asset name can only mean one file, so the
|
||||
// second directory's copy is reported rather than silently overwriting.
|
||||
func TestCollectAssetsNameCollision(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{
|
||||
"bin/tool": "the binary",
|
||||
"assets/tool": "something else with the same name",
|
||||
})
|
||||
assets, skipped := collectAssets(dir)
|
||||
if len(assets) != 1 || assets[0].name != "tool" ||
|
||||
!strings.HasSuffix(assets[0].path, "bin/tool") {
|
||||
t.Errorf("assets = %+v, want only bin/tool", assets)
|
||||
}
|
||||
if len(skipped) != 1 || !strings.Contains(skipped[0], "tool") {
|
||||
t.Errorf("skipped = %v, want the assets/ copy", skipped)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetGitea: a multipart POST under the field name Gitea expects,
|
||||
// with the asset name in the query.
|
||||
func TestUploadAssetGitea(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/mgsh-linux-amd64": "binary-bytes-here"})
|
||||
assets, _ := collectAssets(dir)
|
||||
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
|
||||
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", "gitea")
|
||||
ref := releaseRef{tag: "v1.0", id: "7"}
|
||||
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
|
||||
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
|
||||
}
|
||||
|
||||
req := f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets")
|
||||
if req == nil {
|
||||
t.Fatalf("no upload, requests: %v", f.paths())
|
||||
}
|
||||
if !strings.Contains(req.query, "name=mgsh-linux-amd64") {
|
||||
t.Errorf("query = %q, want the asset name", req.query)
|
||||
}
|
||||
if !strings.HasPrefix(req.ctype, "multipart/form-data") {
|
||||
t.Errorf("content type = %q, want multipart", req.ctype)
|
||||
}
|
||||
body := string(req.raw)
|
||||
if !strings.Contains(body, `name="attachment"`) {
|
||||
t.Errorf("form field is not 'attachment': %q", body)
|
||||
}
|
||||
if !strings.Contains(body, "binary-bytes-here") {
|
||||
t.Errorf("the file contents did not make it: %q", body)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetGitHub: the bytes go raw to the host the release object named,
|
||||
// not to the API host.
|
||||
func TestUploadAssetGitHub(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/tool": "raw-bytes"})
|
||||
assets, _ := collectAssets(dir)
|
||||
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
|
||||
f.route("POST /uploads/repos/mike/mgsh/releases/7/assets", 201, `{"id":1}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", "github")
|
||||
// the template suffix from the release object has to be stripped
|
||||
ref := api.parseReleaseRef("v1.0", []byte(`{"id":7,"upload_url":"`+
|
||||
f.URL+`/uploads/repos/mike/mgsh/releases/7/assets{?name,label}"}`))
|
||||
if ref.id != "7" || strings.Contains(ref.uploadURL, "{") {
|
||||
t.Fatalf("release ref = %+v, want id 7 and a bare upload url", ref)
|
||||
}
|
||||
|
||||
if err := api.uploadAsset("mike", "mgsh", ref, assets[0]); err != nil {
|
||||
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
|
||||
}
|
||||
req := f.find("POST /uploads/repos/mike/mgsh/releases/7/assets")
|
||||
if req == nil {
|
||||
t.Fatalf("no upload to the upload host, requests: %v", f.paths())
|
||||
}
|
||||
if string(req.raw) != "raw-bytes" {
|
||||
t.Errorf("body = %q, want the file verbatim", req.raw)
|
||||
}
|
||||
if !strings.Contains(req.query, "name=tool") {
|
||||
t.Errorf("query = %q, want the asset name", req.query)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetGitHubWithoutUploadURL: without one there is nowhere to put
|
||||
// the bytes, and that has to be said rather than guessed at.
|
||||
func TestUploadAssetGitHubWithoutUploadURL(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/tool": "x"})
|
||||
assets, _ := collectAssets(dir)
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET /api/v3/repos/mike/mgsh/releases/7/assets", 200, `[]`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", "github")
|
||||
err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0])
|
||||
if err == nil || !strings.Contains(err.Error(), "upload_url") {
|
||||
t.Errorf("error = %v, want it to name the missing upload_url", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetGitLab: a GitLab release stores links, not files, so the file
|
||||
// goes into the generic package registry first and the release then points at
|
||||
// it.
|
||||
func TestUploadAssetGitLab(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/tool": "package-bytes"})
|
||||
assets, _ := collectAssets(dir)
|
||||
|
||||
const pkg = "/api/v4/projects/mike%2Fmgsh/packages/generic/mgsh/v1.0/tool"
|
||||
const links = "/api/v4/projects/mike%2Fmgsh/releases/v1.0/assets/links"
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET "+links, 200, `[]`)
|
||||
f.route("PUT "+pkg, 201, `{"message":"201 Created"}`)
|
||||
f.route("POST "+links, 201, `{"id":1}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", "gitlab")
|
||||
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1.0", id: "v1.0"}, assets[0]); err != nil {
|
||||
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
|
||||
}
|
||||
|
||||
up := f.find("PUT " + pkg)
|
||||
if up == nil {
|
||||
t.Fatalf("no package upload, requests: %v", f.paths())
|
||||
}
|
||||
if string(up.raw) != "package-bytes" {
|
||||
t.Errorf("package body = %q, want the file verbatim", up.raw)
|
||||
}
|
||||
link := f.find("POST " + links)
|
||||
if link == nil {
|
||||
t.Fatalf("the release was not linked to the package, requests: %v", f.paths())
|
||||
}
|
||||
if link.body["name"] != "tool" || link.body["link_type"] != "package" {
|
||||
t.Errorf("link body = %v", link.body)
|
||||
}
|
||||
if u, _ := link.body["url"].(string); !strings.HasSuffix(u, pkg) {
|
||||
t.Errorf("link url = %q, want it to point at the package", u)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetReplacesPrevious: re-releasing after a rebuild has to replace
|
||||
// the old file, not fail on it or leave two of each.
|
||||
func TestUploadAssetReplacesPrevious(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/tool": "v2"})
|
||||
assets, _ := collectAssets(dir)
|
||||
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200,
|
||||
`[{"id":41,"name":"other"},{"id":42,"name":"tool"}]`)
|
||||
f.route("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42", 204, ``)
|
||||
f.route("POST /api/v1/repos/mike/mgsh/releases/7/assets", 201, `{"id":43}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", "gitea")
|
||||
if err := api.uploadAsset("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets[0]); err != nil {
|
||||
t.Fatalf("uploadAsset: %v (requests %v)", err, f.paths())
|
||||
}
|
||||
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/42") == nil {
|
||||
t.Errorf("the previous asset was not removed: %v", f.paths())
|
||||
}
|
||||
// only the one with the matching name
|
||||
if f.find("DELETE /api/v1/repos/mike/mgsh/releases/7/assets/41") != nil {
|
||||
t.Errorf("an unrelated asset was deleted: %v", f.paths())
|
||||
}
|
||||
if f.find("POST /api/v1/repos/mike/mgsh/releases/7/assets") == nil {
|
||||
t.Errorf("the replacement was not uploaded: %v", f.paths())
|
||||
}
|
||||
}
|
||||
|
||||
// TestUploadAssetsReportsFailures: one bad file must not stop the others, and
|
||||
// the caller has to hear about it.
|
||||
func TestUploadAssetsReportsFailures(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{"bin/good": "ok", "bin/bad": "boom"})
|
||||
assets, _ := collectAssets(dir)
|
||||
|
||||
f := newFakeProvider(t)
|
||||
f.route("GET /api/v1/repos/mike/mgsh/releases/7/assets", 200, `[]`)
|
||||
// the fake answers 404 for anything unrouted, so the POST fails
|
||||
api := newRemoteAPI(f.URL, "tok", "gitea")
|
||||
|
||||
out := captureStdout(t, func() {
|
||||
if err := api.uploadAssets("mike", "mgsh", releaseRef{tag: "v1", id: "7"}, assets); err == nil {
|
||||
t.Error("failed uploads were not reported")
|
||||
}
|
||||
})
|
||||
// both were attempted, and each was announced before it started
|
||||
for _, name := range []string{"good", "bad"} {
|
||||
if !strings.Contains(out, name) {
|
||||
t.Errorf("%s was not attempted or not announced:\n%s", name, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssetSummaryAndDirList(t *testing.T) {
|
||||
dir := assetTree(t, map[string]string{
|
||||
"bin/a": strings.Repeat("x", 1024),
|
||||
"assets/b": strings.Repeat("y", 512),
|
||||
"src/main.go": "not counted",
|
||||
})
|
||||
assets, _ := collectAssets(dir)
|
||||
if got := assetSummary(assets); got != "2 assets, 1.5K" {
|
||||
t.Errorf("assetSummary = %q, want \"2 assets, 1.5K\"", got)
|
||||
}
|
||||
if got := strings.Join(assetDirList(dir), ","); got != "./bin,./assets" {
|
||||
t.Errorf("assetDirList = %q", got)
|
||||
}
|
||||
// singular reads properly, and a project without the directories lists none
|
||||
if got := assetSummary(assets[:1]); !strings.HasPrefix(got, "1 asset,") {
|
||||
t.Errorf("assetSummary(one) = %q", got)
|
||||
}
|
||||
if got := assetDirList(t.TempDir()); len(got) != 0 {
|
||||
t.Errorf("assetDirList of a bare project = %v", got)
|
||||
}
|
||||
}
|
||||
+65
-25
@@ -56,46 +56,68 @@ func (r *remoteAPI) releaseByTagPath(owner, repo, tag string) string {
|
||||
return r.releasePath(owner, repo) + "/tags/" + url.PathEscape(tag)
|
||||
}
|
||||
|
||||
// findRelease returns the provider's id for the release of tag and whether it
|
||||
// exists at all. GitLab addresses releases by tag, so there the tag is the id.
|
||||
func (r *remoteAPI) findRelease(owner, repo, tag string) (id string, found bool, err error) {
|
||||
// releaseRef identifies a release for the calls that follow creating it.
|
||||
type releaseRef struct {
|
||||
tag string
|
||||
id string // provider id; the tag itself on GitLab
|
||||
// GitHub uploads assets to a different host than its API, and names it in
|
||||
// the release object as an RFC 6570 template
|
||||
uploadURL string
|
||||
}
|
||||
|
||||
// parseReleaseRef reads the identifying fields out of a release object.
|
||||
func (r *remoteAPI) parseReleaseRef(tag string, data []byte) releaseRef {
|
||||
ref := releaseRef{tag: tag, id: tag}
|
||||
if r.kind == kindGitLab {
|
||||
return ref // GitLab addresses a release by its tag throughout
|
||||
}
|
||||
var res struct {
|
||||
ID int64 `json:"id"`
|
||||
UploadURL string `json:"upload_url"`
|
||||
}
|
||||
json.Unmarshal(data, &res)
|
||||
ref.id = strconv.FormatInt(res.ID, 10)
|
||||
// ".../assets{?name,label}" -> ".../assets"
|
||||
if i := strings.IndexByte(res.UploadURL, '{'); i >= 0 {
|
||||
ref.uploadURL = res.UploadURL[:i]
|
||||
} else {
|
||||
ref.uploadURL = res.UploadURL
|
||||
}
|
||||
return ref
|
||||
}
|
||||
|
||||
// findRelease returns the release of tag and whether it exists at all.
|
||||
func (r *remoteAPI) findRelease(owner, repo, tag string) (ref releaseRef, found bool, err error) {
|
||||
code, data, err := r.do("GET", r.releaseByTagPath(owner, repo, tag), nil)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
return releaseRef{}, false, err
|
||||
}
|
||||
switch code {
|
||||
case 404:
|
||||
return "", false, nil
|
||||
return releaseRef{}, false, nil
|
||||
case 200:
|
||||
if r.kind == kindGitLab {
|
||||
return tag, true, nil
|
||||
}
|
||||
var res struct {
|
||||
ID int64 `json:"id"`
|
||||
}
|
||||
json.Unmarshal(data, &res)
|
||||
return strconv.FormatInt(res.ID, 10), true, nil
|
||||
return r.parseReleaseRef(tag, data), true, nil
|
||||
default:
|
||||
return "", false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data))
|
||||
return releaseRef{}, false, fmt.Errorf("checking release failed (HTTP %d): %s", code, firstLine(data))
|
||||
}
|
||||
}
|
||||
|
||||
// createRelease publishes a new release for an already pushed tag.
|
||||
func (r *remoteAPI) createRelease(owner, repo string, rel release) error {
|
||||
func (r *remoteAPI) createRelease(owner, repo string, rel release) (releaseRef, error) {
|
||||
code, data, err := r.do("POST", r.releasePath(owner, repo), r.releaseBody(rel, true))
|
||||
if err != nil {
|
||||
return err
|
||||
return releaseRef{}, err
|
||||
}
|
||||
if code != 200 && code != 201 {
|
||||
return fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data))
|
||||
return releaseRef{}, fmt.Errorf("creating release failed (HTTP %d): %s", code, firstLine(data))
|
||||
}
|
||||
return nil
|
||||
return r.parseReleaseRef(rel.Tag, data), nil
|
||||
}
|
||||
|
||||
// updateRelease rewrites the name and notes of an existing release. Gitea and
|
||||
// GitHub patch it by numeric id, GitLab puts it by tag.
|
||||
func (r *remoteAPI) updateRelease(owner, repo, id string, rel release) error {
|
||||
method, ep := "PATCH", r.releasePath(owner, repo)+"/"+id
|
||||
func (r *remoteAPI) updateRelease(owner, repo string, ref releaseRef, rel release) error {
|
||||
method, ep := "PATCH", r.releasePath(owner, repo)+"/"+ref.id
|
||||
if r.kind == kindGitLab {
|
||||
method, ep = "PUT", r.releaseByTagPath(owner, repo, rel.Tag)
|
||||
}
|
||||
@@ -164,9 +186,20 @@ func handleRelease(args string) {
|
||||
body = releaseNotes(DIR, tag)
|
||||
}
|
||||
|
||||
// ./bin and ./assets ride along when they are there; nothing to configure,
|
||||
// and nothing happens for a project that has neither
|
||||
assets, skipped := collectAssets(DIR)
|
||||
for _, s := range skipped {
|
||||
errorln("skipping " + s + ": another directory already contributes that name")
|
||||
}
|
||||
if len(assets) > 0 {
|
||||
fmt.Printf("%s %s %s\n", col(cGray, "attaching"), col(cYellow, assetSummary(assets)),
|
||||
col(cGray, "from "+strings.Join(assetDirList(DIR), " and ")))
|
||||
}
|
||||
|
||||
done := 0
|
||||
for _, t := range targets {
|
||||
if publishRelease(t, PRJ, tag, body) {
|
||||
if publishRelease(t, PRJ, tag, body, assets) {
|
||||
done++
|
||||
}
|
||||
}
|
||||
@@ -223,7 +256,7 @@ func releaseNotes(dir, tag string) string {
|
||||
// publishRelease pushes the tag to one mirror target and turns it into a
|
||||
// release there. It returns success, so one unreachable server does not stop
|
||||
// the remaining ones.
|
||||
func publishRelease(t RemoteTarget, repo, tag, body string) bool {
|
||||
func publishRelease(t RemoteTarget, repo, tag, body string, assets []releaseAsset) bool {
|
||||
api := newRemoteAPI(t.URL, t.Key, t.Type)
|
||||
|
||||
owner, err := api.authUser()
|
||||
@@ -248,17 +281,17 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
|
||||
}
|
||||
|
||||
rel := release{Tag: tag, Name: tag, Body: body, Prerelease: preReleaseRe.MatchString(tag)}
|
||||
id, found, err := api.findRelease(owner, repo, tag)
|
||||
ref, found, err := api.findRelease(owner, repo, tag)
|
||||
if err != nil {
|
||||
errorln(t.Name + ": " + err.Error())
|
||||
return false
|
||||
}
|
||||
if found {
|
||||
if err := api.updateRelease(owner, repo, id, rel); err != nil {
|
||||
if err := api.updateRelease(owner, repo, ref, rel); err != nil {
|
||||
errorln(t.Name + ": " + err.Error())
|
||||
return false
|
||||
}
|
||||
} else if err := api.createRelease(owner, repo, rel); err != nil {
|
||||
} else if ref, err = api.createRelease(owner, repo, rel); err != nil {
|
||||
errorln(t.Name + ": " + err.Error())
|
||||
return false
|
||||
}
|
||||
@@ -272,5 +305,12 @@ func publishRelease(t RemoteTarget, repo, tag, body string) bool {
|
||||
}
|
||||
fmt.Printf("%s %s %s %s\n", col(cGray, "remote"), col(cYellow, t.Name),
|
||||
col(cGreen, what), col(cCyan, api.repoWebURL(owner, repo)))
|
||||
|
||||
if len(assets) > 0 {
|
||||
if err := api.uploadAssets(owner, repo, ref, assets); err != nil {
|
||||
errorln(t.Name + ": " + err.Error())
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
+15
-11
@@ -9,6 +9,7 @@ package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
@@ -20,6 +21,8 @@ import (
|
||||
// recordedReq is one request the fake provider received.
|
||||
type recordedReq struct {
|
||||
method, path, query string
|
||||
ctype string
|
||||
raw []byte // the body as sent, for the asset uploads
|
||||
body map[string]any
|
||||
}
|
||||
|
||||
@@ -42,11 +45,12 @@ func newFakeProvider(t *testing.T) *fakeProvider {
|
||||
body string
|
||||
}{}}
|
||||
f.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery}
|
||||
rec := recordedReq{method: r.Method, path: r.URL.EscapedPath(), query: r.URL.RawQuery,
|
||||
ctype: r.Header.Get("Content-Type")}
|
||||
if r.Body != nil {
|
||||
var m map[string]any
|
||||
json.NewDecoder(r.Body).Decode(&m)
|
||||
rec.body = m
|
||||
raw, _ := io.ReadAll(r.Body)
|
||||
rec.raw = raw
|
||||
json.Unmarshal(raw, &rec.body)
|
||||
}
|
||||
f.got = append(f.got, rec)
|
||||
|
||||
@@ -106,7 +110,7 @@ func TestCreateReleasePerProvider(t *testing.T) {
|
||||
f.route("POST "+c.wantPath, 201, `{"id":7}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", c.typ)
|
||||
if err := api.createRelease("mike", "mgsh", rel); err != nil {
|
||||
if _, err := api.createRelease("mike", "mgsh", rel); err != nil {
|
||||
t.Fatalf("%s: createRelease: %v (requests: %v)", c.typ, err, f.paths())
|
||||
}
|
||||
|
||||
@@ -145,11 +149,11 @@ func TestFindReleaseAndUpdate(t *testing.T) {
|
||||
f.route(c.wantMethod+" "+c.wantUpdate, 200, `{}`)
|
||||
|
||||
api := newRemoteAPI(f.URL, "tok", c.typ)
|
||||
id, found, err := api.findRelease("mike", "mgsh", "v1.2")
|
||||
ref, found, err := api.findRelease("mike", "mgsh", "v1.2")
|
||||
if err != nil || !found {
|
||||
t.Fatalf("%s: findRelease = %q,%v,%v", c.typ, id, found, err)
|
||||
t.Fatalf("%s: findRelease = %+v,%v,%v", c.typ, ref, found, err)
|
||||
}
|
||||
if err := api.updateRelease("mike", "mgsh", id, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil {
|
||||
if err := api.updateRelease("mike", "mgsh", ref, release{Tag: "v1.2", Name: "v1.2", Body: "new"}); err != nil {
|
||||
t.Fatalf("%s: updateRelease: %v (requests: %v)", c.typ, err, f.paths())
|
||||
}
|
||||
req := f.find(c.wantMethod + " " + c.wantUpdate)
|
||||
@@ -168,9 +172,9 @@ func TestFindReleaseAndUpdate(t *testing.T) {
|
||||
func TestFindReleaseMissing(t *testing.T) {
|
||||
f := newFakeProvider(t) // everything 404s
|
||||
api := newRemoteAPI(f.URL, "tok", "gitea")
|
||||
id, found, err := api.findRelease("mike", "mgsh", "v9")
|
||||
if err != nil || found || id != "" {
|
||||
t.Fatalf("findRelease on empty server = %q,%v,%v", id, found, err)
|
||||
ref, found, err := api.findRelease("mike", "mgsh", "v9")
|
||||
if err != nil || found || ref.id != "" {
|
||||
t.Fatalf("findRelease on empty server = %+v,%v,%v", ref, found, err)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
4.0.53
|
||||
4.0.55
|
||||
|
||||
Reference in New Issue
Block a user