# Fix OCI artifact uploads with`oras`
## Problem
ORAS (OCI Registry As Storage) artifact uploads were failing with several HTTP-related errors when pushing to Forgejo's container registry. This prevented users from storing OCI artifacts like `artifacthub-repo.yaml` in commands like `oras push [...] artifacthub-repo.yaml:application/vnd.cncf.artifacthub.repository-metadata.layer.v1.yaml`.
This has been discussed previously in https://github.com/go-gitea/gitea/issues/25846
## Root Causes and Fixes
### 1. Missing Content-Length for Empty Blobs
**Issue**: Empty blobs (size 0) were not getting the required `Content-Length: 0` header, causing ORAS to fail with "unknown response Content-Length".
**Fix**: Changed the condition in `setResponseHeaders` from `if h.ContentLength != 0` to `if h.ContentLength >= 0` to ensure the Content-Length header is always set for valid blob sizes.
```go
// Before
if h.ContentLength != 0 {
resp.Header().Set("Content-Length", strconv.FormatInt(h.ContentLength, 10))
}
// After
if h.ContentLength >= 0 {
resp.Header().Set("Content-Length", strconv.FormatInt(h.ContentLength, 10))
}
```
### 2. Content-Length Mismatch in JSON Error Responses
**Issue**: The `jsonResponse` function was calling `WriteHeader()` before writing JSON content, causing "wrote more than the declared Content-Length" errors when the HTTP stack calculated a different Content-Length than what was actually written.
**Fix**: Modified `jsonResponse` to buffer JSON content first, calculate the exact Content-Length, then write the complete response.
### 3. Incomplete HTTP Responses in Error Handling
**Issue**: The `apiError` function was only setting response headers without writing any response body, causing EOF errors when clients expected a complete HTTP response.
**Fix**: Updated `apiError` to write proper JSON error responses following the OCI Distribution Specification format with `code` and `message` fields.
### 4. Empty Config Blob Handling for OCI Artifacts
**Issue**: OCI artifacts often have empty config blobs (required by spec but contain no data). The JSON decoder was failing with EOF when trying to parse these empty configs.
**Fix**: Added EOF handling in `parseOCIImageConfig` to return a valid default metadata object for empty config blobs.
```go
if err := json.NewDecoder(r).Decode(&image); err != nil {
// Handle empty config blobs (common in OCI artifacts)
if err == io.EOF {
return &Metadata{
Type: TypeOCI,
Platform: DefaultPlatform,
}, nil
}
return nil, err
}
```
## Testing
Verified that ORAS artifact uploads now work correctly:
```bash
oras push registry/owner/package:artifacthub.io \
--config /dev/null:application/vnd.cncf.artifacthub.config.v1+yaml \
artifacthub-repo.yaml:application/vnd.cncf.artifacthub.repository-metadata.layer.v1.yaml
```
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8070
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: pat-s <patrick.schratz@gmail.com>
Co-committed-by: pat-s <patrick.schratz@gmail.com>
- While doing cross-compiling of Forgejo (with CGO enabled) it was noticed that besides compiling sqlite3, github.com/DataDog/zstd (a CGO wrapper around the reference zstd library) took a long time to compile. Upon investigating why this library was included in the first place I concluded that this library is not even used and compiling this package is a waste of time and CPU cycles.
- https://github.com/sassoftware/go-rpmutils is the library that uses the CGO zstd library, and would use it the pure Go variant (https://github.com/klauspost/compress/zstd) if CGO is enabled (which is the default). It uses zstd to uncompress the payload of a RPM packages. This is a operation that Forgejo does not use in the slightest, hence being unused code.
- It is not possible to force compiling the pure Go variant if CGO is enabled. Therefore forking and removing this code is the only option to avoid compiling the zstd C library. The changes made to the fork can be seen here: 2660c86d57...v1.0.0
- Via [actiongraph](https://github.com/icio/actiongraph) you can precisely see where the Go compiler is spending time, on a beefy machine the compilation takes of the zstd C library takes ~40s. For reference compiling the sqlite3 C library takes ~50s.
- The forgejo binary (build via `make backend`) reduced its size by 835776 bytes (0.8 MiB).
TL;DR forked library to remove unused code to avoid Go spending a considerable amount of time compiling the reference zstd library that in the end is never used.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/7953): <!--number 7953 --><!--line 0 --><!--description cmVwbGFjZSBnby1ycG11dGlscyBsaWJyYXJ5IHdpdGggb3VyIG93bg==-->replace go-rpmutils library with our own<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7953
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Aleksandr Gamzin alexgamz1119@gmail.com
Adds support for the Apt-Rpm registry of the Alt Lunux distribution.
Alt Linux uses RPM packages to store and distribute software to its users. But the logic of the Alt Linux package registry is different from the Red Hat package registry.
I have added support for the Alt Linux package registry.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: Aleksandr Gamzin <gamzin@altlinux.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6351
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Alex619829 <alex619829@noreply.codeberg.org>
Co-committed-by: Alex619829 <alex619829@noreply.codeberg.org>
- When a dependency is renamed, specified via `package="actual-name"` in
Cargo.toml, this should become the name of the depedency when the
package is retrieved from the registery by cargo and the old name should
be available in the `package` field.
- The reference implementation also does this: 490e66a9d6/src/controllers/krate/publish.rs (L702-L705)
- Resolves#5936
- Unit test added.
Fix#28121
I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.
348d1d0f32/routers/api/packages/container/container.go (L712-L717)
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
<img width="600px"
src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>
In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.
```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```
---
Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.
b7fb20e73e/modules/storage/minio.go (L214-L221)
(cherry picked from commit 0690cb076bf63f71988a709f62a9c04660b51a4f)
Conflicts:
- modules/storage/azureblob.go
Dropped the change, as we do not support Azure blob storage.
- modules/storage/helper.go
Resolved by adjusting their `discardStorage` to our
`DiscardStorage`
- routers/api/actions/artifacts.go
routers/api/actions/artifactsv4.go
routers/web/repo/actions/view.go
routers/web/repo/download.go
Resolved the conflicts by manually adding the new `nil`
parameter to the `storage.Attachments.URL()` calls.
Originally conflicted due to differences in the if expression
above these calls.
- [x] add architecture-specific removal support
- [x] Fix upload competition
- [x] Fix not checking input when downloading
docs: https://codeberg.org/forgejo/docs/pulls/874
### Release notes
- [ ] I do not want this change to show in the release notes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5351
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
This PR addresses the missing `bin` field in Composer metadata, which
currently causes vendor-provided binaries to not be symlinked to
`vendor/bin` during installation.
In the current implementation, running `composer install` does not
publish the binaries, leading to issues where expected binaries are not
available.
By properly declaring the `bin` field, this PR ensures that binaries are
correctly symlinked upon installation, as described in the [Composer
documentation](https://getcomposer.org/doc/articles/vendor-binaries.md).
(cherry picked from commit d351a42494e71b5e2da63302c2f9b46c78e6dbde)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
(cherry picked from commit 33cc5837a655ad544b936d4d040ca36d74092588)
Conflicts:
assets/go-licenses.json
go.mod
go.sum
resolved with make tidy
Previous arch package grouping was not well-suited for complex or multi-architecture environments. It now supports the following content:
- Support grouping by any path.
- New support for packages in `xz` format.
- Fix clean up rules
<!--start release-notes-assistant-->
## Draft release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/4903): <!--number 4903 --><!--line 0 --><!--description c3VwcG9ydCBncm91cGluZyBieSBhbnkgcGF0aCBmb3IgYXJjaCBwYWNrYWdl-->support grouping by any path for arch package<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4903
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
Related #30075
CC @thojo0
Example with rendered readme:
(cherry picked from commit 4e7b067a7fdfb3e2c8dfdf87475e3938051fd400)
Enable [unparam](https://github.com/mvdan/unparam) linter.
Often I could not tell the intention why param is unused, so I put
`//nolint` for those cases like webhook request creation functions never
using `ctx`.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit fc2d75f86d77b022ece848acf2581c14ef21d43b)
Conflicts:
modules/setting/config_env.go
modules/storage/azureblob.go
services/webhook/dingtalk.go
services/webhook/discord.go
services/webhook/feishu.go
services/webhook/matrix.go
services/webhook/msteams.go
services/webhook/packagist.go
services/webhook/slack.go
services/webhook/telegram.go
services/webhook/wechatwork.go
run make lint-go and fix Forgejo specific warnings
Noteable additions:
- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches
Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
(cherry picked from commit 74f0c84fa4245a20ce6fb87dac1faf2aeeded2a2)
Conflicts:
.golangci.yml
apply the linter recommendations to Forgejo code as well
Refs: https://codeberg.org/forgejo/forgejo/pulls/2222
(cherry picked from commit 5f837efc15f3d1e0d7fbed7fc569251143266584)
fix: write xml header
(cherry picked from commit a715984a42be9da81c48106d5eae244098ac1108)
fix: optional elements and xml schema
(cherry picked from commit 6ea6895a3616246e7282aa20d8f010fa931b60ea)
fix: pass all other requests to file search
(cherry picked from commit 9bfc74833a3b657453b4519573598432a87e3e3c)
test: add integration test
(cherry picked from commit b798f4ce86daa78e694c5c142e6f5f44938e6cb6)
fix: use xmlResponse
(cherry picked from commit 7f76df0b246c64fac0eeb115642c8cb6eb676f36)
(cherry picked from commit e18d574ca40905aec52fbbe8247ba83fd01874dd)
Resolves https://github.com/go-gitea/gitea/issues/28704
Example of an entry in the generated `APKINDEX` file:
```
C:Q1xCO3H9LTTEbhKt9G1alSC87I56c=
P:hello
V:2.12-r1
A:x86_64
T:The GNU Hello program produces a familiar, friendly greeting
U:https://www.gnu.org/software/hello/
L:GPL-3.0-or-later
S:15403
I:36864
o:hello
m:
t:1705934118
D:so:libc.musl-x86_64.so.1
p:cmd:hello=2.12-r1
i:foobar=1.0 !baz
k:42
```
the `i:` and `k:` entries are new.
---------
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Related #26984
(https://github.com/go-gitea/gitea/pull/26984#issuecomment-1889588912)
Fix admin cleanup message.
Fix models `Get` not respecting default values.
Rebuild RPM repository files after cleanup.
Do not add RPM group to package version name.
Force stable sorting of Alpine/Debian/RPM repository data.
Fix missing deferred `Close`.
Add tests for multiple RPM groups.
Removed non-cached `ReplaceAllStringRegex`.
If there are multiple groups available, it's stated in the package
installation screen:
The current rpm repository places all packages in the same repository,
and different systems (el7,f34) may hit packages that do not belong to
this distribution ( #25304 ) , which now supports grouping of rpm.
Fixes#25304 .
Fixes#27056 .
Refactor: [#25866](https://github.com/go-gitea/gitea/pull/25866)
- Added new tests to cover corner cases
- Replace existing regex with new one
Closes#26551
---
As @silverwind suggested, I started from
[validate-npm-package-name](https://github.com/npm/validate-npm-package-name),
but found this solution too complicated.
Then I tried to fix existing regex myself, but thought, that exclude all
restricted symbols is harder, than set only allowed symbols.
Then I search a bit more and found
[package-name-regex](https://github.com/dword-design/package-name-regex)
and regex from it works for all new test cases.
Let me know, if more information or help with this PR is needed.
Fixes#24723
Direct serving of content aka HTTP redirect is not mentioned in any of
the package registry specs but lots of official registries do that so it
should be supported by the usual clients.
Co-authored-by: @awkwardbunny
This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.
---------
Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Close#23444
Add `Repository` to npm package `Metadata` struct so the `repository` in
`package.json` can be stored and be returned in the endpoint.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
