This makes it such that annexed files are treated like plain git files
in comparisons (e.g. the diff of a commit).
It also changes the image diff viewer to show a more reasonable error
message when one of the annexed files under comparison is missing.
Fixes#56.
Reviewed-on: https://codeberg.org/forgejo-aneksajo/forgejo-aneksajo/pulls/57
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This eliminates the wait time for the first p2phttp connection since
server startup at the cost of adding that time to the startup itself.
Reviewed-on: https://codeberg.org/forgejo-aneksajo/forgejo-aneksajo/pulls/59
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
Otherwise, git-annex tries to use its own default port (9417) and fails.
Fixes#52.
Reviewed-on: https://codeberg.org/forgejo-aneksajo/forgejo-aneksajo/pulls/55
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This adds a new endpoint under `/git-annex-p2phttp` which acts as an
authenticating proxy to git-annex' p2phttp server. This makes it
possible to set `annex+<server-url>/git-annex-p2phttp` as
`remote.<name>.annexurl` and use git-annex fully over http(s) with the
normal credentials and access tokens provided by Forgejo.
Fixes#25.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/42
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
Git-annex emits HEAD requests for keys while trying to drop them from a
repository that was cloned via http. Forgejo asked for authentication
for these HEAD requests. This meant that cloning and getting files was
possible without authentication, but dropping was not.
Since the response to a HEAD request is a subset of the response to a
GET request it is safe to make those unauthenticated as well. That is
what this change does, although limited to the
:username/:reponame/annex/objects endpoint.
Fixes#40.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/41
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
Previously, an external renderer that matched on an annexed file would
only see its content streamed via `STDIN`, or a temporary file with a copy
of its content would be generated and passed-by-filepath (with
`IS_INPUT_FILE=true`). Whether that happens, is also subject to
`MAX_DISPLAY_FILE_SIZE` (which defaults to 8MB).
This was problematic, because annexed files tend to be large. Moreover,
if present, they already exist as write-protected files on the
file-system. Creating a copy is both expensive and serves no particular
purpose.
This commit changes how external renderers are called.
1) With `IS_INPUT_FILE=true`, the renderer is passed the true location
of an annex key, if present, and an empty path, if not.
2) The original, repository-relative path of the rendering target is
made available to the renderer via the `GITEA_RELATIVE_PATH`
environment variable.
To achieve a lean implementation, the `Blob` of the rendering target
is passed on to the `RenderContext` (because the implementation of
the annex-related functionality is centered on this dtype.
This change makes it less costly to increase `MAX_DISPLAY_FILE_SIZE`,
in order to make large, annexed files eligible for markup rendering,
because no content copies will be made any longer.
External renderers can now use the original file path, with the full
original filename, including extensions, for decision making. For
example, to detect particular compression formats based in a file name
extension, or to alter the rendering based on contextual information
encoded in the file path (e.g., a multi-file data structure with a
particular organization pattern).
Apart from the additional environment variable, there is no change to
the handling of renderers that take their input via `STDIN` (i.e.,
`IS_INPUT_FILE=false`).
Fixes#35.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/36
Reviewed-by: matrss <matrss@noreply.codeberg.org>
Co-authored-by: Michael Hanke <michael.hanke@gmail.com>
Co-committed-by: Michael Hanke <michael.hanke@gmail.com>
Previously, trying to view files that were annexed, but missing, just
led to an uninformative error 500. This was rather confusing.
With these changes it now shows the pointer target instead of the
(missing) content of the file, and also indicates this situation in the
"stored with git-annex" message. For semantic correctness views for
missing files return a 404 instead of a 200, as they would with the
content present.
Fixes#7, fixes#13.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/28
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This updates the repo index/file view endpoints so annex files match the way
LFS files are rendered, making annexed files accessible via the web instead of
being black boxes only accessible by git clone.
This mostly just duplicates the existing LFS logic. It doesn't try to combine itself
with the existing logic, to make merging with upstream easier. If upstream ever
decides to accept, I would like to try to merge the redundant logic.
The one bit that doesn't directly copy LFS is my choice to hide annex-symlinks.
LFS files are always _pointer files_ and therefore always render with the "file"
icon and no special label, but annex files come in two flavours: symlinks or
pointer files. I've conflated both kinds to try to give a consistent experience.
The tests in here ensure the correct download link (/media, from the last PR)
renders in both the toolbar and, if a binary file (like most annexed files will be),
in the main pane, but it also adds quite a bit of code to make sure text files
that happen to be annexed are dug out and rendered inline like LFS files are.
Previously, Gitea's LFS support allowed direct-downloads of LFS content,
via http://$HOSTNAME:$PORT/$USER/$REPO/media/branch/$BRANCH/$FILE
Expand that grace to git-annex too. Now /media should provide the
relevant *content* from the .git/annex/objects/ folder.
This adds tests too. And expands the tests to try symlink-based annexing,
since /media implicitly supports both that and pointer-file-based annexing.
Usage of `path` was replaced by `path/filepath` in upstream forgejo, and
it made sense to use that as well where `path` was previously used. The
`setHeaderCacheForever` function and the `sendFile` method had their
signature changed.
This makes HTTP symmetric with SSH clone URLs.
This gives us the fancy feature of _anonymous_ downloads,
so people can access datasets without having to set up an
account or manage ssh keys.
Previously, to access "open access" data shared this way,
users would need to:
1. Create an account on gitea.example.com
2. Create ssh keys
3. Upload ssh keys (and make sure to find and upload the correct file)
4. `git clone git@gitea.example.com:user/dataset.git`
5. `cd dataset`
6. `git annex get`
This cuts that down to just the last three steps:
1. `git clone https://gitea.example.com/user/dataset.git`
2. `cd dataset`
3. `git annex get`
This is significantly simpler for downstream users, especially for those
unfamiliar with the command line.
Unfortunately there's no uploading. While git-annex supports uploading
over HTTP to S3 and some other special remotes, it seems to fail on a
_plain_ HTTP remote. See https://github.com/neuropoly/gitea/issues/7
and https://git-annex.branchable.com/forum/HTTP_uploads/#comment-ce28adc128fdefe4c4c49628174d9b92.
This is not a major loss since no one wants uploading to be anonymous anyway.
To support private repos, I had to hunt down and patch a secret extra security
corner that Gitea only applies to HTTP for some reason (services/auth/basic.go).
This was guided by https://git-annex.branchable.com/tips/setup_a_public_repository_on_a_web_site/
Fixes https://github.com/neuropoly/gitea/issues/3
Co-authored-by: Mathieu Guay-Paquet <mathieu.guaypaquet@polymtl.ca>
[git-annex](https://git-annex.branchable.com/) is a more complicated cousin to
git-lfs, storing large files in an optional-download side content. Unlike lfs,
it allows mixing and matching storage remotes, so the content remote(s) doesn't
need to be on the same server as the git remote, making it feasible to scatter
a collection across cloud storage, old harddrives, or anywhere else storage can
be scavenged. Since this can get complicated, fast, it has a content-tracking
database (`git annex whereis`) to help find everything later.
The use-case we imagine for including it in Gitea is just the simple case, where
we're primarily emulating git-lfs: each repo has its large content at the same URL.
Our motivation is so we can self-host https://www.datalad.org/ datasets, which
currently are only hostable by fragilely scrounging together cloud storage --
and having to manage all the credentials associated with all the pieces -- or at
https://openneuro.org which is fragile in its own ways.
Supporting git-annex also allows multiple Gitea instance to be annex remotes for
each other, mirroring the content or otherwise collaborating the split up the
hosting costs.
Enabling
--------
TODO
HTTP
----
TODO
Permission Checking
-------------------
This tweaks the API in routers/private/serv.go to expose the calling user's
computed permission, instead of just returning HTTP 403.
This doesn't fit in super well. It's the opposite from how the git-lfs support is
done, where there's a complete list of possible subcommands and their matching
permission levels, and then the API compares the requested with the actual level
and returns HTTP 403 if the check fails.
But it's necessary. The main git-annex verbs, 'git-annex-shell configlist' and
'git-annex-shell p2pstdio' are both either read-only or read-write operations,
depending on the state on disk on either end of the connection and what the user
asked it to ask for, with no way to know before git-annex examines the situation.
So tell the level via GIT_ANNEX_READONLY and trust it to handle itself.
In the older Gogs version, the permission was directly read in cmd/serv.go:
```
mode, err = db.UserAccessMode(user.ID, repo)
```
- 966e925cf3/internal/cmd/serv.go (L334)
but in Gitea permission enforcement has been centralized in the API layer.
(perhaps so the cmd layer can avoid making direct DB connections?)
Deletion
--------
git-annex has this "lockdown" feature where it tries
really quite very hard to prevent you deleting its
data, to the point that even an rm -rf won't do it:
each file in annex/objects/ is nested inside a
folder with read-only permissions.
The recommended workaround is to run chmod -R +w when
you're sure you actually want to delete a repo. See
https://git-annex.branchable.com/internals/lockdown
So we edit util.RemoveAll() to do just that, so now
it's `chmod -R +w && rm -rf` instead of just `rm -rf`.
- Do an access check when loading issues for a project column, currently
this is not done and exposes the title, labels and existence of a
private issue that the viewer of the project board may not have access
to.
- The number of issues cannot be calculated in a efficient manner
and stored in the database because their number may vary depending on
the visibility of the repositories participating in the project. The
previous implementation used the pre-calculated numbers stored in each
project, which did not reflect that potential variation.
- The code is derived from https://github.com/go-gitea/gitea/pull/22865
(cherry picked from commit 2193afaeb9954a5778f5a47aafd0e6fbbf48d000)
The web route to update and delete variables of runners did not check if
the ID that was given belonged to the context it was requested in, this
made it possible to update and delete every existing runner variable of
a instance for any authenticated user.
The code has been reworked to always take into account the context of
the request (owner and repository ID).
The web route to delete action runners did not check if the ID that was
given belonged to the context it was requested in, this made it possible
to delete every existing runner of a instance by a authenticated user.
The code was reworked to ensure that the caller of the delete
runner function retrieved the runner by ID and then checks if it belongs
to the context it was requested in, although this is not an optimal
solution it is consistent with the context checking of other code for
runners.
**Backport:** https://codeberg.org/forgejo/forgejo/pulls/6708
- Currently, the explore/organizations page always defaults to using "newest" as its sort. Instead, use the pre-existing config option (`setting.UI.ExploreDefaultSort`) so server administrators can change the default sort order.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
(Sorry, not sure how to add a test for this change)
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
I don't mind either way.
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: John Moon <john.moon@vts-i.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6749
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
Co-committed-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
**Backport:** https://codeberg.org/forgejo/forgejo/pulls/6633
When the change is reverted, the test fails as follows:
```sh
=== TestAPIGetTokens (tests/integration/api_token_test.go:34)
--- FAIL: TestAPIGetTokens (0.17s)
testlogger.go:405: 2025/01/20 14:05:22 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /home/earl-warren/software/forgejo/tests/gitea-lfs-meta
testlogger.go:405: 2025/01/20 14:05:22 ...eb/routing/logger.go:102:func1() [I] router: completed GET /api/v1/users/user2/tokens for test-mock:12345, 200 OK in 2.5ms @ user/app.go:24(user.ListAccessTokens)
testlogger.go:405: 2025/01/20 14:05:22 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/v1/users/user1/tokens for test-mock:12345, 201 Created in 4.7ms @ user/app.go:75(user.CreateAccessToken)
testlogger.go:405: 2025/01/20 14:05:22 ...eb/routing/logger.go:102:func1() [I] router: completed GET /api/v1/users/user2/tokens for test-mock:12345, 401 Unauthorized in 4.9ms @ v1/api.go:413(v1.Routes.func2.5.1.reqBasicOrRevProxyAuth.6)
api_token_test.go:46:
Error Trace: /home/earl-warren/software/forgejo/tests/integration/integration_test.go:556
/home/earl-warren/software/forgejo/tests/integration/api_token_test.go:46
Error: Not equal:
expected: 200
actual : 401
Test: TestAPIGetTokens
Messages: Request: GET /api/v1/users/user2/tokens
api_token_test.go:46: Response: {"message":"auth required","url":"http://localhost:3003/api/swagger"}
testlogger.go:405: 2025/01/20 14:05:22 ...eb/routing/logger.go:102:func1() [I] router: completed DELETE /api/v1/users/user1/tokens/94 for test-mock:12345, 204 No Content in 1.4ms @ user/app.go:145(user.DeleteAccessToken)
```
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/6633): <!--number 6633 --><!--line 0 --><!--description bGlzdGluZyB0b2tlbnMgbXVzdCBub3QgcmVxdWlyZSBiYXNpYyBhdXRo-->listing tokens must not require basic auth<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Earl Warren <contact@earl-warren.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6643
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
Co-committed-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
**Backport:** https://codeberg.org/forgejo/forgejo/pulls/6513
Backport #31851Fix#30171
---
Fixes https://github.com/go-gitea/gitea/issues/30171, this is also a
issue in Forgejo. Backport the implementation that uses the existing
sync module which does not work for multiple instances which is
perfectly fine for Forgejo for now.
(cherry picked from commit 9c990ac043a0167dc59f1c822988ed2316f7c1df)
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6517
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
Co-committed-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
**Backport:** https://codeberg.org/forgejo/forgejo/pulls/6488
This is no longer possible in future go-git versions, so lets hardcode it
Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6492
Co-authored-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
Co-committed-by: forgejo-backport-action <forgejo-backport-action@noreply.codeberg.org>
This commit allows chaning default branch update style through global
and repository settings. The setting affects "Update branch" button
in PR view (button shows when some commits are ahead of master branch).
When default update style is set to "rebase", dropdown button updates branch
by rebase by default. When update style is set to other value, dropdown button
updates branch by merge. Any of these actions may be selected using dropdown
in any case.
Signed-off-by: George Bartolomey <george@bh4.ru>
Added support for searching content in a specific directory or file.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6143
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
Co-committed-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
Fix#32795
If a job uses a matrix, multiple `ActionRunJobs` may have the same
`JobID`. We need to merge the outputs of these jobs to make them
available to the jobs that need them.
(cherry picked from commit 7269130d2878d51dcdf11f7081a591f85bd493e8)
Conflicts:
models/fixtures/action_run.yml
models/fixtures/action_run_job.yml
trivial context conflicts
The R package repository currently does not have support for older
versions of packages which should be stored in a separate /Archive
router. This PR remedies that by adding a new path router.
I am a member of a group that loves using Gitea and this bug has been
annoying us for a long time. Hope it can be merged in time for Gitea
1.23.0.
Any feedback much appreciated.
Fixes#32782
(cherry picked from commit 874b8484aa9f7e10172fd1a8a7c768e70b36c475)
Signed-off-by: Awiteb <a@4rs.nl>
Fixes: #6239
## Checklist
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [X] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [X] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [X] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6240
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Awiteb <a@4rs.nl>
Co-committed-by: Awiteb <a@4rs.nl>
This commit has a fundamental flaw, in order to syncronize if external
users are still active the commit checks if the refresh token is
accepted by the OAuth provider, if that is not the case it sees that as
the user is disabled and sets the is active field to `false` to signal
that. Because it might be possible (this commit makes this a highly
likelyhood) that the OAuth provider still recognizes this user the
commit introduces code to allow users to re-active themselves via the
oauth flow if they were disabled because of this. However this code
makes no distinction in why the user was disabled and always re-actives
the user.
Thus the reactivation via the OAuth flow allows users to bypass the
manually activation setting (`[service].REGISTER_MANUAL_CONFIRM`) or if
the admin for other reasons disabled the user.
This reverts commit 21fdd28f08.
(cherry picked from commit b32f0cdfa05c3a0e34425e1b8a5dfa8b63914a01)
Conflicts:
tests/integration/wiki_test.go
"Long-Page" is missing as well as the tests package
Fix#32683
This PR adds the login endpoint and fixes the documentation links.
(cherry picked from commit 136408307c6de7aac2ab5476f8cddf90f39355dc)
Conflicts:
routers/api/packages/api.go
trivial context conflicts
- I made a mistake when specifying the `FOR` clause for the index hint,
I read it as being an required argument by XORM. The [MariaDB
documention](https://mariadb.com/kb/en/use-index/) tells that it
defaults to the `FOR JOIN` clause hence why I specified `JOIN` (As can
be seen in the previous PR's SQL analyze I didn't specify the `FOR`
clause). However apparently there seems to be some wizardy going on as
we need to tell MariaDB to use this index for the `ORDER BY` clause to
actually force MariaDB to use this index over the `updated_unix` index.
However because it's not actually required by XORM to specify this
value I leave this empty as mariadb is apparently smart enough to figure
out for which type we want to use this index.
- TL;DR make this index hint actually effective for MariaDB.
- Ref: #6146
