This updates the repo index/file view endpoints so annex files match the way
LFS files are rendered, making annexed files accessible via the web instead of
being black boxes only accessible by git clone.
This mostly just duplicates the existing LFS logic. It doesn't try to combine itself
with the existing logic, to make merging with upstream easier. If upstream ever
decides to accept, I would like to try to merge the redundant logic.
The one bit that doesn't directly copy LFS is my choice to hide annex-symlinks.
LFS files are always _pointer files_ and therefore always render with the "file"
icon and no special label, but annex files come in two flavours: symlinks or
pointer files. I've conflated both kinds to try to give a consistent experience.
The tests in here ensure the correct download link (/media, from the last PR)
renders in both the toolbar and, if a binary file (like most annexed files will be),
in the main pane, but it also adds quite a bit of code to make sure text files
that happen to be annexed are dug out and rendered inline like LFS files are.
Previously, Gitea's LFS support allowed direct-downloads of LFS content,
via http://$HOSTNAME:$PORT/$USER/$REPO/media/branch/$BRANCH/$FILE
Expand that grace to git-annex too. Now /media should provide the
relevant *content* from the .git/annex/objects/ folder.
This adds tests too. And expands the tests to try symlink-based annexing,
since /media implicitly supports both that and pointer-file-based annexing.
This makes HTTP symmetric with SSH clone URLs.
This gives us the fancy feature of _anonymous_ downloads,
so people can access datasets without having to set up an
account or manage ssh keys.
Previously, to access "open access" data shared this way,
users would need to:
1. Create an account on gitea.example.com
2. Create ssh keys
3. Upload ssh keys (and make sure to find and upload the correct file)
4. `git clone git@gitea.example.com:user/dataset.git`
5. `cd dataset`
6. `git annex get`
This cuts that down to just the last three steps:
1. `git clone https://gitea.example.com/user/dataset.git`
2. `cd dataset`
3. `git annex get`
This is significantly simpler for downstream users, especially for those
unfamiliar with the command line.
Unfortunately there's no uploading. While git-annex supports uploading
over HTTP to S3 and some other special remotes, it seems to fail on a
_plain_ HTTP remote. See https://github.com/neuropoly/gitea/issues/7
and https://git-annex.branchable.com/forum/HTTP_uploads/#comment-ce28adc128fdefe4c4c49628174d9b92.
This is not a major loss since no one wants uploading to be anonymous anyway.
To support private repos, I had to hunt down and patch a secret extra security
corner that Gitea only applies to HTTP for some reason (services/auth/basic.go).
This was guided by https://git-annex.branchable.com/tips/setup_a_public_repository_on_a_web_site/
Fixes https://github.com/neuropoly/gitea/issues/3
Co-authored-by: Mathieu Guay-Paquet <mathieu.guaypaquet@polymtl.ca>
[git-annex](https://git-annex.branchable.com/) is a more complicated cousin to
git-lfs, storing large files in an optional-download side content. Unlike lfs,
it allows mixing and matching storage remotes, so the content remote(s) doesn't
need to be on the same server as the git remote, making it feasible to scatter
a collection across cloud storage, old harddrives, or anywhere else storage can
be scavenged. Since this can get complicated, fast, it has a content-tracking
database (`git annex whereis`) to help find everything later.
The use-case we imagine for including it in Gitea is just the simple case, where
we're primarily emulating git-lfs: each repo has its large content at the same URL.
Our motivation is so we can self-host https://www.datalad.org/ datasets, which
currently are only hostable by fragilely scrounging together cloud storage --
and having to manage all the credentials associated with all the pieces -- or at
https://openneuro.org which is fragile in its own ways.
Supporting git-annex also allows multiple Gitea instance to be annex remotes for
each other, mirroring the content or otherwise collaborating the split up the
hosting costs.
Enabling
--------
TODO
HTTP
----
TODO
Permission Checking
-------------------
This tweaks the API in routers/private/serv.go to expose the calling user's
computed permission, instead of just returning HTTP 403.
This doesn't fit in super well. It's the opposite from how the git-lfs support is
done, where there's a complete list of possible subcommands and their matching
permission levels, and then the API compares the requested with the actual level
and returns HTTP 403 if the check fails.
But it's necessary. The main git-annex verbs, 'git-annex-shell configlist' and
'git-annex-shell p2pstdio' are both either read-only or read-write operations,
depending on the state on disk on either end of the connection and what the user
asked it to ask for, with no way to know before git-annex examines the situation.
So tell the level via GIT_ANNEX_READONLY and trust it to handle itself.
In the older Gogs version, the permission was directly read in cmd/serv.go:
```
mode, err = db.UserAccessMode(user.ID, repo)
```
- 966e925cf3/internal/cmd/serv.go (L334)
but in Gitea permission enforcement has been centralized in the API layer.
(perhaps so the cmd layer can avoid making direct DB connections?)
Deletion
--------
git-annex has this "lockdown" feature where it tries
really quite very hard to prevent you deleting its
data, to the point that even an rm -rf won't do it:
each file in annex/objects/ is nested inside a
folder with read-only permissions.
The recommended workaround is to run chmod -R +w when
you're sure you actually want to delete a repo. See
https://git-annex.branchable.com/internals/lockdown
So we edit util.RemoveAll() to do just that, so now
it's `chmod -R +w && rm -rf` instead of just `rm -rf`.
- If a branch cannot be renamed due to a protected branch rule, show
this error in the UI instead of throwing an internal server error.
- Add integration test (also simplify the existing one).
- Resolves#2751
Backport #26986 by @norohind
Fix#20175
Current implementation of API does not allow creating pull requests
between branches of the same
repo when you specify *namespace* (owner of the repo) in `head` field in
http request body.
---------
Co-authored-by: norohind <60548839+norohind@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit 408c92938ba6f38eea1210cae2c485e1f19d4982)
Backport #29859 by @buckybytes
The error message:
`editor.file_changed_while_editing = The file contents have changed
since you started editing. <a target="_blank" rel="noopener noreferrer"
href="%s">Click here</a> to see them or <strong>Commit Changes
again</strong> to overwrite them.`
Is re-used in inappropriate contexts. The link in the key goes to a 404
when the key is used in a situation where the file contents have not
changed.
Added two new keys to differentiate commit id mismatch and push out of
date conditions.
Co-authored-by: buckybytes <158571971+buckybytes@users.noreply.github.com>
(cherry picked from commit 00ea9af8e118790a186b4cf78a0ce85ce986bf1d)
Backport #29554 by @lng2020
As title.
The former code directly used `ctx.Repo.GitRepo`, causing 500.
22b4f0c09f/routers/api/v1/repo/release.go (L241)
Co-authored-by: Nanguan Lin <nanguanlin6@gmail.com>
(cherry picked from commit b84303ef6e73e2436f0c4c3985020be6bbbb5d1e)
Backport #29430
Thanks to inferenceus : some sort orders on the "explore/users" page
could list users by their lastlogintime/updatetime.
It leaks user's activity unintentionally. This PR makes that page only
use "supported" sort orders.
Removing the "sort orders" could also be a good solution, while IMO at
the moment keeping the "create time" and "name" orders is also fine, in
case some users would like to find a target user in the search result,
the "sort order" might help.
(cherry picked from commit 2b059f493e46b8b0fb52492623e36a8375cb5fbb)
Backport #29537 by wxiaoguang
This is only a quick fix to make it easier to backport.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 971eab18fa0b29312105df739fba443cf9e84d50)
Backport #29531 by wxiaoguang
Add two "HTMLURL" methods for PackageDescriptor.
And rename "FullWebLink" to "VersionWebLink"
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 8723389028bcb5e96359fca61efd7d6da0d6af99)
Backport #29439 by @sillyguodong
Previously, it will be treated as "re-run all jobs" when `jobIndex ==
0`. So when you click re-run button on the first job, it triggers all
the jobs actually.
Caused by #26535.
Co-authored-by: sillyguodong <33891828+sillyguodong@users.noreply.github.com>
(cherry picked from commit 9456deb512db59025cae26d82812ff880c5ea3bc)
Backport of #2292
When issue templates were moved into services in
def4956122, the code was also refactored
and simplified. Unfortunately, that simplification broke the
`/api/v1/{owner}/{repo}/issue_templates` route, because it was
previously using a helper function that ignored invalid templates, and
after the refactor, the function it called *always* returned non-nil as
the second return value. This, in turn, results in the aforementioned
end point always returning an internal server error.
This change restores the previous behaviour of ignoring invalid files
returned by `issue.GetTemplatesFromDefaultBranch`, and adds a few test
cases to exercise the endpoint.
Other users of `GetTemplatesFromDefaultBranch` already ignore the second
return value, or handle it correctly, so no changes are necessary there.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
(cherry picked from commit be8d16438a)
Backport #29105
`ctx.Error` only displays the text but `ctx.ServerError` renders the
usual error page.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit a1c0b3a02e54e5fe879dabccb71fba9498b64051)
Backport #29347 by @carlosfelgueiras
Fixes#27188.
Introduces a check on the installation that tries to parse the FROM
address. If it fails, shows a new error message to the user.
Co-authored-by: Carlos Felgueiras <carlosfelgueiras@tecnico.ulisboa.pt>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit 35db5a373babd9af157fd63eeb20d6da53320b73)
Follow-up of #2282 and #2296 (which tried to address #2278)
One of the issue with the previous PR is that when a conversation on the Files tab was marked as "resolved", it would fetch all the comments for that line (even the outdated ones, which should not be shown on this page - except when explicitly activated).
To properly fix this, I have changed `FetchCodeCommentsByLine` to `FetchCodeConversation`. Its role is to fetch all comments related to a given (review, path, line) and reverted my changes in the template (which were based on a misunderstanding).
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2306
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Backport #28935 by @silverwind
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:
- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.
Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit b8e6cffd317401d980600e339eb21b15b9bc64c1)
Backport #28905 by @Anthony-Jhoiro
Currently, the `updateMirror` function which update the mirror interval
and enable prune properties is only executed by the `Edit` function. But
it is only triggered if `opts.MirrorInterval` is not null, even if
`opts.EnablePrune` is not null.
With this patch, it is now possible to update the enable_prune property
with a patch request without modifying the mirror_interval.
## Example request with httpie
### Currently:
**Does nothing**
```bash
http PATCH https://gitea.your-server/api/v1/repos/myOrg/myRepo "enable_prune:=false" -A bearer -a $gitea_token
```
**Updates both properties**
```bash
http PATCH https://gitea.your-server/api/v1/repos/myOrg/myRepo "enable_prune:=false" "mirror_interval=10m" -A bearer -a $gitea_token
```
### With the patch
**Updates enable_prune only**
```bash
http PATCH https://gitea.your-server/api/v1/repos/myOrg/myRepo "enable_prune:=false" -A bearer -a $gitea_token
```
Co-authored-by: Anthony Quéré <47711333+Anthony-Jhoiro@users.noreply.github.com>
(cherry picked from commit d6eb6c90f45ebe93709b90691be73ba5c31e843e)
Backport https://github.com/go-gitea/gitea/pull/28726 by @fuxiaohei
Fix Uploaded artifacts should be overwritten
https://github.com/go-gitea/gitea/issues/28549
When upload different content to uploaded artifact, it checks that
content size is not match in db record with previous artifact size, then
the new artifact is refused.
Now if it finds uploading content size is not matching db record when
receiving chunks, it updates db records to follow the latest size value.
(cherry picked from commit 7f0ce2dfc7f4a0c50f6895f6d478f5230089f1c7)
Backport #28877 by @KN4CK3R
Fixes#28875
If `RequireSignInView` is enabled, the ghost user has no access rights.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit b7c944b9e4e9f847719fbce421b2f4fee7281187)
- Backport of #1981
- When the user is not found in `reloadparam`, early return when the
user is not found to avoid calling `IsUserVisibleToViewer` which in turn
avoids causing a NPE.
- This fixes the case that a 500 error and 404 error is shown on the
same page.
- Add integration test for non-existant user RSS.
- Regression by c6366089df
(cherry picked from commit f0e06962786ef8c417b0c6f07940c1909d3b91ba)
(cherry picked from commit 75d806690875a4fc38eb1e3c904096be34657011)
(cherry picked from commit 4d0a1e0637450865c7bbac69e42d92d63b95149c)
(cherry picked from commit 5f40a485da1b2c5f129f32e2ddc2065e3ba9ccd0)
(cherry picked from commit c4cb7812e39add6f7ff3d6f3f2d4e02c66435f0e)
Backport #26745Fixes#26548
This PR refactors the rendering of markup links. The old code uses
`strings.Replace` to change some urls while the new code uses more
context to decide which link should be generated.
The added tests should ensure the same output for the old and new
behaviour (besides the bug).
We may need to refactor the rendering a bit more to make it clear how
the different helper methods render the input string. There are lots of
options (resolve links / images / mentions / git hashes / emojis / ...)
but you don't really know what helper uses which options. For example,
we currently support images in the user description which should not be
allowed I think:
<details>
<summary>Profile</summary>
https://try.gitea.io/KN4CK3R
</details>
(cherry picked from commit 022552d5b6adc792d3cd16df7de6e52cb7b41a72)
Backport #28140 by @earl-warren
- Make use of the `form-fetch-action` for the merge button, which will
automatically prevent the action from happening multiple times and show
a nice loading indicator as user feedback while the merge request is
being processed by the server.
- Adjust the merge PR code to JSON response as this is required for the
`form-fetch-action` functionality.
- Resolves https://codeberg.org/forgejo/forgejo/issues/774
- Likely resolves the cause of
https://codeberg.org/forgejo/forgejo/issues/1688#issuecomment-1313044
(cherry picked from commit 4ec64c19507caefff7ddaad722b1b5792b97cc5a)
Co-authored-by: Earl Warren <109468362+earl-warren@users.noreply.github.com>
Co-authored-by: Gusted <postmaster@gusted.xyz>
(cherry picked from commit fbf29f29b5225be8e5e682e45b6977e7dda9b318)
Backport #28227 by @denyskon
When creating a pull request, allow directly assigning it to a project,
as it is already possible for issues.
After:
---------
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit 80e564087dc1a89b8b74abcdc879c2de444d3278)
Backport #27736 by @lng2020
Fix#27722Fix#27357Fix#25837Fix#28732
1. Fix the typo `BlockingByDependenciesNotPermitted`, which causes the
`not permitted message` not to show. The correct one is `Blocking` or
`BlockedBy`
2. Rewrite the perm check. The perm check uses a very tricky way to
avoid duplicate checks for a slice of issues, which is confusing. In
fact, it's also the reason causing the bug. It uses `lastRepoID` and
`lastPerm` to avoid duplicate checks, but forgets to assign the
`lastPerm` at the end of the code block. So I rewrote this to avoid this
trick.
3. It also reuses the `blocks` slice, which is even more confusing. So I
rewrote this too.
Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
(cherry picked from commit 571822b6eca07832f6b5ff633bec8ab8fdb32624)
Backport #28765 by @jackHay22
Fixes #28756
## Changes
- Require and check API token for `GET
/repos/{owner}/{repo}/subscription` in order to populate `ctx.Doer`.
Co-authored-by: Jack Hay <jack@allspice.io>
(cherry picked from commit cb33623bb6b0cf60417c4a1d84b8f42ef1714ea3)
Backport #28531 by @pulltheflower
- Fix the bug about admin/hooks API that `GET /admin/hooks` can only
fetch system_hooks, `POST /admin/hooks` can only create default_hooks.
Co-authored-by: vincent <38434877+pulltheflower@users.noreply.github.com>
(cherry picked from commit 3c79315cf2308e68caba5bd7f2f94aa3472d9425)
- The current architecture is inherently insecure, because you can
construct the 'secret' cookie value with values that are available in
the database. Thus provides zero protection when a database is
dumped/leaked.
- This patch implements a new architecture that's inspired from: [Paragonie Initiative](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies).
- Integration testing is added to ensure the new mechanism works.
- Removes a setting, because it's not used anymore.
(cherry picked from commit eff097448b1ebd2a280fcdd55d10b1f6081e9ccd)
[GITEA] rework long-term authentication (squash) add migration
Reminder: the migration is run via integration tests as explained
in the commit "[DB] run all Forgejo migrations in integration tests"
(cherry picked from commit 4accf7443c1c59b4d2e7787d6a6c602d725da403)
(cherry picked from commit 99d06e344ebc3b50bafb2ac4473dd95f057d1ddc)
(cherry picked from commit d8bc98a8f021d381bf72790ad246f923ac983ad4)
(cherry picked from commit 6404845df9a63802fff4c5bd6cfe1e390076e7f0)
(cherry picked from commit 72bdd4f3b9f6509d1ff3f10ecb12c621a932ed30)
(cherry picked from commit 4b01bb0ce812b6c59414ff53fed728563d8bc9cc)
(cherry picked from commit c26ac318162b2cad6ff1ae54e2d8f47a4e4fe7c2)
(cherry picked from commit 8d2dab94a6)
Conflicts:
routers/web/auth/auth.go
https://codeberg.org/forgejo/forgejo/issues/2158
- https://github.com/NYTimes/gziphandler doesn't seems to be maintained
anymore and Forgejo already includes
https://github.com/klauspost/compress which provides a maintained and
faster gzip handler fork.
- Enables Jitter to prevent BREACH attacks, as this *seems* to be
possible in the context of Forgejo.
(cherry picked from commit cc2847241d82001babd8d40c87d03169f21c14cd)
(cherry picked from commit 99ba56a8761dd08e08d9499cab2ded1a6b7b970f)
Conflicts:
go.sum
https://codeberg.org/forgejo/forgejo/pulls/1581
(cherry picked from commit 711638193daa2311e2ead6249a47dcec47b4e335)
(cherry picked from commit 9c12a37fde6fa84414bf332ff4a066facdb92d38)
(cherry picked from commit 91191aaaedaf999209695e2c6ca4fb256b396686)
(cherry picked from commit 72be417f844713265a94ced6951f8f4b81d0ab1a)
(cherry picked from commit 98497c84da205ec59079e42274aa61199444f7cd)
(cherry picked from commit fba042adb5c1abcbd8eee6b5a4f735ccb2a5e394)
(cherry picked from commit dd2414f226)
Conflicts:
routers/web/web.go
https://codeberg.org/forgejo/forgejo/issues/2016
Backport #28587, the only conflict is the test file.
The CORS code has been unmaintained for long time, and the behavior is
not correct.
This PR tries to improve it. The key point is written as comment in
code. And add more tests.
Fix#28515Fix#27642Fix#17098
(cherry picked from commit 7a2786ca6cd84633784a2c9986da65a9c4d79c78)
