This updates the repo index/file view endpoints so annex files match the way
LFS files are rendered, making annexed files accessible via the web instead of
being black boxes only accessible by git clone.
This mostly just duplicates the existing LFS logic. It doesn't try to combine itself
with the existing logic, to make merging with upstream easier. If upstream ever
decides to accept, I would like to try to merge the redundant logic.
The one bit that doesn't directly copy LFS is my choice to hide annex-symlinks.
LFS files are always _pointer files_ and therefore always render with the "file"
icon and no special label, but annex files come in two flavours: symlinks or
pointer files. I've conflated both kinds to try to give a consistent experience.
The tests in here ensure the correct download link (/media, from the last PR)
renders in both the toolbar and, if a binary file (like most annexed files will be),
in the main pane, but it also adds quite a bit of code to make sure text files
that happen to be annexed are dug out and rendered inline like LFS files are.
Previously, Gitea's LFS support allowed direct-downloads of LFS content,
via http://$HOSTNAME:$PORT/$USER/$REPO/media/branch/$BRANCH/$FILE
Expand that grace to git-annex too. Now /media should provide the
relevant *content* from the .git/annex/objects/ folder.
This adds tests too. And expands the tests to try symlink-based annexing,
since /media implicitly supports both that and pointer-file-based annexing.
This moves the `annexObjectPath()` helper out of the tests and into a
dedicated sub-package as `annex.ContentLocation()`, and expands it with
`.Pointer()` (which validates using `git annex examinekey`),
`.IsAnnexed()` and `.Content()` to make it a more useful module.
The tests retain their own wrapper version of `ContentLocation()`
because I tried to follow close to the API modules/lfs uses, which in
terms of abstract `git.Blob` and `git.TreeEntry` objects, not in terms
of `repoPath string`s which are more convenient for the tests.
This makes HTTP symmetric with SSH clone URLs.
This gives us the fancy feature of _anonymous_ downloads,
so people can access datasets without having to set up an
account or manage ssh keys.
Previously, to access "open access" data shared this way,
users would need to:
1. Create an account on gitea.example.com
2. Create ssh keys
3. Upload ssh keys (and make sure to find and upload the correct file)
4. `git clone git@gitea.example.com:user/dataset.git`
5. `cd dataset`
6. `git annex get`
This cuts that down to just the last three steps:
1. `git clone https://gitea.example.com/user/dataset.git`
2. `cd dataset`
3. `git annex get`
This is significantly simpler for downstream users, especially for those
unfamiliar with the command line.
Unfortunately there's no uploading. While git-annex supports uploading
over HTTP to S3 and some other special remotes, it seems to fail on a
_plain_ HTTP remote. See https://github.com/neuropoly/gitea/issues/7
and https://git-annex.branchable.com/forum/HTTP_uploads/#comment-ce28adc128fdefe4c4c49628174d9b92.
This is not a major loss since no one wants uploading to be anonymous anyway.
To support private repos, I had to hunt down and patch a secret extra security
corner that Gitea only applies to HTTP for some reason (services/auth/basic.go).
This was guided by https://git-annex.branchable.com/tips/setup_a_public_repository_on_a_web_site/
Fixes https://github.com/neuropoly/gitea/issues/3
Co-authored-by: Mathieu Guay-Paquet <mathieu.guaypaquet@polymtl.ca>
Fixes https://github.com/neuropoly/gitea/issues/11
Tests:
* `git annex init`
* `git annex copy --from origin`
* `git annex copy --to origin`
over:
* ssh
for:
* the owner
* a collaborator
* a read-only collaborator
* a stranger
in a
* public repo
* private repo
And then confirms:
* Deletion of the remote repo (to ensure lockdown isn't messing with us: https://git-annex.branchable.com/internals/lockdown/#comment-0cc5225dc5abe8eddeb843bfd2fdc382)
------
To support all this:
* Add util.FileCmp()
* Patch withKeyFile() so it can be nested in other copies of itself
-------
Many thanks to Mathieu for giving style tips and catching several bugs,
including a subtle one in util.filecmp() which neutered it.
Co-authored-by: Mathieu Guay-Paquet <mathieu.guay-paquet@polymtl.ca>
[git-annex](https://git-annex.branchable.com/) is a more complicated cousin to
git-lfs, storing large files in an optional-download side content. Unlike lfs,
it allows mixing and matching storage remotes, so the content remote(s) doesn't
need to be on the same server as the git remote, making it feasible to scatter
a collection across cloud storage, old harddrives, or anywhere else storage can
be scavenged. Since this can get complicated, fast, it has a content-tracking
database (`git annex whereis`) to help find everything later.
The use-case we imagine for including it in Gitea is just the simple case, where
we're primarily emulating git-lfs: each repo has its large content at the same URL.
Our motivation is so we can self-host https://www.datalad.org/ datasets, which
currently are only hostable by fragilely scrounging together cloud storage --
and having to manage all the credentials associated with all the pieces -- or at
https://openneuro.org which is fragile in its own ways.
Supporting git-annex also allows multiple Gitea instance to be annex remotes for
each other, mirroring the content or otherwise collaborating the split up the
hosting costs.
Enabling
--------
TODO
HTTP
----
TODO
Permission Checking
-------------------
This tweaks the API in routers/private/serv.go to expose the calling user's
computed permission, instead of just returning HTTP 403.
This doesn't fit in super well. It's the opposite from how the git-lfs support is
done, where there's a complete list of possible subcommands and their matching
permission levels, and then the API compares the requested with the actual level
and returns HTTP 403 if the check fails.
But it's necessary. The main git-annex verbs, 'git-annex-shell configlist' and
'git-annex-shell p2pstdio' are both either read-only or read-write operations,
depending on the state on disk on either end of the connection and what the user
asked it to ask for, with no way to know before git-annex examines the situation.
So tell the level via GIT_ANNEX_READONLY and trust it to handle itself.
In the older Gogs version, the permission was directly read in cmd/serv.go:
```
mode, err = db.UserAccessMode(user.ID, repo)
```
- 966e925cf3/internal/cmd/serv.go (L334)
but in Gitea permission enforcement has been centralized in the API layer.
(perhaps so the cmd layer can avoid making direct DB connections?)
Deletion
--------
git-annex has this "lockdown" feature where it tries
really quite very hard to prevent you deleting its
data, to the point that even an rm -rf won't do it:
each file in annex/objects/ is nested inside a
folder with read-only permissions.
The recommended workaround is to run chmod -R +w when
you're sure you actually want to delete a repo. See
https://git-annex.branchable.com/internals/lockdown
So we edit util.RemoveAll() to do just that, so now
it's `chmod -R +w && rm -rf` instead of just `rm -rf`.
Backport #26986 by @norohind
Fix#20175
Current implementation of API does not allow creating pull requests
between branches of the same
repo when you specify *namespace* (owner of the repo) in `head` field in
http request body.
---------
Co-authored-by: norohind <60548839+norohind@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit 408c92938ba6f38eea1210cae2c485e1f19d4982)
Backport #29859 by @buckybytes
The error message:
`editor.file_changed_while_editing = The file contents have changed
since you started editing. <a target="_blank" rel="noopener noreferrer"
href="%s">Click here</a> to see them or <strong>Commit Changes
again</strong> to overwrite them.`
Is re-used in inappropriate contexts. The link in the key goes to a 404
when the key is used in a situation where the file contents have not
changed.
Added two new keys to differentiate commit id mismatch and push out of
date conditions.
Co-authored-by: buckybytes <158571971+buckybytes@users.noreply.github.com>
(cherry picked from commit 00ea9af8e118790a186b4cf78a0ce85ce986bf1d)
Backport #29863 by @lng2020
Sometimes the column name is case-sensitive and it may cause 500.
Co-authored-by: Nanguan Lin <nanguanlin6@gmail.com>
(cherry picked from commit c044510ca8bed67cb2d50ba741b3d0b520aa3c43)
Backport #29813 by @yardenshoham
It didn't include the word picker.
Signed-off-by: Yarden Shoham <git@yardenshoham.com>
(cherry picked from commit 47dc4598a305ea1e6017803b95c7c60d0be64784)
Backport #29732
This adds new lexers and includes some fixes. See
https://github.com/alecthomas/chroma/releases/tag/v2.13.0 for the full
changelog.
Co-authored-by: JakobDev <jakobdev@gmx.de>
(cherry picked from commit 31ab839a6587ea93edfec4d0147282c689d3e312)
Backport #29684 by @silverwind
Fixes: https://github.com/go-gitea/gitea/issues/29414
I see no way for us to catch this error, so downgrade it until
https://github.com/microsoft/monaco-editor/issues/4325 is fixed, which
will likely take a few weeks to propagate up from vscode.
The entries in `updates.config.js` will make
[`updates`](https://github.com/silverwind/updates) not upgrade these
anymore and I think it's good documentation as well to have the reasons
why we don't upgrade these dependencies.
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 462ae88fc2e6295a9376855627ad2a824a6c8f1d)
- Backport of #2696
- Add `form-fetch-action` to indicate that this form POST to an link
that returns JSON and thus should be handled by Javascript code.
- Found by @fnetx
- Regression of https://codeberg.org/forgejo/forgejo/pulls/1793
(cherry picked from commit 9551c1a6f8)
Backport of #2658
Regression of #2507, which switched the HEAD from `pr.GetGitRefName()`
to `pr.HeadCommitID` but it had to be `prInfo.HeadCommitID`. Resolves#2656
I was able to reproduce this locally with _some_ pull requests, haven't
been able to get a reproducer trough integration testing.
(cherry picked from commit a4cc37b46a)
This should fix#2266.
This has apparently be fixed in `main` https://github.com/go-gitea/gitea/pull/27798 (but quite a big PR, which was not backported). I should likely push the test to the main branch as well.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2626
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Backport #29305 by @DanielMatiasCarvalho
This seeks to fix the bug reported on issue #29196.
Cause:
ID's with custom characters (- , _ , etc.), were not linking correctly
in the Markdown file when rendered in the browser because the ID in the
respective destinies would be different than the one in anchor, while
for IDs with only letters, the ID would be the same.
Fix:
It was suggested that to fix this bug, it should more or less like
GitHub does it. While in gitea the anchors would be put in HTML like
this:
```
<p dir="auto"><a href="#user-content-_toc152597800" rel="nofollow">Review</a></p>
<p dir="auto"><a href="#user-content-_toc152597802" rel="nofollow">Staging</a></p>
<p dir="auto"><a href="#user-content-_toc152597803" rel="nofollow">Development</a></p>
<p dir="auto"><a href="#user-content-_toc152597828" rel="nofollow">Testing</a></p>
<p dir="auto"><a href="#user-content-_toc152597829" rel="nofollow">Unit-tests</a></p>
```
In GitHub, the same anchor's href properties would be the same without
"user-content-" trailing behind.
So my code made sure to change those anchors, so it would not include
"user-content-" and then add respective Event Listeners so it would
scroll into the supposed places.
Fixes: #29196
Co-authored-by: DC <106393991+DanielMatiasCarvalho@users.noreply.github.com>
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit bfc7c8a5985825ec28f3feefe1d050ec52525097)
Backport #29672 by @charles7668
Close#29661fix#29656
Co-authored-by: charles <30816317+charles7668@users.noreply.github.com>
(cherry picked from commit 1f897637441a9a5c43e01b84e374d836d9260a00)
Backport #29554 by @lng2020
As title.
The former code directly used `ctx.Repo.GitRepo`, causing 500.
22b4f0c09f/routers/api/v1/repo/release.go (L241)
Co-authored-by: Nanguan Lin <nanguanlin6@gmail.com>
(cherry picked from commit b84303ef6e73e2436f0c4c3985020be6bbbb5d1e)
