Fixes https://github.com/neuropoly/gitea/issues/11
Tests:
* `git annex init`
* `git annex copy --from origin`
* `git annex copy --to origin`
over:
* ssh
for:
* the owner
* a collaborator
* a read-only collaborator
* a stranger
in a
* public repo
* private repo
And then confirms:
* Deletion of the remote repo (to ensure lockdown isn't messing with us: https://git-annex.branchable.com/internals/lockdown/#comment-0cc5225dc5abe8eddeb843bfd2fdc382)
------
To support all this:
* Add util.FileCmp()
* Patch withKeyFile() so it can be nested in other copies of itself
-------
Many thanks to Mathieu for giving style tips and catching several bugs,
including a subtle one in util.filecmp() which neutered it.
Co-authored-by: Mathieu Guay-Paquet <mathieu.guay-paquet@polymtl.ca>
[git-annex](https://git-annex.branchable.com/) is a more complicated cousin to
git-lfs, storing large files in an optional-download side content. Unlike lfs,
it allows mixing and matching storage remotes, so the content remote(s) doesn't
need to be on the same server as the git remote, making it feasible to scatter
a collection across cloud storage, old harddrives, or anywhere else storage can
be scavenged. Since this can get complicated, fast, it has a content-tracking
database (`git annex whereis`) to help find everything later.
The use-case we imagine for including it in Gitea is just the simple case, where
we're primarily emulating git-lfs: each repo has its large content at the same URL.
Our motivation is so we can self-host https://www.datalad.org/ datasets, which
currently are only hostable by fragilely scrounging together cloud storage --
and having to manage all the credentials associated with all the pieces -- or at
https://openneuro.org which is fragile in its own ways.
Supporting git-annex also allows multiple Gitea instance to be annex remotes for
each other, mirroring the content or otherwise collaborating the split up the
hosting costs.
Enabling
--------
TODO
HTTP
----
TODO
Permission Checking
-------------------
This tweaks the API in routers/private/serv.go to expose the calling user's
computed permission, instead of just returning HTTP 403.
This doesn't fit in super well. It's the opposite from how the git-lfs support is
done, where there's a complete list of possible subcommands and their matching
permission levels, and then the API compares the requested with the actual level
and returns HTTP 403 if the check fails.
But it's necessary. The main git-annex verbs, 'git-annex-shell configlist' and
'git-annex-shell p2pstdio' are both either read-only or read-write operations,
depending on the state on disk on either end of the connection and what the user
asked it to ask for, with no way to know before git-annex examines the situation.
So tell the level via GIT_ANNEX_READONLY and trust it to handle itself.
In the older Gogs version, the permission was directly read in cmd/serv.go:
```
mode, err = db.UserAccessMode(user.ID, repo)
```
- 966e925cf3/internal/cmd/serv.go (L334)
but in Gitea permission enforcement has been centralized in the API layer.
(perhaps so the cmd layer can avoid making direct DB connections?)
Deletion
--------
git-annex has this "lockdown" feature where it tries
really quite very hard to prevent you deleting its
data, to the point that even an rm -rf won't do it:
each file in annex/objects/ is nested inside a
folder with read-only permissions.
The recommended workaround is to run chmod -R +w when
you're sure you actually want to delete a repo. See
https://git-annex.branchable.com/internals/lockdown
So we edit util.RemoveAll() to do just that, so now
it's `chmod -R +w && rm -rf` instead of just `rm -rf`.
- Backport https://codeberg.org/forgejo/forgejo/pulls/1839
- Consider executable files as a valid case when returning a downloadURL
for them. They are just regular files with the difference being the
executable permission bit being set.
- Not integration testing as it's not possible without adding adjusting
the existing repositories to have a executable file.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1825
(cherry picked from commit ca32891d548c302b0f3b3072647058278ffb9cbf)
- Backport https://codeberg.org/forgejo/forgejo/pulls/1849
- Hook Forgejo's `EnsureUpToDate` to Gitea's `EnsureUpToDate`, such that
the Forgejo migrations are also being checked to be up to date.
- I'm not sure how I missed this and if this has caused any problems,
but due to the lack of any open issue about it it seems to not be a big
problem.
(cherry picked from commit 6c65b6dcf6ab0d58e5c2d03a866e4e38294f72ad)
backport #28213
This PR will fix some missed checks for private repositories' data on
web routes and API routes.
(cherry picked from commit dfd511faf35fef68557e53763f9b06e5a139565d)
(cherry picked from commit 93a844dd13904c0ba1b7fd4a0a233002194a504b)
(cherry picked from commit 6d6d1a121ce3fc5cf7cd92ad1a38be3bdcbf7088)
(cherry picked from commit 8b101f2860dfbdfd99de71d30740c9e72e1cd9d5)
(cherry picked from commit 3e56212d6d1bca0aecdc1f224c7d78287ef9d35d)
(cherry picked from commit 4f619bc58583892c197ee2588ead929342336217)
