forgejo-aneksajo/modules
Nick Guenther cebfc9cb47 git-annex support
[git-annex](https://git-annex.branchable.com/) is a more complicated cousin to
git-lfs, storing large files in an optional-download side content.  Unlike lfs,
it allows mixing and matching storage remotes, so the content remote(s) doesn't
need to be on the same server as the git remote, making it feasible to scatter
a collection across cloud storage, old harddrives, or anywhere else storage can
be scavenged.  Since this can get complicated, fast, it has a content-tracking
database (`git annex whereis`) to help find everything later.

The use-case we imagine for including it in Gitea is just the simple case, where
we're primarily emulating git-lfs: each repo has its large content at the same URL.

Our motivation is so we can self-host https://www.datalad.org/ datasets, which
currently are only hostable by fragilely scrounging together cloud storage --
and having to manage all the credentials associated with all the pieces -- or at
https://openneuro.org which is fragile in its own ways.

Supporting git-annex also allows multiple Gitea instance to be annex remotes for
each other, mirroring the content or otherwise collaborating the split up the
hosting costs.

Enabling
--------

TODO

HTTP
----

TODO

Permission Checking
-------------------

This tweaks the API in routers/private/serv.go to expose the calling user's
computed permission, instead of just returning HTTP 403.

This doesn't fit in super well. It's the opposite from how the git-lfs support is
done, where there's a complete list of possible subcommands and their matching
permission levels, and then the API compares the requested with the actual level
and returns HTTP 403 if the check fails.

But it's necessary. The main git-annex verbs, 'git-annex-shell configlist' and
'git-annex-shell p2pstdio' are both either read-only or read-write operations,
depending on the state on disk on either end of the connection and what the user
asked it to ask for, with no way to know before git-annex examines the situation.
So tell the level via GIT_ANNEX_READONLY and trust it to handle itself.

In the older Gogs version, the permission was directly read in cmd/serv.go:

```
mode, err = db.UserAccessMode(user.ID, repo)
```
- 966e925cf3/internal/cmd/serv.go (L334)

but in Gitea permission enforcement has been centralized in the API layer.
(perhaps so the cmd layer can avoid making direct DB connections?)

Deletion
--------

git-annex has this "lockdown" feature where it tries
really quite very hard to prevent you deleting its
data, to the point that even an rm -rf won't do it:
each file in annex/objects/ is nested inside a
folder with read-only permissions.

The recommended workaround is to run chmod -R +w when
you're sure you actually want to delete a repo. See
https://git-annex.branchable.com/internals/lockdown

So we edit util.RemoveAll() to do just that, so now
it's `chmod -R +w && rm -rf` instead of just `rm -rf`.
2024-05-24 11:24:11 +02:00
..
actions Merge pull request '[ACTIONS] port scheduled actions from Gitea' (#2826) from earl-warren/forgejo:wip-gitea-schedule into forgejo 2024-03-27 06:23:56 +00:00
activitypub Upgrade to golangci-lint@v1.55.0 (#27756) 2023-10-24 02:54:59 +00:00
analyze Rename code_langauge.go to code_language.go (#26377) 2023-08-07 15:00:53 -04:00
assetfs Use Set[Type] instead of map[Type]bool/struct{}. (#26804) 2023-08-30 06:55:25 +00:00
auth Refactor locale&string&template related code (#29165) 2024-02-16 15:20:52 +01:00
avatar [GITEA] Drop sha256-simd in favor of stdlib 2024-02-05 16:09:40 +01:00
base Refactor StringsToInt64s (#29967) 2024-03-26 19:04:26 +01:00
cache Always enable caches (#28527) 2023-12-19 09:29:05 +00:00
charset s/Gitea/Forgejo in various log messages and comments 2024-04-22 14:41:17 +00:00
container Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
csv Refactor locale&string&template related code (#29165) 2024-02-16 15:20:52 +01:00
emoji Update emoji set to Unicode 15 (#25595) 2023-06-29 16:29:48 +00:00
eventsource Final round of db.DefaultContext refactor (#27587) 2023-10-14 08:37:24 +00:00
generate Port "Use general token signing secret" 2024-02-19 20:49:37 +01:00
git Merge pull request '[gitea] week 2024-20-v7.0 cherry pick (release/v1.22 -> v7.0/forgejo)' (#3772) from earl-warren/wcp/2024-20-v7.0 into v7.0/forgejo 2024-05-16 15:36:08 +00:00
gitgraph models/asymkey: Implement Tag verification 2024-04-01 13:42:11 +00:00
gitrepo Move get/set default branch from git package to gitrepo package to hide repopath (#29126) 2024-03-11 23:36:59 +07:00
graceful [FIX] make pprof labels conformant with prometheus spec 2024-04-01 18:22:11 +00:00
hcaptcha Consume hcaptcha and pwn deps (#22610) 2023-01-29 09:49:51 -06:00
highlight Add option to disable ambiguous unicode characters detection (#28454) 2023-12-17 14:38:54 +00:00
hostmatcher Support allowed hosts for webhook to work with proxy (#27655) 2023-10-18 09:44:36 +00:00
html Refactor backend SVG package and add tests (#26335) 2023-08-05 04:34:59 +00:00
httpcache [BRANDING] add X-Forgejo-* headers 2024-02-05 16:02:14 +01:00
httplib Skip gzip for some well-known compressed file types (#30796) 2024-05-07 07:59:45 +01:00
indexer [FIX] Set max fuzziness to 2 for bleve (#3444) 2024-04-26 08:29:21 +00:00
issue/template Extend issue template yaml engine (#29274) 2024-03-06 12:10:47 +08:00
json Replace interface{} with any (#25686) 2023-07-04 18:36:08 +00:00
label Make label templates have consistent behavior and priority (#23749) 2023-04-10 16:44:02 +08:00
lfs Close file in the Upload func (#30262) (#30270) 2024-04-08 11:50:17 +02:00
log Reduce some allocations in type conversion (#26772) 2023-08-29 00:43:16 +08:00
markup Fix user mention processing 2024-04-30 12:28:49 +00:00
mcaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
metrics Reduce usage of db.DefaultContext (#27073) 2023-09-14 17:09:32 +00:00
migration Refactor locale&string&template related code (#29165) 2024-02-16 15:20:52 +01:00
nosql s/Gitea/Forgejo in various log messages and comments 2024-04-22 14:41:17 +00:00
optional migrate some more "OptionalBool" to "Option[bool]" (#29479) 2024-03-06 12:10:44 +08:00
options Use a general approach to access custom/static/builtin assets (#24022) 2023-04-12 18:16:45 +08:00
packages Some NuGet package enhancements 2024-04-16 06:54:35 +00:00
paginator Use more specific test methods (#24265) 2023-04-22 17:56:27 -04:00
pprof Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
private git-annex support 2024-05-24 11:24:11 +02:00
process [FIX] make pprof labels conformant with prometheus spec 2024-04-01 18:22:11 +00:00
proxy Use proxy for pull mirror (#22771) 2023-02-11 08:39:50 +08:00
proxyprotocol Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
public Refactor CORS handler (#28587) 2023-12-25 20:13:18 +08:00
queue workergroup: gracefully exit on close(popItemErr) 2024-03-24 07:12:31 +01:00
recaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
references Fix issue link does not support quotes (#29484) (#29487) 2024-03-06 12:10:45 +08:00
regexplru Upgrade go dependencies (#25819) 2023-07-14 11:00:31 +08:00
repository Merge pull request '[gitea] week 2024-20-v7.0 cherry pick (release/v1.22 -> v7.0/forgejo)' (#3772) from earl-warren/wcp/2024-20-v7.0 into v7.0/forgejo 2024-05-16 15:36:08 +00:00
secret [GITEA] Drop sha256-simd in favor of stdlib 2024-02-05 16:09:40 +01:00
session Avoid importing modules/web/middleware in modules/session (#30584) (#30589) 2024-04-21 18:16:09 +02:00
setting s/Gitea/Forgejo in various log messages and comments 2024-04-22 14:41:17 +00:00
sitemap Fix sitemap (#22272) 2022-12-30 23:31:00 +08:00
ssh Remove SSH workaround (#27893) 2023-11-03 15:21:05 +00:00
storage Add minio bucket lookup type option 2024-03-06 09:38:02 +00:00
structs Fix no edit history after editing issue's title and content (#30814) 2024-05-07 08:21:38 +01:00
svg Refactor backend SVG package and add tests (#26335) 2023-08-05 04:34:59 +00:00
sync Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
system Replace more db.DefaultContext (#27628) 2023-10-15 17:46:06 +02:00
templates [PORT] gitea##30237: Fix and rewrite contrast color calculation, fix project-related bugs 2024-05-14 15:50:47 +02:00
test [TESTS] add log.Level to test.NewLogChecker 2024-02-05 16:54:44 +01:00
testlogger Merge pull request '[v7.0/forgejo] [FEAT] Mark database errors in tests as failure' (#2978) from bp-v7.0/forgejo-2dabd20 into v7.0/forgejo 2024-04-02 15:53:23 +00:00
timeutil Remove the time-since class (#29826) 2024-03-20 08:46:30 +01:00
translation Rename ByteSize to ReadableSize 2024-04-02 17:29:32 +00:00
turnstile Add new captcha: cloudflare turnstile (#22369) 2023-02-05 15:29:03 +08:00
typesniffer Detect ogg mime-type as audio or video (#26494) 2023-08-15 10:31:25 +08:00
updatechecker [PRIVACY] Add a DNS method to fetch new updates 2024-02-05 15:38:36 +01:00
uri Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
user Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
util git-annex support 2024-05-24 11:24:11 +02:00
validation [GITEA] add option for banning dots in usernames 2024-02-05 16:05:50 +01:00
web Avoid importing modules/web/middleware in modules/session (#30584) (#30589) 2024-04-21 18:16:09 +02:00
webhook [FEAT] sourcehut webhooks 2024-04-05 19:36:04 +00:00