The Forgejo version is derived from the git history, so the image build
needs to happen in the context of a full repository clone.
Also, the post-processing of the version string needs to remove the
second occurrence of "-g", as the first one is now part of the added
"-git-annex" part.
Reviewed-on: https://codeberg.org/forgejo-aneksajo/forgejo-aneksajo/pulls/51
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This adds a new endpoint under `/git-annex-p2phttp` which acts as an
authenticating proxy to git-annex' p2phttp server. This makes it
possible to set `annex+<server-url>/git-annex-p2phttp` as
`remote.<name>.annexurl` and use git-annex fully over http(s) with the
normal credentials and access tokens provided by Forgejo.
Fixes#25.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/42
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This adds some rudimentary tests that drop files in a repository's clone
as well as from a repository on Forgejo.
Fixes#4.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/47
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
This is a prerequisite for #25, as p2phttp requires a newer git-annex
version than is available from NeuroDebian.
Reviewed-on: https://codeberg.org/matrss/forgejo-aneksajo/pulls/45
Co-authored-by: Matthias Riße <m.risse@fz-juelich.de>
Co-committed-by: Matthias Riße <m.risse@fz-juelich.de>
Multiple tests that worked fine on v1.20.4-1 started to fail after the
rebase onto v1.20.5-1. These tests are:
- TestGitAnnexPermissions/Private/Owner/HTTP/Init
- TestGitAnnexPermissions/Private/Owner/HTTP/Download
- TestGitAnnexPermissions/Private/Writer/HTTP/Init
- TestGitAnnexPermissions/Private/Writer/HTTP/Download
- TestGitAnnexPermissions/Private/Reader/HTTP/Init
- TestGitAnnexPermissions/Private/Reader/HTTP/Download
What these tests have in common is that they all operate on a private
repository via http with authentication.
They broke at some point between v1.20.4-1 and v1.20.5-1, so I did a
bisect between these two points running the offending tests. This
brought me to the conclusion that
ee48c0d5ea introduced the issue.
The thing is, this commit does not change any code, it only changes the
test environment. Among other things that didn't look as suspicious, it
changes the container image from a bespoke test_env image based on
debian bullseye to a node image based on debian bookworm. Obviously,
this means that there are many version differences between the two.
The first one I looked at was git. The previous bullseye image used a
manually installed git version 2.40.0, while the bookworm image has
2.39.2 installed. Updating git in the new image did not fix the issue,
however.
The next thing I looked at was the git-annex version. Bullseye had
8.20210223 installed and worked, while bookworm used 10.20230126 when
the tests broke. So I tried my luck upgrading to a more recent version
via neurodebian (10.20240227-1~ndall+1). This still worked fine on
bullseye and now also works fine on bookworm.
I have no idea why this specific version of git-annex broke the tests,
but at least there was a commit to pinpoint this to, which isn't always
the case with docker images silently changing beneath you...
Below are the versions as they are reported by git and git-annex:
bullseye (works):
git version 2.30.2
git-annex version: 8.20210223
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
bullseye + git-annex from neurodebian (works):
git version 2.30.2
git-annex version: 10.20240227-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
bookworm (fails):
git version 2.39.2
git-annex version: 10.20230126
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
bookworm + git-annex from neurodebian (works):
git version 2.39.2
git-annex version: 10.20240227-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
The end-to-end tests will always fail when more than one release is
broken. When trying to fix one, the other will get in the way and vice
versa. The only way to get out of this deadlock is to replace all
broken releases but one by doing the following on forgejo-integration:
* set SKIP_END_TO_END to true in the actions vars tab
* pushing a commit to the corresponding branch, fixing the problem
(cherry picked from commit 54c8ac3e39)
It could be used but then `cp --dereference` would need to be used instead in
the forgejo-build-publish action.
+ docker cp forgejo-amd64:/app/gitea/forgejo-cli forgejo-9.0-test-linux-amd64
+ chmod +x forgejo-9.0-test-linux-amd64
chmod: cannot operate on dangling symlink 'forgejo-9.0-test-linux-amd64'
(cherry picked from commit 1a7a9055e4)
- retrieved by the commit hash
- removes bindata tags from integration tests, because it does not seem
to be required
- due to the missing automatically generated data, the zstd tests fail
(they use repo data including node_modules (!) as input to the test,
there is no apparent reason for the size constants)
includes:
- easier repo declaration for playwright tests by @Gusted
- full backend build for pushing Git repos by @Gusted
- playwright testing (which fails with the current diff algorithm, but
passes with the new)
- disable eslint rule for conditional expect, because it defeats the
purpose (working around it would result in much more complex test code
in our cases)
When the Forgejo CLI binary is `forgejo-cli`, the `--verbose` or `--quiet`
arguments are available globally for all sub-commands. The same
sub-commands can be used with `forgejo forgejo-cli`, those flags are
not available.
If the tag of a stable release is removed from integration, it won't
be properly described when building the test release. It will be:
8.0.0-dev-1648-7b31a541c0+gitea-1.22.0
instead of:
8.0.1-5-7b31a541c0+gitea-1.22.0
The releases are created when:
* a tag is pushed to the integration repository it will create a
vX.Y.Z release
* a new commit is pushed to a branch and mirrored to the integration
repository, it will create a vX.Y-test release named after the branch
When both vX.Y.Z and vX.Y-test release are present, the end-to-end
tests will use vX.Y.Z because it comes first in release sort
order. This ensures that a last round of end-to-end tests is run from
the release built in the integration repository, exactly as it will be
published and signed.
In between stable releases, the vX.Y-test releases are built daily and
must be used instead for end-to-end testing so that problems can be
detected as soon as possible. For that to happen, the stable release
must be removed from the integration repository and this is done 24h
after they were published.
The vX.Y-test releases are removed if they have not been updated in 18
months. As of August 2024 it is possible for a LTS to still be needed
in tests over a year after it was last updated, although it is
unlikely that such a lack of activity happens, there is no reason to
remove the test release before that.
* specify the version targeted by the pull request. The end-to-end
tests previously compiled all known branches which was a waste. The
pull request now must specify which version it is targeting so that
only this version is recompiled and used for testing.
* when building the daily releases, use the release from the
integration organization to ensure the tests are run against the
latest build. Clarify in a comment why the lookup order of
organizations is reversed in this particular case.
Refs: https://code.forgejo.org/forgejo/end-to-end/pulls/239
