davrot/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2025-07-25 02:00:07 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
25981 commits 9 branches 12 tags 216 MiB
Commit graph

110 commits

Author SHA1 Message Date
Jakob Ackermann
a530cca2c5 [history-v1] increase timeout for preparing test environment (#24345) 
GitOrigin-RevId: a11a16f07da61603218ff14bef32172d4f32c469
 2025-03-18 09:05:53 +00:00
Brian Gough
3f7c88108c Merge pull request #24275 from overleaf/bg-fix-pending-change-timestamp 
fix pending change timestamp

GitOrigin-RevId: 9297a4b57ea718e6a2e1ca62388919c62911af6c
 2025-03-18 09:05:08 +00:00
Andrew Rumble
061d67ee4b Emit more specific errors from backupVerifier 
GitOrigin-RevId: 99475608f096be3e35fbaaf1825b99d145ea86f3
 2025-03-12 09:05:31 +00:00
Andrew Rumble
441c7a89a7 Merge pull request #24204 from overleaf/ar-jpa-add-chunk-verification 
[history-v1] add chunk verification

GitOrigin-RevId: 7208ad20872386813bb1c6946283afddb5e8b1cf
 2025-03-11 09:05:57 +00:00
Brian Gough
9ff3a45690 Merge pull request #24142 from overleaf/bg-backup-fix-blob-error-handling 
fix bug which caused errors to be ignored when backing up blobs

GitOrigin-RevId: 64674a484a6e22493cba8f8e47d4671072dc2e9a
 2025-03-07 09:06:26 +00:00
Brian Gough
981e91f012 Merge pull request #21763 from overleaf/bg-backup-script 
initial script for running backups

GitOrigin-RevId: d22c373de30738d8080d40dce10790f0bdcb9f51
 2025-02-24 09:04:32 +00:00
Brian Gough
d2738fda73 Merge pull request #23565 from overleaf/bg-fix-history-metadata-in-projects-collection 
fix history metadata in projects collection

GitOrigin-RevId: 18c821ef5966a8470b24dfa60313b09bdda9707d
 2025-02-14 09:03:33 +00:00
Jakob Ackermann
0a7b2004d2 [misc] silence logger when running tests (#22243) 
* [misc] silence logger when running tests

* [misc] re-enable logging in some tests for scripts

* [misc] make it easy to turn on verbose logging for tests

```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
 2025-02-10 09:06:02 +00:00
Jakob Ackermann
64e8d2b8b3 [history-v1] fix backup deletion for postgres projects (#23466) 
* [history-v1] fix backup deletion for postgres projects

* [history-v1] convert historyId to string on assignment

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 5e1033972745a9b72606638f56ebf2147406cc39
 2025-02-10 09:05:06 +00:00
Eric Mc Sween
ce4c8a4e47 Merge pull request #23398 from overleaf/em-log-doc-hash-mismatches 
Validate content hashes in history (log only)

GitOrigin-RevId: ed772fc4e4d0aa9e980f9693a759647bd937e13a
 2025-02-07 09:05:59 +00:00
Jakob Ackermann
668e9428f8 [history-v1] speed up tests for back-fill script (#23440) 
* [history-v1] speed up tests for back-fill script

* [history-v1] move "should process nothing on re-run" test to the end

GitOrigin-RevId: d5180cfaefca3c46c440230eb3566539de1582bd
 2025-02-07 09:05:16 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889) 
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
 2025-02-07 09:05:01 +00:00
Brian Gough
b2ff14f669 Merge pull request #23375 from overleaf/bg-add-id-checks-for-chunks 
guard against non-postgres projectIds

GitOrigin-RevId: eab2024e4e893591f4b1c6a507b26d935273ae5f
 2025-02-05 09:06:54 +00:00
Jakob Ackermann
3a4c5a0d0f [history-v1] add readOnly lookup for raw chunks (#23318) 
* [history-v1] add readOnly lookup for raw chunks

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [history-v1] reduce min poolsize for readOnly pool to 0

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a711c4ee4f3ea3775bd090e620d1ef52689fa1f4
 2025-02-04 09:04:52 +00:00
Jakob Ackermann
c6c623da78 [project-history] script for fixing-up files/metadata with bulk resync (#23184) 
* [history-v1] add cheap endpoint for checking time of last history write

The /raw endpoint skips the GCS lookup for the chunk.

* [project-history] script for fixing-up files/metadata with bulk resync

* [project-history] upgrade structure only resync when full sync is needed

* [project-history] start resync and process resync updates under lock

* [project-history] stop retrying during graceful shutdown

GitOrigin-RevId: 73184d5786e1d40f5b7e21f387fc37cf43f0ac2d
 2025-02-03 09:05:43 +00:00
Jakob Ackermann
9c377b74c1 [history-v1] add flag for back-filling hashes for projectIds from file (#23005) 
* [history-v1] delay process exit to give logging time to flush

* [history-v1] add flag for back-filling hashes for projectIds from file

GitOrigin-RevId: 887a1e1c72d6f5a13bfc8d0e54023afbf5bc671c
 2025-01-22 09:05:21 +00:00
Jakob Ackermann
6884338a34 [history-v1] fix remaining files with missing hashes (#22985) 
GitOrigin-RevId: df382732954538f9db177180aee0a44c5d5679ca
 2025-01-22 09:05:16 +00:00
Jakob Ackermann
ab4d1e0986 Merge pull request #22870 from overleaf/jpa-back-fill-fix-up 
[history-v1] add script for fixing up back-fill errors

GitOrigin-RevId: 118992a32c1f6da4289cd35399ddd07a741da4ee
 2025-01-20 09:05:14 +00:00
Jakob Ackermann
98e0ed6a06 Merge pull request #22894 from overleaf/jpa-large-chunk 
[history-v1] add test case for large text file

GitOrigin-RevId: 5d7cc37d74c67e1927785ea9c544d897491e6abd
 2025-01-16 09:06:02 +00:00
Jakob Ackermann
70fd6cacbc Merge pull request #22711 from overleaf/jpa-gzip 
[history-v1] compress blobs before sending them to AWS

GitOrigin-RevId: 1ca1dda6f36738fbabbf00fdab62b86230b9e4f9
 2025-01-08 09:04:57 +00:00
Jakob Ackermann
8171ffafc5 Merge pull request #22710 from overleaf/jpa-fix-tests 
[history-v1] fix tests

GitOrigin-RevId: dd16d078a4bfea108e9035c0a83f4fd5c3dd715b
 2025-01-08 09:04:52 +00:00
Andrew Rumble
31e8a908ee Merge pull request #22334 from overleaf/ar-guard-against-integer-like-strings-when-working-with-postgres 
[history-v1] Guard against non-postgres projectIds

GitOrigin-RevId: 5bf75c67424297f52f2abd9d0f0f14a0f79f8921
 2024-12-13 09:04:59 +00:00
Andrew Rumble
a92a37bc3c Merge pull request #22466 from overleaf/ar-backup-files-when-inserting 
[history-v1] backup files when inserting

GitOrigin-RevId: e636bce178604978c6d41c083bf671795d20b5a1
 2024-12-13 09:04:54 +00:00
Andrew Rumble
52254b5695 Merge pull request #22459 from overleaf/revert-22392-ar-backup-files-when-inserting 
Revert "[history-v1] backup files when inserting"

GitOrigin-RevId: f21d49dbc8909ab93bdde78c321672124bb13697
 2024-12-12 09:05:27 +00:00
Andrew Rumble
6404e3047d Merge pull request #22392 from overleaf/ar-backup-files-when-inserting 
[history-v1] backup files when inserting

GitOrigin-RevId: 1649b2828899d67ee37c0ac331917c6d5424c803
 2024-12-12 09:05:11 +00:00
Brian Gough
104ae341b1 Merge pull request #22327 from overleaf/bg-fix-copy-blob 
fix bug that prevents copying blobs between different backends in history-v1

GitOrigin-RevId: 41140ad42d0d7c1beda83e588649127c22603dec
 2024-12-05 09:05:22 +00:00
Jakob Ackermann
ce0d5fd383 Merge pull request #22177 from overleaf/jpa-file-view-hash-1 
[web] migrate file-view to download from history-v1 (via web) 1/2

GitOrigin-RevId: b787e90c57af5e2704b06ba63502aa6fc09ea1df
 2024-11-28 09:06:33 +00:00
Brian Gough
be90a3b2bb Merge pull request #22170 from overleaf/bg-history-v1-copy-blob 
add copyBlob support to history-v1

GitOrigin-RevId: 797ea66c37ca938fc906c4dff7bb1c8bf14c031e
 2024-11-28 09:05:30 +00:00
Jakob Ackermann
3d7254b419 Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal 
[history-v1] backup-verifier-app: initial revision

GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
 2024-11-27 09:04:55 +00:00
Jakob Ackermann
3d0f5188ae Merge pull request #22143 from overleaf/jpa-extra-test 
[history-v1] back_fill_file_hash: add extra test

GitOrigin-RevId: dddc6fdf4a6a7608a8a096537ebf47fe360709a8
 2024-11-27 09:04:34 +00:00
Jakob Ackermann
40603e0561 Merge pull request #22134 from overleaf/jpa-broken-file-trees 
[history-v1] back_fill_file_hash: gracefully handle broken file-trees

GitOrigin-RevId: 463c785e98581364b107f3262951e7fa0fb88a0f
 2024-11-26 09:05:51 +00:00
Jakob Ackermann
b7d37b434a Merge pull request #22123 from overleaf/jpa-process-hashed 
[history-v1] back_fill_file_hash: optionally process hashed files

GitOrigin-RevId: 53ebaa7b03166a6b1aacc0f985bb295c9ec04804
 2024-11-26 09:05:47 +00:00
Jakob Ackermann
0bd8729cc1 Merge pull request #22115 from overleaf/jpa-skip-get 
[history-v1] back_fill_file_hash: optimize obtaining the DEK

GitOrigin-RevId: 6fc5218737551ae4e6152414c32560f545767091
 2024-11-26 09:04:50 +00:00
Jakob Ackermann
838ae23b52 Merge pull request #22091 from overleaf/jpa-bad-file-trees 
[history-v1] back_fill_file_hash: gracefully handle bad file-trees

GitOrigin-RevId: 0419c06b1ccf827f4c6c5170978a38659435f26f
 2024-11-25 09:05:02 +00:00
Jakob Ackermann
c2b876372b Merge pull request #22070 from overleaf/jpa-args 
[history-v1] refactor env vars to args for GNU-parallel usage

GitOrigin-RevId: 1ebfa3dfc25d36f2f86c22fa22e4864d55b511b2
 2024-11-22 09:06:11 +00:00
Jakob Ackermann
69021ca4b0 Merge pull request #22061 from overleaf/jpa-reduce-idle-time 
[history-v1] back_fill_file_hash: reduce idle time between batches

GitOrigin-RevId: 9b24bb882b158f33915d2e0ab2f82273eac09739
 2024-11-22 09:05:55 +00:00
Jakob Ackermann
4f25b0de91 Merge pull request #22028 from overleaf/jpa-logging 
[misc] logging tweaks

GitOrigin-RevId: 7757b81b6c684955374038376035dc2bd272c4e8
 2024-11-21 09:05:41 +00:00
Jakob Ackermann
44e0428496 Merge pull request #22019 from overleaf/jpa-cleanup 
[history-v1] back_fill_file_hash: cleanup BUFFER_DIR when done

GitOrigin-RevId: 6413292bbe8cca470919b740d7e2b45fc5e7d824
 2024-11-21 09:05:05 +00:00
Jakob Ackermann
2e630e50dc Merge pull request #22017 from overleaf/jpa-process-all-projects 
[history-v1] back_fill_file_hash: process all projects

GitOrigin-RevId: 41c3cd59022bbac09552684ef2a99c58f2235ac3
 2024-11-21 09:04:50 +00:00
Jakob Ackermann
1fca37af61 Merge pull request #22016 from overleaf/jpa-fix-flaky-test 
[history-v1] back_fill_file_hash: fix typo in tests and add comments

GitOrigin-RevId: a4055d7dd33ba584a2a86053470585f412392305
 2024-11-21 09:04:46 +00:00
Andrew Rumble
adecf0d5fe Merge pull request #21998 from overleaf/ar-fix-flaky-project-archive-test 
[history-v1] Handle race condition in project archive test

GitOrigin-RevId: c32f4646f08223da0ed42068fa3fb613a6595b95
 2024-11-21 09:03:35 +00:00
Jakob Ackermann
0253130c36 Merge pull request #21972 from overleaf/jpa-get-project-blobs-batch 
[history-v1] implement getProjectBlobsBatch

GitOrigin-RevId: f03dcc690ef63f72400ccf001c6e497bd4fbe790
 2024-11-20 09:05:34 +00:00
Jakob Ackermann
24f2388aa2 Merge pull request #21948 from overleaf/bg-jpa-back-fill-project-blobs 
[history-v1] back_fill_file_hash: process blobs

GitOrigin-RevId: e54d0f8ab537ce43a12f9c972ba2ee82836073c8
 2024-11-20 09:05:04 +00:00
Jakob Ackermann
fb36fff63d Merge pull request #21931 from overleaf/bg-get-all-blobs-for-project 
add getProjectBlobs method to retrieve metadata for all blobs in a project

GitOrigin-RevId: 38f504a4fb56cd8ef8beaff1d8917ead26e85f5a
 2024-11-20 09:04:56 +00:00
Jakob Ackermann
8e74d3c58c Merge pull request #21947 from overleaf/bg-jpa-back-fill-script-tweaks 
[history-v1] back_fill_file_hash: performance tweaks

GitOrigin-RevId: c3d0c7906707fc902addcde64eaf41c24ceeece7
 2024-11-20 09:04:51 +00:00
Jakob Ackermann
27076c50cc Merge pull request #21670 from overleaf/jpa-mongo-backend-types 
[history-v1] add types to mongo BlobStore backend

GitOrigin-RevId: 7d91074eaa781904f7f3b56390aacee1800a7f67
 2024-11-19 09:05:23 +00:00
Jakob Ackermann
ca0a46b5bb Merge pull request #21928 from overleaf/jpa-handle-already-hard-deleted 
[history-v1] backup-deletion-app: use deletedProjectOverleafHistoryId

GitOrigin-RevId: 169ba0fba71c42b0415e5fa40424547b054dd5b0
 2024-11-18 09:06:13 +00:00
Jakob Ackermann
9cc6f2a9d5 Merge pull request #21829 from overleaf/jpa-backup-deletion 
[history-v1] backup-deletion-app: initial revision

GitOrigin-RevId: 9e05c0e16595cb8d180861ef503c7b4522110d39
 2024-11-18 09:05:44 +00:00
Jakob Ackermann
97b39ef98f Merge pull request #21681 from overleaf/jpa-back-fill-file-hash 
[history-v1] add script for back filling hash into project file-tree

GitOrigin-RevId: 70ab8c72f3eea1062708e4471b4ad1c60317ad26
 2024-11-18 09:05:39 +00:00
Jakob Ackermann
61c7c728ad Merge pull request #21755 from overleaf/jpa-history-initialize-project 
[history-v1] tests: use createEmptyProject helper consistently

GitOrigin-RevId: 80d979091d80c53c780919700561849d9703368e
 2024-11-12 09:05:46 +00:00