davrot/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2025-08-01 14:00:23 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
25409 commits 9 branches 12 tags 216 MiB
Commit graph

84 commits

Author SHA1 Message Date
Eric Mc Sween
c18b3f95b2 Merge pull request #25492 from overleaf/em-paginate-changes-1 
Paginate history changes endpoint

GitOrigin-RevId: 2b48044d64244404efcd2e090b682c1f571a5567
 2025-05-22 08:08:09 +00:00
Miguel Serrano
85533a36e9 [history-v1] Disable backups on CE/SP (#25591) 
Disables backup when `backupStore` is not present,
as it's the case for CE/SP

GitOrigin-RevId: a920f041c639e599084fa97d2ef2643a01da70e3
 2025-05-19 12:26:30 +00:00
Eric Mc Sween
732b1d146e Merge pull request #25456 from overleaf/em-concurrency-handling 
Add consistency constraints to the chunk store and Redis buffer

GitOrigin-RevId: 6f983ff207a13d204645e343290c94443dc537b0
 2025-05-13 08:07:36 +00:00
Eric Mc Sween
391fca9e83 Merge pull request #25361 from overleaf/em-load-latest-raw 
Rename loadLatestRaw() to getLatestChunkMetadata()

GitOrigin-RevId: e089dcfa57cbbc43df8666b51eca0d81d595a5a7
 2025-05-09 08:06:14 +00:00
Eric Mc Sween
5717ea7f5c Merge pull request #25306 from overleaf/em-redis-buffer-read-operations 
Add changes from Redis when reading chunks from the chunk store

GitOrigin-RevId: c0ebf0669b91eb2efc5d1091d025e81efdff9fe4
 2025-05-09 08:06:09 +00:00
Brian Gough
6eada92966 Merge pull request #25406 from overleaf/em-content-hash-validation-resync 
Resync project when content hash validation fails

GitOrigin-RevId: ea9b5a78f89c55276fd67835bc262717bc778e92
 2025-05-09 08:05:12 +00:00
Brian Gough
ec91c120b1 Merge pull request #25284 from overleaf/em-queue-changes-verification 
Exercise the Redis buffer when persisting changes

GitOrigin-RevId: a649b9808b6472e7c5dd9c8bfa6e3c98fb6ef4d4
 2025-05-08 08:06:22 +00:00
Brian Gough
f9b36cd5be Merge pull request #25241 from overleaf/bg-remove-existing-chunk-buffer 
remove existing chunk redis backend and chunk buffer

GitOrigin-RevId: 28fb02d1802312de6892e2fb7dd59191e3fc8914
 2025-05-08 08:05:09 +00:00
Brian Gough
19a804d5bf Merge pull request #25147 from overleaf/bg-history-buffer-use-persist-time 
use persist time in history buffer

GitOrigin-RevId: 881c42f86c6cd3cc2ea8373af4371ccc1a89e9ed
 2025-05-01 08:05:21 +00:00
Brian Gough
850da34778 Merge pull request #25086 from overleaf/bg-history-buffer-use-expire-time 
add expire time to redis buffer in history-v1

GitOrigin-RevId: 3d74957c341e62e536dc60869a7ca71ac173e380
 2025-04-29 08:05:14 +00:00
Brian Gough
ed9844b2ec Merge pull request #25088 from overleaf/bg-increase-verify-blob-timeout 
increase script timeout to avoid flaky tests in backupVerifier

GitOrigin-RevId: 4a52178b0e03ca9a69f82f7ddfe866ce689592b7
 2025-04-25 08:05:53 +00:00
Brian Gough
612981bedb Merge pull request #24993 from overleaf/bg-history-buffer-use-cache-in-persist-changes 
use chunkBuffer in persistChanges

GitOrigin-RevId: dd4cdf39ba53c3becf306119fed7eacfe67de15d
 2025-04-25 08:05:12 +00:00
Brian Gough
a7466a7291 Merge pull request #24966 from overleaf/bg-history-buffer-optimised-get 
add getCurrentChunkIfValid function

GitOrigin-RevId: e947a99ac928b58048a87cea0be1da34fcf3a9f8
 2025-04-25 08:05:04 +00:00
Brian Gough
12b96e40a5 Merge pull request #24977 from overleaf/bg-history-buffer-improve-tests 
add more tests for chunk buffer in history-v1

GitOrigin-RevId: 3cfa2492efd67597a2782ca7a5671889a67049d5
 2025-04-24 08:05:53 +00:00
Eric Mc Sween
315bde6f1b Merge pull request #24968 from overleaf/em-chunks-concurrency-postgres 
Handle concurrency during chunk extension in the Postgres backend

GitOrigin-RevId: fd706b73deacf141cbd478d3ed47f298e6c6db72
 2025-04-18 08:05:37 +00:00
Andrew Rumble
cde7ff5d2f Don't run the verifier loop when app created from test 
GitOrigin-RevId: e8a565ae00019de66cbbaf961e0ee3ace90f800e
 2025-04-17 08:06:11 +00:00
Andrew Rumble
5e76a97bc4 Log more information for failing tests 
GitOrigin-RevId: 22ae2151b476b4f810ab9223b42f9cb6fe9d7442
 2025-04-17 08:05:40 +00:00
Eric Mc Sween
aa367bcd1d Merge pull request #24897 from overleaf/em-chunks-concurrency 
Concurrency handling for history chunks with Mongo backend

GitOrigin-RevId: 30abe11237c80e7803c8934a20a57a7223afa85a
 2025-04-17 08:05:36 +00:00
Andrew Rumble
a1a3019d1e Debugging flakey test 
GitOrigin-RevId: 0da9b4ee107162aed88f2f782e80a6325fb48622
 2025-04-17 08:04:51 +00:00
Brian Gough
cedc96bdd7 Merge pull request #24906 from overleaf/bg-history-redis-read-cache 
implement read cache for history-v1 chunks

GitOrigin-RevId: 128de7e9380fd489f68d5045d3333a27018845c2
 2025-04-16 08:06:18 +00:00
Eric Mc Sween
397a546095 Merge pull request #24895 from overleaf/em-fix-history-changes 
Properly serialize changes returned by the history changes endpoint

GitOrigin-RevId: a85fd8aede9316100d2cec901c1ab9b7d1faa9bb
 2025-04-15 08:06:19 +00:00
Brian Gough
835e14b8b2 Merge pull request #24768 from overleaf/bg-history-redis-buffer 
test redis caching when loading latest chunk in history-v1

GitOrigin-RevId: f0ee09e5e9e1d7605e228913cb8539be4134e1f7
 2025-04-15 08:05:03 +00:00
Eric Mc Sween
3aa579f232 Merge pull request #24736 from overleaf/em-history-get-changes 
Endpoint for getting history changes

GitOrigin-RevId: b96afed0492522d62df9c24390f76e5490afbb44
 2025-04-11 08:05:48 +00:00
Eric Mc Sween
0e9c310d1d Merge pull request #24390 from overleaf/em-enforce-content-hash-validation 
Enforce content hash validation in history

GitOrigin-RevId: 90de21ea86ddc6548001059c41139a2af5b27060
 2025-03-24 10:50:01 +00:00
Jakob Ackermann
a530cca2c5 [history-v1] increase timeout for preparing test environment (#24345) 
GitOrigin-RevId: a11a16f07da61603218ff14bef32172d4f32c469
 2025-03-18 09:05:53 +00:00
Brian Gough
3f7c88108c Merge pull request #24275 from overleaf/bg-fix-pending-change-timestamp 
fix pending change timestamp

GitOrigin-RevId: 9297a4b57ea718e6a2e1ca62388919c62911af6c
 2025-03-18 09:05:08 +00:00
Andrew Rumble
061d67ee4b Emit more specific errors from backupVerifier 
GitOrigin-RevId: 99475608f096be3e35fbaaf1825b99d145ea86f3
 2025-03-12 09:05:31 +00:00
Andrew Rumble
441c7a89a7 Merge pull request #24204 from overleaf/ar-jpa-add-chunk-verification 
[history-v1] add chunk verification

GitOrigin-RevId: 7208ad20872386813bb1c6946283afddb5e8b1cf
 2025-03-11 09:05:57 +00:00
Brian Gough
9ff3a45690 Merge pull request #24142 from overleaf/bg-backup-fix-blob-error-handling 
fix bug which caused errors to be ignored when backing up blobs

GitOrigin-RevId: 64674a484a6e22493cba8f8e47d4671072dc2e9a
 2025-03-07 09:06:26 +00:00
Brian Gough
981e91f012 Merge pull request #21763 from overleaf/bg-backup-script 
initial script for running backups

GitOrigin-RevId: d22c373de30738d8080d40dce10790f0bdcb9f51
 2025-02-24 09:04:32 +00:00
Brian Gough
d2738fda73 Merge pull request #23565 from overleaf/bg-fix-history-metadata-in-projects-collection 
fix history metadata in projects collection

GitOrigin-RevId: 18c821ef5966a8470b24dfa60313b09bdda9707d
 2025-02-14 09:03:33 +00:00
Jakob Ackermann
0a7b2004d2 [misc] silence logger when running tests (#22243) 
* [misc] silence logger when running tests

* [misc] re-enable logging in some tests for scripts

* [misc] make it easy to turn on verbose logging for tests

```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
 2025-02-10 09:06:02 +00:00
Jakob Ackermann
64e8d2b8b3 [history-v1] fix backup deletion for postgres projects (#23466) 
* [history-v1] fix backup deletion for postgres projects

* [history-v1] convert historyId to string on assignment

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 5e1033972745a9b72606638f56ebf2147406cc39
 2025-02-10 09:05:06 +00:00
Eric Mc Sween
ce4c8a4e47 Merge pull request #23398 from overleaf/em-log-doc-hash-mismatches 
Validate content hashes in history (log only)

GitOrigin-RevId: ed772fc4e4d0aa9e980f9693a759647bd937e13a
 2025-02-07 09:05:59 +00:00
Jakob Ackermann
668e9428f8 [history-v1] speed up tests for back-fill script (#23440) 
* [history-v1] speed up tests for back-fill script

* [history-v1] move "should process nothing on re-run" test to the end

GitOrigin-RevId: d5180cfaefca3c46c440230eb3566539de1582bd
 2025-02-07 09:05:16 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889) 
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
 2025-02-07 09:05:01 +00:00
Brian Gough
b2ff14f669 Merge pull request #23375 from overleaf/bg-add-id-checks-for-chunks 
guard against non-postgres projectIds

GitOrigin-RevId: eab2024e4e893591f4b1c6a507b26d935273ae5f
 2025-02-05 09:06:54 +00:00
Jakob Ackermann
3a4c5a0d0f [history-v1] add readOnly lookup for raw chunks (#23318) 
* [history-v1] add readOnly lookup for raw chunks

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [history-v1] reduce min poolsize for readOnly pool to 0

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a711c4ee4f3ea3775bd090e620d1ef52689fa1f4
 2025-02-04 09:04:52 +00:00
Jakob Ackermann
c6c623da78 [project-history] script for fixing-up files/metadata with bulk resync (#23184) 
* [history-v1] add cheap endpoint for checking time of last history write

The /raw endpoint skips the GCS lookup for the chunk.

* [project-history] script for fixing-up files/metadata with bulk resync

* [project-history] upgrade structure only resync when full sync is needed

* [project-history] start resync and process resync updates under lock

* [project-history] stop retrying during graceful shutdown

GitOrigin-RevId: 73184d5786e1d40f5b7e21f387fc37cf43f0ac2d
 2025-02-03 09:05:43 +00:00
Jakob Ackermann
9c377b74c1 [history-v1] add flag for back-filling hashes for projectIds from file (#23005) 
* [history-v1] delay process exit to give logging time to flush

* [history-v1] add flag for back-filling hashes for projectIds from file

GitOrigin-RevId: 887a1e1c72d6f5a13bfc8d0e54023afbf5bc671c
 2025-01-22 09:05:21 +00:00
Jakob Ackermann
6884338a34 [history-v1] fix remaining files with missing hashes (#22985) 
GitOrigin-RevId: df382732954538f9db177180aee0a44c5d5679ca
 2025-01-22 09:05:16 +00:00
Jakob Ackermann
ab4d1e0986 Merge pull request #22870 from overleaf/jpa-back-fill-fix-up 
[history-v1] add script for fixing up back-fill errors

GitOrigin-RevId: 118992a32c1f6da4289cd35399ddd07a741da4ee
 2025-01-20 09:05:14 +00:00
Jakob Ackermann
98e0ed6a06 Merge pull request #22894 from overleaf/jpa-large-chunk 
[history-v1] add test case for large text file

GitOrigin-RevId: 5d7cc37d74c67e1927785ea9c544d897491e6abd
 2025-01-16 09:06:02 +00:00
Jakob Ackermann
70fd6cacbc Merge pull request #22711 from overleaf/jpa-gzip 
[history-v1] compress blobs before sending them to AWS

GitOrigin-RevId: 1ca1dda6f36738fbabbf00fdab62b86230b9e4f9
 2025-01-08 09:04:57 +00:00
Jakob Ackermann
8171ffafc5 Merge pull request #22710 from overleaf/jpa-fix-tests 
[history-v1] fix tests

GitOrigin-RevId: dd16d078a4bfea108e9035c0a83f4fd5c3dd715b
 2025-01-08 09:04:52 +00:00
Andrew Rumble
31e8a908ee Merge pull request #22334 from overleaf/ar-guard-against-integer-like-strings-when-working-with-postgres 
[history-v1] Guard against non-postgres projectIds

GitOrigin-RevId: 5bf75c67424297f52f2abd9d0f0f14a0f79f8921
 2024-12-13 09:04:59 +00:00
Andrew Rumble
a92a37bc3c Merge pull request #22466 from overleaf/ar-backup-files-when-inserting 
[history-v1] backup files when inserting

GitOrigin-RevId: e636bce178604978c6d41c083bf671795d20b5a1
 2024-12-13 09:04:54 +00:00
Andrew Rumble
52254b5695 Merge pull request #22459 from overleaf/revert-22392-ar-backup-files-when-inserting 
Revert "[history-v1] backup files when inserting"

GitOrigin-RevId: f21d49dbc8909ab93bdde78c321672124bb13697
 2024-12-12 09:05:27 +00:00
Andrew Rumble
6404e3047d Merge pull request #22392 from overleaf/ar-backup-files-when-inserting 
[history-v1] backup files when inserting

GitOrigin-RevId: 1649b2828899d67ee37c0ac331917c6d5424c803
 2024-12-12 09:05:11 +00:00
Brian Gough
104ae341b1 Merge pull request #22327 from overleaf/bg-fix-copy-blob 
fix bug that prevents copying blobs between different backends in history-v1

GitOrigin-RevId: 41140ad42d0d7c1beda83e588649127c22603dec
 2024-12-05 09:05:22 +00:00