davrot/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2025-07-23 14:00:08 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
25981 commits 9 branches 11 tags 216 MiB
Commit graph

204 commits

Author SHA1 Message Date
Miguel Serrano
85533a36e9 [history-v1] Disable backups on CE/SP (#25591) 
Disables backup when `backupStore` is not present,
as it's the case for CE/SP

GitOrigin-RevId: a920f041c639e599084fa97d2ef2643a01da70e3
 2025-05-19 12:26:30 +00:00
Eric Mc Sween
732b1d146e Merge pull request #25456 from overleaf/em-concurrency-handling 
Add consistency constraints to the chunk store and Redis buffer

GitOrigin-RevId: 6f983ff207a13d204645e343290c94443dc537b0
 2025-05-13 08:07:36 +00:00
Brian Gough
0d70223a48 Merge pull request #25482 from overleaf/bg-fix-bull-exporter-errors 
retain completed and failed jobs for backup queue

GitOrigin-RevId: 7831ce2565dc493e3ce7f55001207daea2140575
 2025-05-13 08:05:40 +00:00
Brian Gough
9762cf95e3 Merge pull request #25463 from overleaf/bg-history-redis-fix-logging-in-expire-worker 
reduce expire_redis_chunks log verbosity in production

GitOrigin-RevId: afcf6edc7154d49714bc60c276c129d70eaa49c7
 2025-05-12 08:05:44 +00:00
Brian Gough
9e07549ecb Merge pull request #25449 from overleaf/bg-histoy-redis-refactor-expire-worker 
refactor the expire worker to make it easier to extend

GitOrigin-RevId: 7b277b243ed51ab3b46316d98b7157af95a9e42b
 2025-05-12 08:05:19 +00:00
Eric Mc Sween
391fca9e83 Merge pull request #25361 from overleaf/em-load-latest-raw 
Rename loadLatestRaw() to getLatestChunkMetadata()

GitOrigin-RevId: e089dcfa57cbbc43df8666b51eca0d81d595a5a7
 2025-05-09 08:06:14 +00:00
Eric Mc Sween
5717ea7f5c Merge pull request #25306 from overleaf/em-redis-buffer-read-operations 
Add changes from Redis when reading chunks from the chunk store

GitOrigin-RevId: c0ebf0669b91eb2efc5d1091d025e81efdff9fe4
 2025-05-09 08:06:09 +00:00
Brian Gough
dc73a18ca4 Merge pull request #25432 from overleaf/em-redis-buffer-strict-apply 
Use strict validation for the Redis buffer

GitOrigin-RevId: 43e73af5deabbf3de9f5eed14f062acc5fa35e36
 2025-05-09 08:05:50 +00:00
Brian Gough
9cf284aefa Merge pull request #25414 from overleaf/bg-history-only-log-content-mismatch-once-per-request 
log warning for content hash mismatch only on first occurrence

GitOrigin-RevId: ff09f8c262461488bd564ea0644d414bb32ff17e
 2025-05-09 08:05:46 +00:00
Brian Gough
6eada92966 Merge pull request #25406 from overleaf/em-content-hash-validation-resync 
Resync project when content hash validation fails

GitOrigin-RevId: ea9b5a78f89c55276fd67835bc262717bc778e92
 2025-05-09 08:05:12 +00:00
Brian Gough
ad94c29659 Merge pull request #25391 from overleaf/em-queue-changes-verification-rollout-stage-2 
queue changes verification rollout stage 2

GitOrigin-RevId: c79a5a252c6fc8caf6fd164a31e6e360b6fc3e73
 2025-05-08 08:06:27 +00:00
Brian Gough
ec91c120b1 Merge pull request #25284 from overleaf/em-queue-changes-verification 
Exercise the Redis buffer when persisting changes

GitOrigin-RevId: a649b9808b6472e7c5dd9c8bfa6e3c98fb6ef4d4
 2025-05-08 08:06:22 +00:00
Brian Gough
f9b36cd5be Merge pull request #25241 from overleaf/bg-remove-existing-chunk-buffer 
remove existing chunk redis backend and chunk buffer

GitOrigin-RevId: 28fb02d1802312de6892e2fb7dd59191e3fc8914
 2025-05-08 08:05:09 +00:00
Brian Gough
4464320757 Merge pull request #25224 from overleaf/em-disable-chunk-buffer 
Bypass chunk buffer when loading the latest chunk

GitOrigin-RevId: 98a15b496b0d52802f9b61cefb60a7b8df653fb2
 2025-05-02 08:05:17 +00:00
Brian Gough
19a804d5bf Merge pull request #25147 from overleaf/bg-history-buffer-use-persist-time 
use persist time in history buffer

GitOrigin-RevId: 881c42f86c6cd3cc2ea8373af4371ccc1a89e9ed
 2025-05-01 08:05:21 +00:00
Brian Gough
850da34778 Merge pull request #25086 from overleaf/bg-history-buffer-use-expire-time 
add expire time to redis buffer in history-v1

GitOrigin-RevId: 3d74957c341e62e536dc60869a7ca71ac173e380
 2025-04-29 08:05:14 +00:00
Brian Gough
612981bedb Merge pull request #24993 from overleaf/bg-history-buffer-use-cache-in-persist-changes 
use chunkBuffer in persistChanges

GitOrigin-RevId: dd4cdf39ba53c3becf306119fed7eacfe67de15d
 2025-04-25 08:05:12 +00:00
Brian Gough
a7466a7291 Merge pull request #24966 from overleaf/bg-history-buffer-optimised-get 
add getCurrentChunkIfValid function

GitOrigin-RevId: e947a99ac928b58048a87cea0be1da34fcf3a9f8
 2025-04-25 08:05:04 +00:00
Brian Gough
626416ed02 Merge pull request #24945 from overleaf/bg-redis-buffer-stats-script 
add script for listing buffer stats from redis

GitOrigin-RevId: 7836563d51a5c6ded264d0e709d5cfcda70596e3
 2025-04-24 08:05:49 +00:00
Eric Mc Sween
315bde6f1b Merge pull request #24968 from overleaf/em-chunks-concurrency-postgres 
Handle concurrency during chunk extension in the Postgres backend

GitOrigin-RevId: fd706b73deacf141cbd478d3ed47f298e6c6db72
 2025-04-18 08:05:37 +00:00
Eric Mc Sween
aa367bcd1d Merge pull request #24897 from overleaf/em-chunks-concurrency 
Concurrency handling for history chunks with Mongo backend

GitOrigin-RevId: 30abe11237c80e7803c8934a20a57a7223afa85a
 2025-04-17 08:05:36 +00:00
Brian Gough
cedc96bdd7 Merge pull request #24906 from overleaf/bg-history-redis-read-cache 
implement read cache for history-v1 chunks

GitOrigin-RevId: 128de7e9380fd489f68d5045d3333a27018845c2
 2025-04-16 08:06:18 +00:00
Brian Gough
d6c2188f2d Merge pull request #24903 from overleaf/bg-fix-backup-scheduler 
remove history.id check from processPendingProjects function

GitOrigin-RevId: 017ab7c9bf693ed4077d4619574154166af04fe3
 2025-04-16 08:05:06 +00:00
Brian Gough
62c8af2a93 Merge pull request #24856 from overleaf/bg-history-redis-buffer-tweaks 
fix error logging for chunk cache mismatches

GitOrigin-RevId: 85344c4025fdaa6ee916c5438ff38c7c49f4bce3
 2025-04-15 08:06:27 +00:00
Brian Gough
3850e97446 Merge pull request #24857 from overleaf/bg-backup-scheduler-fix-for-broken-projects 
handle broken projects in history backup

GitOrigin-RevId: a295d9d7a29715807a8172dae46e0fe3c14ecf12
 2025-04-15 08:06:23 +00:00
Brian Gough
835e14b8b2 Merge pull request #24768 from overleaf/bg-history-redis-buffer 
test redis caching when loading latest chunk in history-v1

GitOrigin-RevId: f0ee09e5e9e1d7605e228913cb8539be4134e1f7
 2025-04-15 08:05:03 +00:00
Brian Gough
4ba0e97b95 Merge pull request #24749 from overleaf/bg-history-redis-buffer 
add history redis to history-v1

GitOrigin-RevId: 70dc1aee809ad17902c93c020f3804c0f1429238
 2025-04-10 08:05:35 +00:00
Brian Gough
d85dbe429d Merge pull request #24745 from overleaf/bg-history-use-consistent-import-for-chunk-store 
use consistent import for chunk_store

GitOrigin-RevId: 427b148c53c9d0913b2cdfdc634273a1d8ece060
 2025-04-10 08:05:31 +00:00
Brian Gough
040f70471c Merge pull request #24636 from overleaf/bg-history-backup-fix-broken-projects 
add --fix option to backup script

GitOrigin-RevId: 568c9158669bb1cede0f0dd75e7507b10e8ff5a2
 2025-04-08 08:05:59 +00:00
Brian Gough
f08532dfb0 Merge pull request #24637 from overleaf/bg-history-backup-uninitialised-projects 
backup uninitialised projects

GitOrigin-RevId: 9310ef9f803decffbd674024a1ffd33d1960a2c4
 2025-04-08 08:05:54 +00:00
Andrew Rumble
dabf610764 Extract getEndDateForRPO method to utils 
This will allow sharing with other functionality.

GitOrigin-RevId: a6e11447180511cc3741ca03f4996ef7ceb45ea5
 2025-03-27 14:16:28 +00:00
Eric Mc Sween
0e9c310d1d Merge pull request #24390 from overleaf/em-enforce-content-hash-validation 
Enforce content hash validation in history

GitOrigin-RevId: 90de21ea86ddc6548001059c41139a2af5b27060
 2025-03-24 10:50:01 +00:00
Andrew Rumble
2779691cd9 Add a script for verifying a set of blobs have been backed up 
GitOrigin-RevId: 8f7af647ce0ac86ee9800cdf3d386fb78debed7a
 2025-03-24 10:48:39 +00:00
Brian Gough
0292bc418d Merge pull request #24335 from overleaf/bg-warn-old-pending-changes 
add warning for backups outside rpo threshold

GitOrigin-RevId: a8421529ae64693d860b0325961b010a132426da
 2025-03-18 09:05:49 +00:00
Brian Gough
3f7c88108c Merge pull request #24275 from overleaf/bg-fix-pending-change-timestamp 
fix pending change timestamp

GitOrigin-RevId: 9297a4b57ea718e6a2e1ca62388919c62911af6c
 2025-03-18 09:05:08 +00:00
Andrew Rumble
f134746c9c Extend script to allow multiple blobs to be processed from csv 
GitOrigin-RevId: ad47eb754436ddc7f56b27ceda627268c3a030a1
 2025-03-18 09:05:03 +00:00
Andrew Rumble
b5d6484991 Add a script to backup a single blob from a project 
GitOrigin-RevId: 464e6d69093b87891497e07d1627cd20e2285380
 2025-03-18 09:04:59 +00:00
Andrew Rumble
78481e010e Add verification looper and handle shutdown signals 
Shutdown signals become more relevant now that we are looping as we want
to gracefully stop processing records rather than continue looping.

GitOrigin-RevId: dbb499388c86d552d77954988f8fc27d140da3f1
 2025-03-18 09:04:54 +00:00
Andrew Rumble
9d72eeeeac Add new strategy to verify_sampled_projects 
GitOrigin-RevId: d967da41250bb5945d5b8668b212d4a61b4f9d69
 2025-03-18 09:04:50 +00:00
Brian Gough
2d48c86e61 Merge pull request #24269 from overleaf/bg-backup-replication-smoothing 
extend backup scheduler to delay jobs for load levelling

GitOrigin-RevId: be62c458d3be464bf80ca590fcc107c2760aa061
 2025-03-13 09:06:28 +00:00
Brian Gough
a178c0f400 Merge pull request #24268 from overleaf/bg-backup-replication-lag-metrics 
add backup replication metrics

GitOrigin-RevId: a734435e3c6ce56350b2286bd218a5e2324d93a9
 2025-03-13 09:06:24 +00:00
Brian Gough
cf105cf01d Merge pull request #24267 from overleaf/bg-fix-init-option 
update backup script to use index when finding uninitialised projects

GitOrigin-RevId: 04ce0654b3d42de1c1a9bb542482c2dd53540628
 2025-03-13 09:06:20 +00:00
Brian Gough
3b93efdf5c Merge pull request #24256 from overleaf/bg-backup-fix-sample 
fix backup sample script

GitOrigin-RevId: 4bf4c15324d9b3b7ebd809f7b212e52d75ab216d
 2025-03-13 09:06:16 +00:00
Andrew Rumble
273ae4aecd Split healthCheck out into separate module 
GitOrigin-RevId: 847d00b696fe6d82f4bd5fea8f9130437c68e7b2
 2025-03-13 09:04:47 +00:00
Andrew Rumble
b5f8bfa28e Switch health check to use projects instead of blobs 
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: db1a1c8ce5968e558b0754e5e0da50af89fd80db
 2025-03-13 09:04:43 +00:00
Andrew Rumble
19eefebe95 Revert "Switch health check to use projects instead of blobs" 
This reverts commit c318b70397ed5e2fcbb07fa019412b56844260ef.

GitOrigin-RevId: 087ae9d21be83bf3dae47c4e6d27eb4e74f387df
 2025-03-12 09:06:34 +00:00
Andrew Rumble
087a9daf34 Revert "Split healthCheck out into separate module" 
This reverts commit 96061812977d5c854e494cd44163b16a96722b17.

GitOrigin-RevId: f30a185b65a4f1346ed13fa0c6e9ea0852d44335
 2025-03-12 09:06:30 +00:00
Andrew Rumble
a7be1f3430 Split healthCheck out into separate module 
GitOrigin-RevId: 96061812977d5c854e494cd44163b16a96722b17
 2025-03-12 09:06:22 +00:00
Andrew Rumble
c373db4f86 Switch health check to use projects instead of blobs 
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: c318b70397ed5e2fcbb07fa019412b56844260ef
 2025-03-12 09:06:18 +00:00
Andrew Rumble
149b590413 Fix script name in usage prompt 
GitOrigin-RevId: ab4ed7479c96d3d1db82b23ef3c883706c61995e
 2025-03-12 09:05:44 +00:00