Brian Gough
efd53e567c
Merge pull request #24237 from overleaf/bg-fix-backup-worker
...
fixes for backup worker
GitOrigin-RevId: 79c99e5b2d99d1b0298f69d968e0e2cb89281dd6
2025-03-12 09:05:04 +00:00
Brian Gough
c233243948
Merge pull request #24200 from overleaf/bg-backup-queue-pending-jobs
...
fix backup worker and backup scheduler to handle pending projects
GitOrigin-RevId: a97e011615666b3ae2b8fafe26a96d41b3609edd
2025-03-11 09:06:05 +00:00
Andrew Rumble
441c7a89a7
Merge pull request #24204 from overleaf/ar-jpa-add-chunk-verification
...
[history-v1] add chunk verification
GitOrigin-RevId: 7208ad20872386813bb1c6946283afddb5e8b1cf
2025-03-11 09:05:57 +00:00
Brian Gough
9ff3a45690
Merge pull request #24142 from overleaf/bg-backup-fix-blob-error-handling
...
fix bug which caused errors to be ignored when backing up blobs
GitOrigin-RevId: 64674a484a6e22493cba8f8e47d4671072dc2e9a
2025-03-07 09:06:26 +00:00
Brian Gough
281168fd52
Merge pull request #24080 from overleaf/bg-serialise-backup-initialisation
...
fix backup initialization to avoid incorrect use of batchedUpdate
GitOrigin-RevId: 6984f3510c6b03b3dfda35efea8173f848e58eff
2025-03-07 09:06:22 +00:00
Brian Gough
154de1a0f9
Merge pull request #24102 from overleaf/bg-backup-stats-script
...
add backup sampling script
GitOrigin-RevId: cfd546d421cdcfb9f8ad2111703a437a7272bd54
2025-03-07 09:06:17 +00:00
Brian Gough
893294e6b8
Merge pull request #24069 from overleaf/bg-backup-errors
...
more tweaks for backup errors
GitOrigin-RevId: 0f7c7bb5004923c3c22c6e3471bb7152cc3e05e2
2025-03-05 09:05:50 +00:00
Brian Gough
34b397073f
Merge pull request #24053 from overleaf/bg-backup-queue-tweaks-II
...
additional backup queue tweaks
GitOrigin-RevId: b54e0c5f4a23bb9962c5062bd1b80f9f124c3c87
2025-03-05 09:05:15 +00:00
Brian Gough
c815106d62
Merge pull request #24037 from overleaf/bg-fix-backup-worker-errors
...
backup worker tweaks
GitOrigin-RevId: 7901bb0257c7c7320afd08ef965f124347a1de30
2025-03-04 09:05:12 +00:00
Brian Gough
1dbf5dca10
Merge pull request #24019 from overleaf/bg-backup-add-error-logging
...
add logging of backup errors in batch mode
GitOrigin-RevId: 96ae7a2354f82451b3bbe8c5459c4d4cf47e5459
2025-03-04 09:04:44 +00:00
Brian Gough
4720e8f2c5
Merge pull request #23989 from overleaf/bg-backup-worker-reduce-logging
...
reduce log level from info to debug for backup logging
GitOrigin-RevId: 10f541b0e0068aafd50710226fb8425527fee7c5
2025-03-03 09:05:22 +00:00
Brian Gough
8d0fa5be65
Merge pull request #23977 from overleaf/bg-backup-worker-load-global-blobs
...
ensure global blobs are loaded in backup worker
GitOrigin-RevId: b80791ef38e7fa8db364bdd3a9c461757778409d
2025-03-03 09:05:08 +00:00
Brian Gough
b37b409994
Merge pull request #23946 from overleaf/bg-backup-queue-implementation
...
additional backup queue implementation
GitOrigin-RevId: 03754e57a6c6798a07dcca6a5248dec61b3cdc7a
2025-02-28 09:05:50 +00:00
Brian Gough
763e5ba82c
Merge pull request #23920 from overleaf/bg-backup-queue
...
initial history backup queue worker app
GitOrigin-RevId: e9647a2ec3caeb1fff843cead12164ba89fdb1f8
2025-02-28 09:04:46 +00:00
Brian Gough
6e55a9fa81
Merge pull request #23837 from overleaf/bg-redis-redact-config
...
sanitise password in redis config output
GitOrigin-RevId: 45d8bd13f01c3c4f138549c8a9f47dc0db29d0b0
2025-02-25 09:05:22 +00:00
Brian Gough
0acc067205
Merge pull request #23836 from overleaf/bg-backup-extend-debugging-script
...
add support for fetching chunks in backup debugging script
GitOrigin-RevId: 641298088d62148693525259748dcddf4d67d14b
2025-02-25 09:05:18 +00:00
Brian Gough
9d8f9e5358
Merge pull request #23835 from overleaf/bg-backup-show-errors
...
add error cause logging for missing chunks and blobs in backup comparison
GitOrigin-RevId: 9a1bf84b64621add71fb1d1e0cb013598b226bc8
2025-02-25 09:05:14 +00:00
Brian Gough
dd8a683e9e
Merge pull request #23834 from overleaf/bg-backup-graceful-shutdown
...
add graceful shutdown handling to backup script
GitOrigin-RevId: f0fb9a3867b26c7800edefab126fd05fbabdae02
2025-02-25 09:05:10 +00:00
Andrew Rumble
b0ed2efdf9
Merge pull request #23812 from overleaf/ar-allow-debugging-for-remove-backup-blobs-script-errors
...
[history-v1] Minor improvements to AWS cleanup script
GitOrigin-RevId: bd01a72169b9cbc018612173294cfbf8649a9fbd
2025-02-24 09:06:15 +00:00
Andrew Rumble
a0d79d5871
Merge pull request #23742 from overleaf/ar-script-for-removing-blobs-in-wrong-place
...
[history-v1] Script for deleting blobs backed up to wrong location
GitOrigin-RevId: 6abe1dba5164f8fa7d41c4ee3e4a07764e73b7e1
2025-02-24 09:05:53 +00:00
Brian Gough
981e91f012
Merge pull request #21763 from overleaf/bg-backup-script
...
initial script for running backups
GitOrigin-RevId: d22c373de30738d8080d40dce10790f0bdcb9f51
2025-02-24 09:04:32 +00:00
Brian Gough
32983da29c
Merge pull request #23700 from overleaf/bg-history-redis
...
add redis to history-v1
GitOrigin-RevId: def900c4b560ea0a297b11b6c0ecc8cc8caa5b49
2025-02-21 09:04:06 +00:00
Brian Gough
8831fb9a18
Merge pull request #23169 from overleaf/bg-export-global-blobs
...
Add script to export global blobs to CSV
GitOrigin-RevId: 86c885981b150aae4541770a094c822e1fb364b9
2025-02-03 09:05:53 +00:00
Brian Gough
d6e89c7338
Merge pull request #23058 from overleaf/bg-export-backed-up-blobs
...
export backed up blobs to csv file
GitOrigin-RevId: 5a176055dc85c56ced64cbf13aa705b56071cdae
2025-01-24 09:05:31 +00:00
Jakob Ackermann
9c377b74c1
[history-v1] add flag for back-filling hashes for projectIds from file ( #23005 )
...
* [history-v1] delay process exit to give logging time to flush
* [history-v1] add flag for back-filling hashes for projectIds from file
GitOrigin-RevId: 887a1e1c72d6f5a13bfc8d0e54023afbf5bc671c
2025-01-22 09:05:21 +00:00
Jakob Ackermann
6884338a34
[history-v1] fix remaining files with missing hashes ( #22985 )
...
GitOrigin-RevId: df382732954538f9db177180aee0a44c5d5679ca
2025-01-22 09:05:16 +00:00
Jakob Ackermann
ab4d1e0986
Merge pull request #22870 from overleaf/jpa-back-fill-fix-up
...
[history-v1] add script for fixing up back-fill errors
GitOrigin-RevId: 118992a32c1f6da4289cd35399ddd07a741da4ee
2025-01-20 09:05:14 +00:00
Andrew Rumble
a92a37bc3c
Merge pull request #22466 from overleaf/ar-backup-files-when-inserting
...
[history-v1] backup files when inserting
GitOrigin-RevId: e636bce178604978c6d41c083bf671795d20b5a1
2024-12-13 09:04:54 +00:00
Jakob Ackermann
3d7254b419
Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal
...
[history-v1] backup-verifier-app: initial revision
GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
2024-11-27 09:04:55 +00:00
Jakob Ackermann
40603e0561
Merge pull request #22134 from overleaf/jpa-broken-file-trees
...
[history-v1] back_fill_file_hash: gracefully handle broken file-trees
GitOrigin-RevId: 463c785e98581364b107f3262951e7fa0fb88a0f
2024-11-26 09:05:51 +00:00
Jakob Ackermann
b7d37b434a
Merge pull request #22123 from overleaf/jpa-process-hashed
...
[history-v1] back_fill_file_hash: optionally process hashed files
GitOrigin-RevId: 53ebaa7b03166a6b1aacc0f985bb295c9ec04804
2024-11-26 09:05:47 +00:00
Jakob Ackermann
0bd8729cc1
Merge pull request #22115 from overleaf/jpa-skip-get
...
[history-v1] back_fill_file_hash: optimize obtaining the DEK
GitOrigin-RevId: 6fc5218737551ae4e6152414c32560f545767091
2024-11-26 09:04:50 +00:00
Jakob Ackermann
ed27af11f8
Merge pull request #22107 from overleaf/jpa-error-handling
...
[history-v1] back_fill_file_hash: error handling
GitOrigin-RevId: 2dbf52d015206cf42ca434b1f16221d6e0f2cc2b
2024-11-26 09:04:45 +00:00
Jakob Ackermann
1a6bd670d4
Merge pull request #22104 from overleaf/jpa-graceful-shutdown-abort-retries
...
[history-v1] back_fill_file_hash: abort retries during graceful shutdown
GitOrigin-RevId: 0d52dde32e4a7bf2293f341644b7416e669b2bdc
2024-11-25 09:05:31 +00:00
Jakob Ackermann
838ae23b52
Merge pull request #22091 from overleaf/jpa-bad-file-trees
...
[history-v1] back_fill_file_hash: gracefully handle bad file-trees
GitOrigin-RevId: 0419c06b1ccf827f4c6c5170978a38659435f26f
2024-11-25 09:05:02 +00:00
Jakob Ackermann
c2b876372b
Merge pull request #22070 from overleaf/jpa-args
...
[history-v1] refactor env vars to args for GNU-parallel usage
GitOrigin-RevId: 1ebfa3dfc25d36f2f86c22fa22e4864d55b511b2
2024-11-22 09:06:11 +00:00
Jakob Ackermann
69021ca4b0
Merge pull request #22061 from overleaf/jpa-reduce-idle-time
...
[history-v1] back_fill_file_hash: reduce idle time between batches
GitOrigin-RevId: 9b24bb882b158f33915d2e0ab2f82273eac09739
2024-11-22 09:05:55 +00:00
Jakob Ackermann
4f25b0de91
Merge pull request #22028 from overleaf/jpa-logging
...
[misc] logging tweaks
GitOrigin-RevId: 7757b81b6c684955374038376035dc2bd272c4e8
2024-11-21 09:05:41 +00:00
Jakob Ackermann
44e0428496
Merge pull request #22019 from overleaf/jpa-cleanup
...
[history-v1] back_fill_file_hash: cleanup BUFFER_DIR when done
GitOrigin-RevId: 6413292bbe8cca470919b740d7e2b45fc5e7d824
2024-11-21 09:05:05 +00:00
Jakob Ackermann
2e630e50dc
Merge pull request #22017 from overleaf/jpa-process-all-projects
...
[history-v1] back_fill_file_hash: process all projects
GitOrigin-RevId: 41c3cd59022bbac09552684ef2a99c58f2235ac3
2024-11-21 09:04:50 +00:00
Jakob Ackermann
6de1817ef5
Merge pull request #22013 from overleaf/jpa-flush-mongo-queues-early
...
[history-v1] back_fill_file_hash: flush mongo write queues early
GitOrigin-RevId: 9b6c6ff9861945e69e42d15dc19f4c03c39193e9
2024-11-21 09:04:42 +00:00
Jakob Ackermann
0253130c36
Merge pull request #21972 from overleaf/jpa-get-project-blobs-batch
...
[history-v1] implement getProjectBlobsBatch
GitOrigin-RevId: f03dcc690ef63f72400ccf001c6e497bd4fbe790
2024-11-20 09:05:34 +00:00
Jakob Ackermann
4964d6414b
Merge pull request #21982 from overleaf/jpa-cleanup
...
[history-v1] back_fill_file_hash: cleanup fs and graceful shutdown
GitOrigin-RevId: 362669ff988ad71fda713bf4896b1abcb36caf35
2024-11-20 09:05:30 +00:00
Jakob Ackermann
24f2388aa2
Merge pull request #21948 from overleaf/bg-jpa-back-fill-project-blobs
...
[history-v1] back_fill_file_hash: process blobs
GitOrigin-RevId: e54d0f8ab537ce43a12f9c972ba2ee82836073c8
2024-11-20 09:05:04 +00:00
Jakob Ackermann
8e74d3c58c
Merge pull request #21947 from overleaf/bg-jpa-back-fill-script-tweaks
...
[history-v1] back_fill_file_hash: performance tweaks
GitOrigin-RevId: c3d0c7906707fc902addcde64eaf41c24ceeece7
2024-11-20 09:04:51 +00:00
Jakob Ackermann
97b39ef98f
Merge pull request #21681 from overleaf/jpa-back-fill-file-hash
...
[history-v1] add script for back filling hash into project file-tree
GitOrigin-RevId: 70ab8c72f3eea1062708e4471b4ad1c60317ad26
2024-11-18 09:05:39 +00:00
Antoine Clausse
7f48c67512
Add prefer-node-protocol ESLint rule ( #21532 )
...
* Add `unicorn/prefer-node-protocol`
* Fix `unicorn/prefer-node-protocol` ESLint errors
* Run `npm run format:fix`
* Add sandboxed-module sourceTransformers in mocha setups
Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`
* Remove `node:` in the SandboxedModule requires
* Fix new linting errors with `node:`
GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Eric Mc Sween
b8195f537d
Merge pull request #18065 from overleaf/em-doc-versions-recovery-fix
...
Doc versions recovery for Server Pro 5.0.3
GitOrigin-RevId: 226b09383f9499c7eb1c20bf9e8e8e549bdbccb0
2024-04-25 08:04:50 +00:00
Eric Mc Sween
a2d5e030c6
Merge pull request #18041 from overleaf/em-jpa-recovery-script
...
[history-v1] add doc version recovery script
GitOrigin-RevId: 3f240f313465ce5fa9c53f72a992807f9396ebb4
2024-04-23 08:05:40 +00:00
John Lees-Miller
2d9db134a2
Merge pull request #14544 from overleaf/jlm-add-history-restore-script
...
Add project recovery script
GitOrigin-RevId: b98fb2988245d2b67f8fc0711742294e895b1f07
2023-08-31 08:04:31 +00:00
