davrot/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2025-07-23 05:00:07 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
25981 commits 9 branches 11 tags 216 MiB
Commit graph

204 commits

Author SHA1 Message Date
Jakob Ackermann
81f0807fc6 [web] prepare filestore migration for Server Pro/CE (#27230) 
* [web] prepare filestore migration for Server Pro/CE

* [history-v1] remove unused USER_FILES_BUCKET_NAME env var from script

* [server-ce] tests: write default docker-compose.override.yml on startup

* [server-ce] tests: extend access logging of host-admin for response

* [server-ce] tests: test text and binary file upload

* [server-ce] tests: add tests for filestore migration

* [web] simplify feature gate for filestore/project-history-blobs logic

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [server-ce] test: fix flaky test helper

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: f89bdab2749e2b7a49d609e2eac6bf621c727966
 2025-07-22 08:06:58 +00:00
Jakob Ackermann
bf43d4f709 [history-v1] make back_fill_file_hash_fix_up compatible with Server Pro (#27280) 
* [history-v1] move MockFilestore into shared place

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [history-v1] make back_fill_file_hash_fix_up compatible with Server Pro

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 70ea57e1503031d9f14dcd60c4c110e746450587
 2025-07-22 08:06:41 +00:00
Brian Gough
5208ad39ec Merge pull request #27173 from overleaf/bg-filestore-migration-remove-backups 
remove backup code from back_fill_file_hash script and tests

GitOrigin-RevId: 364eefc47670e6e7f2314de810ea259b609ff976
 2025-07-18 08:06:14 +00:00
Brian Gough
5669a7d1c8 Merge pull request #27147 from overleaf/bg-filestore-migration-for-server-pro-II 
add support for fetching files via http from filestore in back_fill_file_hash script and tests

GitOrigin-RevId: 8dea6383ed6fe9ee6786a5695e2deee93b1cdd84
 2025-07-17 08:05:30 +00:00
Eric Mc Sween
1833bd3d00 Merge pull request #27126 from overleaf/em-fix-persist-changes 
Fix chunk creation over a one change chunk

GitOrigin-RevId: aecae334849522975b83c77224ee27db64de4ed8
 2025-07-16 08:05:40 +00:00
Brian Gough
8fab1b54a3 Merge pull request #27025 from overleaf/bg-delete-redis-buffer-when-project-deleted 
delete redis buffer when project deleted

GitOrigin-RevId: eef7b6fdeb04cb556ae47794379d83e659f89b2e
 2025-07-11 08:06:08 +00:00
Jakob Ackermann
d7a0cbefea [history-v1] fix typo in metrics name in redis chunk store (#26865) 
GitOrigin-RevId: 5b37e3b22a3250fcfbd7411e635cc11ed38dcb27
 2025-07-08 08:05:52 +00:00
Jakob Ackermann
cf472f54d0 [history-v1] use POST requests for expiring redis buffer from cron (#26568) 
* [history-v1] use POST requests for expiring redis buffer from cron

(cherry picked from commit 15780ac54e36b96e1aed9fd9eb6dfe9d4fbf842f)

* [history-v1] remove double claim of expire job

GitOrigin-RevId: 8b2eab07006a5819a47eed3f646b2a4d75f86e5b
 2025-07-08 08:05:47 +00:00
Eric Mc Sween
735cc2272f Merge pull request #26505 from overleaf/em-persist-buffer-batch 
Persist Redis buffer in batches

GitOrigin-RevId: 1de22807dae3554b3274ec103783b0868b1439d9
 2025-06-30 08:05:25 +00:00
Eric Mc Sween
a38eefd2ab Merge pull request #26559 from overleaf/em-redis-buffer-delete 
Delete project from redis buffer when expiring

GitOrigin-RevId: 66c48adc388a4a4e0b1b54b581dc945a38a41356
 2025-06-30 08:05:20 +00:00
Andrew Rumble
903277c222 Handle rangeHashes 
GitOrigin-RevId: 9b9a67d8e45f17e948d003aed33cef6128a25744
 2025-06-25 08:06:50 +00:00
Andrew Rumble
34b674aa6f Handle errors when converting lazyfiles to eagerfiles 
GitOrigin-RevId: 6ada48e2f2b7ffee9c8560856436dc5e2076099d
 2025-06-25 08:06:45 +00:00
Andrew Rumble
745043ca92 Add more detail for the modes 
GitOrigin-RevId: 44b63b76b1edb8bfb78049c9c8560934e340ef96
 2025-06-25 08:06:41 +00:00
Andrew Rumble
6ed488cc65 Change raw mode to just fetch all raw content from backup 
GitOrigin-RevId: 397060d63c824dbe688d187312dd78d1f3e4bf5d
 2025-06-25 08:06:36 +00:00
Andrew Rumble
f2b0a982ac Use BackupBlobStore to get chunk content 
GitOrigin-RevId: 91de53101ea289b43bdb08352aecc09ae89d0f74
 2025-06-25 08:06:32 +00:00
Andrew Rumble
797f29d40a Handle errors when processing command line arguments 
GitOrigin-RevId: 7c1f011e705d6e4b6fbf0afee3dc3aab749f0de4
 2025-06-25 08:06:27 +00:00
Andrew Rumble
392037efd6 Improve types and export functions we want to use 
GitOrigin-RevId: 25fd93214f88c425bee23d43de113cf754f89524
 2025-06-25 08:06:18 +00:00
Andrew Rumble
3600aa4b75 Add script for restoring projects from backup 
GitOrigin-RevId: b639d74989afa17638bbcba0f8ee34c978fcd5d9
 2025-06-25 08:06:14 +00:00
Andrew Rumble
25a911d4cb Export loadChunk from verifier to allow code reuse 
GitOrigin-RevId: fc5f2de50583ab2469e8ab3c86776c55501ba2e2
 2025-06-25 08:06:09 +00:00
Eric Mc Sween
1042092144 Merge pull request #26547 from overleaf/em-revert-expiry-post 
Revert "use POST requests for expiring redis buffer from cron"

GitOrigin-RevId: 95e9fd1be7b73699d6fac24035437d467c273d0a
 2025-06-19 08:06:58 +00:00
Eric Mc Sween
150dfd6cba Merge pull request #26539 from overleaf/jpa-post 
[history-v1] use POST requests for expiring redis buffer from cron

GitOrigin-RevId: 51c9a25b998e581ed20c0e113bd4989537a1e6ef
 2025-06-19 08:06:50 +00:00
Eric Mc Sween
fd9fd9f0e7 Merge pull request #26545 from overleaf/jpa-fix-resync 
[history-v1] use mongo projectId for initiating resync

GitOrigin-RevId: f93f2358695782fb222d23ba3720d98724b9a291
 2025-06-19 08:06:45 +00:00
Eric Mc Sween
7bdc4291fc Merge pull request #26533 from overleaf/jpa-queue 
[history-v1] use POST requests for flushing history queues

GitOrigin-RevId: ae87a046a7844b25cc123d08ac6c8d1624608394
 2025-06-19 08:06:16 +00:00
Eric Mc Sween
f68bf5a69f Merge pull request #26477 from overleaf/em-redis-buffer-resync 
Handle invalid content hash when persisting changes

GitOrigin-RevId: 5259190396c8c261cad1abcd5de66314c1e871fb
 2025-06-18 08:06:17 +00:00
Eric Mc Sween
9cb4ef4d7d Merge pull request #26353 from overleaf/bg-history-redis-extend-persist-worker 
extend persist worker to make parallel requests

GitOrigin-RevId: 8def7d5a8b5c9fcbe5fe45ac8f3ace503d31877a
 2025-06-17 08:05:55 +00:00
Brian Gough
2f44a4eb5a Merge pull request #26351 from overleaf/bg-history-redis-convert-persist-worker-to-esm 
history redis convert persist worker to esm

GitOrigin-RevId: edcbac6e3f1d3dde3fa8239378995f3ff3afcfdd
 2025-06-13 08:06:32 +00:00
Brian Gough
d189c91c59 Merge pull request #26378 from overleaf/bg-history-redis-fix-misc 
add type check to scan.js

GitOrigin-RevId: a306ec841b8801a2a5eab8c35f246180206f7231
 2025-06-13 08:06:27 +00:00
Brian Gough
eed6a982f7 Merge pull request #26377 from overleaf/bg-history-redis-remove-unwanted-parameters 
remove unwanted parameters in queueChangesFake and queueChangesFakeOnlyIfExists functions

GitOrigin-RevId: 5946cd1f81db7076eb545b5a0aca28b81fa19be4
 2025-06-13 08:06:22 +00:00
Brian Gough
2e4b57bf81 Merge pull request #26357 from overleaf/bg-history-redis-fix-history-buffer-level 
remove incorrect parameter from queueChanges call in queueChangesFake function

GitOrigin-RevId: 4567be5eb5634c0c419f7f8758f1a6f27586a9af
 2025-06-12 08:05:58 +00:00
Brian Gough
8a90173aa7 Merge pull request #26271 from overleaf/bg-history-redis-deployment-refactor 
introduce history-v1 buffering levels

GitOrigin-RevId: 7709935a5ceb19ef6c5723ded647217b7399759a
 2025-06-12 08:05:39 +00:00
Brian Gough
b290e93441 Merge pull request #26270 from overleaf/bg-history-redis-commit-change-manager 
replace redis logic in persistChanges with new commitChanges method

GitOrigin-RevId: e06f9477b9d5548fa92ef87fb6e1f4f672001a35
 2025-06-11 08:07:45 +00:00
Brian Gough
b946c2abff Merge pull request #26304 from overleaf/bg-history-redis-clear-persist-time-on-persist 
add persist time handling to setPersistedVersion method

GitOrigin-RevId: 5e115b49116ee4604e3e478c206c7e9cf147cbc8
 2025-06-11 08:07:32 +00:00
Brian Gough
fdd0d95554 Merge pull request #26293 from overleaf/bg-history-redis-fix-persist-worker 
add missing load global blobs from persist worker

GitOrigin-RevId: ae9393f2353fb4d5afe349aa7d0a26bab80c7f53
 2025-06-11 08:06:59 +00:00
Brian Gough
2a833aa23a Merge pull request #26250 from overleaf/bg-history-redis-add-return-value-to-persistBuffer 
provide return value from persistBuffer

GitOrigin-RevId: ba52ff42b91ffe9adc23ab0461fa836540735563
 2025-06-11 08:06:54 +00:00
Brian Gough
fec6dde00f Merge pull request #26203 from overleaf/bg-history-redis-fix-loadAtVersion 
Extend loadAtVersion to handle nonpersisted versions

GitOrigin-RevId: 22060605ea7bb89a8d4d61bafab8f63b94d59067
 2025-06-11 08:06:50 +00:00
Brian Gough
c81cc4055e Merge pull request #26220 from overleaf/bg-history-redis-fix-loadAtVersion-startVersion 
correct startVersion calculation in loadAtVersion

GitOrigin-RevId: b81c30dcab90b137169a4bddef3c22f44a957f68
 2025-06-11 08:06:45 +00:00
Brian Gough
2d0706591b Merge pull request #26219 from overleaf/bg-history-redis-fix-loadAtTimestamp 
correct startVersion calculation in loadAtTimestamp

GitOrigin-RevId: ad46aae47c0769943e787199d68e895cf139bb56
 2025-06-11 08:06:41 +00:00
Brian Gough
f904933d68 Merge pull request #26180 from overleaf/bg-history-redis-add-queueChanges 
add queueChanges method to history-v1

GitOrigin-RevId: fb6da79bd5ca40e7cbdcb077ad3a036cc5509ced
 2025-06-11 08:06:36 +00:00
Brian Gough
c23e84eb37 Merge pull request #26273 from overleaf/bg-history-redis-add-persist-worker-to-cron 
modify existing run-chunk-lifecycle cron job to persist and expire redis queues

GitOrigin-RevId: afb94b3e2fba7368cfec11997dfd5b2bbd6321a9
 2025-06-11 08:06:23 +00:00
Brian Gough
2eb695f4c3 Merge pull request #26122 from overleaf/bg-history-redis-make-persist-buffer-consistent 
make persistBuffer export consistent with other methods

GitOrigin-RevId: 24536e521e1d20ef63cc74bd9ba40e095025d512
 2025-06-09 08:05:39 +00:00
Brian Gough
d280f40885 Merge pull request #26116 from overleaf/bg-history-redis-show-buffer 
add script to display redis buffer for a given history ID

GitOrigin-RevId: 71c2e79480c0873d30801ed3c13aa9a7fc7873f6
 2025-06-09 08:05:35 +00:00
Brian Gough
54c0eb7fdc Merge pull request #25958 from overleaf/bg-history-redis-check-persisted-version-on-update 
prevent setPersistedVersion from setting an out of bounds version

GitOrigin-RevId: 9561b7b96399bed901db5c2ac20a0cdbf4c67395
 2025-06-04 08:07:19 +00:00
Brian Gough
393cee7af5 Merge pull request #25993 from overleaf/bg-history-refactor-persist-buffer-limits 
refactor persist buffer to add limits

GitOrigin-RevId: 4a40a7a8812acf5bb7f98bfd7b94d81ebe19fc57
 2025-06-04 08:06:19 +00:00
Brian Gough
50df3862e9 Merge pull request #25954 from overleaf/bg-history-expire-worker-fix 
fix expire_redis_chunks to only clear job on error

GitOrigin-RevId: f7ec435edda95958b453fba501686dcfd84426f7
 2025-06-04 08:06:14 +00:00
Brian Gough
a80203f748 Merge pull request #25909 from overleaf/bg-history-persist-worker 
add history persist worker

GitOrigin-RevId: b9e31e7bdd84570efc0b87b9f5e90b4078551a8c
 2025-06-04 08:06:10 +00:00
Brian Gough
cb350ecc65 Merge pull request #25907 from overleaf/bg-history-redis-persist-buffer 
add a `persistBuffer` method to history-v1

GitOrigin-RevId: 71a34e48e9ebe378e2f765f3216023e505a58a5d
 2025-06-04 08:06:06 +00:00
Brian Gough
3296fc15da Merge pull request #25905 from overleaf/bg-history-redis-fix-import-path 
fix import path for Job errors in history-v1

GitOrigin-RevId: f5f88bd34e713cd2ed78185ed4ce917e10d09caf
 2025-05-29 08:06:06 +00:00
Brian Gough
b525a80d28 Merge pull request #25470 from overleaf/bg-history-redis-downgrade-job-related-errors 
downgrade expected job errors in scanAndProcessDueItems

GitOrigin-RevId: 0a2689699bfc6512c5017c7f5e51ac4f80c409fe
 2025-05-29 08:06:01 +00:00
Jakob Ackermann
11b94593c2 [web] remove deledFiles collection (#25750) 
* [history-v1] remove processing of deleted files when back-filling hashes

* [web] remove deledFiles collection

GitOrigin-RevId: 7c080e564f7d7acb33ebe7ebe012f415a847d0df
 2025-05-23 08:05:01 +00:00
Eric Mc Sween
c18b3f95b2 Merge pull request #25492 from overleaf/em-paginate-changes-1 
Paginate history changes endpoint

GitOrigin-RevId: 2b48044d64244404efcd2e090b682c1f571a5567
 2025-05-22 08:08:09 +00:00