Commit graph

792 commits

Author SHA1 Message Date
Liangjun Song
2f87db9c0d Merge pull request #24790 from overleaf/ls-use-script-runner
Update some scripts to use Script Runner

GitOrigin-RevId: aaa11f94dcfd328c158bb02d1b9fb2adfb1bb146
2025-05-23 08:05:23 +00:00
Antoine Clausse
436dcc977f Update Node to 22.15.1 (#25785)
GitOrigin-RevId: 52428d2d7e67c3135a1604fa487dd142aa08bf15
2025-05-22 08:07:42 +00:00
Alf Eaton
827fb19df7 Define Change.toRaw result as RawChange (#25676)
GitOrigin-RevId: 0fa23806e9d7ee015b5a2a542d52382499edf9ab
2025-05-22 08:07:07 +00:00
Antoine Clausse
67ab5a749a Update Node to 22.15.0 (#24699)
* Run `bin/update_node 20.18.2 22.15.0`

* Remove expects on `fetchMock.callHistory.done()` to fix tests: are they necessary?

* Set node version to `22.x` in linked-url-proxy

* Increase test timeout to 30s in `github-sync`, Add waiting steps

* Define `navigator.onLine` in tests setup

GitOrigin-RevId: 75eb556e9f51b665e57497a0879b6915d14069ce
2025-05-16 08:06:26 +00:00
Eric Mc Sween
2dd054d602 Merge pull request #25606 from overleaf/em-o-error-types
OError: handle unknown error types
GitOrigin-RevId: a52dd92a4f95cd738556612599805f41f24de69f
2025-05-16 08:06:22 +00:00
Antoine Clausse
fee5ea8411 Remove returns of promises within functions with callbacks (address DeprecationWarning) (#25603)
* [document-updater] Don't return promises within functions with callbacks

Remove the errors:
DeprecationWarning: Calling promisify on a function that returns a Promise is likely a mistake
https://cloudlogging.app.goo.gl/YHDhoarvLEw2w9rXA

* Remove some more unnecessary returns in functions with callbacks, for consistency

* Add `sendCanaryAppliedOp` to excluded methods for promisification

GitOrigin-RevId: fa6d3e47c4e6561dc29d4c15e57c3289fc1f3dfa
2025-05-16 08:05:33 +00:00
Jakob Ackermann
e8b5ee2ff9 [history-ot] initial implementation of using doc-level history-ot (#25054)
* [history-v1-ot] initial implementation of using doc-level history-v1-ot

* [web] fix advancing of the otMigrationStage

Use 'nextStage' for the user provided, desired stage when advancing.

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [document-updater] document size check in editor-core

* [history-ot] rename history-v1-ot to history-ot and add types

* [history-ot] apply review feedback

- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData

Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>

* Revert "[document-updater] add safe rollback point for history-ot (#25283)"

This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

---------

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
2025-05-08 08:05:44 +00:00
Antoine Clausse
81941ff335 Update some dependencies so they're compatible with Node 22 (#25317)
* `"@google-cloud/profiler": "^6.0.3"`

* `bin/npm update pprof`

* `bin/npm update nan`

* `bin/npm update @google-cloud/profiler`

* Ignore false positive of `@typescript-eslint/return-await`

> Returning an awaited value that is not a promise is not allowed
Though the function was promisified

GitOrigin-RevId: 24dbe3e8df2b55c0b9583ac79a61e0956ac3fac0
2025-05-07 08:06:57 +00:00
Antoine Clausse
9a2847dbee [web] Add startup metrics (#25277)
* [web] refactor startup sequence

The primary objective here is to call loadGlobalBlobs() only once.
But to get there, we need to reorder things and add extra try/catch
sections to ensure we are not letting the global uncaughtException
handler catch startup errors.

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>

* [web] add metrics for startup steps

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: c73edea02516e919d55b896588dcd1862835fedf
2025-05-07 08:06:52 +00:00
Jakob Ackermann
07b2255426 [misc] cleanup .dockerignore and .gitignore files (#25312)
- Remove settings ignore, they are inconsistent and break local prettier
- Remove .dockerignore files, only root ignore file is used
- Move .idea/.run/*.swp/coverage to root
- Remove .npmrc entries, we are no longer writing the rc file
- Remove node_modules/.DS_Store, is contained in root
- Remove cruft

GitOrigin-RevId: 3025fd5acaef343312f55149466c638e759a6e1f
2025-05-07 08:05:24 +00:00
Jakob Ackermann
478e264817 [fetch-utils] fix leak of abort event handlers in AbortSignal (#25172)
GitOrigin-RevId: 992496010eb1cbe571b2e87fab8e7227b0d64538
2025-04-30 08:06:05 +00:00
Brian Gough
fe8d6392d5 Merge pull request #24793 from overleaf/bg-update-build-scripts-to-include-history-redis
update build scripts to include HISTORY_REDIS_HOST

GitOrigin-RevId: da0f317c80401067c0f4aa772196cb2f24849b8e
2025-04-11 08:06:03 +00:00
Brian Gough
9f527f10e1 Merge pull request #24104 from overleaf/bg-batched-update-concurrency-warning
prevent concurrent execution of batchedUpdate

GitOrigin-RevId: 90853ccd83943d8cd7b01fd11f152512d806e9a7
2025-04-04 08:05:00 +00:00
Andrew Rumble
d2d556ddf6 Allow cached persistor to get size of a blob
GitOrigin-RevId: 7e2c14381cf717c21325917ef4f78acb340a50d9
2025-03-24 10:48:35 +00:00
Jakob Ackermann
70601db76f [fetch-utils] export fetchRedirectWithResponse (#24372)
GitOrigin-RevId: 7cd30479fbe70fce68f2d6ef573187fc4f412749
2025-03-24 10:46:19 +00:00
Jakob Ackermann
e99cd74cca [fetch-utils] add fetchRedirectWithResponse (#24341)
GitOrigin-RevId: 9b1e9b02399beea87dbae0f87d1d7d897d145178
2025-03-18 09:05:44 +00:00
Andrew Rumble
d04bfbcdea Add promiseMapSettledWithLimit utility
GitOrigin-RevId: e34102de17f28e43deb383d630088c6e753e2ec1
2025-03-12 09:05:22 +00:00
Eric Mc Sween
290bdf4361 Merge pull request #23993 from overleaf/em-remove-fixed-remove-change-flag
Remove fixedRemoveChange flag from editor updates

GitOrigin-RevId: bf74e1137560184c4b024a3b5c6ede5a841d3559
2025-03-04 09:04:48 +00:00
Andrew Rumble
767591973c Merge pull request #23905 from overleaf/ar-mocha-11
[all] Bump mocha everywhere

GitOrigin-RevId: bf5587047f9ceb88872cbd356c2029717bb8ca9d
2025-02-28 09:05:40 +00:00
Miguel Serrano
2e72aa8a47 Merge pull request #23652 from overleaf/msm-update-path-to-regexp
[monorepo] Update express to 4.1.2

GitOrigin-RevId: b3db75257e1edc1a25a1a657def352ecca926920
2025-02-21 09:05:26 +00:00
Eric Mc Sween
879aa33cbe Merge pull request #23519 from overleaf/em-remove-ordered-rejections-flag
Remove orderedRejections flag on ops

GitOrigin-RevId: df464023e7e6c3b5899e6a424f423a374fd33df6
2025-02-13 09:05:53 +00:00
Eric Mc Sween
510ad0ce9a Merge pull request #23509 from overleaf/em-fix-remove-change
Fix RangesTracker._removeChange() when multiple changes have the same id

GitOrigin-RevId: 2dafcf275b283da320bca70b460aba3051ca9575
2025-02-12 09:05:29 +00:00
Jakob Ackermann
0a7b2004d2 [misc] silence logger when running tests (#22243)
* [misc] silence logger when running tests

* [misc] re-enable logging in some tests for scripts

* [misc] make it easy to turn on verbose logging for tests

```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
2025-02-10 09:06:02 +00:00
Eric Mc Sween
ce4c8a4e47 Merge pull request #23398 from overleaf/em-log-doc-hash-mismatches
Validate content hashes in history (log only)

GitOrigin-RevId: ed772fc4e4d0aa9e980f9693a759647bd937e13a
2025-02-07 09:05:59 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889)
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
2025-02-07 09:05:01 +00:00
Jakob Ackermann
6b8a50eb92 [project-history] trim logging (#23408)
* [project-history] trim logging when projectVersion is out of order

* [editor-core] record reason rather than full pathname when not clean

* [editor-core] remove pathname from error messages

* [project-history] avoid logging updates when historyId is inconsistent

* [project-history] avoid logging failed history-v1 requests repeatedly

* [project-history] log once when doc version is out of order in chunk

GitOrigin-RevId: 5ac431ac87b57c00fac9d5ab486ca305b1775372
2025-02-06 09:04:35 +00:00
Eric Mc Sween
39842c9857 Merge pull request #23250 from overleaf/em-project-history-doc-hash
Propagate the doc hash from project-history to history

GitOrigin-RevId: 341df52df41f7a5f8dbebbce53a47f9d5e1f8175
2025-02-04 09:05:03 +00:00
Jakob Ackermann
ba60f885a4 [project-history] fetch most recent chunk once when processing updates (#23261)
GitOrigin-RevId: 08ddd1f953b0dbae0541a60b45fec0f88e0a1d06
2025-01-31 09:05:14 +00:00
Antoine Clausse
272108a213 Run bin/update_node 20.18.0 20.18.2 (#23074)
GitOrigin-RevId: 3f1ab14c4c36ea54408e0ce2c404a323bfe8d9a6
2025-01-24 09:06:05 +00:00
Andrew Rumble
47db5c1236 Merge pull request #23039 from overleaf/ar-bump-mongo-6.12
[many] Bump mongodb to 6.12.0 universally

GitOrigin-RevId: 6df934518e95217b6157ca5a41cf8acf4beb7689
2025-01-23 09:06:13 +00:00
Eric Mc Sween
e9c1c0f9c8 Merge pull request #22650 from overleaf/em-tracked-deletes-at-same-position
Handle multiple tracked deletes at same position

GitOrigin-RevId: 3cbf1c418bcd50cf08e1b90ce6ba3bc480236079
2025-01-14 09:05:34 +00:00
Eric Mc Sween
a323a5c915 Merge pull request #22527 from overleaf/revert-22471-em-tracked-deletes-at-same-position
Revert "Improve handling of tracked delete rejections"

GitOrigin-RevId: 444a5a73aa2b93162036dfc463cd76d9c463aadb
2024-12-16 09:05:10 +00:00
Eric Mc Sween
6737cfa38d Merge pull request #22471 from overleaf/em-tracked-deletes-at-same-position
Improve handling of tracked delete rejections

GitOrigin-RevId: 35857d0a3f739c0531223737b2b649c9e8033157
2024-12-16 09:04:58 +00:00
Rebeka Dekany
7493936bad Merge pull request #22310 from overleaf/ar-bump-mongoose-8.8.3
[web] Upgrade mongoose to 8.8.3

GitOrigin-RevId: 5540f7384b74daafa01522c23633587fc56a7faa
2024-12-06 09:05:25 +00:00
Antoine Clausse
a6ec075f78 Increase checkSummaryValues ranges to fix flakiness (#22227)
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
GitOrigin-RevId: d6da0b1e66cf5cb4ef4dbed91ae468ffe5807e37
2024-12-02 09:06:22 +00:00
Jakob Ackermann
3d7254b419 Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal
[history-v1] backup-verifier-app: initial revision

GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
2024-11-27 09:04:55 +00:00
Brian Gough
6672372828 remove sentry from backend services (#20752)
* remove sentry from backend services - no longer required

* Remove Sentry integration from logging manager

* Remove Sentry from clsi default settings

* Remove `initializeErrorReporting` in libraries/logger

* Remove `@sentry/node` from `libraries/logger`

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 8149a885f5258804b93ae39cde7b7333e992532a
2024-11-27 09:04:50 +00:00
Jakob Ackermann
0bd8729cc1 Merge pull request #22115 from overleaf/jpa-skip-get
[history-v1] back_fill_file_hash: optimize obtaining the DEK

GitOrigin-RevId: 6fc5218737551ae4e6152414c32560f545767091
2024-11-26 09:04:50 +00:00
Jakob Ackermann
ed27af11f8 Merge pull request #22107 from overleaf/jpa-error-handling
[history-v1] back_fill_file_hash: error handling

GitOrigin-RevId: 2dbf52d015206cf42ca434b1f16221d6e0f2cc2b
2024-11-26 09:04:45 +00:00
Jakob Ackermann
69021ca4b0 Merge pull request #22061 from overleaf/jpa-reduce-idle-time
[history-v1] back_fill_file_hash: reduce idle time between batches

GitOrigin-RevId: 9b24bb882b158f33915d2e0ab2f82273eac09739
2024-11-22 09:05:55 +00:00
Jakob Ackermann
4f25b0de91 Merge pull request #22028 from overleaf/jpa-logging
[misc] logging tweaks

GitOrigin-RevId: 7757b81b6c684955374038376035dc2bd272c4e8
2024-11-21 09:05:41 +00:00
Jakob Ackermann
8e74d3c58c Merge pull request #21947 from overleaf/bg-jpa-back-fill-script-tweaks
[history-v1] back_fill_file_hash: performance tweaks

GitOrigin-RevId: c3d0c7906707fc902addcde64eaf41c24ceeece7
2024-11-20 09:04:51 +00:00
Antoine Clausse
87e7e3017a Migrate missing node:18 to node:20 (#21903)
GitOrigin-RevId: 0315d5df0ab1154d5ccadcbf11b09ec53df5728f
2024-11-20 09:04:47 +00:00
Jakob Ackermann
27076c50cc Merge pull request #21670 from overleaf/jpa-mongo-backend-types
[history-v1] add types to mongo BlobStore backend

GitOrigin-RevId: 7d91074eaa781904f7f3b56390aacee1800a7f67
2024-11-19 09:05:23 +00:00
Jakob Ackermann
dd15ade2b9 Merge pull request #21880 from overleaf/jpa-history-store
[history-v1] make HistoryStore generic and add backupHistoryStore

GitOrigin-RevId: 65d275de182dbcf5d4b6bf3c610b71b58db68e70
2024-11-18 09:05:48 +00:00
Jakob Ackermann
97b39ef98f Merge pull request #21681 from overleaf/jpa-back-fill-file-hash
[history-v1] add script for back filling hash into project file-tree

GitOrigin-RevId: 70ab8c72f3eea1062708e4471b4ad1c60317ad26
2024-11-18 09:05:39 +00:00
Jakob Ackermann
d6e94d2586 Merge pull request #21908 from overleaf/jpa-storage-class
[object-persistor] s3: add support for setting storage class on upload

GitOrigin-RevId: dc5480a95ec7f1551f08848f7fa45f760d97ab22
2024-11-18 09:05:10 +00:00
Jakob Ackermann
2be894c18a Merge pull request #21892 from overleaf/jpa-auto-gunzip
[object-persistor] add autoGunzip option to getObjectStream

GitOrigin-RevId: 21cd6c9ab349017ddb28d165238371d967ab9a59
2024-11-15 09:06:32 +00:00
Jakob Ackermann
bd855044af Merge pull request #21891 from overleaf/jpa-consistent-getObjectSize-type
[object-persistor] gcs: return a number from getObjectSize

GitOrigin-RevId: f3e0e3269e789b0077c82014c42a4ca63cd84b39
2024-11-15 09:06:28 +00:00
Jakob Ackermann
38b83d7edf Merge pull request #21881 from overleaf/jpa-missing-break
[object-persistor] s3SSEC: missing break statement in rootKEK discovery

GitOrigin-RevId: 3116a36006c9c7ae0b15593d8938fd559794a898
2024-11-15 09:06:14 +00:00