Commit graph

55 commits

Author SHA1 Message Date
Jakob Ackermann
1e6b13f9d5 [history-ot] rename remaining history-v1-ot references to history-ot (#25428)
* [history-ot] rename remaining history-v1-ot references to history-ot

* [web] rename History-v1 OT -> History OT in admin panel

* [web] rename OT Migration -> History OT Migration in admin panel

GitOrigin-RevId: 103ce816d5320d6379d51009cdc08b8a71aa48e6
2025-05-20 08:06:46 +00:00
Jakob Ackermann
e8b5ee2ff9 [history-ot] initial implementation of using doc-level history-ot (#25054)
* [history-v1-ot] initial implementation of using doc-level history-v1-ot

* [web] fix advancing of the otMigrationStage

Use 'nextStage' for the user provided, desired stage when advancing.

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [document-updater] document size check in editor-core

* [history-ot] rename history-v1-ot to history-ot and add types

* [history-ot] apply review feedback

- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData

Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>

* Revert "[document-updater] add safe rollback point for history-ot (#25283)"

This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

---------

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
2025-05-08 08:05:44 +00:00
Jakob Ackermann
f0edc7ba00 [web] update the projects lastUpdated timestamp when changing file-tree (#24867)
* [misc] freeze time before any other unit test setup steps

Freezing it after other work (notably sandboxed-module imports) will
result in flaky tests.

* [web] update the projects lastUpdated timestamp when changing file-tree

GitOrigin-RevId: b82b2ff74dc31886f3c4bd300375117eead6e0cd
2025-04-16 08:05:14 +00:00
Jakob Ackermann
3f10b29869 [real-time] backwards compatibility fix for redis 6.2 (#24734)
GitOrigin-RevId: ffc51a8280e0d0708e7dcb2638cabed2b7adfbf5
2025-04-09 08:05:37 +00:00
Jakob Ackermann
ca111771c2 [real-time] rename metric label to align with common name (#24732)
GitOrigin-RevId: fd161f2345f0c30aa82395dbca673025698f13fe
2025-04-09 08:05:33 +00:00
Jakob Ackermann
dadf6f0ed4 [real-time] record metrics on number of connected clients per project (#24727)
Note: "connected" here means across all real-time pods.

- editing_session_mode, counter
  - mode=connect - a client connected
  - mode=disconnect - a client disconnect
  - mode=update - continuous editing
  - status=empty - all clients disconnected
  - status=single - a single client is connected
  - status=multi - multiple clients are connected

- project_not_empty_since histogram with buckets [0,1h,2h,1d,2d,1w,30d]
  - status=empty/single/multi as described above

GitOrigin-RevId: 1cc42e72bbb5aae754399bdbc3f8771642f35c22
2025-04-09 08:05:29 +00:00
Domagoj Kriskovic
5a447aa349 [real-time] ensure tc is enabled when reviewer role is used (#22245)
* Support for adding reviewer role

* show reviewer in track changes user list

* added "review" in assertClientCanViewProject

* test if reviewer can read project

* added collaboratorsGetter tests

* eit toggle-track-changes when track changes changes

* [real-time] ensure tc is enabled when reviewer role is used

* use assertClientCanReviewProjectAndDoc, refactor _assertClientCanApplyUpdate

GitOrigin-RevId: 158bd1ff0d4b4977da950134f8ad8b3740855290
2024-12-10 09:05:18 +00:00
Domagoj Kriskovic
13e3a941f4 [real-time] make sure reviewer has permissions to read a project (#22317)
* added "review" in assertClientCanViewProject

* test if reviewer can read project

GitOrigin-RevId: 92a30f8b456bec9f0fc81a1361e454eecff838bd
2024-12-10 09:05:14 +00:00
Domagoj Kriskovic
511fe60a9c Revert "Support for adding reviewer role (#22137)" (#22293)
This reverts commit 54064a7f961fe06f188ab449cd469cdaaf01b20a.

GitOrigin-RevId: 1dabc635756aff465d2fe9aa0d18121b081df3b0
2024-12-04 09:05:12 +00:00
Domagoj Kriskovic
f676eca2b8 Support for adding reviewer role (#22137)
* Support for adding reviewer role

* show reviewer in track changes user list

* added "review" in assertClientCanViewProject

* test if reviewer can read project

* added collaboratorsGetter tests

* eit toggle-track-changes when track changes changes

* Support for changing privilege to reviewers for invited users (#22159)

* Add reviewer in change privilege level handler

* added reviewer translation

* added acceptance tests

* fix tests

* Set track changes state permissions for reviewer role (#22167)

* Add reviewer in change privilege level handler

* added reviewer translation

* added acceptance tests

* fix tests

* Set track changes state permissions for reviewer role

* added authorization helper tests

* added ensureUserCanReviewProjectContent middleware

* allow changing track changes only with write permissions

* removed canUserReviewProjectContent

* List projects where user is added as a reviewer (#22249)

* List projects where user is added as reviewer

* list projects in /user/projects

* fix tests

GitOrigin-RevId: 54064a7f961fe06f188ab449cd469cdaaf01b20a
2024-12-04 09:05:00 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Jakob Ackermann
0f0d562786 Merge pull request #19455 from overleaf/jpa-metrics
[misc] add metrics for document processing/broadcasting

GitOrigin-RevId: d81de0dfb7a91863547631580f3c85f569718130
2024-07-19 08:04:34 +00:00
Brian Gough
49dc94192a Merge pull request #18848 from overleaf/bg-cookie-session-metrics-for-real-time
add cookie session metrics for real-time

GitOrigin-RevId: 6b482dafc19fca46a375ea00a9b2297e20d915ce
2024-06-13 08:04:24 +00:00
roo hutton
d53a3e315e Merge pull request #17879 from overleaf/rh-refresh-collab
Refresh websocket on collaborator change

GitOrigin-RevId: b05444d592f5952a08bad51c9f529ed9183d042f
2024-04-17 08:05:18 +00:00
Jakob Ackermann
385d3f9c1b Merge pull request #16980 from overleaf/jpa-join-project-remove-sl-2
[misc] joinProject: pass userId and anonymous access token in body 2/2

GitOrigin-RevId: b1a11941a6d9c7cc779769d1e97d3a0f03bed610
2024-02-13 09:03:39 +00:00
Jakob Ackermann
974069bf1c Merge pull request #16979 from overleaf/jpa-join-project-remove-sl-1
[misc] joinProject: pass userId and anonymous access token in body 1/2

GitOrigin-RevId: 5d7832246c7262c004c2cd465d62488384b35ee3
2024-02-12 09:03:16 +00:00
Miguel Serrano
51de84664d Merge pull request #16346 from overleaf/msm-remove-underscore
Remove `underscore` in monorepo

GitOrigin-RevId: fd3b92e3e8a940a7a086669232102320c34a89e2
2024-01-09 09:03:58 +00:00
June Kelly
d68ed0efdf Merge pull request #13140 from overleaf/jk-real-time-disconnect-link-sharing
[real-time] Disconnect relevant users when link-sharing is turned off

GitOrigin-RevId: cf44a30a235717b658a759e8a74ae4d0e5abae47
2023-05-31 08:05:07 +00:00
June Kelly
5d0ce8c4cd Merge pull request #13052 from overleaf/jk-realtime-disconnect-users
[real-time] Disconnect client when user is removed from project

GitOrigin-RevId: e3d31b9192c639b2988e488e34e19050e7caeb4a
2023-05-17 12:04:48 +00:00
Eric Mc Sween
d9e0215aee Merge pull request #12211 from overleaf/em-camel-case-real-time
Camel case variables in real-time

GitOrigin-RevId: 0a35d2e39d54c258bed017ea4dcdbf9a19a2a9b1
2023-03-21 09:05:37 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
2021-10-28 08:03:26 +00:00
Brian Gough
6cece12369 Merge pull request #5109 from overleaf/bg-fix-handshake-exception
fix handshake exception

GitOrigin-RevId: 917518ccac3f5fddf4029f1b1ec3e3806dde9a29
2021-09-16 08:03:08 +00:00
Brian Gough
618cf99548 Merge pull request #4950 from overleaf/bg-realtime-log-relevelling
realtime log re-levelling

GitOrigin-RevId: 81d86ba648e7fc1436b638b67371b35fd831a62f
2021-09-15 08:03:25 +00:00
Jakob Ackermann
b8fcb265b2 [misc] EventLogger: drop explicit metrics.inc amount
The module has a hard-coded increment of 1, which is the only value the
 prometheus backend supports.
2021-07-13 12:40:46 +01:00
Jakob Ackermann
bd5e8b8f71 [misc] fix chai assertions .equal.true -> .equals(true) 2021-07-13 12:34:56 +01:00
Jakob Ackermann
7e8e231059 [misc] run format_fix and lint:fix 2021-07-13 12:04:45 +01:00
Jakob Ackermann
a26ae73597 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:18 +01:00
Eric Mc Sween
b9e7cbf73b Add global test setup
Configure chai and SandboxedModule in a central place. Configure
SandboxedModule globals that are needed in Node 12.
2021-03-18 16:41:31 -04:00
Henry Oswald
46b389e8b3 fix off by 1 error in key sharding 2021-02-09 10:48:40 +00:00
Henry Oswald
31e1808dd8 shard the pending-updates-list queue 2021-02-08 16:02:41 +00:00
Henry Oswald
c308e60b6b speed up DocumentUpdaterControllerTests by directly injecting redis into
tests
2021-02-08 11:13:22 +00:00
Jakob Ackermann
746c5aeb80 [misc] bump metrics module to 3.4.1
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:22 +00:00
Jakob Ackermann
15af5c7977 [misc] bump @overleaf/redis-wrapper to version 2.0.0 2020-11-11 16:24:22 +00:00
Jakob Ackermann
c6f2a3b387 Merge pull request #189 from overleaf/jpa-relevel-log
[misc] re-level expected error log messages
2020-09-02 10:46:08 +02:00
Jakob Ackermann
4960bdd6fe [misc] re-level log: 404 from web -> WARN and emit 'project not found'
A stale browser tab tried to join a deleted project.
Emitting 'project not found'/'ProjectNotFound' will trigger a page
 reload in the frontend, upon web can render a 404.
See frontend: ConnectionManager.joinProject callback
2020-08-27 11:51:57 +01:00
Jakob Ackermann
55d938ba14 [misc] add tests for web replying with a 404 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
884b340c75 [misc] re-level log: 403 from web goes to WARN and emit 'not authorized'
Users will get redirected to the login page and will see a 'restricted'
 page after logging in again.
See frontend: ConnectionManager.reportConnectionError
2020-08-27 11:51:56 +01:00
Jakob Ackermann
d2a2b9d46e [misc] add tests for web replying with a 403 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
2ce7b36c95 [misc] drop duplicate log line for unauthorized applyOtUpdate calls
The violation is logged in Router._handleError.
2020-08-27 10:22:31 +01:00
Jakob Ackermann
425052ff91 Merge pull request #187 from overleaf/jpa-o-error-tagging
[misc] migrate to OError tagging/wrapping
2020-08-25 11:46:28 +02:00
Jakob Ackermann
ee3d3b09ed [misc] wrap redis errors as tagging does not work with them
ioredis may reuse the error instance for multiple callbacks. E.g. when
 the connection to redis fails, the queue is flushed with the same
 MaxRetriesPerRequestError instance.
2020-08-24 10:12:20 +01:00
Jakob Ackermann
537e97be73 [misc] OError.tag all the errors in async contexts
See the docs of OError.tag:
https://github.com/overleaf/o-error#long-stack-traces-with-oerrortag
(currently at 221dd902e7bfa0ee92de1ea5a3cbf3152c3ceeb4)

I am tagging all errors at each async hop. Most of the controller code
 will only ever see already tagged errors -- or new errors created in
 our app code. They should have enough info that we do not need to tag
 them again.
2020-08-24 10:12:06 +01:00
Jakob Ackermann
4cb8cc4a85 [DocumentUpdaterManager] use a new ClientRequestedMissingOpsError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
8abfdb87ff [DocumentUpdaterManager] use a new DocumentUpdaterRequestFailedError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
68bc9d0d23 [WebApiManager] use a new WebApiRequestFailedError 2020-08-21 12:47:06 +01:00
Jakob Ackermann
af50f9b02c [DocumentUpdaterManager] use a new UpdateTooLargeError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
5950b26a42 [SafeJsonParse] migrate to OError and use a new DataTooLargeToParseError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
f82177a46a [Errors] migrate to OError 2020-08-21 12:47:04 +01:00
Brian Gough
831d794bf4 clean up join/leave handling
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
2020-08-12 10:54:22 +01:00
Jakob Ackermann
8601084a2e [misc] WebsocketControllerTests: remove debugging code 2020-07-01 11:36:36 +01:00