Commit graph

745 commits

Author SHA1 Message Date
yu-i-i
b035613237 Add Template Gallery support 2025-07-22 19:22:09 +02:00
Andrew Rumble
1a1f283245 Update node to 22.17.0
GitOrigin-RevId: 7a15f7a0e95e3331c589955aca1fbc5dfb13a568
2025-07-09 08:06:01 +00:00
Antoine Clausse
436dcc977f Update Node to 22.15.1 (#25785)
GitOrigin-RevId: 52428d2d7e67c3135a1604fa487dd142aa08bf15
2025-05-22 08:07:42 +00:00
Antoine Clausse
b667cef262 Revert "Update defaultHighWaterMark to 64KiB (Node 22's default) (#25522)" (#25789)
This reverts commit 19d731abf683066654027de3a4f9ac0b8916f22c.

GitOrigin-RevId: eb7c45ab45e02054601b607a4bfeb432424a1837
2025-05-22 08:07:38 +00:00
Antoine Clausse
67ab5a749a Update Node to 22.15.0 (#24699)
* Run `bin/update_node 20.18.2 22.15.0`

* Remove expects on `fetchMock.callHistory.done()` to fix tests: are they necessary?

* Set node version to `22.x` in linked-url-proxy

* Increase test timeout to 30s in `github-sync`, Add waiting steps

* Define `navigator.onLine` in tests setup

GitOrigin-RevId: 75eb556e9f51b665e57497a0879b6915d14069ce
2025-05-16 08:06:26 +00:00
Antoine Clausse
aa002369cb Update defaultHighWaterMark to 64KiB (Node 22's default) (#25522)
* Set defaultHighWaterMark to 16KiB

This is already the default in Node 20

* Set defaultHighWaterMark to 64KiB

Per https://github.com/overleaf/internal/pull/25522#issuecomment-2872035192

GitOrigin-RevId: 19d731abf683066654027de3a4f9ac0b8916f22c
2025-05-14 08:05:16 +00:00
Jakob Ackermann
07b2255426 [misc] cleanup .dockerignore and .gitignore files (#25312)
- Remove settings ignore, they are inconsistent and break local prettier
- Remove .dockerignore files, only root ignore file is used
- Move .idea/.run/*.swp/coverage to root
- Remove .npmrc entries, we are no longer writing the rc file
- Remove node_modules/.DS_Store, is contained in root
- Remove cruft

GitOrigin-RevId: 3025fd5acaef343312f55149466c638e759a6e1f
2025-05-07 08:05:24 +00:00
Christopher Hoskin
94e12ec404 Merge pull request #24971 from overleaf/csh-issue-19131-diable-filestore-endpoints
Disable the old filestore endpoints if user_files not defined

GitOrigin-RevId: f7d188cef3fe53835070903448690baff4ebad98
2025-04-18 08:04:59 +00:00
Christopher Hoskin
778221c0af Merge pull request #24924 from overleaf/csh-issue-18692-404-if-templates-not-configured
Only define the template endpoints if configured

GitOrigin-RevId: 9ec9d411d6aed8774a880e80d0559a3491e832de
2025-04-17 08:06:03 +00:00
Brian Gough
fe8d6392d5 Merge pull request #24793 from overleaf/bg-update-build-scripts-to-include-history-redis
update build scripts to include HISTORY_REDIS_HOST

GitOrigin-RevId: da0f317c80401067c0f4aa772196cb2f24849b8e
2025-04-11 08:06:03 +00:00
Andrew Rumble
767591973c Merge pull request #23905 from overleaf/ar-mocha-11
[all] Bump mocha everywhere

GitOrigin-RevId: bf5587047f9ceb88872cbd356c2029717bb8ca9d
2025-02-28 09:05:40 +00:00
Antoine Clausse
fe03d8d7fd Add shellcheck to web, server-pro, server-ce (#23823)
* Update shellcheck files to include non ".sh" files having the sh shebang

* Add shellcheck to root Makefile

* `make shellcheck_fix`

* Add shellcheck to server-ce and server-pro

* Exclude SC1091 (Not following) in SP/CE

* Fix errors in SP

* Fix errors in CE

* Update root shellcheck to ignore all failing rules. We can later remove rules one by one.

* Add shellcheck to web

* Add shellcheck step for server-ce and server-pro in cloudbuild.yaml

* Revert "Add shellcheck to root Makefile"

This reverts commit e0fa56f2

* Revert "`make shellcheck_fix`"

This reverts commit eb179245c109a9e742a7fdeeb75a4bdd03963587.

* `make shellcheck_fix` in server-ce

* Fix: Use $(...) notation instead of legacy backticked

```
In init_scripts/100_set_docker_host_ipaddress.sh line 5:
echo "`route -n | awk '/UG[ \t]/{print $2}'` dockerhost" >> /etc/hosts
      ^-- SC2006 (style): Use $(...) notation instead of legacy backticks `...`.

Did you mean:
echo "$(route -n | awk '/UG[ \t]/{print $2}') dockerhost" >> /etc/hosts

For more information:
  https://www.shellcheck.net/wiki/SC2006 -- Use $(...) notation instead of le...
```

* `make shellcheck_fix` in web

* Manual corrections on autofix

* Update SHELLCHECK_FILES to Perl regex so it matches at start of file

GitOrigin-RevId: 815d53f319a3792efa22703beb75570de5977450
2025-02-26 09:05:02 +00:00
Miguel Serrano
2e72aa8a47 Merge pull request #23652 from overleaf/msm-update-path-to-regexp
[monorepo] Update express to 4.1.2

GitOrigin-RevId: b3db75257e1edc1a25a1a657def352ecca926920
2025-02-21 09:05:26 +00:00
Jakob Ackermann
0a7b2004d2 [misc] silence logger when running tests (#22243)
* [misc] silence logger when running tests

* [misc] re-enable logging in some tests for scripts

* [misc] make it easy to turn on verbose logging for tests

```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
2025-02-10 09:06:02 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889)
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
2025-02-07 09:05:01 +00:00
Jakob Ackermann
5404dcb93d [misc] update build scripts (#23316)
* [misc] update build scripts

* [misc] make shellcheck happy

GitOrigin-RevId: 343655fda54aa7e835dd666dcd4218d61c58fd09
2025-02-04 09:04:37 +00:00
Antoine Clausse
272108a213 Run bin/update_node 20.18.0 20.18.2 (#23074)
GitOrigin-RevId: 3f1ab14c4c36ea54408e0ce2c404a323bfe8d9a6
2025-01-24 09:06:05 +00:00
Andrew Rumble
47db5c1236 Merge pull request #23039 from overleaf/ar-bump-mongo-6.12
[many] Bump mongodb to 6.12.0 universally

GitOrigin-RevId: 6df934518e95217b6157ca5a41cf8acf4beb7689
2025-01-23 09:06:13 +00:00
Rebeka Dekany
7493936bad Merge pull request #22310 from overleaf/ar-bump-mongoose-8.8.3
[web] Upgrade mongoose to 8.8.3

GitOrigin-RevId: 5540f7384b74daafa01522c23633587fc56a7faa
2024-12-06 09:05:25 +00:00
Jakob Ackermann
d19c5e236f Merge pull request #22208 from overleaf/jpa-clsi-hash
[misc] clsi: read files from history-v1 with fallback to filestore

GitOrigin-RevId: c54bb128780198c14e7a63818f39fad62ce65d4e
2024-11-29 09:05:39 +00:00
Brian Gough
6672372828 remove sentry from backend services (#20752)
* remove sentry from backend services - no longer required

* Remove Sentry integration from logging manager

* Remove Sentry from clsi default settings

* Remove `initializeErrorReporting` in libraries/logger

* Remove `@sentry/node` from `libraries/logger`

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 8149a885f5258804b93ae39cde7b7333e992532a
2024-11-27 09:04:50 +00:00
Antoine Clausse
2b1df12052 [filestore] Replace delayShutdownMs by gracefulShutdownDelayInMs (#22077)
* Add a default 30s timeout for `delayShutdownMs`

`settings.delayShutdownMs` doesn't seem to be defined anywhere

Logs typically often this pair of entries:
```
INFO 2024-11-21T15:51:42.115Z [resource.labels.containerName: filestore] received interrupt, cleaning up
INFO 2024-11-21T15:51:42.120Z [resource.labels.containerName: filestore] shutdown timed out, exiting
```
This indicates that there is no delay between the interrupt signal and the shutdown. The 100ms delay also doesn't happen. We believe that the `server.close` callback is called after `server.closeAllConnections()`, and the `server closed` log is usually lost because of the process exiting immediately.

See: https://cloudlogging.app.goo.gl/WJQ6mc3gWwotVQya7

* Add timeout before exiting

* Replace `delayShutdownMs` by `gracefulShutdownDelayInMs`.

Looks like the error came from a wrong merge in https://github.com/overleaf/internal/pull/18756

We don't want a default 30s timeout for the shutdown, per https://github.com/overleaf/internal/pull/16888

GitOrigin-RevId: c1bdc8986f78a6e18e8b8b1fe60b33aa6ffef909
2024-11-22 09:06:27 +00:00
Andrew Rumble
77831b60bf Merge pull request #22059 from overleaf/ar-update-node-18-in-recent-history-changes
[history-v1] Update more missed node 18 usages

GitOrigin-RevId: 481e3de95dc34ef2bca38302a3ff595a92cea6ed
2024-11-22 09:06:00 +00:00
Antoine Clausse
87e7e3017a Migrate missing node:18 to node:20 (#21903)
GitOrigin-RevId: 0315d5df0ab1154d5ccadcbf11b09ec53df5728f
2024-11-20 09:04:47 +00:00
Jakob Ackermann
d6e94d2586 Merge pull request #21908 from overleaf/jpa-storage-class
[object-persistor] s3: add support for setting storage class on upload

GitOrigin-RevId: dc5480a95ec7f1551f08848f7fa45f760d97ab22
2024-11-18 09:05:10 +00:00
Jakob Ackermann
2be894c18a Merge pull request #21892 from overleaf/jpa-auto-gunzip
[object-persistor] add autoGunzip option to getObjectStream

GitOrigin-RevId: 21cd6c9ab349017ddb28d165238371d967ab9a59
2024-11-15 09:06:32 +00:00
Jakob Ackermann
bd855044af Merge pull request #21891 from overleaf/jpa-consistent-getObjectSize-type
[object-persistor] gcs: return a number from getObjectSize

GitOrigin-RevId: f3e0e3269e789b0077c82014c42a4ca63cd84b39
2024-11-15 09:06:28 +00:00
Andrew Rumble
73b17dd694 Merge pull request #21627 from overleaf/ar-node-20
[all] node 20 upgrade v2

GitOrigin-RevId: aed13a800894717a796062e3dd4050e045040258
2024-11-15 09:04:32 +00:00
Jakob Ackermann
805c4f8321 Merge pull request #21764 from overleaf/jpa-esm-build-scripts
[misc] prepare linting for ESM files in services

GitOrigin-RevId: 468983ebff40617158bac28514381a6aaf725893
2024-11-12 09:05:50 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Jakob Ackermann
122d89a831 Merge pull request #21660 from overleaf/jpa-s3-https
[object-persistor] s3: simplify using a custom CA for HTTPS endpoints

GitOrigin-RevId: 2c6a5312a842582e5e40e917ccc586392087cb7a
2024-11-08 09:07:00 +00:00
Jakob Ackermann
65dc6bf940 Merge pull request #21631 from overleaf/jpa-live-s3
[object-persistor] s3SSEC: make compatible with AWS S3

GitOrigin-RevId: 3cd1f5ec43141f15defe081fc916d2ca2d78ca01
2024-11-08 09:06:55 +00:00
Jakob Ackermann
d5478c11ea Merge pull request #21623 from overleaf/jpa-s3-ssec-static-dek-path
[object-persistor] s3SSEC: use static path for DEK inside project folder

GitOrigin-RevId: 0c43ef8964c16d3e7638f17ff47b1c4641e439df
2024-11-08 09:06:51 +00:00
Jakob Ackermann
a67560c26b Merge pull request #21589 from overleaf/jpa-s3-ssec-hkdf
[object-persistor] s3SSEC: add HKDF layer to KEK

GitOrigin-RevId: 1def9e378e1dfd90f449ad392b0db2101584e17f
2024-11-08 09:06:46 +00:00
Jakob Ackermann
bec73ddfae Merge pull request #21543 from overleaf/jpa-s3-seec-kek-rotation
[object-persistor] s3SSEC: add support for (automatic) KEK rotation

GitOrigin-RevId: 315082e894c74e276a8efbc46b41ec7e102f9010
2024-11-08 09:06:42 +00:00
Jakob Ackermann
859901ac0c Merge pull request #21380 from overleaf/jpa-s3-ssec-backend
[object-persistor] add backend for SSE-C with S3 using KEK and DEK

GitOrigin-RevId: 9676f5cd5e08107c8c284b68b8d450a1c05bf1b1
2024-11-08 09:06:37 +00:00
Andrew Rumble
57d12dfb17 Update to node 18.20.4
GitOrigin-RevId: 4b613ea3973cb7944c3d414ea91bcf367dbf50a6
2024-11-07 09:04:31 +00:00
Jakob Ackermann
5bb90dc6cb Merge pull request #21442 from overleaf/jpa-align-mongo-version
[misc] align all the mongodb dependency versions

GitOrigin-RevId: 1194fe57601af98bb61250a285bfc85b4b8179dd
2024-11-01 09:05:23 +00:00
Jakob Ackermann
7b3e39f63f Merge pull request #21375 from overleaf/jpa-if-none-match
[object-persistor] add support for ifNoneMatch=* in sendStream

GitOrigin-RevId: 268f054ac1b6452105b02757cdec32bad00702fd
2024-11-01 09:05:11 +00:00
Jakob Ackermann
a551a0e9f7 Merge pull request #21361 from overleaf/jpa-filestore-minio
[filestore] migrate to minio as S3 backend for running tests against

GitOrigin-RevId: aa098d8baa4445f5dec7d651b6cf5ed081b0a331
2024-11-01 09:05:07 +00:00
Andrew Rumble
79d80cc266 Merge pull request #21462 from overleaf/revert-18828-ac-node-20
Revert "Upgrade Node from `18.20.2` to `20.17.0`"

GitOrigin-RevId: cc12c33b4588caa2959817a113547d20d437d6f4
2024-10-30 09:05:07 +00:00
Andrew Rumble
a480df8a89 Merge pull request #18828 from overleaf/ac-node-20
Upgrade Node from `18.20.2` to `20.17.0`

GitOrigin-RevId: 0ba9b0921fc246a97d3da0aad7d48635c0e24ae8
2024-10-30 09:04:38 +00:00
Jakob Ackermann
52edad6f12 Merge pull request #21322 from overleaf/jpa-skip-converted-cleanup
[filestore] remove unnecessary cleanup of converted files

GitOrigin-RevId: a46aaee473938209dc56d9fea90f96d09952ce47
2024-10-25 08:06:07 +00:00
Jakob Ackermann
29381cf054 Merge pull request #21321 from overleaf/jpa-reconfigure-filestore
[filestore] refactor acceptance tests

GitOrigin-RevId: a6bb1527220b1c062d980c79d2ccb62973b99d2c
2024-10-25 08:06:02 +00:00
Jakob Ackermann
d1d65e65ad Merge pull request #21320 from overleaf/jpa-filestore-controller
[filestore] only call handler from controller

GitOrigin-RevId: 1d1bbe4e961b300a919ae79e3c760322304783bc
2024-10-25 08:05:57 +00:00
Jakob Ackermann
e56e5442fd Merge pull request #21295 from overleaf/jpa-fix-filestore-tests
[filestore] fix conditional tests following sharing changes

GitOrigin-RevId: 7d804acb14942fb6d06ee38f782fc78796e9182e
2024-10-25 08:05:53 +00:00
Antoine Clausse
6c2cf20125 Merge pull request #20552 from overleaf/ac-update-node-fetch-2
Reapply "Upgrade node-fetch to 2.7.0", Fix fetch-utils tests

GitOrigin-RevId: b42a2d2c50ce73f474e39755845e4df065f30b48
2024-10-01 08:05:18 +00:00
Liangjun Song
cfd0b39b96 Merge pull request #20557 from overleaf/ls-bump-express
update express and other dependencies

GitOrigin-RevId: 22107eab0fa95d534ab07360548ae2b2123776db
2024-09-27 08:04:18 +00:00
Jakob Ackermann
e3feafb7b0 Merge pull request #20332 from overleaf/jpa-small-runner
[misc] CI improvements

GitOrigin-RevId: 819d4d357fcb130f133337668644eedcf5f4a688
2024-09-26 08:04:38 +00:00
Jakob Ackermann
8fa676082e Merge pull request #20299 from overleaf/jpa-object-persistor-metrics-dep
[object-persistor] depend on @overleaf/metrics directly

GitOrigin-RevId: eb0c07af8101d44def14154abb552bc77254e074
2024-09-24 08:05:50 +00:00