* [mics] fix "app" label in clsi-cache metrics in dev-env
* [clsi-cache] validate filePath when processing file
* [clsi-cache] meter ingress and egress bandwidth
Files are downloaded directly from nginx, hence we cannot meter egress
in clsi-cache easily.
GitOrigin-RevId: 24de8c41728f0e9c984113c1470dec6153e75f20
* Replace `diskusage` by `fs` in clsi
* Replace `diskusage` by `fs` in clsi-cache
* Update disk space calculations to include block size in bytes
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
* Add warning comments about Docker-for-Mac fs stats being off by a factor
---------
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 02ea07e531b89bb3d10ddfe780348b19cbddad1f
* [clsi-cache] initial revision of the clsi-cache service
* [clsi] send output files to clsi-cache and import from clsi-cache
* [web] pass editorId to clsi
* [web] clear clsi-cache when clearing clsi cache
* [web] add split-tests for controlling clsi-cache rollout
* [web] populate clsi-cache when cloning/creating project from template
* [clsi-cache] produce less noise when populating cache hits 404
* [clsi-cache] push docker image to AR
* [clsi-cache] push docker image to AR
* [clsi-cache] allow compileGroup in job payload
* [clsi-cache] set X-Zone header from latest endpoint
* [clsi-cache] use method POST for /enqueue endpoint
* [web] populate clsi-cache in zone b with template data
* [clsi-cache] limit number of editors per project/user folder to 10
* [web] clone: populate the clsi-cache unless the TeXLive release changed
* [clsi-cache] keep user folder when clearing cache as anonymous user
* [clsi] download old output.tar.gz when synctex finds empty compile dir
* [web] fix lint
* [clsi-cache] multi-zonal lookup of single build output
* [clsi-cache] add more validation and limits
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [clsi] do not include clsi-cache tar-ball in output.zip
* [clsi-cache] fix reference after remaining constant
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
* [web] consolidate validation of filename into ClsiCacheHandler
* [clsi-cache] extend metrics and event tracking
- break down most of the clsi metrics by label
- compile=initial - new compile dir without previous output files
- compile=recompile - recompile in existing compile dir
- compile=from-cache - compile using previous clsi-cache
- extend segmentation on compile-result-backend event
- isInitialCompile=true - found new compile dir at start of request
- restoredClsiCache=true - restored compile dir from clsi-cache
* [clsi] rename metrics labels for download of clsi-cache
This is in preparation for synctex changes.
* [clsi] use constant for limit of entries in output.tar.gz
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* [clsi-cache] fix cloning of project cache
---------
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 4901a65497af13be1549af7f38ceee3188fcf881
* Set `node-fetch` to `^2.7.0`
* Update package-lock.json
```
# root
bin/npm update node-fetch
bin/npm update cross-fetch
# in other path in docker
npm update node-fetch
```
* Update node-fetch patch
* [fetch-utils] Skip the test: destroys the request body if it doesn't get consumed
```
1) fetch-utils
fetchJson
destroys the request body if it doesn't get consumed:
FetchError: Invalid response body while trying to fetch http://example.com:30001/json/ignore-request: write EPIPE
at PassThrough.<anonymous> (/overleaf/node_modules/node-fetch/lib/index.js:400:12)
at PassThrough.emit (node:events:529:35)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
```
* [fetch-utils] Delete the test: destroys the request body if it doesn't get consumed
* Remove the `setTimeout` in the node-fetch patch
Fixes a test and doesn't break filestore acceptance tests
* Update node-fetch patch again: bring changes from e87b093/src/index.js
* Update node-fetch patch again: bring changes from e87b093/src/index.js
* Update node-fetch patches back to single lines
Per https://github.com/overleaf/internal/pull/20165#discussion_r1739035513
GitOrigin-RevId: 945e5a12e838673b7bc87b588b7aca1bcd9109e2
* Revert "Revert "Bump async to 3.2.2 (#7618)""
This reverts commit 75153a555211d654744c2e61e27fe21085826c22.
* [web] fix usage of async.queue.drain in script
* [clsi] fix usage of async.queue.drain
* [spelling] fix usage of async.queue.drain
* [redis-wrapper] fix usage of async.queue.drain
* [web] Test that LockManager queue is cleared
This protects against a regression found when upgrading the
async package. Here we test that the `queue.drain` callback
is really getting called, and the lock is being removed from
the LOCK_QUEUES map.
* [redis-wrapper] Upgrade async to 3.2.2
GitOrigin-RevId: df921e6d7f1d505bd467f22e58600ba1aff48869
