Commit graph

21 commits

Author SHA1 Message Date
Jakob Ackermann
b538d56591 [clsi-cache] backend (#24388)
* [clsi-cache] initial revision of the clsi-cache service

* [clsi] send output files to clsi-cache and import from clsi-cache

* [web] pass editorId to clsi

* [web] clear clsi-cache when clearing clsi cache

* [web] add split-tests for controlling clsi-cache rollout

* [web] populate clsi-cache when cloning/creating project from template

* [clsi-cache] produce less noise when populating cache hits 404

* [clsi-cache] push docker image to AR

* [clsi-cache] push docker image to AR

* [clsi-cache] allow compileGroup in job payload

* [clsi-cache] set X-Zone header from latest endpoint

* [clsi-cache] use method POST for /enqueue endpoint

* [web] populate clsi-cache in zone b with template data

* [clsi-cache] limit number of editors per project/user folder to 10

* [web] clone: populate the clsi-cache unless the TeXLive release changed

* [clsi-cache] keep user folder when clearing cache as anonymous user

* [clsi] download old output.tar.gz when synctex finds empty compile dir

* [web] fix lint

* [clsi-cache] multi-zonal lookup of single build output

* [clsi-cache] add more validation and limits

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [clsi] do not include clsi-cache tar-ball in output.zip

* [clsi-cache] fix reference after remaining constant

Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>

* [web] consolidate validation of filename into ClsiCacheHandler

* [clsi-cache] extend metrics and event tracking

- break down most of the clsi metrics by label
  - compile=initial - new compile dir without previous output files
  - compile=recompile - recompile in existing compile dir
  - compile=from-cache - compile using previous clsi-cache
- extend segmentation on compile-result-backend event
  - isInitialCompile=true - found new compile dir at start of request
  - restoredClsiCache=true - restored compile dir from clsi-cache

* [clsi] rename metrics labels for download of clsi-cache

This is in preparation for synctex changes.

* [clsi] use constant for limit of entries in output.tar.gz

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [clsi-cache] fix cloning of project cache

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 4901a65497af13be1549af7f38ceee3188fcf881
2025-04-10 08:05:17 +00:00
Jakob Ackermann
1117ea1b3e [clsi] add helper function for detecting extraneous files (#24376)
* [clsi] add helper function for detecting extraneous files

* [clsi] output.pdfxref and output.synctex.gz are extraneous as well

* [clsi] minor code cleanup

Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>

---------

Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: 0d6c96c99b6be3adaee1558b1f958da885e6307a
2025-03-24 10:46:43 +00:00
Jakob Ackermann
d19c5e236f Merge pull request #22208 from overleaf/jpa-clsi-hash
[misc] clsi: read files from history-v1 with fallback to filestore

GitOrigin-RevId: c54bb128780198c14e7a63818f39fad62ce65d4e
2024-11-29 09:05:39 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Eric Mc Sween
16fee6d7d2 Merge pull request #12203 from overleaf/em-camel-case-clsi
Camel case variables in clsi

GitOrigin-RevId: 1542a6b7f8886e5fef496ce31acd2bba537befa9
2023-03-23 09:04:46 +00:00
Eric Mc Sween
77aa2baa9d Merge pull request #8714 from overleaf/em-promisify-compile-manager
Promisify CompileManager

GitOrigin-RevId: 644ed061ae139d6196b24f8ead38579de6b844a3
2022-07-08 08:04:22 +00:00
Eric Mc Sween
e0d91eaa26 Merge pull request #7906 from overleaf/em-downgrade-logs
Downgrade all INFO logs to DEBUG

GitOrigin-RevId: 05ed582ef0721fcada059f0ad158565f50feca27
2022-05-17 08:05:26 +00:00
Jakob Ackermann
264b107e39 Merge pull request #6927 from overleaf/jpa-split-clsi-metrics
[misc] split clsi metrics into multiple categories

GitOrigin-RevId: 964ab57b344b92383a2937beff50139bae273a0e
2022-03-02 09:02:52 +00:00
Jakob Ackermann
958216d578 Merge pull request #6926 from overleaf/jpa-clsi-bump-logger-metrics
[clsi] upgrade logger and metrics module

GitOrigin-RevId: 85c346b5eed683672a77d86d6a434dc313b7824b
2022-03-02 09:02:50 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
2021-10-28 08:03:26 +00:00
Jakob Ackermann
02918e7483 Merge pull request #4649 from overleaf/jpa-fs-based-caching
[perf] UrlCache: pure fs based cache state for downloads

GitOrigin-RevId: d19afc396324d4c3318b31620c8ad0c04e0544ce
2021-10-07 08:03:18 +00:00
Jakob Ackermann
f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann
64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann
18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Jakob Ackermann
1d1b9ebebc [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:11 +01:00
Brian Gough
6d5dfb7758 clean up log file deletion and add unit test 2020-05-20 14:12:08 +01:00
Brian Gough
3db513cfc9 record latexmk output 2020-05-15 16:08:10 +01:00
Jakob Ackermann
6777432569 [misc] add a metric for failing downloads 2020-04-03 12:35:40 +02:00
mserranom
cf6c8ab496 removed mkdirp dependency and replaced with fs.mkdir 2020-03-12 10:22:08 +01:00
mserranom
8729acd48c prettier: convert app/js decaffeinated files to Prettier format 2020-02-19 12:14:37 +01:00
mserranom
3af6bdd588 decaffeinate: rename app/coffee dir to app/js 2020-02-19 12:14:34 +01:00
Renamed from services/clsi/app/coffee/ResourceWriter.js (Browse further)