diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/bug_report.md similarity index 83% rename from .github/ISSUE_TEMPLATE.md rename to .github/ISSUE_TEMPLATE/bug_report.md index 3a375bcbe9..9c0577106e 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,10 +1,19 @@ +--- +name: Bug report +about: Report a bug +title: '' +labels: type:bug +assignees: '' + +--- + diff --git a/README.md b/README.md index 4895254926..33e7a8956b 100644 --- a/README.md +++ b/README.md @@ -14,39 +14,52 @@ License
-- Figure 1: A screenshot of a project being edited in Overleaf Community Edition. + Figure 1: A screenshot of a project being edited in Overleaf Extended Community Edition.
## Community Edition -[Overleaf](https://www.overleaf.com) is an open-source online real-time collaborative LaTeX editor. We run a hosted version at [www.overleaf.com](https://www.overleaf.com), but you can also run your own local version, and contribute to the development of Overleaf. +[Overleaf](https://www.overleaf.com) is an open-source online real-time collaborative LaTeX editor. Overleaf runs a hosted version at [www.overleaf.com](https://www.overleaf.com), but you can also run your own local version, and contribute to the development of Overleaf. + +## Extended Community Edition + +The present "extended" version of Overleaf CE includes: + +- Template Gallery +- Sandboxed Compiles with TeX Live image selection +- LDAP authentication +- SAML authentication +- OpenID Connect authentication +- Real-time track changes and comments +- Autocomplete of reference keys +- Symbol Palette +- "From External URL" feature + +> [!CAUTION] +> Overleaf Community Edition is intended for use in environments where **all** users are trusted. Community Edition is **not** appropriate for scenarios where isolation of users is required due to Sandbox Compiles not being available. When not using Sandboxed Compiles, users have full read and write access to the `sharelatex` container resources (filesystem, network, environment variables) when running LaTeX compiles. +Therefore, in any environment where not all users can be fully trusted, it is strongly recommended to enable the Sandboxed Compiles feature available in the Extended Community Edition. + +For more information on Sandbox Compiles check out Overleaf [documentation](https://docs.overleaf.com/on-premises/configuration/overleaf-toolkit/server-pro-only-configuration/sandboxed-compiles). ## Enterprise -If you want help installing and maintaining Overleaf in your lab or workplace, we offer an officially supported version called [Overleaf Server Pro](https://www.overleaf.com/for/enterprises). It also includes more features for security (SSO with LDAP or SAML), administration and collaboration (e.g. tracked changes). [Find out more!](https://www.overleaf.com/for/enterprises) - -## Keeping up to date - -Sign up to the [mailing list](https://mailchi.mp/overleaf.com/community-edition-and-server-pro) to get updates on Overleaf releases and development. +If you want help installing and maintaining Overleaf in your lab or workplace, Overleaf offers an officially supported version called [Overleaf Server Pro](https://www.overleaf.com/for/enterprises). ## Installation -We have detailed installation instructions in the [Overleaf Toolkit](https://github.com/overleaf/toolkit/). - -## Upgrading - -If you are upgrading from a previous version of Overleaf, please see the [Release Notes section on the Wiki](https://github.com/overleaf/overleaf/wiki#release-notes) for all of the versions between your current version and the version you are upgrading to. +Detailed installation instructions can be found in the [Overleaf Toolkit](https://github.com/overleaf/toolkit/). +Configuration details and release history for the Extended Community Edition can be found on the [Extended CE Wiki Page](https://github.com/yu-i-i/overleaf-cep/wiki). ## Overleaf Docker Image This repo contains two dockerfiles, [`Dockerfile-base`](server-ce/Dockerfile-base), which builds the -`sharelatex/sharelatex-base` image, and [`Dockerfile`](server-ce/Dockerfile) which builds the -`sharelatex/sharelatex` (or "community") image. +`sharelatex/sharelatex-base:ext-ce` image, and [`Dockerfile`](server-ce/Dockerfile) which builds the +`sharelatex/sharelatex:ext-ce` image. The Base image generally contains the basic dependencies like `wget`, plus `texlive`. -We split this out because it's a pretty heavy set of +This is split out because it's a pretty heavy set of dependencies, and it's nice to not have to rebuild all of that every time. The `sharelatex/sharelatex` image extends the base image and adds the actual Overleaf code @@ -54,20 +67,16 @@ and services. Use `make build-base` and `make build-community` from `server-ce/` to build these images. -We use the [Phusion base-image](https://github.com/phusion/baseimage-docker) -(which is extended by our `base` image) to provide us with a VM-like container +The [Phusion base-image](https://github.com/phusion/baseimage-docker) +(which is extended by the `base` image) provides a VM-like container in which to run the Overleaf services. Baseimage uses the `runit` service -manager to manage services, and we add our init-scripts from the `server-ce/runit` -folder. - - -## Contributing - -Please see the [CONTRIBUTING](CONTRIBUTING.md) file for information on contributing to the development of Overleaf. +manager to manage services, and init scripts from the `server-ce/runit` +folder are added. ## Authors -[The Overleaf Team](https://www.overleaf.com/about) +[The Overleaf Team](https://www.overleaf.com/about) +[yu-i-i](https://github.com/yu-i-i/overleaf-cep) — Extensions for CE unless otherwise noted ## License diff --git a/develop/README.md b/develop/README.md index 568259c4e3..8e3f89862c 100644 --- a/develop/README.md +++ b/develop/README.md @@ -42,7 +42,7 @@ To do this, use the included `bin/dev` script: bin/dev ``` -This will start all services using `nodemon`, which will automatically monitor the code and restart the services as necessary. +This will start all services using `node --watch`, which will automatically monitor the code and restart the services as necessary. To improve performance, you can start only a subset of the services in development mode by providing a space-separated list to the `bin/dev` script: @@ -77,6 +77,7 @@ each service: | `filestore` | 9235 | | `notifications` | 9236 | | `real-time` | 9237 | +| `references` | 9238 | | `history-v1` | 9239 | | `project-history` | 9240 | diff --git a/develop/dev.env b/develop/dev.env index aae91497db..b003e4e0eb 100644 --- a/develop/dev.env +++ b/develop/dev.env @@ -6,14 +6,18 @@ DOCUMENT_UPDATER_HOST=document-updater FILESTORE_HOST=filestore GRACEFUL_SHUTDOWN_DELAY_SECONDS=0 HISTORY_V1_HOST=history-v1 +HISTORY_REDIS_HOST=redis LISTEN_ADDRESS=0.0.0.0 MONGO_HOST=mongo MONGO_URL=mongodb://mongo/sharelatex?directConnection=true NOTIFICATIONS_HOST=notifications PROJECT_HISTORY_HOST=project-history +QUEUES_REDIS_HOST=redis REALTIME_HOST=real-time REDIS_HOST=redis +REFERENCES_HOST=references SESSION_SECRET=foo +V1_HISTORY_HOST=history-v1 WEBPACK_HOST=webpack WEB_API_PASSWORD=overleaf WEB_API_USER=overleaf diff --git a/develop/docker-compose.dev.yml b/develop/docker-compose.dev.yml index 4432a24162..65e15ef07b 100644 --- a/develop/docker-compose.dev.yml +++ b/develop/docker-compose.dev.yml @@ -112,8 +112,19 @@ services: - ../services/real-time/app.js:/overleaf/services/real-time/app.js - ../services/real-time/config:/overleaf/services/real-time/config + references: + command: ["node", "--watch", "app.js"] + environment: + - NODE_OPTIONS=--inspect=0.0.0.0:9229 + ports: + - "127.0.0.1:9238:9229" + volumes: + - ../services/references/app:/overleaf/services/references/app + - ../services/references/config:/overleaf/services/references/config + - ../services/references/app.js:/overleaf/services/references/app.js + web: - command: ["node", "--watch", "app.js", "--watch-locales"] + command: ["node", "--watch", "app.mjs", "--watch-locales"] environment: - NODE_OPTIONS=--inspect=0.0.0.0:9229 ports: diff --git a/develop/docker-compose.yml b/develop/docker-compose.yml index d0dc8ec6da..e5b84c38b3 100644 --- a/develop/docker-compose.yml +++ b/develop/docker-compose.yml @@ -1,6 +1,5 @@ volumes: clsi-cache: - clsi-output: filestore-public-files: filestore-template-files: filestore-uploads: @@ -26,15 +25,16 @@ services: env_file: - dev.env environment: - - DOCKER_RUNNER=true - TEXLIVE_IMAGE=texlive-full # docker build texlive -t texlive-full - - COMPILES_HOST_DIR=${PWD}/compiles + - SANDBOXED_COMPILES=true + - SANDBOXED_COMPILES_HOST_DIR_COMPILES=${PWD}/compiles + - SANDBOXED_COMPILES_HOST_DIR_OUTPUT=${PWD}/output user: root volumes: - ${PWD}/compiles:/overleaf/services/clsi/compiles + - ${PWD}/output:/overleaf/services/clsi/output - ${DOCKER_SOCKET_PATH:-/var/run/docker.sock}:/var/run/docker.sock - clsi-cache:/overleaf/services/clsi/cache - - clsi-output:/overleaf/services/clsi/output contacts: build: @@ -123,7 +123,7 @@ services: dockerfile: services/real-time/Dockerfile env_file: - dev.env - + redis: image: redis:5 ports: @@ -131,6 +131,13 @@ services: volumes: - redis-data:/data + references: + build: + context: .. + dockerfile: services/references/Dockerfile + env_file: + - dev.env + web: build: context: .. @@ -140,7 +147,7 @@ services: - dev.env environment: - APP_NAME=Overleaf Community Edition - - ENABLED_LINKED_FILE_TYPES=project_file,project_output_file + - ENABLED_LINKED_FILE_TYPES=project_file,project_output_file,url - EMAIL_CONFIRMATION_DISABLED=true - NODE_ENV=development - OVERLEAF_ALLOW_PUBLIC_ACCESS=true @@ -161,6 +168,7 @@ services: - notifications - project-history - real-time + - references webpack: build: diff --git a/doc/logo.png b/doc/logo.png index 106926b095..f154ac0cca 100644 Binary files a/doc/logo.png and b/doc/logo.png differ diff --git a/doc/screenshot.png b/doc/screenshot.png index 1c1f339630..92633192a5 100644 Binary files a/doc/screenshot.png and b/doc/screenshot.png differ diff --git a/docker-compose.yml b/docker-compose.yml index 08d6db6fe7..962adfb5d8 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -32,7 +32,7 @@ services: OVERLEAF_REDIS_HOST: redis REDIS_HOST: redis - ENABLED_LINKED_FILE_TYPES: 'project_file,project_output_file' + ENABLED_LINKED_FILE_TYPES: 'project_file,project_output_file,url' # Enables Thumbnail generation using ImageMagick ENABLE_CONVERSIONS: 'true' @@ -73,11 +73,19 @@ services: ## Server Pro ## ################ - ## Sandboxed Compiles: https://github.com/overleaf/overleaf/wiki/Server-Pro:-Sandboxed-Compiles + ## The Community Edition is intended for use in environments where all users are trusted and is not appropriate for + ## scenarios where isolation of users is required. Sandboxed Compiles are not available in the Community Edition, + ## so the following environment variables must be commented out to avoid compile issues. + ## + ## Sandboxed Compiles: https://docs.overleaf.com/on-premises/configuration/overleaf-toolkit/server-pro-only-configuration/sandboxed-compiles SANDBOXED_COMPILES: 'true' - SANDBOXED_COMPILES_SIBLING_CONTAINERS: 'true' ### Bind-mount source for /var/lib/overleaf/data/compiles inside the container. - SANDBOXED_COMPILES_HOST_DIR: '/home/user/sharelatex_data/data/compiles' + SANDBOXED_COMPILES_HOST_DIR_COMPILES: '/home/user/sharelatex_data/data/compiles' + ### Bind-mount source for /var/lib/overleaf/data/output inside the container. + SANDBOXED_COMPILES_HOST_DIR_OUTPUT: '/home/user/sharelatex_data/data/output' + ### Backwards compatibility (before Server Pro 5.5) + DOCKER_RUNNER: 'true' + SANDBOXED_COMPILES_SIBLING_CONTAINERS: 'true' ## Works with test LDAP server shown at bottom of docker compose # OVERLEAF_LDAP_URL: 'ldap://ldap:389' diff --git a/libraries/access-token-encryptor/.dockerignore b/libraries/access-token-encryptor/.dockerignore deleted file mode 100644 index c2658d7d1b..0000000000 --- a/libraries/access-token-encryptor/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -node_modules/ diff --git a/libraries/access-token-encryptor/.gitignore b/libraries/access-token-encryptor/.gitignore deleted file mode 100644 index 66936c4121..0000000000 --- a/libraries/access-token-encryptor/.gitignore +++ /dev/null @@ -1,46 +0,0 @@ -compileFolder - -Compiled source # -################### -*.com -*.class -*.dll -*.exe -*.o -*.so - -# Packages # -############ -# it's better to unpack these files and commit the raw source -# git has its own built in compression methods -*.7z -*.dmg -*.gz -*.iso -*.jar -*.rar -*.tar -*.zip - -# Logs and databases # -###################### -*.log -*.sql -*.sqlite - -# OS generated files # -###################### -.DS_Store? -ehthumbs.db -Icon? -Thumbs.db - -/node_modules/* -data/*/* - -**.swp - -/log.json -hash_folder - -.npmrc diff --git a/libraries/access-token-encryptor/.nvmrc b/libraries/access-token-encryptor/.nvmrc index 0254b1e633..fc37597bcc 100644 --- a/libraries/access-token-encryptor/.nvmrc +++ b/libraries/access-token-encryptor/.nvmrc @@ -1 +1 @@ -20.18.2 +22.17.0 diff --git a/libraries/access-token-encryptor/buildscript.txt b/libraries/access-token-encryptor/buildscript.txt index 967c3063ad..8f08720e7e 100644 --- a/libraries/access-token-encryptor/buildscript.txt +++ b/libraries/access-token-encryptor/buildscript.txt @@ -1,10 +1,10 @@ access-token-encryptor --dependencies=None ---docker-repos=gcr.io/overleaf-ops +--docker-repos=us-east1-docker.pkg.dev/overleaf-ops/ol-docker --env-add= --env-pass-through= --esmock-loader=False --is-library=True ---node-version=20.18.2 +--node-version=22.17.0 --public-repo=False ---script-version=4.5.0 +--script-version=4.7.0 diff --git a/libraries/fetch-utils/.dockerignore b/libraries/fetch-utils/.dockerignore deleted file mode 100644 index c2658d7d1b..0000000000 --- a/libraries/fetch-utils/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -node_modules/ diff --git a/libraries/fetch-utils/.gitignore b/libraries/fetch-utils/.gitignore deleted file mode 100644 index edb0f85350..0000000000 --- a/libraries/fetch-utils/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ - -# managed by monorepo$ bin/update_build_scripts -.npmrc diff --git a/libraries/fetch-utils/.nvmrc b/libraries/fetch-utils/.nvmrc index 0254b1e633..fc37597bcc 100644 --- a/libraries/fetch-utils/.nvmrc +++ b/libraries/fetch-utils/.nvmrc @@ -1 +1 @@ -20.18.2 +22.17.0 diff --git a/libraries/fetch-utils/buildscript.txt b/libraries/fetch-utils/buildscript.txt index 716d3f0d5c..7008a533a6 100644 --- a/libraries/fetch-utils/buildscript.txt +++ b/libraries/fetch-utils/buildscript.txt @@ -1,10 +1,10 @@ fetch-utils --dependencies=None ---docker-repos=gcr.io/overleaf-ops +--docker-repos=us-east1-docker.pkg.dev/overleaf-ops/ol-docker --env-add= --env-pass-through= --esmock-loader=False --is-library=True ---node-version=20.18.2 +--node-version=22.17.0 --public-repo=False ---script-version=4.5.0 +--script-version=4.7.0 diff --git a/libraries/fetch-utils/index.js b/libraries/fetch-utils/index.js index 643dcc752b..60e8180c7d 100644 --- a/libraries/fetch-utils/index.js +++ b/libraries/fetch-utils/index.js @@ -23,11 +23,11 @@ async function fetchJson(url, opts = {}) { } async function fetchJsonWithResponse(url, opts = {}) { - const { fetchOpts } = parseOpts(opts) + const { fetchOpts, detachSignal } = parseOpts(opts) fetchOpts.headers = fetchOpts.headers ?? {} fetchOpts.headers.Accept = fetchOpts.headers.Accept ?? 'application/json' - const response = await performRequest(url, fetchOpts) + const response = await performRequest(url, fetchOpts, detachSignal) if (!response.ok) { const body = await maybeGetResponseBody(response) throw new RequestFailedError(url, opts, response, body) @@ -53,8 +53,8 @@ async function fetchStream(url, opts = {}) { } async function fetchStreamWithResponse(url, opts = {}) { - const { fetchOpts, abortController } = parseOpts(opts) - const response = await performRequest(url, fetchOpts) + const { fetchOpts, abortController, detachSignal } = parseOpts(opts) + const response = await performRequest(url, fetchOpts, detachSignal) if (!response.ok) { const body = await maybeGetResponseBody(response) @@ -76,8 +76,8 @@ async function fetchStreamWithResponse(url, opts = {}) { * @throws {RequestFailedError} if the response has a failure status code */ async function fetchNothing(url, opts = {}) { - const { fetchOpts } = parseOpts(opts) - const response = await performRequest(url, fetchOpts) + const { fetchOpts, detachSignal } = parseOpts(opts) + const response = await performRequest(url, fetchOpts, detachSignal) if (!response.ok) { const body = await maybeGetResponseBody(response) throw new RequestFailedError(url, opts, response, body) @@ -108,9 +108,9 @@ async function fetchRedirect(url, opts = {}) { * @throws {RequestFailedError} if the response has a non redirect status code or missing Location header */ async function fetchRedirectWithResponse(url, opts = {}) { - const { fetchOpts } = parseOpts(opts) + const { fetchOpts, detachSignal } = parseOpts(opts) fetchOpts.redirect = 'manual' - const response = await performRequest(url, fetchOpts) + const response = await performRequest(url, fetchOpts, detachSignal) if (response.status < 300 || response.status >= 400) { const body = await maybeGetResponseBody(response) throw new RequestFailedError(url, opts, response, body) @@ -142,8 +142,8 @@ async function fetchString(url, opts = {}) { } async function fetchStringWithResponse(url, opts = {}) { - const { fetchOpts } = parseOpts(opts) - const response = await performRequest(url, fetchOpts) + const { fetchOpts, detachSignal } = parseOpts(opts) + const response = await performRequest(url, fetchOpts, detachSignal) if (!response.ok) { const body = await maybeGetResponseBody(response) throw new RequestFailedError(url, opts, response, body) @@ -178,13 +178,14 @@ function parseOpts(opts) { const abortController = new AbortController() fetchOpts.signal = abortController.signal + let detachSignal = () => {} if (opts.signal) { - abortOnSignal(abortController, opts.signal) + detachSignal = abortOnSignal(abortController, opts.signal) } if (opts.body instanceof Readable) { abortOnDestroyedRequest(abortController, fetchOpts.body) } - return { fetchOpts, abortController } + return { fetchOpts, abortController, detachSignal } } function setupJsonBody(fetchOpts, json) { @@ -208,6 +209,9 @@ function abortOnSignal(abortController, signal) { abortController.abort(signal.reason) } signal.addEventListener('abort', listener) + return () => { + signal.removeEventListener('abort', listener) + } } function abortOnDestroyedRequest(abortController, stream) { @@ -226,11 +230,12 @@ function abortOnDestroyedResponse(abortController, response) { }) } -async function performRequest(url, fetchOpts) { +async function performRequest(url, fetchOpts, detachSignal) { let response try { response = await fetch(url, fetchOpts) } catch (err) { + detachSignal() if (fetchOpts.body instanceof Readable) { fetchOpts.body.destroy() } @@ -239,6 +244,7 @@ async function performRequest(url, fetchOpts) { method: fetchOpts.method ?? 'GET', }) } + response.body.on('close', detachSignal) if (fetchOpts.body instanceof Readable) { response.body.on('close', () => { if (!fetchOpts.body.readableEnded) { diff --git a/libraries/fetch-utils/test/unit/FetchUtilsTests.js b/libraries/fetch-utils/test/unit/FetchUtilsTests.js index e9fd0ff231..691e90778d 100644 --- a/libraries/fetch-utils/test/unit/FetchUtilsTests.js +++ b/libraries/fetch-utils/test/unit/FetchUtilsTests.js @@ -1,6 +1,9 @@ const { expect } = require('chai') +const fs = require('node:fs') +const events = require('node:events') const { FetchError, AbortError } = require('node-fetch') const { Readable } = require('node:stream') +const { pipeline } = require('node:stream/promises') const { once } = require('node:events') const { TestServer } = require('./helpers/TestServer') const selfsigned = require('selfsigned') @@ -203,6 +206,31 @@ describe('fetch-utils', function () { ).to.be.rejectedWith(AbortError) expect(stream.destroyed).to.be.true }) + + it('detaches from signal on success', async function () { + const signal = AbortSignal.timeout(10_000) + for (let i = 0; i < 20; i++) { + const s = await fetchStream(this.url('/hello'), { signal }) + expect(events.getEventListeners(signal, 'abort')).to.have.length(1) + await pipeline(s, fs.createWriteStream('/dev/null')) + expect(events.getEventListeners(signal, 'abort')).to.have.length(0) + } + }) + + it('detaches from signal on error', async function () { + const signal = AbortSignal.timeout(10_000) + for (let i = 0; i < 20; i++) { + try { + await fetchStream(this.url('/500'), { signal }) + } catch (err) { + if (err instanceof RequestFailedError && err.response.status === 500) + continue + throw err + } finally { + expect(events.getEventListeners(signal, 'abort')).to.have.length(0) + } + } + }) }) describe('fetchNothing', function () { @@ -391,9 +419,16 @@ async function* infiniteIterator() { async function abortOnceReceived(func, server) { const controller = new AbortController() const promise = func(controller.signal) + expect(events.getEventListeners(controller.signal, 'abort')).to.have.length(1) await once(server.events, 'request-received') controller.abort() - return await promise + try { + return await promise + } finally { + expect(events.getEventListeners(controller.signal, 'abort')).to.have.length( + 0 + ) + } } async function expectRequestAborted(req) { diff --git a/libraries/logger/.dockerignore b/libraries/logger/.dockerignore deleted file mode 100644 index c2658d7d1b..0000000000 --- a/libraries/logger/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -node_modules/ diff --git a/libraries/logger/.gitignore b/libraries/logger/.gitignore deleted file mode 100644 index 2006c875a4..0000000000 --- a/libraries/logger/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -node_modules - -.npmrc diff --git a/libraries/logger/.nvmrc b/libraries/logger/.nvmrc index 0254b1e633..fc37597bcc 100644 --- a/libraries/logger/.nvmrc +++ b/libraries/logger/.nvmrc @@ -1 +1 @@ -20.18.2 +22.17.0 diff --git a/libraries/logger/buildscript.txt b/libraries/logger/buildscript.txt index 7db4c0c4c3..e2520c4800 100644 --- a/libraries/logger/buildscript.txt +++ b/libraries/logger/buildscript.txt @@ -1,10 +1,10 @@ logger --dependencies=None ---docker-repos=gcr.io/overleaf-ops +--docker-repos=us-east1-docker.pkg.dev/overleaf-ops/ol-docker --env-add= --env-pass-through= --esmock-loader=False --is-library=True ---node-version=20.18.2 +--node-version=22.17.0 --public-repo=False ---script-version=4.5.0 +--script-version=4.7.0 diff --git a/libraries/metrics/.dockerignore b/libraries/metrics/.dockerignore deleted file mode 100644 index c2658d7d1b..0000000000 --- a/libraries/metrics/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -node_modules/ diff --git a/libraries/metrics/.gitignore b/libraries/metrics/.gitignore deleted file mode 100644 index 2006c875a4..0000000000 --- a/libraries/metrics/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -node_modules - -.npmrc diff --git a/libraries/metrics/.nvmrc b/libraries/metrics/.nvmrc index 0254b1e633..fc37597bcc 100644 --- a/libraries/metrics/.nvmrc +++ b/libraries/metrics/.nvmrc @@ -1 +1 @@ -20.18.2 +22.17.0 diff --git a/libraries/metrics/buildscript.txt b/libraries/metrics/buildscript.txt index 2919cb8335..d1e272c356 100644 --- a/libraries/metrics/buildscript.txt +++ b/libraries/metrics/buildscript.txt @@ -1,10 +1,10 @@ metrics --dependencies=None ---docker-repos=gcr.io/overleaf-ops +--docker-repos=us-east1-docker.pkg.dev/overleaf-ops/ol-docker --env-add= --env-pass-through= --esmock-loader=False --is-library=True ---node-version=20.18.2 +--node-version=22.17.0 --public-repo=False ---script-version=4.5.0 +--script-version=4.7.0 diff --git a/libraries/metrics/initialize.js b/libraries/metrics/initialize.js index 1028ee06c3..f1a77666c7 100644 --- a/libraries/metrics/initialize.js +++ b/libraries/metrics/initialize.js @@ -5,6 +5,8 @@ * before any other module to support code instrumentation. */ +const metricsModuleImportStartTime = performance.now() + const APP_NAME = process.env.METRICS_APP_NAME || 'unknown' const BUILD_VERSION = process.env.BUILD_VERSION const ENABLE_PROFILE_AGENT = process.env.ENABLE_PROFILE_AGENT === 'true' @@ -103,3 +105,5 @@ function recordProcessStart() { const metrics = require('.') metrics.inc('process_startup') } + +module.exports = { metricsModuleImportStartTime } diff --git a/libraries/metrics/package.json b/libraries/metrics/package.json index 384e58cfe5..19b566c2b0 100644 --- a/libraries/metrics/package.json +++ b/libraries/metrics/package.json @@ -9,7 +9,7 @@ "main": "index.js", "dependencies": { "@google-cloud/opentelemetry-cloud-trace-exporter": "^2.1.0", - "@google-cloud/profiler": "^6.0.0", + "@google-cloud/profiler": "^6.0.3", "@opentelemetry/api": "^1.4.1", "@opentelemetry/auto-instrumentations-node": "^0.39.1", "@opentelemetry/exporter-trace-otlp-http": "^0.41.2", diff --git a/libraries/mongo-utils/.dockerignore b/libraries/mongo-utils/.dockerignore deleted file mode 100644 index c2658d7d1b..0000000000 --- a/libraries/mongo-utils/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -node_modules/ diff --git a/libraries/mongo-utils/.gitignore b/libraries/mongo-utils/.gitignore deleted file mode 100644 index edb0f85350..0000000000 --- a/libraries/mongo-utils/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ - -# managed by monorepo$ bin/update_build_scripts -.npmrc diff --git a/libraries/mongo-utils/.nvmrc b/libraries/mongo-utils/.nvmrc index 0254b1e633..fc37597bcc 100644 --- a/libraries/mongo-utils/.nvmrc +++ b/libraries/mongo-utils/.nvmrc @@ -1 +1 @@ -20.18.2 +22.17.0 diff --git a/libraries/mongo-utils/batchedUpdate.js b/libraries/mongo-utils/batchedUpdate.js index 5e97f45aca..41af41f0d4 100644 --- a/libraries/mongo-utils/batchedUpdate.js +++ b/libraries/mongo-utils/batchedUpdate.js @@ -16,6 +16,7 @@ let VERBOSE_LOGGING let BATCH_RANGE_START let BATCH_RANGE_END let BATCH_MAX_TIME_SPAN_IN_MS +let BATCHED_UPDATE_RUNNING = false /** * @typedef {import("mongodb").Collection} Collection @@ -34,6 +35,7 @@ let BATCH_MAX_TIME_SPAN_IN_MS * @property {string} [BATCH_RANGE_START] * @property {string} [BATCH_SIZE] * @property {string} [VERBOSE_LOGGING] + * @property {(progress: string) => Promise