Add hotfixes 6.1.1 and 6.1.2 (adapted)

- enable Projects button in navbar for admin projects page
- enable copy of user ID in user info modal
- add null check in UserListController (thanks @moschlar)

.github/ISSUE_TEMPLATE/bug_report.md

@@ -9,12 +9,8 @@ assignees: ''
 
 <!--
 
-Note: If you are using www.overleaf.com and have a problem,
-      or if you would like to request a new feature please contact
-      the support team at support@overleaf.com
-
-      This form should only be used to report bugs in the
-      Community Edition release of Overleaf.
+Note: This form should only be used to report bugs in the
+      Extended CE release of Overleaf.
 
 -->
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -4,8 +4,3 @@
 
 ## Related issues / Pull Requests
 <!-- Fixes #xyz, Contributes to #xyz, Related to #xyz-->
-
-
-## Contributor Agreement
-
-- [ ] I confirm I have signed the [Contributor License Agreement](https://github.com/overleaf/overleaf/blob/main/CONTRIBUTING.md#contributor-license-agreement)

CONTRIBUTING.md

@@ -1,43 +1,33 @@
-Contributing to Overleaf
-==========================
+Contributing to Overleaf CE+
+============================
 
 Thank you for reading this! If you'd like to report a bug or join in the development
-of Overleaf, then here are some notes on how to do that.
+of Overleaf CE+, then here are some notes on how to do that.
 
 Reporting bugs and opening issues
 ---------------------------------
 
-If you'd like to report a bug or open an issue, please **[check if there is an existing issue](https://github.com/overleaf/overleaf/issues).**
+If you'd like to report a bug or open an issue, please **[check if there is an existing issue](https://github.com/yu-i-i/overleaf-cep).**
 If there is then please add any more information that you have, or give it a 👍.
 
 When submitting an issue please describe the issue as clearly as possible, including how to
 reproduce the bug, which situations it appears in, what you expected to happen, and what actually happens.
 If you can include a screenshot for front end issues that is very helpful.
 
-**Note**: If you are using [www.overleaf.com](www.overleaf.com) and have a problem, or if you would like to request a new feature, please contact the Support team at support@overleaf.com. Raise an issue here only to report bugs in the Community Edition release of Overleaf.
 
 Pull Requests
 -------------
 
-See [our wiki](https://github.com/overleaf/overleaf/wiki)
+See [Overleaf wiki](https://github.com/overleaf/overleaf/wiki)
 for how to manage the Overleaf development environment and for our developer guidelines.
 
 We love pull requests, so be bold with them! Don't be afraid of going ahead
 and changing something, or adding a new feature. We're very happy to work with you
 to get your changes merged into Overleaf.
 
-If you're looking for something to work on, have a look at the [open issues](https://github.com/overleaf/overleaf/issues).
+If you're looking for something to work on, have a look at the [open issues](https://github.com/yu-i-i/overleaf-cep/issues).
 
 Security
 --------
 
-Please see [our security policy](https://github.com/overleaf/overleaf/security/policy) if you would like to report a potential security vulnerability.
-
-Contributor License Agreement
------------------------------
-
-Before we can accept any contributions of code, we need you to agree to our
-[Contributor License Agreement](https://docs.google.com/forms/d/e/1FAIpQLSef79XH3mb7yIiMzZw-yALEegS-wyFetvjTiNBfZvf_IHD2KA/viewform?usp=sf_link).
-This is to ensure that you own the copyright of your contribution, and that you
-agree to give us a license to use it in both the open source version, and the version
-of Overleaf running at www.overleaf.com, which may have additional changes.
+Please see [Overleaf security policy](https://github.com/overleaf/overleaf/security/policy) if you would like to report a potential security vulnerability.

CREDITS.md

@@ -0,0 +1,70 @@
+# CREDITS
+
+This project includes ideas and adapted code from several open-source projects.\
+In most cases the original code has been modified, optimized, or extended.
+
+## Symbol palette
+
+The symbol palette feature is based on the original Overleaf implementation:
+
+https://github.com/overleaf/web/tree/master/frontend/js/features/symbol-palette
+
+The original code was slightly improved, particularly in the parts related to keyboard input.
+
+## LDAP authentication
+
+LDAP authentication was inspired by:
+
+https://github.com/smhaller/ldap-overleaf-sl
+
+The project provided the idea and part of the implementation for adding LDAP users to the user's contacts.
+
+## Autocomplete of reference keys
+
+Autocomplete of reference keys is based on:
+
+https://github.com/lcpu-club/overleaf
+
+The implementation largely follows the referenced code, except that a different `.bib` file parser is used.
+
+## Real-time track changes and comments
+
+The Track Changes and Comments feature exists largely in the original Overleaf codebase.\
+The missing parts were implemented based on:
+
+https://github.com/ertuil/overleaf
+
+The referenced code was fixed, optimized, and extended.
+
+## Sandboxed compiles
+
+The Sandboxed Compiles feature was until recently largely present in the original Overleaf codebase.\
+The missing parts were implemented in this project.
+
+## Import file from external URL
+
+The "From External URL" feature exists in the original Overleaf code.\
+The missing proxy component was implemented in this project.
+
+## Git integration
+
+Git integration is based on:
+
+https://github.com/ayaka-notes/overleaf-pro/tree/feat-git-bridge
+
+The backend was completely rewritten.\
+The frontend uses the referenced code, which was optimized and improved.
+
+## Original Features
+
+The following features are original work implemented in this project:
+
+- Template Gallery
+- Advanced administrator tools for managing user accounts and projects
+- SAML authentication
+- OpenID Connect authentication
+
+## Acknowledgments
+
+Thanks to the users of the project for valuable feedback, suggestions,
+and help in identifying and fixing bugs.

README.md

@@ -21,19 +21,21 @@
 
 [Overleaf](https://www.overleaf.com) is an open-source online real-time collaborative LaTeX editor. Overleaf runs a hosted version at [www.overleaf.com](https://www.overleaf.com), but you can also run your own local version, and contribute to the development of Overleaf.
 
-## Extended Community Edition
+## Extended Community Edition (CE+)
 
 The present "extended" version of Overleaf CE includes:
 
-- Template Gallery
-- Sandboxed Compiles with TeX Live image selection
+- Template gallery
+- Sandboxed compiles with TeX Live image selection
 - LDAP authentication
 - SAML authentication
 - OpenID Connect authentication
 - Real-time track changes and comments
 - Autocomplete of reference keys
-- Symbol Palette
-- "From External URL" feature
+- Symbol palette
+- Import file from external URL
+- Advanced administrator tools for managing user accounts and projects
+- Git integration
 
 > [!CAUTION]
 > Overleaf Community Edition is intended for use in environments where **all** users are trusted. Community Edition is **not** appropriate for scenarios where isolation of users is required due to Sandbox Compiles not being available. When not using Sandboxed Compiles, users have full read and write access to the `sharelatex` container resources (filesystem, network, environment variables) when running LaTeX compiles. 
@@ -73,11 +75,14 @@ folder are added.
 
 ## Authors
 
-[The Overleaf Team](https://www.overleaf.com/about)  
-[yu-i-i](https://github.com/yu-i-i/overleaf-cep) — Extensions for CE unless otherwise noted
+[The Overleaf Team](https://www.overleaf.com/about)\
+[yu-i-i](https://github.com/yu-i-i/overleaf-cep) — CE extensions; references to adapted code are listed in [`CREDITS`](CREDITS.md)
 
 ## License
 
 The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the [`LICENSE`](LICENSE) file.
 
-Copyright (c) Overleaf, 2014-2025.
+Copyright (c) Overleaf, 2014-2026.\
+Copyright (c) yu-i-i, 2024-2026, for CE extensions.
+
+Portions of the code are derived from other open-source projects; see [`CREDITS`](CREDITS.md).

develop/docker-compose.dev.yml

@@ -88,7 +88,7 @@ services:
     volumes:
       - ../services/linked-url-proxy/app:/overleaf/services/linked-url-proxy/app
       - ../services/linked-url-proxy/config:/overleaf/services/linked-url-proxy/config
-      - ../services/linked-url-proxy/app.js:/overleaf/services/linked-url-proxy/app.js
+      - ../services/linked-url-proxy/app.mjs:/overleaf/services/linked-url-proxy/app.mjs
 
   notifications:
     command: ["node", "--watch", "app.ts"]

develop/docker-compose.yml

@@ -129,7 +129,7 @@ services:
       dockerfile: services/real-time/Dockerfile
     env_file:
       - dev.env
- 
+
   redis:
     image: redis:5
     ports:
@@ -187,3 +187,18 @@ services:
       - "127.0.0.1:80:3808"
     volumes:
       - ./webpack.config.dev-env.js:/overleaf/services/web/webpack.config.dev-env.js
+
+  git-bridge:
+    image: quay.io/sharelatex/git-bridge:latest
+    expose:
+      - "8000"
+    ports:
+      - "8000:8000"
+    environment:
+      GIT_BRIDGE_API_BASE_URL: "http://web:3000/api/v0"
+      GIT_BRIDGE_OAUTH2_SERVER: "http://web:3000"
+      GIT_BRIDGE_POSTBACK_BASE_URL: "http://git-bridge:8000"
+      GIT_BRIDGE_ROOT_DIR: "/data/git-bridge"
+      LOG_LEVEL: "DEBUG"
+    volumes:
+      - ./data/git-bridge:/data/git-bridge

develop/webpack.config.dev-env.js

@@ -14,6 +14,11 @@ module.exports = merge(base, {
         target: 'http://real-time:3026',
         ws: true,
       },
+      {
+        context: '/git/**',
+        target: 'http://git-bridge:8000',
+        pathRewrite: { '^/git': '' }
+      },
       {
         context: ['!**/*.js', '!**/*.css', '!**/*.json'],
         target: 'http://web:3000',

libraries/access-token-encryptor/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/access-token-encryptor/**
-package-lock.json
-package.json
-patches/**

libraries/access-token-encryptor/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/access-token-encryptor/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/access-token-encryptor monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/access-token-encryptor/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'access-token-encryptor-eslint', name: 'access-token-encryptor eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/access-token-encryptor/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/access-token-encryptor monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/access-token-encryptor monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/access-token-encryptor monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'access-token-encryptor test results', testResults: 'libraries/access-token-encryptor/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/access-token-encryptor/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/access-token-encryptor/buildscript.txt

@@ -5,4 +5,5 @@ access-token-encryptor
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/access-token-encryptor/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/eslint-plugin/index.js

@@ -0,0 +1,11 @@
+module.exports = {
+  rules: {
+    'no-unnecessary-trans': require('./no-unnecessary-trans'),
+    'prefer-kebab-url': require('./prefer-kebab-url'),
+    'should-unescape-trans': require('./should-unescape-trans'),
+    'no-generated-editor-themes': require('./no-generated-editor-themes'),
+    'require-script-runner': require('./require-script-runner'),
+    'require-vi-doMock-valid-path': require('./require-vi-doMock-valid-path'),
+    'require-loading-label': require('./require-loading-label'),
+  },
+}

libraries/eslint-plugin/no-generated-editor-themes.js

@@ -0,0 +1,21 @@
+module.exports = {
+  meta: {
+    type: 'error',
+    docs: {
+      description:
+        'Prohibit CodeMirror themes that are generated in a function',
+    },
+  },
+  create(context) {
+    return {
+      ':matches(ArrowFunctionExpression, FunctionDeclaration, FunctionExpression) CallExpression > MemberExpression[object.name="EditorView"]:matches([property.name="theme"],[property.name="baseTheme"])'(
+        node
+      ) {
+        context.report({
+          node,
+          message: `EditorView.theme and EditorView.baseTheme each add CSS to the page for every instance of the theme. Store the theme in a variable and reuse it instead.`,
+        })
+      },
+    }
+  },
+}

libraries/eslint-plugin/no-unnecessary-trans.js

@@ -0,0 +1,43 @@
+module.exports = {
+  meta: {
+    type: 'problem',
+    fixable: 'code',
+    docs: {
+      description: 'Prohibit Trans with no components or values',
+    },
+  },
+  create(context) {
+    return {
+      'JSXOpeningElement[name.name="Trans"]'(node) {
+        const attributes = new Map(
+          node.attributes.map(attr => [attr.name.name, attr])
+        )
+
+        if (!attributes.has('components')) {
+          if (node.parent.children.length > 0) {
+            context.report({
+              node,
+              message: `Trans components must not have child elements`,
+            })
+          } else if (attributes.has('values')) {
+            context.report({
+              node,
+              message: `Use t('…') when there are no components`,
+            })
+          } else {
+            context.report({
+              node,
+              message: `Use t('…') when there are no components`,
+              fix(fixer) {
+                const i18nKey = attributes.get('i18nKey').value.value
+
+                // Note: Prettier can fix indentation
+                return fixer.replaceText(node.parent, `{t('${i18nKey}')}`)
+              },
+            })
+          }
+        }
+      },
+    }
+  },
+}

libraries/eslint-plugin/package.json

@@ -0,0 +1,17 @@
+{
+  "name": "@overleaf/eslint-plugin",
+  "version": "0.1.0",
+  "author": "Overleaf (https://www.overleaf.com)",
+  "license": "AGPL-3.0-only",
+  "main": "index.js",
+  "dependencies": {
+    "eslint": "^8.51.0",
+    "lodash": "^4.17.21"
+  },
+  "devDependencies": {
+    "@typescript-eslint/parser": "^8.50.0"
+  },
+  "scripts": {
+    "test": "node rules.test.js"
+  }
+}

libraries/eslint-plugin/prefer-kebab-url-ignore.js

@@ -0,0 +1,83 @@
+// URL parts should be kebab-case, but we didn't have this rule in the past.
+// The ESLint rule `prefer-kebab-url` will ignore these "legacy" URL parts.
+
+const ignoreWords = {
+  snake: new Set([
+    'clear_saml_data',
+    'confirm_link',
+    'confirm_university_domain',
+    'create_recurly_account',
+    'current_history_content',
+    'current_user',
+    'default_email',
+    'disable_managed_users',
+    'doc_snapshot',
+    'enable_history_ranges_support',
+    'features_override',
+    'generate_password_reset_url',
+    'get_assignment',
+    'get_clone',
+    'health_check',
+    'institutional_emails',
+    'latest_template',
+    'link_after_saml_response',
+    'linked_file',
+    'metrics_segmentation',
+    'new_users',
+    'no_autostart_post_gateway',
+    'personal_info',
+    'planned_maintenance',
+    'refresh_features',
+    'register_admin',
+    'register_ldap_admin',
+    'register_saml_admin',
+    'restore_file',
+    'revert_file',
+    'saved_vers',
+    'send_test_email',
+    'session_maintenance',
+    'set_in_session',
+    'sign_in_to_link',
+    'split_test',
+    'sso_configuration_test',
+    'sso_email',
+    'sso_enrollment',
+    'track_changes',
+    'update_admin',
+    'user_details',
+  ]),
+  camel: new Set([
+    'addWorkflowScope',
+    'aiErrorAssistant',
+    'beginAuth',
+    'brandVariationId',
+    'closeEditor',
+    'completeRegistration',
+    'deactivateOldProjects',
+    'deletedSubscription',
+    'disconnectAllUsers',
+    'editingSession',
+    'emailSubscription',
+    'enableManagedUsers',
+    'externalCollaboration',
+    'flushProjectToTpds',
+    'indexAll',
+    'offboardManagedUser',
+    'openEditor',
+    'perfTest',
+    'pollDropboxForUser',
+    'resendInvite',
+    'resendManagedUserInvite',
+    'salesContactForm',
+    'showSupport',
+  ]),
+  other: new Set([
+    'Project',
+    'disableSSO',
+    'enableSSO',
+    'resendSSOLinkInvite',
+    'usersCSV',
+  ]),
+}
+
+module.exports = { ignoreWords }

libraries/eslint-plugin/prefer-kebab-url.js

@@ -0,0 +1,91 @@
+const _ = require('lodash')
+const { ignoreWords } = require('./prefer-kebab-url-ignore')
+
+const removeTextBetweenBrackets = text => {
+  while (text.includes('[') || text.includes('(')) {
+    text = text.replaceAll(/\[[^[\]]*]/g, '')
+    text = text.replaceAll(/\([^()]*\)/g, '')
+  }
+  return text
+}
+
+const shouldIgnoreWord = str =>
+  str.includes(':') ||
+  str.includes('(') ||
+  str === '*' ||
+  str.match(/^[a-z0-9.]+$/) ||
+  ignoreWords.snake.has(str) ||
+  ignoreWords.camel.has(str) ||
+  ignoreWords.other.has(str)
+
+const getSuggestion = routePath => {
+  if (typeof routePath === 'string') {
+    const kebabed = routePath
+      .split('/')
+      .map(word => (shouldIgnoreWord(word) ? word : _.kebabCase(word)))
+      .join('/')
+    return kebabed === routePath ? null : `'${kebabed}'`
+  }
+
+  if (routePath instanceof RegExp) {
+    const words = removeTextBetweenBrackets(routePath.source).match(/[\w-]+/g)
+    if (!words) return routePath
+
+    let newSource = routePath.source
+    for (const word of words) {
+      if (!shouldIgnoreWord(word)) {
+        newSource = newSource.replaceAll(
+          new RegExp(`\\b${word}\\b`, 'g'),
+          _.kebabCase(word)
+        )
+      }
+    }
+
+    const kebabed = new RegExp(newSource, routePath.flags)
+    return kebabed.source.toString() === routePath.source.toString()
+      ? null
+      : kebabed
+  }
+}
+
+module.exports = {
+  meta: {
+    type: 'problem',
+    fixable: 'code',
+    hasSuggestions: true,
+    docs: {
+      description: 'Enforce using kebab-case for URL paths',
+    },
+  },
+  create: context => ({
+    CallExpression(node) {
+      if (
+        node.callee.type === 'MemberExpression' &&
+        node.arguments[0]?.type === 'Literal' &&
+        [/app/i, /router/i].some(callee =>
+          typeof callee === 'string'
+            ? node.callee.object.name === callee
+            : callee.test(node.callee.object.name)
+        ) &&
+        ['get', 'post', 'put', 'delete'].includes(node.callee.property.name)
+      ) {
+        const routePath = node.arguments[0].value
+
+        const suggestion = getSuggestion(routePath)
+
+        if (suggestion) {
+          context.report({
+            node: node.arguments[0],
+            message: 'Route path should be in kebab-case.',
+            suggest: [
+              {
+                desc: `Change to kebab-case: ${suggestion}`,
+                fix: fixer => fixer.replaceText(node.arguments[0], suggestion),
+              },
+            ],
+          })
+        }
+      }
+    },
+  }),
+}

libraries/eslint-plugin/require-loading-label.js

@@ -0,0 +1,49 @@
+module.exports = {
+  meta: {
+    type: 'problem',
+    fixable: null,
+    docs: {
+      description:
+        'Require loadingLabel prop when isLoading is specified on OLButton',
+    },
+    schema: [],
+  },
+  create(context) {
+    return {
+      'JSXOpeningElement[name.name="OLButton"]'(node) {
+        const attributes = new Map(
+          node.attributes.map(attr => [attr.name?.name, attr])
+        )
+
+        const isLoadingAttr = attributes.get('isLoading')
+        const loadingLabelAttr = attributes.get('loadingLabel')
+
+        if (isLoadingAttr && !loadingLabelAttr) {
+          const isLoadingValue = isLoadingAttr.value
+
+          if (
+            !isLoadingValue ||
+            (isLoadingValue.type === 'JSXExpressionContainer' &&
+              isLoadingValue.expression.type === 'Literal' &&
+              isLoadingValue.expression.value === true)
+          ) {
+            context.report({
+              node: isLoadingAttr,
+              message:
+                'Button with isLoading prop must also specify loadingLabel',
+            })
+          } else if (
+            isLoadingValue.type === 'JSXExpressionContainer' &&
+            isLoadingValue.expression.type !== 'Literal'
+          ) {
+            context.report({
+              node: isLoadingAttr,
+              message:
+                'Button with isLoading prop must also specify loadingLabel',
+            })
+          }
+        }
+      },
+    }
+  },
+}

libraries/eslint-plugin/require-script-runner.js

@@ -0,0 +1,28 @@
+module.exports = {
+  meta: {
+    type: 'suggestion',
+    docs: {
+      description: 'Require Script Runner for scripts',
+    },
+  },
+  create(context) {
+    let hasImport = false
+
+    return {
+      ImportDeclaration(node) {
+        if (node.source.value.endsWith('lib/ScriptRunner.mjs')) {
+          hasImport = true
+        }
+      },
+      'Program:exit'() {
+        if (!hasImport) {
+          context.report({
+            loc: { line: 1, column: 0 },
+            message:
+              'Please use Script Runner for scripts. Refer to the developer manual (https://manual.dev-overleaf.com/development/code/web_scripts/#monitor-script-execution-and-usage-with-script-runner) for more information.',
+          })
+        }
+      },
+    }
+  },
+}

libraries/eslint-plugin/require-vi-doMock-valid-path.js

@@ -0,0 +1,138 @@
+const path = require('node:path')
+const fs = require('node:fs')
+
+module.exports = {
+  meta: {
+    type: 'problem',
+    docs: {
+      description: 'Ensure vi.doMock first argument is a resolvable path.',
+      category: 'Best Practices',
+      recommended: false,
+      url: '',
+    },
+    fixable: 'code',
+    hasSuggestions: true,
+    schema: [],
+    messages: {
+      unresolvablePath:
+        'The path "{{pathValue}}" in vi.doMock() cannot be resolved relative to the current file.',
+      notAStringLiteral:
+        'The first argument of vi.doMock() must be (or resolve to) a string literal representing a path.',
+      noArguments: 'vi.doMock() called with no arguments.',
+    },
+  },
+  create(context) {
+    const currentFilePath = context.getFilename()
+    // ESLint can sometimes pass <text> or <input> for snippets not in a file
+    if (currentFilePath === '<text>' || currentFilePath === '<input>') {
+      return {}
+    }
+    const currentDirectory = path.dirname(currentFilePath)
+
+    function canResolve(modulePath) {
+      try {
+        require.resolve(path.resolve(currentDirectory, modulePath))
+        return true
+      } catch (e) {
+        const absolutePath = path.resolve(currentDirectory, modulePath)
+        const extensions = [
+          '',
+          '.js',
+          '.mjs',
+          '.ts',
+          '.jsx',
+          '.tsx',
+          '.json',
+          '.node',
+          '/index.js',
+          '/index.ts',
+        ] // Add common extensions
+        for (const ext of extensions) {
+          if (fs.existsSync(absolutePath + ext)) {
+            return true
+          }
+        }
+        return false
+      }
+    }
+
+    return {
+      CallExpression(node) {
+        if (
+          node.callee.type === 'MemberExpression' &&
+          node.callee.object.type === 'Identifier' &&
+          node.callee.object.name === 'vi' &&
+          node.callee.property.type === 'Identifier' &&
+          node.callee.property.name === 'doMock'
+        ) {
+          if (node.arguments.length === 0) {
+            context.report({
+              node,
+              messageId: 'noArguments',
+            })
+            return
+          }
+
+          const firstArg = node.arguments[0]
+          let pathValue = firstArg.value
+
+          if (
+            firstArg.type !== 'Literal' ||
+            typeof firstArg.value !== 'string'
+          ) {
+            if (firstArg.type === 'Identifier') {
+              const variable = context
+                .getScope()
+                .variables.find(v => v.name === firstArg.name)
+              if (
+                variable &&
+                variable.defs.length > 0 &&
+                variable.defs[0].node.init &&
+                variable.defs[0].node.init.type === 'Literal' &&
+                typeof variable.defs[0].node.init.value === 'string'
+              ) {
+                pathValue = variable.defs[0].node.init.value
+                if (canResolve(pathValue)) {
+                  return
+                }
+                // If the first argument was a variable that didn't resolve then we can't auto-fix it
+              }
+            }
+            context.report({
+              node: firstArg,
+              messageId: 'notAStringLiteral',
+            })
+            return
+          }
+
+          if (!pathValue.startsWith('.')) {
+            return
+          }
+
+          if (!canResolve(pathValue)) {
+            const mjsPath = pathValue.replace('.js', '.mjs')
+            const additionalReportOptions = {}
+            if (canResolve(mjsPath)) {
+              additionalReportOptions.fix = fixer =>
+                fixer.replaceText(firstArg, `'${mjsPath}'`)
+              additionalReportOptions.suggest = [
+                {
+                  desc: `Replace with "${pathValue.replace('.js', '.mjs')}"`,
+                  fix: fixer => fixer.replaceText(firstArg, `'${mjsPath}'`),
+                },
+              ]
+            }
+            context.report({
+              node: firstArg,
+              messageId: 'unresolvablePath',
+              data: {
+                pathValue,
+              },
+              ...additionalReportOptions,
+            })
+          }
+        }
+      },
+    }
+  },
+}

libraries/eslint-plugin/rules.test.js

@@ -0,0 +1,168 @@
+const { RuleTester } = require('eslint')
+const preferKebabUrl = require('./prefer-kebab-url')
+const noUnnecessaryTrans = require('./no-unnecessary-trans')
+const shouldUnescapeTrans = require('./should-unescape-trans')
+const noGeneratedEditorThemes = require('./no-generated-editor-themes')
+const viDoMockValidPath = require('./require-vi-doMock-valid-path')
+
+const ruleTester = new RuleTester({
+  parser: require.resolve('@typescript-eslint/parser'),
+  parserOptions: {
+    ecmaVersion: 'latest',
+    ecmaFeatures: { jsx: true },
+  },
+})
+
+ruleTester.run('prefer-kebab-url', preferKebabUrl, {
+  valid: [
+    { code: `app.get('/foo-bar')` },
+    { code: `app.get('/foo-bar/:id')` },
+    { code: `router.post('/foo-bar')` },
+    { code: `router.get('/foo-bar/:id/:name/:age')` },
+    { code: `webRouter.get('/foo-bar/:user_id/(ProjectName)/get-info')` },
+    { code: `webApp.post('/foo-bar/:user_id/(ProjectName)/get-info')` },
+    {
+      code: `router.get(/^\\/download\\/project\\/([^/]*)\\/output\\/output\\.pdf$/)`,
+    },
+    {
+      code: `webRouter.get(/^\\/project\\/([^/]*)\\/user\\/([0-9a-f]+)\\/build\\/([0-9a-f-]+)\\/output\\/(.*)$/)`,
+    },
+  ],
+  invalid: [
+    {
+      code: `app.get('/fooBar')`,
+      errors: [{ message: 'Route path should be in kebab-case.' }],
+    },
+    {
+      code: `app.get('/fooBar/:id')`,
+      errors: [{ message: 'Route path should be in kebab-case.' }],
+    },
+    {
+      code: `webRouter.get('/foo_bar/:id/FooBar/:name/fooBar')`,
+      errors: [{ message: 'Route path should be in kebab-case.' }],
+    },
+    {
+      code: `router.get(/^\\/downLoad\\/pro-ject\\/([^/]*)\\/OutPut\\/out-put\\.pdf$/)`,
+      errors: [{ message: 'Route path should be in kebab-case.' }],
+    },
+  ],
+})
+
+ruleTester.run('no-unnecessary-trans', noUnnecessaryTrans, {
+  valid: [
+    { code: `<Trans i18nKey="test" components={{ strong: <strong/> }}/>` },
+  ],
+  invalid: [
+    {
+      code: `<Trans i18nKey="test" values={{ test: 'foo '}}/>`,
+      errors: [{ message: `Use t('…') when there are no components` }],
+    },
+    {
+      code: `<Trans i18nKey="test" />`,
+      errors: [{ message: `Use t('…') when there are no components` }],
+      output: `{t('test')}`,
+    },
+  ],
+})
+
+ruleTester.run('should-unescape-trans', shouldUnescapeTrans, {
+  valid: [
+    {
+      code: `<Trans i18nKey="test" components={{ strong: <strong/> }}/>`,
+    },
+    {
+      code: `<Trans i18nKey="test" values={{ foo: 'bar' }} components={{ strong: <strong/> }} shouldUnescape tOptions={{ interpolation: { escapeValue: true } }}/>`,
+    },
+  ],
+  invalid: [
+    {
+      code: `<Trans i18nKey="test" values={{ foo: 'bar' }} components={{ strong: <strong/> }} />`,
+      errors: [{ message: 'Trans with values must have shouldUnescape' }],
+      output: `<Trans i18nKey="test" values={{ foo: 'bar' }}\nshouldUnescape components={{ strong: <strong/> }} />`,
+    },
+    {
+      code: `<Trans i18nKey="test" values={{ foo: 'bar' }} components={{ strong: <strong/> }} shouldUnescape />`,
+      errors: [
+        {
+          message:
+            'Trans with shouldUnescape must have tOptions.interpolation.escapeValue',
+        },
+      ],
+      output: `<Trans i18nKey="test" values={{ foo: 'bar' }} components={{ strong: <strong/> }} shouldUnescape\ntOptions={{ interpolation: { escapeValue: true } }} />`,
+    },
+  ],
+})
+
+const noGeneratedEditorThemesError =
+  'EditorView.theme and EditorView.baseTheme each add CSS to the page for every instance of the theme. Store the theme in a variable and reuse it instead.'
+ruleTester.run('no-generated-editor-themes', noGeneratedEditorThemes, {
+  valid: [
+    {
+      code: `EditorView.theme({ '.cm-editor': { color: 'black' } })`,
+    },
+    {
+      code: `const theme = EditorView.theme({ '.cm-editor': { color: 'black' } })`,
+    },
+  ],
+  invalid: [
+    {
+      code: `function createTheme() { return EditorView.theme({ '.cm-editor': { color: 'black' } }) }`,
+      errors: [
+        {
+          message: noGeneratedEditorThemesError,
+        },
+      ],
+    },
+    {
+      code: `() => EditorView.theme({ '.cm-editor': { color: 'black' } })`,
+      errors: [
+        {
+          message: noGeneratedEditorThemesError,
+        },
+      ],
+    },
+    {
+      code: `class Foo { createTheme() { return EditorView.theme({ '.cm-editor': { color: 'black' } }) } }`,
+      errors: [
+        {
+          message: noGeneratedEditorThemesError,
+        },
+      ],
+    },
+  ],
+})
+
+ruleTester.run('domock-require-valid-path', viDoMockValidPath, {
+  valid: [
+    {
+      code: 'vi.doMock("./require-vi-doMock-valid-path.js")',
+      filename: __filename,
+    },
+    {
+      code: 'const filename = "./require-vi-doMock-valid-path.js"; vi.doMock(filename);',
+      filename: __filename,
+    },
+  ],
+  invalid: [
+    {
+      code: "vi.doMock('./require-vi-doMock-valid-path2')",
+      filename: __filename,
+      errors: [
+        {
+          message:
+            'The path "./require-vi-doMock-valid-path2" in vi.doMock() cannot be resolved relative to the current file.',
+        },
+      ],
+    },
+    {
+      code: 'const filename = "./require-vi-doMock-valid-path2.js"; vi.doMock(filename);',
+      filename: __filename,
+      errors: [
+        {
+          message:
+            'The first argument of vi.doMock() must be (or resolve to) a string literal representing a path.',
+        },
+      ],
+    },
+  ],
+})

libraries/eslint-plugin/should-unescape-trans.js

@@ -0,0 +1,60 @@
+module.exports = {
+  meta: {
+    type: 'problem',
+    fixable: 'code',
+    docs: {
+      description: 'Ensure that Trans with values has shouldUnescape',
+    },
+  },
+  create(context) {
+    return {
+      'JSXOpeningElement[name.name="Trans"]'(node) {
+        const attributes = new Map(
+          node.attributes.map(attr => [attr.name.name, attr])
+        )
+
+        if (attributes.has('values') && !attributes.has('shouldUnescape')) {
+          context.report({
+            node,
+            message: 'Trans with values must have shouldUnescape',
+            fix(fixer) {
+              return fixer.insertTextAfter(
+                attributes.get('values'),
+                '\nshouldUnescape' // Note: Prettier can fix indentation
+              )
+            },
+          })
+        }
+
+        if (attributes.has('values') && attributes.has('shouldUnescape')) {
+          const tOptions = attributes.get('tOptions')
+          if (!tOptions) {
+            context.report({
+              node,
+              message:
+                'Trans with shouldUnescape must have tOptions.interpolation.escapeValue',
+              fix(fixer) {
+                return fixer.insertTextAfter(
+                  attributes.get('shouldUnescape'),
+                  '\ntOptions={{ interpolation: { escapeValue: true } }}' // Note: Prettier can fix indentation
+                )
+              },
+            })
+          } else {
+            const property = tOptions.value.expression.properties
+              .find(p => p.key.name === 'interpolation')
+              ?.value.properties.find(p => p.key.name === 'escapeValue')
+
+            if (property?.value.value !== true) {
+              context.report({
+                node,
+                message:
+                  'Trans with shouldUnescape must have tOptions.interpolation.escapeValue set to true',
+              })
+            }
+          }
+        }
+      },
+    }
+  },
+}

libraries/eslint-plugin/tsconfig.json

@@ -0,0 +1,6 @@
+{
+  "extends": "../../tsconfig.backend.json",
+  "include": [
+    "**/*.js",
+  ]
+}

libraries/fetch-utils/.jenkinsIncludeFile

@@ -1,8 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/fetch-utils/**
-libraries/o-error/**
-package-lock.json
-package.json
-patches/**

libraries/fetch-utils/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/fetch-utils/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/fetch-utils monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/fetch-utils/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'fetch-utils-eslint', name: 'fetch-utils eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/fetch-utils/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/fetch-utils monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/fetch-utils monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/fetch-utils monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'fetch-utils test results', testResults: 'libraries/fetch-utils/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/fetch-utils/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/fetch-utils/buildscript.txt

@@ -5,4 +5,5 @@ fetch-utils
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/fetch-utils/package.json

@@ -17,10 +17,10 @@
   "license": "AGPL-3.0-only",
   "devDependencies": {
     "@types/node-fetch": "^2.6.13",
-    "body-parser": "^1.20.3",
+    "body-parser": "1.20.4",
     "chai": "^4.3.6",
     "chai-as-promised": "^7.1.1",
-    "express": "^4.21.2",
+    "express": "4.22.1",
     "mocha": "^11.1.0",
     "mocha-junit-reporter": "^2.2.1",
     "mocha-multi-reporters": "^1.5.1",

libraries/fetch-utils/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/logger/.jenkinsIncludeFile

@@ -1,9 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/fetch-utils/**
-libraries/logger/**
-libraries/o-error/**
-package-lock.json
-package.json
-patches/**

libraries/logger/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/logger/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/logger monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/logger/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'logger-eslint', name: 'logger eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/logger/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/logger monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/logger monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/logger monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'logger test results', testResults: 'libraries/logger/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/logger/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/logger/buildscript.txt

@@ -5,4 +5,5 @@ logger
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/logger/test-log-recorder.js

@@ -0,0 +1,20 @@
+const logger = require('./')
+const bunyan = require('bunyan')
+const serializers = require('./serializers')
+
+function testLogRecorder() {
+  const currentTest = this.currentTest
+  for (const level of ['error', 'fatal']) {
+    logger[level] = (info, msg) => {
+      const entry = { level, ...info, msg }
+      for (const [name, fn] of Object.entries(serializers)) {
+        if (name in entry) entry[name] = fn(entry[name])
+      }
+      currentTest.consoleErrors = (currentTest.consoleErrors || []).concat(
+        JSON.stringify(info, bunyan.safeCycles())
+      )
+    }
+  }
+}
+
+module.exports = testLogRecorder

libraries/logger/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/metrics/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/metrics/**
-package-lock.json
-package.json
-patches/**

libraries/metrics/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/metrics/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/metrics monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/metrics/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'metrics-eslint', name: 'metrics eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/metrics/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/metrics monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/metrics monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/metrics monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'metrics test results', testResults: 'libraries/metrics/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/metrics/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/metrics/buildscript.txt

@@ -5,4 +5,5 @@ metrics
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/metrics/index.js

@@ -97,6 +97,12 @@ function histogram(key, value, buckets, labels = {}) {
 }
 
 class Timer {
+  /**
+   * @param {string} key
+   * @param {number} sampleRate
+   * @param {Record<string, any>} labels
+   * @param {Array<number>} buckets
+   */
   constructor(key, sampleRate = 1, labels = {}, buckets = undefined) {
     if (typeof sampleRate === 'object') {
       // called with (key, labels, buckets)

libraries/metrics/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/mongo-utils/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/mongo-utils/**
-package-lock.json
-package.json
-patches/**

libraries/mongo-utils/Jenkinsfile

@@ -1,72 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/mongo-utils/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/mongo-utils monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/mongo-utils/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'mongo-utils-eslint', name: 'mongo-utils eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/mongo-utils/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/mongo-utils monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/mongo-utils monorepo npm run types:check'
-          }
-        }
-      }
-    }
-  }
-  post {
-    cleanup {
-      sh 'rm -rf libraries/mongo-utils/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/mongo-utils/buildscript.txt

@@ -5,4 +5,5 @@ mongo-utils
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/o-error/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/o-error/**
-package-lock.json
-package.json
-patches/**

libraries/o-error/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/o-error/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/o-error monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/o-error/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'o-error-eslint', name: 'o-error eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/o-error/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/o-error monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/o-error monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/o-error monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'o-error test results', testResults: 'libraries/o-error/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/o-error/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/o-error/buildscript.txt

@@ -5,4 +5,5 @@ o-error
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/o-error/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/object-persistor/.jenkinsIncludeFile

@@ -1,12 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/fetch-utils/**
-libraries/logger/**
-libraries/metrics/**
-libraries/o-error/**
-libraries/object-persistor/**
-libraries/stream-utils/**
-package-lock.json
-package.json
-patches/**

libraries/object-persistor/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/object-persistor/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/object-persistor monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/object-persistor/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'object-persistor-eslint', name: 'object-persistor eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/object-persistor/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/object-persistor monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/object-persistor monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/object-persistor monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'object-persistor test results', testResults: 'libraries/object-persistor/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/object-persistor/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/object-persistor/buildscript.txt

@@ -5,4 +5,5 @@ object-persistor
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/object-persistor/package.json

@@ -30,7 +30,7 @@
     "@overleaf/o-error": "*",
     "@overleaf/stream-utils": "*",
     "fast-crc32c": "overleaf/node-fast-crc32c#aae6b2a4c7a7a159395df9cc6c38dfde702d6f51",
-    "glob": "^7.1.6",
+    "glob": "^12.0.0",
     "range-parser": "^1.2.1",
     "tiny-async-pool": "^1.1.0"
   },

libraries/object-persistor/src/AbstractPersistor.js

@@ -160,4 +160,38 @@ module.exports = class AbstractPersistor {
       name,
     })
   }
+
+  /**
+   * List objects in a directory, returning the full keys.
+   *
+   * Suitable only for directories where the number of keys is known to be small.
+   *
+   * @param {string} location
+   * @param {string} prefix
+   * @returns {Promise<Array<string>>}
+   */
+  async listDirectoryKeys(location, prefix) {
+    throw new NotImplementedError('method not implemented in persistor', {
+      method: 'listDirectoryKeys',
+      location,
+      prefix,
+    })
+  }
+
+  /**
+   * List objects in a directory, returning key and size information.
+   *
+   * Suitable only for directories where the number of keys is known to be small.
+   *
+   * @param {string} location
+   * @param {string} prefix
+   * @returns {Promise<Array<{key: string, size: number}>>}
+   */
+  async listDirectoryStats(location, prefix) {
+    throw new NotImplementedError('method not implemented in persistor', {
+      method: 'listDirectoryStats',
+      location,
+      prefix,
+    })
+  }
 }

libraries/object-persistor/src/FSPersistor.js

@@ -1,18 +1,15 @@
 const crypto = require('node:crypto')
 const fs = require('node:fs')
 const fsPromises = require('node:fs/promises')
-const globCallbacks = require('glob')
+const { glob } = require('glob')
 const Path = require('node:path')
 const { PassThrough } = require('node:stream')
 const { pipeline } = require('node:stream/promises')
-const { promisify } = require('node:util')
 
 const AbstractPersistor = require('./AbstractPersistor')
 const { ReadError, WriteError, NotImplementedError } = require('./Errors')
 const PersistorHelper = require('./PersistorHelper')
 
-const glob = promisify(globCallbacks)
-
 module.exports = class FSPersistor extends AbstractPersistor {
   constructor(settings = {}) {
     if (settings.storageClass) {
@@ -197,6 +194,50 @@ module.exports = class FSPersistor extends AbstractPersistor {
     }
   }
 
+  async listDirectoryKeys(location, name) {
+    const fsPath = this._getFsPath(location, name)
+    const paths = await this._listDirectory(fsPath)
+
+    // Filter to only return files, not directories
+    const files = []
+    for (const path of paths) {
+      try {
+        const stat = await fsPromises.stat(path)
+        if (stat.isFile()) {
+          files.push(path)
+        }
+      } catch (err) {
+        // ignore files that may have just been deleted
+        if (err.code !== 'ENOENT') {
+          throw err
+        }
+      }
+    }
+    return files
+  }
+
+  async listDirectoryStats(location, name) {
+    const fsPath = this._getFsPath(location, name)
+    const paths = await this._listDirectory(fsPath)
+
+    // Filter to only return files, not directories, with their sizes
+    const stats = []
+    for (const path of paths) {
+      try {
+        const stat = await fsPromises.stat(path)
+        if (stat.isFile()) {
+          stats.push({ key: path, size: stat.size })
+        }
+      } catch (err) {
+        // ignore files that may have just been deleted
+        if (err.code !== 'ENOENT') {
+          throw err
+        }
+      }
+    }
+    return stats
+  }
+
   async checkIfObjectExists(location, name) {
     const fsPath = this._getFsPath(location, name)
     try {
@@ -305,10 +346,8 @@ module.exports = class FSPersistor extends AbstractPersistor {
 
   async _listDirectory(path) {
     if (this.useSubdirectories) {
-      // eslint-disable-next-line @typescript-eslint/return-await
       return await glob(Path.join(path, '**'))
     } else {
-      // eslint-disable-next-line @typescript-eslint/return-await
       return await glob(`${path}_*`)
     }
   }

libraries/object-persistor/src/GcsPersistor.js

@@ -288,23 +288,25 @@ module.exports = class GcsPersistor extends AbstractPersistor {
     } while (query)
   }
 
-  async directorySize(bucketName, key) {
-    let files
-    const prefix = ensurePrefixIsDirectory(key)
-
+  async #listDirectory(bucketName, prefix) {
     try {
       const [response] = await this.storage
         .bucket(bucketName)
         .getFiles({ prefix })
-      files = response
+      return response
     } catch (err) {
       throw PersistorHelper.wrapError(
         err,
         'failed to list objects in GCS',
-        { bucketName, key },
+        { bucketName, prefix },
         ReadError
       )
     }
+  }
+
+  async directorySize(bucketName, key) {
+    const prefix = ensurePrefixIsDirectory(key)
+    const files = await this.#listDirectory(bucketName, prefix)
 
     return files.reduce(
       (acc, file) => parseInt(file.metadata.size, 10) + acc,
@@ -312,6 +314,25 @@ module.exports = class GcsPersistor extends AbstractPersistor {
     )
   }
 
+  async listDirectoryKeys(bucketName, prefix) {
+    const files = await this.#listDirectory(
+      bucketName,
+      ensurePrefixIsDirectory(prefix)
+    )
+    return files.map(file => file.name)
+  }
+
+  async listDirectoryStats(bucketName, prefix) {
+    const files = await this.#listDirectory(
+      bucketName,
+      ensurePrefixIsDirectory(prefix)
+    )
+    return files.map(file => ({
+      key: file.name,
+      size: parseInt(file.metadata.size, 10),
+    }))
+  }
+
   async checkIfObjectExists(bucketName, key) {
     try {
       const [response] = await this.storage

libraries/object-persistor/src/PerProjectEncryptedS3Persistor.js

@@ -417,10 +417,20 @@ class CachedPerProjectEncryptedS3Persistor {
    *
    * @param {string} bucketName
    * @param {string} path
-   * @return {Promise<ListDirectoryResult>}
+   * @return {Promise<string[]>}
    */
-  async listDirectory(bucketName, path) {
-    return await this.#parent.listDirectory(bucketName, path)
+  async listDirectoryKeys(bucketName, path) {
+    return await this.#parent.listDirectoryKeys(bucketName, path)
+  }
+
+  /**
+   *
+   * @param {string} bucketName
+   * @param {string} path
+   * @return {Promise<Array<{key: string, size: number}>>}
+   */
+  async listDirectoryStats(bucketName, path) {
+    return await this.#parent.listDirectoryStats(bucketName, path)
   }
 
   /**

libraries/object-persistor/src/ProjectKey.js

@@ -1,4 +1,3 @@
-// Keep in sync with services/filestore/app/js/project_key.js
 const path = require('node:path')
 
 //
@@ -21,3 +20,4 @@ function naiveReverse(string) {
 
 exports.format = format
 exports.pad = pad
+exports.naiveReverse = naiveReverse

libraries/object-persistor/src/S3Persistor.js

@@ -266,7 +266,7 @@ class S3Persistor extends AbstractPersistor {
    * @return {Promise<void>}
    */
   async deleteDirectory(bucketName, key, continuationToken) {
-    const { contents, response } = await this.listDirectory(
+    const { contents, response } = await this.#listDirectory(
       bucketName,
       key,
       continuationToken
@@ -309,7 +309,7 @@ class S3Persistor extends AbstractPersistor {
    * @param {string} [continuationToken]
    * @return {Promise<ListDirectoryResult>}
    */
-  async listDirectory(bucketName, key, continuationToken) {
+  async #listDirectory(bucketName, key, continuationToken) {
     let response
     const options = { Bucket: bucketName, Prefix: key }
     if (continuationToken) {
@@ -535,6 +535,59 @@ class S3Persistor extends AbstractPersistor {
     }
   }
 
+  /**
+   * @param {string} bucketName
+   * @param {string} prefix
+   * @return {Promise<Array<string>>}
+   */
+  async listDirectoryKeys(bucketName, prefix) {
+    const keys = []
+    let continuationToken
+
+    do {
+      const { contents, response } = await this.#listDirectory(
+        bucketName,
+        prefix,
+        continuationToken
+      )
+      keys.push(...contents.map(item => item.Key || ''))
+      continuationToken = response.IsTruncated
+        ? response.NextContinuationToken
+        : undefined
+    } while (continuationToken)
+
+    return keys
+  }
+
+  /**
+   * @param {string} bucketName
+   * @param {string} prefix
+   * @return {Promise<Array<{key: string, size: number}>>}
+   */
+  async listDirectoryStats(bucketName, prefix) {
+    const stats = []
+    let continuationToken
+
+    do {
+      const { contents, response } = await this.#listDirectory(
+        bucketName,
+        prefix,
+        continuationToken
+      )
+      stats.push(
+        ...contents.map(item => ({
+          key: item.Key || '',
+          size: item.Size ?? -1,
+        }))
+      )
+      continuationToken = response.IsTruncated
+        ? response.NextContinuationToken
+        : undefined
+    } while (continuationToken)
+
+    return stats
+  }
+
   /**
    * @param {string} bucket
    * @return {S3Client}

libraries/object-persistor/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/object-persistor/test/setup.js

@@ -27,7 +27,7 @@ SandboxedModule.configure({
   globals: { Buffer, Math, console, process, URL },
   sourceTransformers: {
     removeNodePrefix: function (source) {
-      return source.replace(/require\(['"]node:/g, "require('")
+      return source.replace(/require\((['"])node:/g, 'require($1')
     },
   },
 })

libraries/object-persistor/test/unit/FSPersistorTests.js

@@ -400,6 +400,56 @@ describe('FSPersistorTests', function () {
           ).to.equal(0)
         })
       })
+
+      describe('listDirectoryKeys', function () {
+        beforeEach(async function () {
+          for (const file of Object.values(files)) {
+            await persistor.sendFile(location, file, '/uploads/info.txt')
+          }
+        })
+
+        it('should list directory keys', async function () {
+          const keys = await persistor.listDirectoryKeys(location, 'animals')
+          expect(keys).to.have.lengthOf(2)
+          expect(keys).to.include(scenario.fsPath(files.wombat))
+          expect(keys).to.include(scenario.fsPath(files.giraffe))
+        })
+
+        it('should return empty array for non-existing directories', async function () {
+          const keys = await persistor.listDirectoryKeys(
+            location,
+            'does-not-exist'
+          )
+          expect(keys).to.deep.equal([])
+        })
+      })
+
+      describe('listDirectoryStats', function () {
+        beforeEach(async function () {
+          for (const file of Object.values(files)) {
+            await persistor.sendFile(location, file, '/uploads/info.txt')
+          }
+        })
+
+        it('should list directory stats', async function () {
+          const stats = await persistor.listDirectoryStats(location, 'animals')
+          expect(stats).to.have.lengthOf(2)
+          const keys = stats.map(s => s.key)
+          expect(keys).to.include(scenario.fsPath(files.wombat))
+          expect(keys).to.include(scenario.fsPath(files.giraffe))
+          for (const stat of stats) {
+            expect(stat.size).to.equal(localFiles['/uploads/info.txt'].length)
+          }
+        })
+
+        it('should return empty array for non-existing directories', async function () {
+          const stats = await persistor.listDirectoryStats(
+            location,
+            'does-not-exist'
+          )
+          expect(stats).to.deep.equal([])
+        })
+      })
     })
   }
 })

libraries/object-persistor/test/unit/GcsPersistorTests.js

@@ -723,4 +723,127 @@ describe('GcsPersistorTests', function () {
       })
     })
   })
+
+  describe('listDirectoryKeys', function () {
+    describe('with valid parameters', function () {
+      let keys
+
+      beforeEach(async function () {
+        const filesWithNames = files.map((file, i) => ({
+          ...file,
+          name: i === 0 ? 'llama' : 'hippo',
+        }))
+        GcsBucket.getFiles.resolves([filesWithNames])
+        keys = await GcsPersistor.listDirectoryKeys(bucket, key)
+      })
+
+      it('should list the objects in the directory', function () {
+        expect(Storage.prototype.bucket).to.have.been.calledWith(bucket)
+        expect(GcsBucket.getFiles).to.have.been.calledWith({
+          prefix: `${key}/`,
+        })
+      })
+
+      it('should return the keys', function () {
+        expect(keys).to.deep.equal(['llama', 'hippo'])
+      })
+    })
+
+    describe('when there are no files', function () {
+      let keys
+
+      beforeEach(async function () {
+        GcsBucket.getFiles.resolves([[]])
+        keys = await GcsPersistor.listDirectoryKeys(bucket, key)
+      })
+
+      it('should return an empty array', function () {
+        expect(keys).to.deep.equal([])
+      })
+    })
+
+    describe('when there is an error listing the objects', function () {
+      let error
+
+      beforeEach(async function () {
+        GcsBucket.getFiles.rejects(genericError)
+        try {
+          await GcsPersistor.listDirectoryKeys(bucket, key)
+        } catch (err) {
+          error = err
+        }
+      })
+
+      it('should generate a ReadError', function () {
+        expect(error).to.be.an.instanceOf(Errors.ReadError)
+      })
+
+      it('should wrap the error', function () {
+        expect(error.cause).to.equal(genericError)
+      })
+    })
+  })
+
+  describe('listDirectoryStats', function () {
+    describe('with valid parameters', function () {
+      let stats
+
+      beforeEach(async function () {
+        const filesWithNames = files.map((file, i) => ({
+          ...file,
+          name: i === 0 ? 'llama' : 'hippo',
+        }))
+        GcsBucket.getFiles.resolves([filesWithNames])
+        stats = await GcsPersistor.listDirectoryStats(bucket, key)
+      })
+
+      it('should list the objects in the directory', function () {
+        expect(Storage.prototype.bucket).to.have.been.calledWith(bucket)
+        expect(GcsBucket.getFiles).to.have.been.calledWith({
+          prefix: `${key}/`,
+        })
+      })
+
+      it('should return the stats', function () {
+        expect(stats).to.deep.equal([
+          { key: 'llama', size: 11 },
+          { key: 'hippo', size: 22 },
+        ])
+      })
+    })
+
+    describe('when there are no files', function () {
+      let stats
+
+      beforeEach(async function () {
+        GcsBucket.getFiles.resolves([[]])
+        stats = await GcsPersistor.listDirectoryStats(bucket, key)
+      })
+
+      it('should return an empty array', function () {
+        expect(stats).to.deep.equal([])
+      })
+    })
+
+    describe('when there is an error listing the objects', function () {
+      let error
+
+      beforeEach(async function () {
+        GcsBucket.getFiles.rejects(genericError)
+        try {
+          await GcsPersistor.listDirectoryStats(bucket, key)
+        } catch (err) {
+          error = err
+        }
+      })
+
+      it('should generate a ReadError', function () {
+        expect(error).to.be.an.instanceOf(Errors.ReadError)
+      })
+
+      it('should wrap the error', function () {
+        expect(error.cause).to.equal(genericError)
+      })
+    })
+  })
 })

libraries/object-persistor/test/unit/ProjectKeyTests.js

@@ -1,8 +1,6 @@
-'use strict'
-
 const { expect } = require('chai')
 
-const { format, pad } = require('../../../../storage/lib/project_key')
+const { format, pad } = require('../../src/ProjectKey.js')
 
 describe('projectKey', function () {
   it('reverses padded keys', function () {

libraries/object-persistor/test/unit/S3PersistorTests.js

@@ -1046,4 +1046,215 @@ describe('S3PersistorTests', function () {
       })
     })
   })
+
+  describe('listDirectoryKeys', function () {
+    describe('with valid parameters', function () {
+      let keys
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, {
+          Contents: files.map(file => ({ Key: file.Key })),
+          IsTruncated: false,
+        })
+        keys = await S3Persistor.listDirectoryKeys(bucket, key)
+      })
+
+      it('should list the objects in the directory', function () {
+        S3.assertSendCalledWith(S3.ListObjectsV2Command, {
+          Bucket: bucket,
+          Prefix: key,
+        })
+      })
+
+      it('should return the keys', function () {
+        expect(keys).to.deep.equal(['llama', 'hippo'])
+      })
+    })
+
+    describe('when there are no files', function () {
+      let keys
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, {
+          Contents: [],
+          IsTruncated: false,
+        })
+        keys = await S3Persistor.listDirectoryKeys(bucket, key)
+      })
+
+      it('should return an empty array', function () {
+        expect(keys).to.deep.equal([])
+      })
+    })
+
+    describe('when there are more files available', function () {
+      const continuationToken = 'wombat'
+      let keys
+
+      beforeEach(async function () {
+        S3.mockSend(
+          S3.ListObjectsV2Command,
+          {
+            Contents: [files[0]].map(file => ({ Key: file.Key })),
+            IsTruncated: true,
+            NextContinuationToken: continuationToken,
+          },
+          {
+            nextResponses: [
+              {
+                Contents: [files[1]].map(file => ({ Key: file.Key })),
+                IsTruncated: false,
+              },
+            ],
+          }
+        )
+        keys = await S3Persistor.listDirectoryKeys(bucket, key)
+      })
+
+      it('should list the objects in multiple calls', function () {
+        S3.assertSendCallCount(S3.ListObjectsV2Command, 2)
+        expect(S3.s3ClientStub.send.firstCall.args[0].payload).to.deep.equal({
+          Bucket: bucket,
+          Prefix: key,
+        })
+        expect(S3.s3ClientStub.send.secondCall.args[0].payload).to.deep.equal({
+          Bucket: bucket,
+          Prefix: key,
+          ContinuationToken: continuationToken,
+        })
+      })
+
+      it('should return all keys', function () {
+        expect(keys).to.deep.equal(['llama', 'hippo'])
+      })
+    })
+
+    describe('when there is an error listing the objects', function () {
+      let error
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, genericError, { rejects: true })
+        try {
+          await S3Persistor.listDirectoryKeys(bucket, key)
+        } catch (err) {
+          error = err
+        }
+      })
+
+      it('should generate a ReadError', function () {
+        expect(error).to.be.an.instanceOf(Errors.ReadError)
+      })
+
+      it('should wrap the error', function () {
+        expect(error.cause).to.exist
+      })
+    })
+  })
+
+  describe('listDirectoryStats', function () {
+    describe('with valid parameters', function () {
+      let stats
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, {
+          Contents: files.map(file => ({ Key: file.Key, Size: file.Size })),
+          IsTruncated: false,
+        })
+        stats = await S3Persistor.listDirectoryStats(bucket, key)
+      })
+
+      it('should list the objects in the directory', function () {
+        S3.assertSendCalledWith(S3.ListObjectsV2Command, {
+          Bucket: bucket,
+          Prefix: key,
+        })
+      })
+
+      it('should return the stats', function () {
+        expect(stats).to.deep.equal([
+          { key: 'llama', size: 11 },
+          { key: 'hippo', size: 22 },
+        ])
+      })
+    })
+
+    describe('when there are no files', function () {
+      let stats
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, {
+          Contents: [],
+          IsTruncated: false,
+        })
+        stats = await S3Persistor.listDirectoryStats(bucket, key)
+      })
+
+      it('should return an empty array', function () {
+        expect(stats).to.deep.equal([])
+      })
+    })
+
+    describe('when there are more files available', function () {
+      const continuationToken = 'wombat'
+      let stats
+
+      beforeEach(async function () {
+        S3.mockSend(
+          S3.ListObjectsV2Command,
+          {
+            Contents: [files[0]].map(file => ({
+              Key: file.Key,
+              Size: file.Size,
+            })),
+            IsTruncated: true,
+            NextContinuationToken: continuationToken,
+          },
+          {
+            nextResponses: [
+              {
+                Contents: [files[1]].map(file => ({
+                  Key: file.Key,
+                  Size: file.Size,
+                })),
+                IsTruncated: false,
+              },
+            ],
+          }
+        )
+        stats = await S3Persistor.listDirectoryStats(bucket, key)
+      })
+
+      it('should list the objects in multiple calls', function () {
+        S3.assertSendCallCount(S3.ListObjectsV2Command, 2)
+      })
+
+      it('should return all stats', function () {
+        expect(stats).to.deep.equal([
+          { key: 'llama', size: 11 },
+          { key: 'hippo', size: 22 },
+        ])
+      })
+    })
+
+    describe('when there is an error listing the objects', function () {
+      let error
+
+      beforeEach(async function () {
+        S3.mockSend(S3.ListObjectsV2Command, genericError, { rejects: true })
+        try {
+          await S3Persistor.listDirectoryStats(bucket, key)
+        } catch (err) {
+          error = err
+        }
+      })
+
+      it('should generate a ReadError', function () {
+        expect(error).to.be.an.instanceOf(Errors.ReadError)
+      })
+
+      it('should wrap the error', function () {
+        expect(error.cause).to.exist
+      })
+    })
+  })
 })

libraries/overleaf-editor-core/.jenkinsIncludeFile

@@ -1,8 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/o-error/**
-libraries/overleaf-editor-core/**
-package-lock.json
-package.json
-patches/**

libraries/overleaf-editor-core/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/overleaf-editor-core/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/overleaf-editor-core monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/overleaf-editor-core/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'overleaf-editor-core-eslint', name: 'overleaf-editor-core eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/overleaf-editor-core/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/overleaf-editor-core monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/overleaf-editor-core monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/overleaf-editor-core monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'overleaf-editor-core test results', testResults: 'libraries/overleaf-editor-core/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/overleaf-editor-core/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/overleaf-editor-core/buildscript.txt

@@ -5,4 +5,5 @@ overleaf-editor-core
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🥑 Core
 --public-repo=False

libraries/overleaf-editor-core/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/promise-utils/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/promise-utils/**
-package-lock.json
-package.json
-patches/**

libraries/promise-utils/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/promise-utils/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/promise-utils monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/promise-utils/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'promise-utils-eslint', name: 'promise-utils eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/promise-utils/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/promise-utils monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/promise-utils monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/promise-utils monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'promise-utils test results', testResults: 'libraries/promise-utils/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/promise-utils/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/promise-utils/buildscript.txt

@@ -5,4 +5,5 @@ promise-utils
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/promise-utils/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/ranges-tracker/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/ranges-tracker/**
-package-lock.json
-package.json
-patches/**

libraries/ranges-tracker/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/ranges-tracker/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/ranges-tracker monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/ranges-tracker/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'ranges-tracker-eslint', name: 'ranges-tracker eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/ranges-tracker/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/ranges-tracker monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/ranges-tracker monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/ranges-tracker monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'ranges-tracker test results', testResults: 'libraries/ranges-tracker/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/ranges-tracker/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/ranges-tracker/buildscript.txt

@@ -5,4 +5,5 @@ ranges-tracker
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🥑 Core
 --public-repo=False

libraries/ranges-tracker/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/redis-wrapper/.jenkinsIncludeFile

@@ -1,10 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/fetch-utils/**
-libraries/logger/**
-libraries/o-error/**
-libraries/redis-wrapper/**
-package-lock.json
-package.json
-patches/**

libraries/redis-wrapper/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/redis-wrapper/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/redis-wrapper monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/redis-wrapper/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'redis-wrapper-eslint', name: 'redis-wrapper eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/redis-wrapper/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/redis-wrapper monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/redis-wrapper monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/redis-wrapper monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'redis-wrapper test results', testResults: 'libraries/redis-wrapper/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/redis-wrapper/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/redis-wrapper/buildscript.txt

@@ -5,4 +5,5 @@ redis-wrapper
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/redis-wrapper/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/settings/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/settings/**
-package-lock.json
-package.json
-patches/**

libraries/settings/Jenkinsfile

@@ -1,72 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/settings/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/settings monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/settings/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'settings-eslint', name: 'settings eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/settings/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/settings monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/settings monorepo npm run types:check'
-          }
-        }
-      }
-    }
-  }
-  post {
-    cleanup {
-      sh 'rm -rf libraries/settings/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/settings/buildscript.txt

@@ -5,4 +5,5 @@ settings
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/stream-utils/.jenkinsIncludeFile

@@ -1,7 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/stream-utils/**
-package-lock.json
-package.json
-patches/**

libraries/stream-utils/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/stream-utils/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/stream-utils monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/stream-utils/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'stream-utils-eslint', name: 'stream-utils eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/stream-utils/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/stream-utils monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/stream-utils monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/stream-utils monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'stream-utils test results', testResults: 'libraries/stream-utils/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/stream-utils/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/stream-utils/buildscript.txt

@@ -5,4 +5,5 @@ stream-utils
 --esmock-loader=False
 --is-library=True
 --node-version=22.18.0
+--pipeline-owner=🚉 Platform
 --public-repo=False

libraries/stream-utils/test/mocha-multi-reporters.cjs

@@ -4,5 +4,6 @@ module.exports = {
     mochaFile: `reports/junit-mocha-${process.env.MOCHA_GREP}.xml`,
     includePending: true,
     jenkinsMode: true,
+    output: true,
   },
 }

libraries/validation-tools/.jenkinsIncludeFile

@@ -1,8 +0,0 @@
-# Autogenerated by build scripts. Do not edit.
-.eslint*
-.prettier*
-libraries/o-error/**
-libraries/validation-tools/**
-package-lock.json
-package.json
-patches/**

libraries/validation-tools/Jenkinsfile

@@ -1,82 +0,0 @@
-// Autogenerated by build scripts. Do not edit.
-
-pipeline {
-  agent {
-    node {
-      label 'jenkins-agent-web'
-      customWorkspace '/workspace'
-    }
-  }
-  options {
-    timestamps()
-    timeout(time: 15, unit: 'MINUTES')
-  }
-  environment {
-    BRANCH_NAME = "${env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME}"
-    JENKINS_BUILD_NUMBER = "${BUILD_NUMBER}"
-    BUILD_NUMBER = "${SHORT_SHA}_${BUILD_NUMBER}"
-    COMMIT_SHA = "${GIT_COMMIT}"
-    SHORT_SHA = "${GIT_COMMIT.take(7)}"
-  }
-  stages {
-    stage('Stage 1') {
-      parallel {
-        stage('Install monorepo') {
-          steps {
-            retry(count: 3) {
-              sh 'make monorepo_setup'
-            }
-          }
-        }
-        stage('Create reports folder') {
-          steps {
-            sh 'mkdir libraries/validation-tools/reports'
-          }
-        }
-      }
-    }
-    stage('Stage 2') {
-      parallel {
-        stage('Lint') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/validation-tools monorepo npm run lint -- --format json --output-file reports/eslint.json'
-          }
-          post {
-            always {
-              sh """
-                sed -i 's_"filePath":"/overleaf_"filePath":"/workspace_g' libraries/validation-tools/reports/eslint.json
-              """
-              recordIssues checksAnnotationScope: 'ALL', enabledForFailure: true, failOnError: true, id: 'validation-tools-eslint', name: 'validation-tools eslint', qualityGates: [[integerThreshold: 1, threshold: 1.0, type: 'TOTAL']], sourceCodeRetention: 'LAST_BUILD', tools: [esLint(pattern: 'libraries/validation-tools/reports/eslint.json')]
-            }
-          }
-        }
-        stage('Format') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/validation-tools monorepo npm run format'
-          }
-        }
-        stage('Typecheck') {
-          steps {
-            sh 'bin/run -w /overleaf/libraries/validation-tools monorepo npm run types:check'
-          }
-        }
-        stage('Test') {
-          steps {
-            retry(count: 3) {
-              sh 'bin/run -w /overleaf/libraries/validation-tools monorepo npm run test:ci'
-            }
-          }
-        }
-      }
-    }
-  }
-  post {
-    always {
-      junit checksName: 'validation-tools test results', testResults: 'libraries/validation-tools/reports/junit-*.xml'
-    }
-    cleanup {
-      sh 'rm -rf libraries/validation-tools/reports'
-      sh 'make clean_jenkins -j10'
-    }
-  }
-}

libraries/validation-tools/handleValidationError.js

@@ -0,0 +1,17 @@
+const { isZodErrorLike, fromError } = require('zod-validation-error')
+
+function createHandleValidationError(statusCode = 400) {
+  return [
+    (err, req, res, next) => {
+      if (!isZodErrorLike(err)) {
+        return next(err)
+      }
+
+      res.status(statusCode).json({ ...fromError(err), statusCode })
+    },
+  ]
+}
+
+const handleValidationError = createHandleValidationError(400)
+
+module.exports = { handleValidationError, createHandleValidationError }

libraries/validation-tools/index.js

@@ -1,11 +1,19 @@
 const { ParamsError } = require('./Errors')
 const { z } = require('zod')
 const { zz } = require('./zodHelpers')
-const { validateReq } = require('./validateReq')
+const { parseReq } = require('./parseReq')
+const { validateSchema } = require('./validateSchema')
+const {
+  handleValidationError,
+  createHandleValidationError,
+} = require('./handleValidationError')
 
 module.exports = {
   z,
   zz,
-  validateReq,
+  validateSchema,
+  parseReq,
+  handleValidationError,
+  createHandleValidationError,
   ParamsError,
 }

libraries/validation-tools/package.json

@@ -22,10 +22,11 @@
   "dependencies": {
     "@overleaf/o-error": "*",
     "mongodb": "^6.12.0",
-    "zod": "^4.1.8"
+    "zod": "^4.1.8",
+    "zod-validation-error": "^4.0.1"
   },
   "devDependencies": {
     "typescript": "^5.0.4",
-    "vitest": "^3.2.4"
+    "vitest": "^4.0.0"
   }
 }

libraries/validation-tools/parseReq.js

@@ -12,14 +12,14 @@ const { ParamsError } = require('./Errors')
  */
 
 /**
- * Validate a request against a zod schema
+ * Parse and validate a request against a Zod schema
  *
  * @template {ZodType} T
  * @param {Request} req - The Express request object
  * @param {T} schema - The Zod schema to validate against
  * @returns {output<T>} The validated request object
  */
-function validateReq(req, schema) {
+function parseReq(req, schema) {
   const parsed = schema.safeParse(req)
   if (parsed.success) {
     return parsed.data
@@ -31,4 +31,4 @@ function validateReq(req, schema) {
   }
 }
 
-module.exports = { validateReq }
+module.exports = { parseReq }

libraries/validation-tools/test/unit/src/validateReq.test.ts

@@ -1,10 +1,10 @@
-import { validateReq } from '../../../validateReq'
+import { parseReq } from '../../../parseReq'
 import { describe, expect, it } from 'vitest'
 import { z } from 'zod'
 import type { Request } from 'express'
 import { zz } from '../../../zodHelpers'
 
-describe('validateReq', () => {
+describe('parseReq', () => {
   describe('with a request that is valid for the schema', () => {
     it('should return the parsed request', () => {
       const req = {
@@ -25,7 +25,7 @@ describe('validateReq', () => {
         }),
       })
 
-      const result = validateReq(req, schema)
+      const result = parseReq(req, schema)
 
       expect(result).toEqual({
         params: {
@@ -46,7 +46,7 @@ describe('validateReq', () => {
       } as Request<{ id: string }>
 
       expect(() =>
-        validateReq(
+        parseReq(
           req,
           z.object({
             params: z.object({
@@ -65,7 +65,7 @@ describe('validateReq', () => {
       } as Request
 
       expect(() =>
-        validateReq(
+        parseReq(
           req,
           z.object({
             body: z.object({

libraries/validation-tools/test/unit/src/zodHelpers.test.ts

@@ -41,4 +41,180 @@ describe('zodHelpers', () => {
       expect(parsed.data?.toString()).toBe('507f1f77bcf86cd799439011')
     })
   })
+  describe('datetime', () => {
+    it('parses valid ISO 8601 datetime strings', () => {
+      const parsed = zz.datetime().safeParse('2024-01-01T12:00:00Z')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00Z'))
+    })
+
+    it('parses a valid ISO 8601 datetime with offset', () => {
+      const parsed = zz
+        .datetime({ offset: true })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00+00:00'))
+    })
+
+    it('parses a valid Date object', () => {
+      const date = new Date('2024-01-01T12:00:00Z')
+      const parsed = zz.datetime().safeParse(date)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(date)
+    })
+
+    it('fails to parse datetime with offset when offset option is false', () => {
+      const parsed = zz
+        .datetime({ offset: false })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+
+    it('fails to parse null when schema is not nullable', () => {
+      const parsed = zz.datetime().safeParse(null)
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.message).toContain(
+        'Invalid input: expected date, received null'
+      )
+    })
+
+    it('fails to parse invalid datetime strings', () => {
+      const parsed = zz.datetime().safeParse('invalid-datetime')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+  })
+  describe('datetimeNullable', () => {
+    it('parses valid ISO 8601 datetime strings', () => {
+      const parsed = zz.datetimeNullable().safeParse('2024-01-01T12:00:00Z')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00Z'))
+    })
+
+    it('parses a valid ISO 8601 datetime with offset', () => {
+      const parsed = zz
+        .datetimeNullable({ offset: true })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00+00:00'))
+    })
+
+    it('parses a valid Date object', () => {
+      const date = new Date('2024-01-01T12:00:00Z')
+      const parsed = zz.datetimeNullable().safeParse(date)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(date)
+    })
+
+    it('fails to parse datetime with offset when offset option is false', () => {
+      const parsed = zz
+        .datetimeNullable({ offset: false })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+
+    it('parses null when schema is nullable and input is null', () => {
+      const parsed = zz.datetimeNullable().safeParse(null)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toBeNull()
+    })
+
+    it('fails to parse invalid datetime strings', () => {
+      const parsed = zz.datetimeNullable().safeParse('invalid-datetime')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+  })
+  describe('datetimeNullish', () => {
+    it('parses valid ISO 8601 datetime strings', () => {
+      const parsed = zz.datetimeNullish().safeParse('2024-01-01T12:00:00Z')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00Z'))
+    })
+
+    it('parses a valid ISO 8601 datetime with offset', () => {
+      const parsed = zz
+        .datetimeNullish({ offset: true })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(new Date('2024-01-01T12:00:00+00:00'))
+    })
+
+    it('parses a valid Date object', () => {
+      const date = new Date('2024-01-01T12:00:00Z')
+      const parsed = zz.datetimeNullish().safeParse(date)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toEqual(date)
+    })
+
+    it('parses null when schema is nullable and input is null', () => {
+      const parsed = zz.datetimeNullish().safeParse(null)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toBeNull()
+    })
+
+    it('parses undefined when schema is nullish and input is undefined', () => {
+      const parsed = zz.datetimeNullish().safeParse(undefined)
+      expect(parsed.success).toBe(true)
+      expect(parsed.data).toBeUndefined()
+    })
+
+    it('fails to parse datetime with offset when offset option is false', () => {
+      const parsed = zz
+        .datetimeNullish({ offset: false })
+        .safeParse('2024-01-01T12:00:00+00:00')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+
+    it('fails to parse invalid datetime strings', () => {
+      const parsed = zz.datetimeNullish().safeParse('invalid-datetime')
+      expect(parsed.success).toBe(false)
+      expect(parsed.error?.issues).toHaveLength(1)
+      expect(parsed.error?.issues).toMatchObject([
+        expect.objectContaining({
+          code: 'invalid_format',
+          format: 'datetime',
+          message: 'Invalid ISO datetime',
+        }),
+      ])
+    })
+  })
 })

libraries/validation-tools/types.d.ts

@@ -0,0 +1,6 @@
+import z from 'zod'
+
+export interface DatetimeSchemaOptions extends z.core.$ZodISODateTimeParams {
+  allowNull?: boolean
+  allowUndefined?: boolean
+}

libraries/validation-tools/validateSchema.js

@@ -0,0 +1,66 @@
+// @ts-check
+const { isZodErrorLike } = require('zod-validation-error')
+
+/**
+ * @typedef {import('zod').ZodType} ZodType
+ */
+/**
+ * @template T
+ * @typedef {import('zod').output<T>} output<T>
+ */
+
+/**
+ * A helper function to safely get a nested value from an object
+ * using a path array (e.g., ["query", "resource_type"])
+ * @param {any} data
+ * @param {Array<PropertyKey>} path
+ */
+function getPathValue(data, path) {
+  let current = data
+  for (const key of path) {
+    if (current === null || typeof current !== 'object') {
+      return undefined
+    }
+    current = current[key]
+  }
+  return current
+}
+
+/**
+ * @param {any} issue
+ * @param {any} value
+ */
+const isRequiredError = (issue, value) =>
+  value === undefined &&
+  (issue.code === 'invalid_type' || issue.code === 'invalid_union')
+
+/**
+ * Validates data against a Zod schema and throws a user-friendly error.
+ *
+ * @template {ZodType} T
+ * @param {T} schema - The Zod schema
+ * @param {unknown} data - The data to validate
+ * @returns {output<T>} The validated (and transformed) data
+ */
+function validateSchema(schema, data) {
+  try {
+    return schema.parse(data)
+  } catch (err) {
+    if (isZodErrorLike(err)) {
+      const errorMessages = err.issues.map(issue => {
+        const value = getPathValue(data, issue.path)
+        const fieldName = String(issue.path[issue.path.length - 1])
+        if (isRequiredError(issue, value)) {
+          return `"${fieldName}" is required`
+        }
+        return `"${fieldName}" - ` + issue.message
+      })
+
+      throw new Error(errorMessages.join('; '))
+    }
+
+    throw err
+  }
+}
+
+module.exports = { validateSchema }

libraries/validation-tools/zodHelpers.js

@@ -3,6 +3,23 @@ const mongodb = require('mongodb')
 
 const { ObjectId } = mongodb
 
+/**
+ * @import { DatetimeSchemaOptions } from './types'
+ */
+
+/**
+ * @param {DatetimeSchemaOptions} options
+ */
+const datetimeSchema = ({ allowNull, allowUndefined, ...zodOptions } = {}) => {
+  const union = [z.date(), z.iso.datetime(zodOptions)]
+  if (allowNull) union.push(z.null())
+  if (allowUndefined) union.push(z.undefined())
+  return z.union(union).transform(dt => {
+    if (allowNull && !dt) return dt === null ? null : undefined
+    return dt instanceof Date ? dt : new Date(dt)
+  })
+}
+
 const zz = {
   objectId: () =>
     z.string().refine(ObjectId.isValid, { message: 'invalid Mongo ObjectId' }),
@@ -12,6 +29,10 @@ const zz = {
       .refine(ObjectId.isValid, { message: 'invalid Mongo ObjectId' })
       .transform(val => new ObjectId(val)),
   hex: () => z.string().regex(/^[0-9a-f]*$/),
+  datetime: options => datetimeSchema(options),
+  datetimeNullable: options => datetimeSchema({ ...options, allowNull: true }),
+  datetimeNullish: options =>
+    datetimeSchema({ ...options, allowNull: true, allowUndefined: true }),
 }
 
 module.exports = { zz }

package.json

@@ -2,14 +2,16 @@
   "name": "overleaf",
   "private": true,
   "dependencies": {
+    "passport-ldapauth": "^3.0.1",
     "patch-package": "^8.0.0"
   },
   "devDependencies": {
     "@types/chai": "^4.3.0",
     "@types/chai-as-promised": "^7.1.8",
     "@types/mocha": "^10.0.6",
-    "@typescript-eslint/eslint-plugin": "^8.30.1",
-    "@typescript-eslint/parser": "^8.30.1",
+    "@typescript-eslint/eslint-plugin": "8.50.0",
+    "@typescript-eslint/parser": "^8.50.0",
+    "@vitest/eslint-plugin": "^1.5.0",
     "eslint": "^8.15.0",
     "eslint-config-prettier": "^8.5.0",
     "eslint-config-standard": "^17.0.0",
@@ -22,22 +24,21 @@
     "eslint-plugin-prettier": "^4.0.0",
     "eslint-plugin-promise": "^6.0.0",
     "eslint-plugin-unicorn": "^56.0.0",
-    "prettier": "3.6.2",
+    "prettier": "3.7.4",
     "prettier-plugin-groovy": "0.2.1",
-    "typescript": "^5.8.3"
+    "typescript": "^5.9.3"
   },
   "engines": {
     "npm": "11.4.2"
   },
   "overrides": {
-    "swagger-tools@0.10.4": {
-      "path-to-regexp": "3.3.0",
-      "body-parser": "1.20.3",
-      "multer": "2.0.2"
-    },
     "request@2.88.2": {
       "tough-cookie": "5.1.2",
-      "form-data": "2.5.5"
+      "form-data": "2.5.5",
+      "qs": "6.14.1"
+    },
+    "streamdown": {
+      "mermaid": "file:./services/web/stubs/mermaid-stub"
     }
   },
   "scripts": {
@@ -67,6 +68,7 @@
     "services/history-v1",
     "services/idp",
     "services/latexqc",
+    "services/linked-url-proxy",
     "services/notifications",
     "services/project-history",
     "services/real-time",
@@ -76,6 +78,7 @@
     "services/third-party-references",
     "services/tpdsworker",
     "services/web",
+    "tools/dependency-management",
     "tools/saas-e2e",
     "tools/migrations"
   ]

patches/@google-cloud+storage++teeny-request+8.0.3.patch

@@ -1,37 +1,35 @@
 diff --git a/node_modules/@google-cloud/storage/node_modules/teeny-request/build/src/index.js b/node_modules/@google-cloud/storage/node_modules/teeny-request/build/src/index.js
-index a2251ca..e29e796 100644
+index af5d15e..8a9bf77 100644
 --- a/node_modules/@google-cloud/storage/node_modules/teeny-request/build/src/index.js
 +++ b/node_modules/@google-cloud/storage/node_modules/teeny-request/build/src/index.js
-@@ -166,27 +166,27 @@ function teenyRequest(reqOpts, callback) {
-     }
-     if (callback === undefined) {
+@@ -168,31 +168,34 @@ function teenyRequest(reqOpts, callback) {
          // Stream mode
--        const requestStream = streamEvents(new stream_1.PassThrough());
--        // eslint-disable-next-line @typescript-eslint/no-explicit-any
+         const requestStream = streamEvents(new stream_1.PassThrough());
+         // eslint-disable-next-line @typescript-eslint/no-explicit-any
 -        let responseStream;
 -        requestStream.once('reading', () => {
 -            if (responseStream) {
--                responseStream.pipe(requestStream);
+-                (0, stream_1.pipeline)(responseStream, requestStream, () => { });
 -            }
 -            else {
 -                requestStream.once('response', () => {
--                    responseStream.pipe(requestStream);
+-                    (0, stream_1.pipeline)(responseStream, requestStream, () => { });
 -                });
 -            }
 -        });
-+        const requestStream = new stream_1.PassThrough();
-+        // // eslint-disable-next-line @typescript-eslint/no-explicit-any
 +        // let responseStream;
 +        // requestStream.once('reading', () => {
-+        //     if (responseStream) {
-+        //         responseStream.pipe(requestStream);
-+        //     }
-+        //     else {
-+        //         requestStream.once('response', () => {
-+        //             responseStream.pipe(requestStream);
-+        //         });
-+        //     }
++        //    if (responseStream) {
++        //        (0, stream_1.pipeline)(responseStream, requestStream, () => { });
++        //    }
++        //    else {
++        //        requestStream.once('response', () => {
++        //            (0, stream_1.pipeline)(responseStream, requestStream, () => { });
++        //        });
++        //    }
 +        // });
++
++
          options.compress = false;
          teenyRequest.stats.requestStarting();
          (0, node_fetch_1.default)(uri, options).then(res => {
@@ -41,10 +39,17 @@ index a2251ca..e29e796 100644
 -                requestStream.emit('error', err);
 -            });
 +            teenyRequest.stats.requestFinished(); stream_1.pipeline(res.body, requestStream, () => {});
-+            // responseStream = res.body;
-+            // responseStream.on('error', (err) => {
-+            //     requestStream.emit('error', err);
-+            // });
++        //    responseStream = res.body;
++        //    responseStream.on('error', (err) => {
++        //        requestStream.emit('error', err);
++        //    });
              const response = fetchToRequestResponse(options, res);
              requestStream.emit('response', response);
          }, err => {
+             teenyRequest.stats.requestFinished();
+             requestStream.emit('error', err);
+         });
++
+         // fetch doesn't supply the raw HTTP stream, instead it
+         // returns a PassThrough piped from the HTTP response
+         // stream.