Commit Graph
402 Commits
Author SHA1 Message Date
Rebeka Dekany a648015db8 Centralize prettier configuration to root level (#30501)
* Merge all .prettierignore files into top-level config

* Merge all .prettierrc files into top-level config

* Replace service-specific glob patterns in package.json format scripts with `prettier .`

* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore

* Ignore GitHub templates

* Ignore PUG templates to format them separately with `format:pug`

* Encourage double quotes for YAML, YML files

* Move prettier for PUG source format script to the root

* Move prettier for styles source format script to the root

* Remove prettier for jenkins files from web

* Remove prettier source format script from all services

* Make .prettierrc more readable

* Update format scripts by file type

* Organise `.prettierignore`

* Add `--cache` flag to prettier scripts for faster runs

* Format all files with prettier

* Format all or format services

* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`

* Avoid conlficts with yamllint configuration

* Remove `--cache` flag from prettier scripts

* Update all service Makefiles to use root-level prettier configuration

* Update all Jenkinsfile to use root-level prettier configuration

* Ignore auto-generated files by build_scripts

* Update package-lock.json

* Update root Makefile format targets

* Update SP Jenkinsfile format target

* Update E2E Makefile format script

* Udpate `format_js` to work in both local and CI env

* Add docker-mailtrap to .prettierignore

docker-mailtrap is a third-party git-ignored directory used for testing

* Added Docker env detection to prevent nested Docker spawning

* Ignore handlebars templates

* Add cryptographic files and test output to `.prettierignore`

* Add terraform modules to `.gitignore`

* Remove prettier-plugin-groovy

* Use npx directly instead of Docker for local formatting for faster formatting

* Auto-generate Makefiles

* Revert "Remove prettier-plugin-groovy"

This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.

* Mount monorepo root in RUN_LINT_FORMAT for prettier config access

* Prettier ignores all `node_modules` by default regardless of location

* Show only changed files in format output

* Ignore LICENSE files

* Enable prettier on rendered build_scripts outputs

* Ignoring all the template folders by prettier

* Remove the public/minjs entry since it does not exist

* Remove all non-existent paths

* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files

* Revert "Auto-generate Makefiles"

This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.

* Revert "Use npx directly instead of Docker for local formatting for faster formatting"

This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.

* Ignore dashboard JSON files

* Ignore files generated by bin/update_build_scripts

* Remove unsupported file types from `.prettierignore`

* Ignore test fixture generated files

* Ignore README file types by prettier

* Ignore generate snapshots by prettier

* Allow to format generated bin/update_build_scripts by prettier

* Ensure build script outputs prettier-compatible tsconfig.json

* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double

* Don't read cryptographic files by prettier

* Ignore google verification files by prettier

* Revert npx prettier formatting

* Ignore domain verification files

* Show only changed files in format output

* Make `.github` prettier

* Allow all files to be formatted in jobs by prettier

* Allow server-ce/server-pro files to be formatted by prettier

* Ignore more folders in clsi, filestory, git-bridge by prettier

* Update build script with `RUN_LINTING_CI_MONOREPO`

* Ignore docker-mailtrap and downloads in server-ce by prettier

* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)

* Source format

GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
2026-02-04 09:08:22 +00:00
Anna Claire Fields ee4b5f515c Handle ERR_STREAM_UNABLE_TO_PIPE alongside ERR_STREAM_PREMATURE_CLOSE (#31174)
GitOrigin-RevId: bbf49237b177d7a58a9b13efc6f38f5eecfb745c
2026-02-03 09:05:50 +00:00
Anna Claire Fields e80ce42896 Upgrade Node.js from 22.18 to 24.13 (#30509)
GitOrigin-RevId: c2405a0583e68c61351f52b32ba28faec994fbc0
2026-01-30 09:05:28 +00:00
Mathias Jakobsen 8859aaaeb9 Merge pull request #31011 from overleaf/mj-build-scripts-elastic-search
[build-scripts] Remove elastic search (es) dependency from build scripts

GitOrigin-RevId: db21887149dfb6eab3508762695f73f7ecacb492
2026-01-28 09:07:54 +00:00
Andrew Rumble f58949a440 Merge pull request #30779 from overleaf/ar-split-analytics-queues
[analytics] split analytics queues

GitOrigin-RevId: cb719fbeba91d69503f315e8c32dd8ab5e872421
2026-01-20 09:05:53 +00:00
Alf Eaton 03a3518aae Merge pull request #30703 from overleaf/ae-prettier
Upgrade Prettier to v3.7.4

GitOrigin-RevId: 0f4434019bc7d12f2d5b7ecbb833ee20570d0706
2026-01-16 09:56:07 +00:00
Andrew Rumble 4271744bfd Merge pull request #30606 from overleaf/ar/bump-qs-to-6.14.1
[monorepo] bump qs to 6.14.1

GitOrigin-RevId: 4b1f8b1cf739c3c0195392fa6707fc233d036b70
2026-01-15 14:20:06 +00:00
Tim Down 64da16f0d1 Merge pull request #28670 from overleaf/td-rename-validate-req
Rename validateReq to parseReq

GitOrigin-RevId: a935aaa3f89495e164ce5d10b0bd80436571440f
2026-01-15 09:05:09 +00:00
Jakob Ackermann 18cff6e1ac [object-persistor] import ProjectKey helper from history-v1 (#30600)
GitOrigin-RevId: c72aa4bf91569904a2072c74d6ed2f3c764d97bb
2026-01-13 09:06:55 +00:00
Jakob Ackermann 425e7b1e5b [web] enable mongo notablescan in CI (#29501)
* [monorepo] record ERROR/FATAL log messages in junit report

* [web] put SaaS specific code behind feature flag

* [web] use split test cache for getting user assignments

The unit tests needed updating as they did not replicate any of the
 mongo filtering. The acceptance tests cover this logic.

* [web] make better use of existing indexes

* [web] avoid col-scan in tests of notifications module

* [web] remove cleanup of empty feedbacks collection

* [web] add assertion for reason of rejected request in launchpad test

* [web] add missing indexes

* [web] enable mongo notablescan

* [web] make emailNotifications tests compatible with notablescan

GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
2026-01-13 09:06:38 +00:00
Mathias Jakobsen e895ff8153 Merge pull request #30549 from overleaf/mj-build-script-minio-certs
[build-scripts] Handle expired certs in minio

GitOrigin-RevId: 7e0d03112790fbbfd0c2a1718e272aeeb99c8767
2026-01-12 09:06:36 +00:00
Anna Claire Fields 06a0cf2391 Merge pull request #30339 from overleaf/acf-migration5-cleanup
(5) Remove swagger dependencies from history-v1

GitOrigin-RevId: d3d79dd9d3c8ccb59d6a59aeede7186daf18bdb0
2025-12-18 09:05:37 +00:00
Anna Claire Fields 65c164c73d Merge pull request #30053 from overleaf/acf-migration4-controllers-and-params
(4) Update controllers and tests for Zod migration

GitOrigin-RevId: 876fd64f96e1f5d7244ac1d45053c7db9857d46b
2025-12-16 09:06:44 +00:00
Brian Gough 42d09010aa Merge pull request #30174 from overleaf/bg-history-backup-allow-stalled-jobs-to-be-retried
Allow stalled jobs to be retried in backup queue settings

GitOrigin-RevId: c68605aa89e9f8d4c2bc11fc14739a7d0774096d
2025-12-10 09:06:18 +00:00
Andrew Rumble f3c9b2f2f0 Merge pull request #30182 from overleaf/ar/update-jws-to-3.2.3_or_4.0.1
[monorepo] update jws to 4.0.1

GitOrigin-RevId: b3e778b26b2860112fe80220d6e7354520994082
2025-12-10 09:05:25 +00:00
Brian Gough b9888957e4 Merge pull request #30000 from overleaf/bg-extend-backup-blob-script
extend backup blob script

GitOrigin-RevId: 226e624f0fd93bfe4890acce728ce8005f9787c5
2025-12-08 09:06:05 +00:00
Anna Claire Fields 56c98ac99c Merge pull request #29968 from overleaf/acf-migration3-express-routes
(3) Create Express routes (replaces swagger-router)

GitOrigin-RevId: e27973b630509312b198be899148310e69b94777
2025-12-08 09:05:20 +00:00
Brian Gough 7a57ef00cb Merge pull request #30045 from overleaf/bg-object-persistor-make-list-directory-safer
Improve safety of object persistor

GitOrigin-RevId: bced9814de6613b388ca288a5f72cd42cff6c1d3
2025-12-05 09:06:42 +00:00
Anna Claire Fields c1dc70fc92 Merge pull request #29966 from overleaf/acf-migration2-security-middleware
(2) Create Express authentication middleware (replaces swagger-security)

GitOrigin-RevId: 0a15ec7ca601430a1ae00c7f81da456948650a0c
2025-12-04 09:06:06 +00:00
Anna Claire Fields c25e49782f Merge pull request #29965 from overleaf/acf-migration1-validation
Add Zod validation (replaces swagger-validator + swagger-metadata)

GitOrigin-RevId: 2e4ed742e401bdfe49c6f7dc9d0fceeba20cfc7f
2025-12-03 09:06:15 +00:00
Miguel Serrano a3ec5b2797 Merge pull request #30018 from overleaf/msm-fix-esm-import
[web] Fix ESM import

GitOrigin-RevId: 887f4927248241cb3d237e0bdad36b05928657ef
2025-12-03 09:05:32 +00:00
Brian Gough 06f696ced0 Merge pull request #29980 from overleaf/bg-history-extend-backup-comparison-III
Check file tree hashes in backup comparison

GitOrigin-RevId: 4bd1f36afa34f326d4b8934c8bb0ea00a52cf1d9
2025-12-01 09:06:18 +00:00
Brian Gough e0d6d7cb48 Merge pull request #29952 from overleaf/bg-history-extend-backup-comparison-II
Add batch comparison options for backup verification

GitOrigin-RevId: fe12270b4063c4105f7e7a950c6a58cd39a62843
2025-11-28 09:06:11 +00:00
Brian Gough 70909c126c Merge pull request #29950 from overleaf/bg-history-extend-backup-comparison
Add fast comparison option for blob backups

GitOrigin-RevId: e1383425487ac1b4439248f976e6106567bae07d
2025-11-28 09:06:06 +00:00
Gernot Schulz 74efa0e345 Merge pull request #29898 from overleaf/gs-jenkins-issues-all-pipelines
Enable Jenkins hooks for creating GitHub issues for all pipelines with owners

GitOrigin-RevId: ba46db1d3137db12ee5c78f09b126fb9927d9c49
2025-11-27 09:06:04 +00:00
Brian Gough 1f356754de Merge pull request #29801 from overleaf/bg-history-refactor-backup-worker-shutdown
refactor history-v1-backup worker shutdown

GitOrigin-RevId: 9666a99b00b30e98844e7dd25932f1590d0879b3
2025-11-24 09:05:49 +00:00
Brian Gough e822ac0ee0 Merge pull request #29775 from overleaf/bg-history-use-configurable-concurrency-in-backup-worker
allow configurable concurrency for history-v1 backup worker

GitOrigin-RevId: 59c734b013f99e215cc84688142cb0fbe45b064b
2025-11-24 09:05:44 +00:00
Brian Gough 3227502aeb Merge pull request #29718 from overleaf/bg-history-fix-backup-errors
fix backupBlob error handling and add tests for upload failures

GitOrigin-RevId: 3abda1b70012fda20fc403593d0ee6ce04152e24
2025-11-19 09:05:19 +00:00
Brian Gough 581a3b7ce7 Merge pull request #29700 from overleaf/bg-history-v1-log-stalled-uploads
Add timeout warnings for long-running blob and chunk uploads

GitOrigin-RevId: 5a57ebaaea9dfcaf23153f7e6f27190af541eb16
2025-11-18 09:06:02 +00:00
John Lees-Miller 0d71c825a9 Bump fake-gcs version and fix resumable uploads
GitOrigin-RevId: b1f3cf74f59ff53961665e19826b2f10ac235016
2025-11-18 09:05:21 +00:00
Thomas c059a3c5b0 Handle slashes in branch names for docker build tags (#29529)
* Refactor v1 Makefile to use DOCKER_COMPOSE variable for Docker commands with tag safe branch name

* Handle slashes in branch names for docker build tags

GitOrigin-RevId: 463940e8435845978aced745575905f3bfbb8e1c
2025-11-14 09:05:25 +00:00
Gernot Schulz b834b8bcb7 Merge pull request #29537 from overleaf/gs-jenkins-pr-build-tags-templates
Tag images with branch head instead of merge commit (templates)

GitOrigin-RevId: 53e616aeec058863e25b356841a59f3ebe13c87f
2025-11-10 09:05:00 +00:00
Jakob Ackermann 6691c5dc05 [monorepo] enable mongo notablescan for all services but web (#29497)
* [migrations] make old migrations compatible with --notablescan in mongo

Re-running the migration for db.users.emails.reversedHostname on all
Server Pro/CE users will not be harmfull.

* [monorepo] enable mongo notablescan for all services but web

GitOrigin-RevId: e2a710c9de08de9ac2aa9df1f3082d5f1c33ea05
2025-11-05 09:06:35 +00:00
Brian Gough cc7f0c1b9b Merge pull request #29321 from overleaf/bg-ignore-deleted-projects-in-history-backup
handle deleted projects in history-v1 backup worker

GitOrigin-RevId: f4392045074248137f15d082d922c18b1ef9232f
2025-10-28 09:05:34 +00:00
729e0f5ac9 move migrations to shared location (#28306)
* fix: correct typedef for Document in helpers.mjs

* add move-migrations codemod

* update migration paths to use shared migrations directory

* move migrations to shared location

* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory

* feat: add migrations tool to workspaces in package.json

* [monorepo] Fix order of docker ignore rules

* [web] remove unused docker ignore file

* [monorepo] replace old references to migrations folder

* [server-ce] copy migrations from new place

* [migrations] Inline web scripts

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] move three web scripts over

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] add missing collection

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] remove lodash dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] avoid mongodb-legacy dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [monorepo] run migrations from tools/migrations

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] simplify migration for adding gitBridge feature to users

* [monorepo] run migrations from tests in all the services

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] fixup running web migrations everywhere

* [monorepo] trigger Jenkins builds on changes to mongo migrations

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] build scripts: update devDependencies before deps scanning

* [monorepo] build scripts: formerly depend on tools/migrations

* [monorepo] run eslint on .mjs files

* [migrations] enable more eslint rules and fix all the errors

* [rake] fix migrations:list task

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
2025-10-16 08:07:37 +00:00
Brian Gough 2cae9c4635 Merge pull request #29097 from overleaf/bg-reduce-retry-backup-script-concurrency
reduce concurrency for retrying failed backups

GitOrigin-RevId: aeb1f05b7ed26b80c00f12fe07a5f30a14786c30
2025-10-16 08:05:56 +00:00
Jakob Ackermann 0182ad0046 [monorepo] compute IMAGE_CACHE lazily (#29087)
GitOrigin-RevId: de287c89c1cea858d0609265945266be19d7ea26
2025-10-15 08:07:47 +00:00
Jakob Ackermann 15a6e8ba04 [monorepo] use content hash as docker image cache key (#29046)
* [monorepo] remove docker-repos flag from build scripts

* [monorepo] use content hash as docker image cache key

* [packer] jenkins-worker: populate build cache for all services

* [v1] adopt smarter docker caching

* [latexqc] adopt smarter docker caching

* [monorepo] refresh docker cache in Jenkins

* [packer] jenkins-worker: increase disk size

* [monorepo] run jenkins_docker_build_cache as Jenkins user for v1

* [monorepo] define MONOREPO at the top of all the Makefiles

* [monorepo] add --build-arg BUILDKIT_INLINE_CACHE=1 everywhere

GitOrigin-RevId: 8110c6e68727a52ea80f3ec71711d30e17d69499
2025-10-15 08:06:41 +00:00
Brian Gough f6229d6fe1 Merge pull request #29032 from overleaf/bg-retry-failed-backups
Add script and cronjobs to retry failed history-v1 backups

GitOrigin-RevId: c32102086235d4e22b947f4a80089b186c093cea
2025-10-14 08:06:28 +00:00
Jakob Ackermann fd0d21fbbd [monorepo] migrate build scripts to Jenkins (#29019)
* [monorepo] migrate build scripts to Jenkins

* [fraud-dash] fix integration with monorepo

GitOrigin-RevId: 6a30ec5ea68a1eae9b663faf9ccdc741ec7f9cf8
2025-10-13 08:07:11 +00:00
Tim Down 7aa66b260c Merge pull request #28541 from overleaf/td-notifications-app-ts
Change notifications app.js to TypeScript

GitOrigin-RevId: cb6195e2e8c8cd89e1a954bfcb1911929440d6ca
2025-10-13 08:05:36 +00:00
Miguel Serrano b6d116e957 Merge pull request #28912 from overleaf/msm-fix-certs-build
Fix `certs` image build

GitOrigin-RevId: 02af78b29915276d55e86001a1cdc4703fc830b5
2025-10-09 08:05:26 +00:00
Miguel Serrano 3a35b8680e Merge pull request #28554 from overleaf/msm-force-s3-lib-storage-uploads
[object-persistor] Use `@aws-sdk/lib-storage` for all uploads

GitOrigin-RevId: ab8e54a7bae843f9e6b05ed9cf936130a36b8c2f
2025-10-08 08:05:55 +00:00
Jakob Ackermann 5fe9d3d6e9 [monorepo] migrate monorepo checks to Jenkins (#28765)
GitOrigin-RevId: ebc3db95cebe99226f8de0f66c1830eb5d78e26c
2025-10-01 08:06:28 +00:00
Andrew RumbleandJakob Ackermann 0200ad7515 [monorepo] format ts files (#28678)
* Add ts files to format package script

* Apply changes to service package.json files

* [monorepo] only support plain .ts files in all the services

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 882cf274da86a0dbe1fff395441f6f2458c1405c
2025-09-25 08:05:30 +00:00
Eric Mc Sween 98069966ba Merge pull request #28672 from overleaf/em-reapply-restore-optimization
Reapply file restore optimization

GitOrigin-RevId: c4479141d890e3e0af746f406a0186bf21f3d918
2025-09-25 08:05:25 +00:00
9b2fcbe960 [monorepo] optimize Jenkins build triggers for services (#28477)
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: ea7d6c435d4a81a0f42ea9b608a0cc79087cdffd
2025-09-24 08:06:43 +00:00
Domagoj Kriskovic e9b1c63ed7 Revert "File restore optimization - using snapshot timestamp and file paths (#28546)"
This reverts commit 376a53e1f927cb56544e6782b47d80345655038c.

GitOrigin-RevId: 8e4ab1d4042fec8df1b62ab1c5418d873dc6f5eb
2025-09-24 08:06:06 +00:00
Domagoj KriskovicandEric Mc Sween 005eba7502 File restore optimization - using snapshot timestamp and file paths (#28546)
* Use snapshot timestamp, remove request for paths at version

* Add timestamp to core Snapshot object

* Avoid mutating function argument

* Explain assumption about editable files

* snapshot.toRaw() in getContentAtVersion

* fix project-history acceptance test

* fix history_v1 test

* fix web tests

* Include the snapshot timestamp in stored chunks

---------

Co-authored-by: Eric Mc Sween <5454374+emcsween@users.noreply.github.com>
GitOrigin-RevId: 376a53e1f927cb56544e6782b47d80345655038c
2025-09-24 08:06:02 +00:00
Brian Gough ff267ca246 Merge pull request #28640 from overleaf/bg-fix-chunk-store-metrics
update metrics for getChangesSinceVersion to include status and hasMore flag

GitOrigin-RevId: 389353425a12d6d7c92aa3ae37d6553fc1c7fc1d
2025-09-23 08:08:09 +00:00