Commit Graph

1136 Commits

Author SHA1 Message Date
Andrew Rumble
028bca6e87 Merge pull request #30798 from overleaf/ar-reapply-request-overrides
[monorepo] reapply request overrides

GitOrigin-RevId: 55d19cc011435a00ffc1e5c63f523473cd0fc2dc
2026-01-16 09:56:13 +00:00
Andrew Rumble
4271744bfd Merge pull request #30606 from overleaf/ar/bump-qs-to-6.14.1
[monorepo] bump qs to 6.14.1

GitOrigin-RevId: 4b1f8b1cf739c3c0195392fa6707fc233d036b70
2026-01-15 14:20:06 +00:00
Andrew Rumble
2cdbdc1291 Merge pull request #30785 from overleaf/ar-add-logs-for-no-projectHistoryId-from-web
[document-updater] add log for missing projectHistoryId from web

GitOrigin-RevId: 750150312e341ef09b6c82b9df467900beb4b784
2026-01-15 14:19:37 +00:00
Andrew Rumble
a071cd74bc Merge pull request #30784 from overleaf/revert-30675-ar-jpa-log-missing-history-id-docupdater
Revert "[document-updater] add additional logging when doc is missing projectHistoryId"

GitOrigin-RevId: 725508a8f88ad3354b8b4644f4ee1e91efbecde2
2026-01-15 14:19:24 +00:00
Andrew Rumble
258c6224f4 Merge pull request #30675 from overleaf/ar-jpa-log-missing-history-id-docupdater
[document-updater] add additional logging when doc is missing projectHistoryId

GitOrigin-RevId: fba311cb10e0eeeb0a8165ce4e519c7704a7458f
2026-01-15 14:19:11 +00:00
Jakob Ackermann
425e7b1e5b [web] enable mongo notablescan in CI (#29501)
* [monorepo] record ERROR/FATAL log messages in junit report

* [web] put SaaS specific code behind feature flag

* [web] use split test cache for getting user assignments

The unit tests needed updating as they did not replicate any of the
 mongo filtering. The acceptance tests cover this logic.

* [web] make better use of existing indexes

* [web] avoid col-scan in tests of notifications module

* [web] remove cleanup of empty feedbacks collection

* [web] add assertion for reason of rejected request in launchpad test

* [web] add missing indexes

* [web] enable mongo notablescan

* [web] make emailNotifications tests compatible with notablescan

GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
2026-01-13 09:06:38 +00:00
Domagoj Kriskovic
0a84cefc96 Revert "Record project notification timestamp in Redis on applyUpdate (#29509)"
This reverts commit faa7a97fa929941cade5d62d2d680c6c3f34cdc8.

GitOrigin-RevId: 31c88ee836fb5e1ab3950da590c28e24b1397edb
2026-01-12 09:06:50 +00:00
Domagoj Kriskovic
c461d4d8c3 Rename doc updater script to use .mts extension
GitOrigin-RevId: 43d84434ee91aa579cb049081a31cb3769844320
2026-01-12 09:06:12 +00:00
Domagoj Kriskovic
8bca2545ee Record project notification timestamp in Redis on applyUpdate (#29509)
* Record project notification timestamp in Redis on applyUpdate

* Refactor project notification timestamp recording in Redis

* Remove tests which are catching errors that are no longer used

GitOrigin-RevId: faa7a97fa929941cade5d62d2d680c6c3f34cdc8
2026-01-12 09:06:07 +00:00
Mathias Jakobsen
82cb0e2839 Merge pull request #30312 from overleaf/mj-mock-web-api-async
[document-updater] Remove callbacks from MockWebApi

GitOrigin-RevId: 0189d5735f3e90160e593fb04043a10b1e922685
2025-12-18 09:05:55 +00:00
Mathias Jakobsen
ea4dcd4452 Merge pull request #30311 from overleaf/mj-mock-project-history-async
[document-updater] Remove callbacks from MockProjectHistoryApi

GitOrigin-RevId: b85702bfeeba2dc124880c2b590148514b4e89cb
2025-12-18 09:05:51 +00:00
Mathias Jakobsen
6f67ae4312 Merge pull request #30310 from overleaf/mj-async-mock-docstore-api
[document-updater] Remove callbacks from MockDocstoreApi

GitOrigin-RevId: 86342f293e9bbbcaddd6c44bcca62c301db23670
2025-12-18 09:05:46 +00:00
Brian Gough
a45c39389c Merge pull request #29891 from overleaf/bg-docupdater-make-max-unflushed-age-configurable
Make document-updater max unflushed age configurable via settings

GitOrigin-RevId: 3421f2467b7d8459ba8cdb1859e1b0695b0a01f3
2025-12-08 09:05:08 +00:00
Miguel Serrano
a3ec5b2797 Merge pull request #30018 from overleaf/msm-fix-esm-import
[web] Fix ESM import

GitOrigin-RevId: 887f4927248241cb3d237e0bdad36b05928657ef
2025-12-03 09:05:32 +00:00
Gernot Schulz
74efa0e345 Merge pull request #29898 from overleaf/gs-jenkins-issues-all-pipelines
Enable Jenkins hooks for creating GitHub issues for all pipelines with owners

GitOrigin-RevId: ba46db1d3137db12ee5c78f09b126fb9927d9c49
2025-11-27 09:06:04 +00:00
Thomas
c059a3c5b0 Handle slashes in branch names for docker build tags (#29529)
* Refactor v1 Makefile to use DOCKER_COMPOSE variable for Docker commands with tag safe branch name

* Handle slashes in branch names for docker build tags

GitOrigin-RevId: 463940e8435845978aced745575905f3bfbb8e1c
2025-11-14 09:05:25 +00:00
Domagoj Kriskovic
c84cfc815a Document updater script for notifying web on project updates (#29490)
* Document updater script for notifying web on project updates

* use lua script for deleting keys

* define jobId when adding to queue

* removeOnFail age

* mongo check if collaborator exists

GitOrigin-RevId: f67a79c890a03ccf44fd84243c9a3f380e7afb43
2025-11-11 09:06:04 +00:00
Gernot Schulz
b834b8bcb7 Merge pull request #29537 from overleaf/gs-jenkins-pr-build-tags-templates
Tag images with branch head instead of merge commit (templates)

GitOrigin-RevId: 53e616aeec058863e25b356841a59f3ebe13c87f
2025-11-10 09:05:00 +00:00
Jakob Ackermann
6691c5dc05 [monorepo] enable mongo notablescan for all services but web (#29497)
* [migrations] make old migrations compatible with --notablescan in mongo

Re-running the migration for db.users.emails.reversedHostname on all
Server Pro/CE users will not be harmfull.

* [monorepo] enable mongo notablescan for all services but web

GitOrigin-RevId: e2a710c9de08de9ac2aa9df1f3082d5f1c33ea05
2025-11-05 09:06:35 +00:00
Brian Gough
729e0f5ac9 move migrations to shared location (#28306)
* fix: correct typedef for Document in helpers.mjs

* add move-migrations codemod

* update migration paths to use shared migrations directory

* move migrations to shared location

* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory

* feat: add migrations tool to workspaces in package.json

* [monorepo] Fix order of docker ignore rules

* [web] remove unused docker ignore file

* [monorepo] replace old references to migrations folder

* [server-ce] copy migrations from new place

* [migrations] Inline web scripts

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] move three web scripts over

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] add missing collection

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] remove lodash dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] avoid mongodb-legacy dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [monorepo] run migrations from tools/migrations

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] simplify migration for adding gitBridge feature to users

* [monorepo] run migrations from tests in all the services

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] fixup running web migrations everywhere

* [monorepo] trigger Jenkins builds on changes to mongo migrations

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] build scripts: update devDependencies before deps scanning

* [monorepo] build scripts: formerly depend on tools/migrations

* [monorepo] run eslint on .mjs files

* [migrations] enable more eslint rules and fix all the errors

* [rake] fix migrations:list task

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
2025-10-16 08:07:37 +00:00
Jakob Ackermann
0182ad0046 [monorepo] compute IMAGE_CACHE lazily (#29087)
GitOrigin-RevId: de287c89c1cea858d0609265945266be19d7ea26
2025-10-15 08:07:47 +00:00
Jakob Ackermann
15a6e8ba04 [monorepo] use content hash as docker image cache key (#29046)
* [monorepo] remove docker-repos flag from build scripts

* [monorepo] use content hash as docker image cache key

* [packer] jenkins-worker: populate build cache for all services

* [v1] adopt smarter docker caching

* [latexqc] adopt smarter docker caching

* [monorepo] refresh docker cache in Jenkins

* [packer] jenkins-worker: increase disk size

* [monorepo] run jenkins_docker_build_cache as Jenkins user for v1

* [monorepo] define MONOREPO at the top of all the Makefiles

* [monorepo] add --build-arg BUILDKIT_INLINE_CACHE=1 everywhere

GitOrigin-RevId: 8110c6e68727a52ea80f3ec71711d30e17d69499
2025-10-15 08:06:41 +00:00
Jakob Ackermann
fd0d21fbbd [monorepo] migrate build scripts to Jenkins (#29019)
* [monorepo] migrate build scripts to Jenkins

* [fraud-dash] fix integration with monorepo

GitOrigin-RevId: 6a30ec5ea68a1eae9b663faf9ccdc741ec7f9cf8
2025-10-13 08:07:11 +00:00
Tim Down
7aa66b260c Merge pull request #28541 from overleaf/td-notifications-app-ts
Change notifications app.js to TypeScript

GitOrigin-RevId: cb6195e2e8c8cd89e1a954bfcb1911929440d6ca
2025-10-13 08:05:36 +00:00
Jakob Ackermann
dd3c1b686e [document-updater] fix test after other async/await work (#28944)
GitOrigin-RevId: 7338f340924b3355dac39a86c40327a2964c3020
2025-10-09 08:08:02 +00:00
Jakob Ackermann
d648c96603 [document-updater] migrate HistoryManager to async/await (#28789)
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.

GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
2025-10-09 08:07:36 +00:00
Jakob Ackermann
b0b9733a42 [document-updater] migrate ProjectFlusher to async/await (#28796)
GitOrigin-RevId: 24f61d6c0fab5d65b962cc7031ce0b8c84d5a915
2025-10-09 08:07:31 +00:00
Tim Down
c104aa454e Merge pull request #28845 from overleaf/td-async-await-doc-updater-client
Convert DocUpdateClient in document-updater acceptance tests to async/await

GitOrigin-RevId: 8f2352119f8f1175c2703ed90dbbc483ed039e86
2025-10-08 08:05:26 +00:00
Jakob Ackermann
5fe9d3d6e9 [monorepo] migrate monorepo checks to Jenkins (#28765)
GitOrigin-RevId: ebc3db95cebe99226f8de0f66c1830eb5d78e26c
2025-10-01 08:06:28 +00:00
Andrew Rumble
0200ad7515 [monorepo] format ts files (#28678)
* Add ts files to format package script

* Apply changes to service package.json files

* [monorepo] only support plain .ts files in all the services

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 882cf274da86a0dbe1fff395441f6f2458c1405c
2025-09-25 08:05:30 +00:00
Jakob Ackermann
9b2fcbe960 [monorepo] optimize Jenkins build triggers for services (#28477)
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: ea7d6c435d4a81a0f42ea9b608a0cc79087cdffd
2025-09-24 08:06:43 +00:00
Andrew Rumble
423c2a47e5 Merge pull request #28482 from overleaf/ar-move-notifications-changes-into-buildscripts
[monorepo] move notifications changes into buildscripts

GitOrigin-RevId: b5948702627d0bc87bffd3acc3d7c2ab9f4ac23d
2025-09-23 08:05:53 +00:00
Jakob Ackermann
a6e9a5c7e9 [monorepo] run prettier on Jenkinsfiles (#28565)
* [monorepo] consolidate .editorconfig files at the root

* [monorepo] run prettier on Jenkinsfiles

* [saas-e2e] increase timeout for XeTeX compile

GitOrigin-RevId: 48aa82f7c81611899837753ae92c7732998d4ca5
2025-09-19 08:07:49 +00:00
Eric Mc Sween
543f3fafcc Merge pull request #28431 from overleaf/em-docupdater-get-ranges
Look in docupdater when obtaining comments for DSMP

GitOrigin-RevId: acaead280f725da8e0c438d5b429fa983c3cad86
2025-09-15 08:05:29 +00:00
Eric Mc Sween
97b857ceb8 Merge pull request #28311 from overleaf/em-promisify-redis-manager
Promisify RedisManager in document-updater

GitOrigin-RevId: 87c86d3b8904e17e76756789d2ea3e09fab26d02
2025-09-10 08:05:57 +00:00
Eric Mc Sween
2995b29829 Merge pull request #28205 from overleaf/em-promisify-project-manager
Promisify ProjectManager

GitOrigin-RevId: d10807a9a7f11428dd6bfe41210bfd0ce32c37ba
2025-09-03 08:06:42 +00:00
Eric Mc Sween
ca9455d7f2 Merge pull request #28182 from overleaf/em-promisify-http-controller
Promisify HttpController in document-updater

GitOrigin-RevId: fb87a38be856d64781401f7391b7f2a2c35c89fa
2025-09-01 08:05:12 +00:00
Eric Mc Sween
8f0913fafe Merge pull request #27936 from overleaf/em-unit-tests-mongo
Make Mongo available to unit tests in all services

GitOrigin-RevId: b65bbb69883d5bba31d09802b89f35bdc523fe4d
2025-08-25 08:05:19 +00:00
Andrew Rumble
6190005f9b [monorepo] Upgrade to node 22.18.0 (#27977)
* Upgrade to node 22.18.0

* Switch projects-data fixture to strip-types compatible form

GitOrigin-RevId: 4a7ec56f4a82395d2be8870526df0c679de38ab5
2025-08-22 08:05:17 +00:00
Domagoj Kriskovic
704e577c23 Add endpoint to retrieve document with history ranges and use it in dsmp API (#27564)
GitOrigin-RevId: 3d2ac33cdc903a07b8ec67f7fb6f723ae9c81a26
2025-08-15 08:05:17 +00:00
Jakob Ackermann
99feb94d4b [monorepo] add workaround for flaky mongo db init (#27833)
* [monorepo] add workaround for flaky mongo db init

Upstream issue: https://github.com/docker-library/mongo/issues/730
Upstream PR: https://github.com/docker-library/mongo/pull/731

* [server-ce] revert entrypoint changes

* [monorepo] fix typo in path

* [server-ce] run E2E tests from internal monorepo in cloudbuild

* [monorepo] avoid repeated changes to mongo entrypoint

GitOrigin-RevId: 617db9caee1600c8a506a7a147723a306e57ded3
2025-08-14 08:05:58 +00:00
Jakob Ackermann
836e8b5144 [monorepo] extend cleanup in Jenkins VMs (#27793)
* [monorepo] extend docker compose cleanup

* [web] cleanup docker images

Only retain the branch builds for dependencies.

* [server-ce] jenkins: remove build specific image

Retain the branch image as cache.

* [monorepo] jenkins: prune untagged docker images and docker build cache

* [monorepo] jenkins: remove temporary files from monorepo

* [monorepo] jenkins: remove containers from sandboxed compiles

GitOrigin-RevId: c847015986a060c5c84c39cd06236fd73bbf6f3a
2025-08-12 08:06:27 +00:00
Jakob Ackermann
ac9c20e8d2 [monorepo] tweaks for Jenkins/CI (#27761)
* [packer] jenkins-worker: install htop and lsof

* [web] trim tailing space in Jenkinsfile

* [web] cleanup after finishing Jenkins pipeline

* [server-ce] test: cleanup after finishing Jenkins pipeline

* [saas-e2e] add timeout to waiting for rail/web startup

* [monorepo] add 60s timeout for waiting on mongo to startup

* [saas-e2e] cleanup after finishing Jenkins pipeline

GitOrigin-RevId: 9954ab17b4db6a95c444e8e0b55cebb8d5daae59
2025-08-11 08:06:31 +00:00
Domagoj Kriskovic
49bcfdd575 [ds-mobile-app] Implement reject changes in document updater (#27483)
* [ds-mobile-app] Implement reject changes in document updater

* Return rejected change ids

GitOrigin-RevId: 7915170af9164852ce39d783776106e722aaa925
2025-08-11 08:05:29 +00:00
Alf Eaton
03a4bbb96b Remove "version" from docker-compose files (#27494)
GitOrigin-RevId: 8d9a0e58e3a9c6c4296e4585115d1b259e26a6cd
2025-07-31 08:05:10 +00:00
Christopher Hoskin
7ecee2e0aa Merge pull request #27255 from overleaf/revert-27252-revert-26843-csh-issue-26608-mongo8-dev-ci
Revert "Revert "Upgrade the dev environment and CI to mongo 8""

GitOrigin-RevId: 5074b012504e65240017f1fde9b0d8d04c7b8b61
2025-07-22 08:05:25 +00:00
Christopher Hoskin
f4dc8f7ebc Merge pull request #27252 from overleaf/revert-26843-csh-issue-26608-mongo8-dev-ci
Revert "Upgrade the dev environment and CI to mongo 8"

GitOrigin-RevId: f2145812a5c1cf8d3d3ac31c76cc4aed4ea9d46d
2025-07-21 08:05:01 +00:00
Christopher Hoskin
456f751a18 Merge pull request #26843 from overleaf/csh-issue-26608-mongo8-dev-ci
Upgrade the dev environment and CI to mongo 8

GitOrigin-RevId: 5fc33863094ea2bdb9235372efdc7d0cb492c34a
2025-07-21 08:04:57 +00:00
Andrew Rumble
1a1f283245 Update node to 22.17.0
GitOrigin-RevId: 7a15f7a0e95e3331c589955aca1fbc5dfb13a568
2025-07-09 08:06:01 +00:00
Jakob Ackermann
adf399fb95 [project-history] add support for resync of history-ot ranges (#26475)
* [project-history] add support for resync of history-ot ranges

* [project-history] avoid compressing sharejs and history-ot upgrades

* [document-updater] improve error message of some assertions

... by migrating the assertions like this:
```diff
-stub.calledWith().should.equal(true)
+stub.should.have.been.calledWith()
```
```diff
-stub.called.should.equal(false)
+stub.should.not.have.been.called
```

* [document-updater] move content field in resyncDocContent

* [document-updater] add support for resync of history-ot ranges

GitOrigin-RevId: e6104686a26934a5f25a8f095cbe00c163fbbaa7
2025-06-19 08:05:18 +00:00