501 Commits

Author SHA1 Message Date
Jimmy Domagala-Tang
bc2f5ae746 Reject tracked changes notifications (#32917)
* [web] Reject tracked changes notifications

feat: adding new tests

feat: adding rejected changes notifications

feat: adding tests for rejectchanges

feat: updating tests for rejecting notifications;

feat: adding in rejecting user, and improving subject and activity line

fix: moving to a params object instead of positionals for email building

feat: updating to use events triggered from applyUpdate in document-updater

feat: updating to send rejected author ids with rejected change notification instead of change ids

feat: moving rejected author notification determination to updateManager instead of RangesManager, which is used by other paths

feat: only map to author if changes were made

* fix: gate by user status not project status

* fix: unit tests post-rebase

---------

Co-authored-by: Kristina Hjertberg <kristina.hjertberg@overleaf.com>
GitOrigin-RevId: f992e1885c47d1a6cf776740769d6d4763f3cb7c
2026-05-14 08:05:35 +00:00
Domagoj Kriskovic
8f0979d6a2 Trigger reject tracked changes notification from document-updater
GitOrigin-RevId: 9ac47490d052b3058931ca250f4090e6576f56b2
2026-05-07 08:06:57 +00:00
Jimmy Domagala-Tang
994932b8e3 [Web + Doc-Updater] Add track changes accepted notification (#32752)
* feat: update doc manager to return a list of contributors to the accepted change

* feat: add new notification type for accepting a tracked change

* update email with tracked changes accepted

* feat: update tests

* fix: feedback on consistent api and returns

* feat: adding new tests

* feat: self accepted changes shouldnt trigger notification, and using existing changesAccepted hook

* Add better subject and activity list for track change accepted (#33094)

* feat: add better activity list entry and subject header for accepted changes, to match other notifications

* feat: updating tests

* feat: updating accepting_user_id to just user_id

* fix: adding users in emailBuilder test to userCache

GitOrigin-RevId: 6114f77916b5f503b7bbbb5ca8fed99e58edc31b
2026-04-30 08:06:19 +00:00
Brian Gough
15a24db3c9 Merge pull request #31650 from overleaf/bg-handle-timeouts-in-persistence-manager
add missing handling of timeouts in PersistenceManager

GitOrigin-RevId: 7fe74068f3ea647b27103393c3f0b243b4b25fe3
2026-02-20 09:05:50 +00:00
Brian Gough
a9c94c4184 Merge pull request #31444 from overleaf/bg-jpa-use-fetch-in-persistence-manager
remove requestretry from PersistenceManager

GitOrigin-RevId: 1fc9ffdfa7879d7ab4f0f4683544d09fe8526f3d
2026-02-19 09:05:45 +00:00
Jakob Ackermann
86b29819c2 [document-updater] add missing await (#31034)
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: f0602801ce80ca617027004ad36f71c1e6cdcdd2
2026-01-27 09:06:59 +00:00
Jakob Ackermann
425e7b1e5b [web] enable mongo notablescan in CI (#29501)
* [monorepo] record ERROR/FATAL log messages in junit report

* [web] put SaaS specific code behind feature flag

* [web] use split test cache for getting user assignments

The unit tests needed updating as they did not replicate any of the
 mongo filtering. The acceptance tests cover this logic.

* [web] make better use of existing indexes

* [web] avoid col-scan in tests of notifications module

* [web] remove cleanup of empty feedbacks collection

* [web] add assertion for reason of rejected request in launchpad test

* [web] add missing indexes

* [web] enable mongo notablescan

* [web] make emailNotifications tests compatible with notablescan

GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
2026-01-13 09:06:38 +00:00
Domagoj Kriskovic
0a84cefc96 Revert "Record project notification timestamp in Redis on applyUpdate (#29509)"
This reverts commit faa7a97fa929941cade5d62d2d680c6c3f34cdc8.

GitOrigin-RevId: 31c88ee836fb5e1ab3950da590c28e24b1397edb
2026-01-12 09:06:50 +00:00
Domagoj Kriskovic
8bca2545ee Record project notification timestamp in Redis on applyUpdate (#29509)
* Record project notification timestamp in Redis on applyUpdate

* Refactor project notification timestamp recording in Redis

* Remove tests which are catching errors that are no longer used

GitOrigin-RevId: faa7a97fa929941cade5d62d2d680c6c3f34cdc8
2026-01-12 09:06:07 +00:00
Mathias Jakobsen
82cb0e2839 Merge pull request #30312 from overleaf/mj-mock-web-api-async
[document-updater] Remove callbacks from MockWebApi

GitOrigin-RevId: 0189d5735f3e90160e593fb04043a10b1e922685
2025-12-18 09:05:55 +00:00
Mathias Jakobsen
ea4dcd4452 Merge pull request #30311 from overleaf/mj-mock-project-history-async
[document-updater] Remove callbacks from MockProjectHistoryApi

GitOrigin-RevId: b85702bfeeba2dc124880c2b590148514b4e89cb
2025-12-18 09:05:51 +00:00
Mathias Jakobsen
6f67ae4312 Merge pull request #30310 from overleaf/mj-async-mock-docstore-api
[document-updater] Remove callbacks from MockDocstoreApi

GitOrigin-RevId: 86342f293e9bbbcaddd6c44bcca62c301db23670
2025-12-18 09:05:46 +00:00
Brian Gough
a45c39389c Merge pull request #29891 from overleaf/bg-docupdater-make-max-unflushed-age-configurable
Make document-updater max unflushed age configurable via settings

GitOrigin-RevId: 3421f2467b7d8459ba8cdb1859e1b0695b0a01f3
2025-12-08 09:05:08 +00:00
Brian Gough
729e0f5ac9 move migrations to shared location (#28306)
* fix: correct typedef for Document in helpers.mjs

* add move-migrations codemod

* update migration paths to use shared migrations directory

* move migrations to shared location

* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory

* feat: add migrations tool to workspaces in package.json

* [monorepo] Fix order of docker ignore rules

* [web] remove unused docker ignore file

* [monorepo] replace old references to migrations folder

* [server-ce] copy migrations from new place

* [migrations] Inline web scripts

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] move three web scripts over

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] add missing collection

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] remove lodash dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] avoid mongodb-legacy dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [monorepo] run migrations from tools/migrations

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] simplify migration for adding gitBridge feature to users

* [monorepo] run migrations from tests in all the services

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] fixup running web migrations everywhere

* [monorepo] trigger Jenkins builds on changes to mongo migrations

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] build scripts: update devDependencies before deps scanning

* [monorepo] build scripts: formerly depend on tools/migrations

* [monorepo] run eslint on .mjs files

* [migrations] enable more eslint rules and fix all the errors

* [rake] fix migrations:list task

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
2025-10-16 08:07:37 +00:00
Jakob Ackermann
fd0d21fbbd [monorepo] migrate build scripts to Jenkins (#29019)
* [monorepo] migrate build scripts to Jenkins

* [fraud-dash] fix integration with monorepo

GitOrigin-RevId: 6a30ec5ea68a1eae9b663faf9ccdc741ec7f9cf8
2025-10-13 08:07:11 +00:00
Jakob Ackermann
dd3c1b686e [document-updater] fix test after other async/await work (#28944)
GitOrigin-RevId: 7338f340924b3355dac39a86c40327a2964c3020
2025-10-09 08:08:02 +00:00
Jakob Ackermann
d648c96603 [document-updater] migrate HistoryManager to async/await (#28789)
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.

GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
2025-10-09 08:07:36 +00:00
Tim Down
c104aa454e Merge pull request #28845 from overleaf/td-async-await-doc-updater-client
Convert DocUpdateClient in document-updater acceptance tests to async/await

GitOrigin-RevId: 8f2352119f8f1175c2703ed90dbbc483ed039e86
2025-10-08 08:05:26 +00:00
Eric Mc Sween
97b857ceb8 Merge pull request #28311 from overleaf/em-promisify-redis-manager
Promisify RedisManager in document-updater

GitOrigin-RevId: 87c86d3b8904e17e76756789d2ea3e09fab26d02
2025-09-10 08:05:57 +00:00
Eric Mc Sween
2995b29829 Merge pull request #28205 from overleaf/em-promisify-project-manager
Promisify ProjectManager

GitOrigin-RevId: d10807a9a7f11428dd6bfe41210bfd0ce32c37ba
2025-09-03 08:06:42 +00:00
Eric Mc Sween
ca9455d7f2 Merge pull request #28182 from overleaf/em-promisify-http-controller
Promisify HttpController in document-updater

GitOrigin-RevId: fb87a38be856d64781401f7391b7f2a2c35c89fa
2025-09-01 08:05:12 +00:00
Domagoj Kriskovic
704e577c23 Add endpoint to retrieve document with history ranges and use it in dsmp API (#27564)
GitOrigin-RevId: 3d2ac33cdc903a07b8ec67f7fb6f723ae9c81a26
2025-08-15 08:05:17 +00:00
Domagoj Kriskovic
49bcfdd575 [ds-mobile-app] Implement reject changes in document updater (#27483)
* [ds-mobile-app] Implement reject changes in document updater

* Return rejected change ids

GitOrigin-RevId: 7915170af9164852ce39d783776106e722aaa925
2025-08-11 08:05:29 +00:00
Jakob Ackermann
adf399fb95 [project-history] add support for resync of history-ot ranges (#26475)
* [project-history] add support for resync of history-ot ranges

* [project-history] avoid compressing sharejs and history-ot upgrades

* [document-updater] improve error message of some assertions

... by migrating the assertions like this:
```diff
-stub.calledWith().should.equal(true)
+stub.should.have.been.calledWith()
```
```diff
-stub.called.should.equal(false)
+stub.should.not.have.been.called
```

* [document-updater] move content field in resyncDocContent

* [document-updater] add support for resync of history-ot ranges

GitOrigin-RevId: e6104686a26934a5f25a8f095cbe00c163fbbaa7
2025-06-19 08:05:18 +00:00
Jakob Ackermann
842f6c289f [document-updater] make setDoc aware of tracked deletes in history-ot (#26126)
GitOrigin-RevId: efa1a94f2f435058b553f639e43832454c58591d
2025-06-06 08:05:23 +00:00
Jakob Ackermann
64984ee86a [history-ot] flush history for projects with short queues ASAP (#25776)
* [document-updater] flush history for projects with short queues ASAP

* [k8s] document-updater: enable short history queue for history-ot demo

* [project-history] flush history for projects with short queues ASAP

* [project-history] wait for mongo before running acceptance tests

* [k8s] project-history: enable short history queue for history-ot demo

* [project-history] change wait-for-mongo step in tests

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 3e989c409e8e9887655b35f2659ce0829e61b357
2025-05-23 08:05:35 +00:00
Jakob Ackermann
1e6b13f9d5 [history-ot] rename remaining history-v1-ot references to history-ot (#25428)
* [history-ot] rename remaining history-v1-ot references to history-ot

* [web] rename History-v1 OT -> History OT in admin panel

* [web] rename OT Migration -> History OT Migration in admin panel

GitOrigin-RevId: 103ce816d5320d6379d51009cdc08b8a71aa48e6
2025-05-20 08:06:46 +00:00
Jakob Ackermann
e8b5ee2ff9 [history-ot] initial implementation of using doc-level history-ot (#25054)
* [history-v1-ot] initial implementation of using doc-level history-v1-ot

* [web] fix advancing of the otMigrationStage

Use 'nextStage' for the user provided, desired stage when advancing.

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [document-updater] document size check in editor-core

* [history-ot] rename history-v1-ot to history-ot and add types

* [history-ot] apply review feedback

- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData

Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>

* Revert "[document-updater] add safe rollback point for history-ot (#25283)"

This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

---------

Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
2025-05-08 08:05:44 +00:00
Jakob Ackermann
a5e2708eae [document-updater] add safe rollback point for history-ot (#25283)
GitOrigin-RevId: d7230dd14a379a27d2c6ab03a006463a18979d06
2025-05-06 08:05:28 +00:00
Jakob Ackermann
7f086b21c8 [document-updater] modernize ApplyingUpdatesToADocTests (#25038)
- use beforeEach to ensure tests do not interfere with each other
  Notably, the 'when the ops come in a single linear order' test suite
  had state-changing tests that were dependent on the correct order.
  Also, the assigment of 'this.firstOpTimestamp' was in a test.
- consolidate populating project and doc ids
  The doc reference in this.update was undefined.
- fix doc reference in updates
  There were two misuses of 'doc_id' instead of 'doc'.
- Move mocking of MockWebApi.getDocument to the top and use
  sinon.resetHistory() or sinon.restore() for controlling the stub.
- Add another test for simple transforming of updates
  See 'when another client is sending a concurrent update'.

GitOrigin-RevId: 61ca8a1b0172920ad6ab1b604a9b9259cebddaad
2025-04-24 08:05:10 +00:00
Jakob Ackermann
ff78f687d8 [document-updater] add endpoint for project wide last updated timestamp (#24352)
* [document-updater] fix acceptance test for sending two updates

The Object.create() invocation yields an empty object. The following v
assignment works as expected. The effective update is { v: 43 }.
Processing that fails as no operations were included.

* [document-updater] add endpoint project wide last updated timestamp

* [document-updater] apply review feedback

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 81397537bfd85c2077f19669860b1391c15b34a3
2025-03-24 10:46:38 +00:00
Jakob Ackermann
0a7b2004d2 [misc] silence logger when running tests (#22243)
* [misc] silence logger when running tests

* [misc] re-enable logging in some tests for scripts

* [misc] make it easy to turn on verbose logging for tests

```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
2025-02-10 09:06:02 +00:00
Jakob Ackermann
b0c9176634 [project-history] add resync for project structure only (#23271)
* [project-history] add resync for project structure only

* [project-history] block resyncProjectStructureOnly that update docs

* [project-history] tweak test description

GitOrigin-RevId: ce2749566c36e04dab21c26dd60dd75d93c0d4c0
2025-02-03 09:05:39 +00:00
Eric Mc Sween
76dd6d1e20 Merge pull request #23212 from overleaf/em-docupdater-send-doc-hash
Send doc hash with history updates from docupdater

GitOrigin-RevId: 119475d4198c6603cecd4fd579a64ff4448261ce
2025-01-30 09:05:21 +00:00
Eric Mc Sween
e9c1c0f9c8 Merge pull request #22650 from overleaf/em-tracked-deletes-at-same-position
Handle multiple tracked deletes at same position

GitOrigin-RevId: 3cbf1c418bcd50cf08e1b90ce6ba3bc480236079
2025-01-14 09:05:34 +00:00
Domagoj Kriskovic
30ebad91b7 Allow reviewers to resolve their own comments (#22582)
* Allow reviewers to resolve their own comments

* check if reviewer is comment author

* add missing translation

* add CommentsController tests

* added DocumentManagerTests

* added HttpControllerTests

* Add AuthorizationManagerTests

* added AuthorizationMiddlewareTests

* added DocumentUpdaterHandler test

* fix test descriptions

* remove returns from CommentsControllerTests

* use ensureUserCanResolveThread in authorizationMiddleware

* move canResolveThread to AuthorizationManager

* commentId as param in NotFoundError

* refactor canUserResolveThread

GitOrigin-RevId: 131c3d1eb9ac916eaaa9221d351a92bc07b80cdc
2025-01-14 09:05:11 +00:00
Eric Mc Sween
a323a5c915 Merge pull request #22527 from overleaf/revert-22471-em-tracked-deletes-at-same-position
Revert "Improve handling of tracked delete rejections"

GitOrigin-RevId: 444a5a73aa2b93162036dfc463cd76d9c463aadb
2024-12-16 09:05:10 +00:00
Eric Mc Sween
6737cfa38d Merge pull request #22471 from overleaf/em-tracked-deletes-at-same-position
Improve handling of tracked delete rejections

GitOrigin-RevId: 35857d0a3f739c0531223737b2b649c9e8033157
2024-12-16 09:04:58 +00:00
Mathias Jakobsen
1063dabf33 [web+document-updater] Allow appending to documents (#20745)
Co-authored-by: David Powell <david.powell@overleaf.com>
GitOrigin-RevId: f66283926e7da3edf83ada9316c3a001287e1b42
2024-12-12 09:05:31 +00:00
Andrew Rumble
ad8026038c Merge pull request #22397 from overleaf/revert-22320-revert-22213-ar-avoid-duplicate-blob-writes
Revert "Revert "Avoid duplicate blob writes""

GitOrigin-RevId: f6a0083c0edf9ede14dfb52dbf1809a843aa5ff4
2024-12-10 09:05:07 +00:00
Andrew Rumble
ffa28b1a58 Merge pull request #22320 from overleaf/revert-22213-ar-avoid-duplicate-blob-writes
Revert "Avoid duplicate blob writes"

GitOrigin-RevId: 9f86bcea654cd3fa5f66fbdf42080e7f6e2861a7
2024-12-05 09:04:48 +00:00
Andrew Rumble
40b781eb0a Merge pull request #22213 from overleaf/ar-avoid-duplicate-blob-writes
Avoid duplicate blob writes

GitOrigin-RevId: 5e7a472b235bae5ef84389dbbe791189e951908a
2024-12-04 09:05:08 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Eric Mc Sween
7444026cc3 Merge pull request #21310 from overleaf/em-validate-tracked-changes
Reapply "Sanity check for tracked changes in document-updater"

GitOrigin-RevId: e7b38d192f5202006f61bd015bba81d751af5413
2024-10-30 09:04:46 +00:00
Brian Gough
4920af44b0 Merge pull request #20679 from overleaf/bg-issue19022
Move flush_all_projects endpoint in document-updater into script

GitOrigin-RevId: cb774d860b5928b7fece1a8e21b0b76aecae73ff
2024-10-14 11:10:42 +00:00
Eric Mc Sween
731ba80b47 Merge pull request #20888 from overleaf/revert-20834-em-validate-tracked-deletes
Revert "Sanity check for tracked changes in document-updater"

GitOrigin-RevId: 1b7592b1f2d60374ce9ba54d9db5e34c9636ea9a
2024-10-14 11:05:01 +00:00
Eric Mc Sween
dc445121fe Merge pull request #20834 from overleaf/em-validate-tracked-deletes
Sanity check for tracked changes in document-updater

GitOrigin-RevId: 5f4c5cb931a6dae5257fed2f21e40777cb466309
2024-10-14 11:04:04 +00:00
Eric Mc Sween
a93223c70b Merge pull request #20826 from overleaf/revert-20799-em-ranges-tracker-sanity-checks
Revert "Sanity check for tracked changes in document-updater"

GitOrigin-RevId: 7876d57298d0f5dbd54929fdf69bce2976f16a9f
2024-10-14 11:01:33 +00:00
Eric Mc Sween
b881a96b84 Merge pull request #20799 from overleaf/em-ranges-tracker-sanity-checks
Sanity check for tracked changes in document-updater

GitOrigin-RevId: 5094eee8c279eb194114ac6f7fa36f86c9e16ca7
2024-10-14 11:01:24 +00:00
Eric Mc Sween
4b7f58861a Merge pull request #20694 from overleaf/em-tracked-changes-accept-timestamp
Fix history timestamp when accepting tracked changes

GitOrigin-RevId: d68aff0996e7d639f3dd4a68784c0dc0482c55ab
2024-10-01 08:05:23 +00:00