Commit Graph
526 Commits
Author SHA1 Message Date
Malik Glossop 863e33af78 Merge pull request #35982 from overleaf/mg-history-ot-comments
HistoryOT - Fix comments being dropped and crashing on detached ranges

GitOrigin-RevId: 0906a55a51a6fda9ac828efcbe72989573161261
2026-09-17 12:03:46 +00:00
Jakob Ackermann 4e80d65efc [ranges-tracker] add legacy orderedRejections field in tracked-changes (#36565)
* [docstore] use more realistic test stubs for ranges

* [monorepo] define ranges schemas in @overleaf/ranges-tracker

* [ranges-tracker] add legacy orderedRejections field in tracked-changes

* [monorepo] resolve @overleaf/ranges-tracker subpath imports in typescript

* [real-time] validate comment thread ids and tracked change id seeds

* [ranges-tracker] require ObjectId ids and tracked change metadata in ranges

* [monorepo] require ObjectId tracked change ids on the accept routes

* [web] use ObjectId comment thread ids in the DocumentController tests

GitOrigin-RevId: 58b20dec311d85b917ef63f9efbe745a265a2de6
2026-09-17 12:01:19 +00:00
KristinaandJimmy Domagala-Tang 9c005e4e54 [web/document-updater] display preview for accepted/rejected tracked changes (#36240)
* [web] add preview payload to track-change notification types

* [doc-updater] pass accepted/rejected change data through to web for previews

* [web] snapshot change preview onto accepted/rejected notifications

* [web] render resolved change previews instead of accepted/rejected count lines

* [web] add unit coverage for accepted/rejected change preview flow

* fix: notification carries multiple previews per accept/reject batch; extract shared group helper

* fix: one preview per accept/reject action, drop adjacency grouping

* fix: move buildResolvedChangePreviews out of the email builder class

* fix: type error, make tests more explicit

* fix: render resolver below preview

* fix: overflow text, alignment

* fix: test error

* fix: resolved count

* [document-updater/web] send sparse change preview instead of full doc lines

* fix: store sparse preview, hydrate at render, drop CE gate

* fix: advance slice cursor past tracked-delete text in sparse preview

* fix: inline buildChangeGroupPreview

* fix: changePreviewBuilder -> sparseChangePreview

* fix: rm change_contributors

* fix: copilot flagged testing issues

* fix: hydrate and truncate change preview at write time

* fix: avoid materializing full doc text when slicing preview window

* fix: update reject preview test to hydrated shape

* fix: accepted delete removing context

* [document-updater/web] split resolved change previews into one per change cluster

* fix: narrow cluster userIds to string[] for typecheck

* feat: updating to use zod schema for previews

---------

Co-authored-by: Jimmy Domagala-Tang <jimmy.domagala-tang@overleaf.com>
GitOrigin-RevId: 0740713c715a61a220180f353e497b27da6a5b0f
2026-09-17 11:58:14 +00:00
Andrew Rumble 66355e4372 Merge pull request #36361 from overleaf/ar-fix-postdoc-validation-errors
[docstore][document-updater] fix POST doc validation errors

GitOrigin-RevId: 9b5fbc667a3b56b30077ea8de9d81c17253db273
2026-09-17 11:51:12 +00:00
Jakob Ackermann 0776357d1a [document-updater] address zod violations (#36147)
* [editor-core] address zod violation for group_id

* [monorepo] use discriminated union in zod schemas where possible

* [editor-core] v1_source_doc_id is a number

GitOrigin-RevId: 38f591b07cca4e22954bfe30bf855545a6af675d
2026-09-17 11:46:42 +00:00
Jakob Ackermann 4e80098424 [document-updater] adopt strict zod types and extend test coverage (#36077)
* [document-updater] adopt strict zod types and extend test coverage

* [saas-e2e] add test for project restore with binary and linked file

GitOrigin-RevId: a8e45bd1e46b6ac8a293da24c8b8ad7b236fd9bd
2026-09-17 11:46:05 +00:00
Anna Claire Fields a63fd554c3 Replace MOCHA_GREP with TEST_NAME_PATTERN (#34404)
GitOrigin-RevId: b1b2d1be3d0725f930d83ef035836482ffb72dcd
2026-09-17 11:45:29 +00:00
Jakob Ackermann e8070def26 [document-updater] fix typo in test stub (#35950)
tid does not exist anywhere else in document-updater at
sha 3d76f4b9bf when
the typo was added. The comment id was read from .t only.
Doc-snapshots are copied the ranges literally without any
normalizing of nested structures.

GitOrigin-RevId: f76fbc930fdf02d9367603b5b1d6414de8d9f559
2026-09-17 11:40:01 +00:00
Jakob Ackermann 8fa62d2864 [document-updater] drop support for tc in briefly (#35864)
GitOrigin-RevId: 00d04d911249b81abc5eb4f9d9dfde4c7f2316b3
2026-09-17 11:39:22 +00:00
Jakob Ackermann 7a0a9c3ce5 [document-updater] implement HTTP endpoints for comments (#35833)
GitOrigin-RevId: 06b4f222a55e21b3def432d6e84d9ab4bf65b64d
2026-09-17 11:39:19 +00:00
Jakob Ackermann 8be65f1f93 [document-updater] add support for flush/restore of tc with history-ot (#35823)
GitOrigin-RevId: a79b0ca97c930ca2e2f3ffeea20defbc6973468c
2026-09-17 11:39:15 +00:00
Jakob Ackermann 00a2765d01 [document-updater] add track-changes support to setDoc (#35820)
GitOrigin-RevId: 5d39dd1945e710ec3f7068830fb2476e59f0f2a5
2026-09-17 11:39:12 +00:00
Jakob Ackermann aab08b4c28 [document-updater] remove doc lock from per-doc operations (#35696)
GitOrigin-RevId: c45c1dbf11049c0fab8a7d788006a698ede0d510
2026-09-17 11:34:14 +00:00
Jakob Ackermann bc9d25d385 [monorepo] clean up the per-project pub/sub migration (#35686)
GitOrigin-RevId: 4bbb97d1d2750f21751ef54e206e3e720b88615a
2026-09-17 11:33:51 +00:00
Jakob Ackermann 3565f51a84 [monorepo] prepare for per-project pub/sub for applied ops (#35663)
GitOrigin-RevId: 11d45addf366c3449cad432272ed03d4c8dae49d
2026-09-17 11:33:01 +00:00
Jakob Ackermann e1358c4507 [document-updater] remove unused script (#35614)
GitOrigin-RevId: 95717884ce4b5207ef7289108eb64a84d4ed5dd4
2026-09-17 11:32:11 +00:00
Jakob Ackermann 324f011d80 [document-updater] add script for flushing/discarding stuck docs (#35603)
GitOrigin-RevId: 099e1e0a38515b35decbdc608acd75f59de0ec6f
2026-09-17 11:31:56 +00:00
Jakob Ackermann 68352783b0 [monorepo] clean up per-project queue migration (#35573)
GitOrigin-RevId: 988df4cdc7d929efc921e7e99d803c6ce6e52c57
2026-09-17 11:31:52 +00:00
Jakob Ackermann cdba372dfc [monorepo] prepare per-project queues (#35386)
GitOrigin-RevId: ab020603ec04c705d08be466563adac0ce30bc04
2026-09-17 11:26:50 +00:00
Jakob Ackermann 89f12e6b1f [document-updater] wrap doc locks in project lock (#35416)
* [document-updater] wrap doc locks in project lock

* [document-updater] use BlockingProject as redis key prefix

* [redis-wrapper] update comment

GitOrigin-RevId: 0ce50e76386871e6c29481494febc8d6b37e7df5
2026-09-17 11:24:34 +00:00
Lucie Germain 210ced0d31 Merge pull request #33361 from overleaf/copilot/remove-direct-use-of-body-parser
Remove direct use of body-parser in favour of express built-ins

GitOrigin-RevId: 656c5f216daf68e86d53ae07d272e080bd5ab27f
2026-09-17 11:14:46 +00:00
Jakob Ackermann 1b553c0455 [migrations] only apply non-blocking migrations in CI (#34665)
* [migrations] only apply non-blocking migrations in CI

* [migrations] use "yarn run migrations" instead of "npm run migrations"

GitOrigin-RevId: d9f96db398adf2febcd82333295fb8d5eab741bb
2026-06-26 08:08:05 +00:00
Anna Claire Fields 7035a2e025 Revert "Acf pnp migration master" (#34709)
GitOrigin-RevId: ec7d63e1212e05dd6dfc7a0ddf785c11eca76ad6
2026-06-17 08:07:56 +00:00
Anna Claire Fields 8f07308342 Acf pnp migration master (#34040)
GitOrigin-RevId: 418eab9bc733f27a6e0f8f04a2a8e21c2f5a491f
2026-06-17 08:07:16 +00:00
Domagoj Kriskovic bf1dd6986f Record project notification timestamp in Redis on applyUpdate (reverted) (#30814)
* Revert "Revert "Record project notification timestamp in Redis on applyUpdate (#29509)""

This reverts commit 31c88ee836fb5e1ab3950da590c28e24b1397edb.

GitOrigin-RevId: fc012324f6035156585fab468aca72b900a9710b
2026-05-22 08:06:26 +00:00
Jimmy Domagala-TangandKristina Hjertberg bc2f5ae746 Reject tracked changes notifications (#32917)
* [web] Reject tracked changes notifications

feat: adding new tests

feat: adding rejected changes notifications

feat: adding tests for rejectchanges

feat: updating tests for rejecting notifications;

feat: adding in rejecting user, and improving subject and activity line

fix: moving to a params object instead of positionals for email building

feat: updating to use events triggered from applyUpdate in document-updater

feat: updating to send rejected author ids with rejected change notification instead of change ids

feat: moving rejected author notification determination to updateManager instead of RangesManager, which is used by other paths

feat: only map to author if changes were made

* fix: gate by user status not project status

* fix: unit tests post-rebase

---------

Co-authored-by: Kristina Hjertberg <kristina.hjertberg@overleaf.com>
GitOrigin-RevId: f992e1885c47d1a6cf776740769d6d4763f3cb7c
2026-05-14 08:05:35 +00:00
Domagoj Kriskovic 8f0979d6a2 Trigger reject tracked changes notification from document-updater
GitOrigin-RevId: 9ac47490d052b3058931ca250f4090e6576f56b2
2026-05-07 08:06:57 +00:00
Jimmy Domagala-Tang 994932b8e3 [Web + Doc-Updater] Add track changes accepted notification (#32752)
* feat: update doc manager to return a list of contributors to the accepted change

* feat: add new notification type for accepting a tracked change

* update email with tracked changes accepted

* feat: update tests

* fix: feedback on consistent api and returns

* feat: adding new tests

* feat: self accepted changes shouldnt trigger notification, and using existing changesAccepted hook

* Add better subject and activity list for track change accepted (#33094)

* feat: add better activity list entry and subject header for accepted changes, to match other notifications

* feat: updating tests

* feat: updating accepting_user_id to just user_id

* fix: adding users in emailBuilder test to userCache

GitOrigin-RevId: 6114f77916b5f503b7bbbb5ca8fed99e58edc31b
2026-04-30 08:06:19 +00:00
Brian Gough 15a24db3c9 Merge pull request #31650 from overleaf/bg-handle-timeouts-in-persistence-manager
add missing handling of timeouts in PersistenceManager

GitOrigin-RevId: 7fe74068f3ea647b27103393c3f0b243b4b25fe3
2026-02-20 09:05:50 +00:00
Brian Gough a9c94c4184 Merge pull request #31444 from overleaf/bg-jpa-use-fetch-in-persistence-manager
remove requestretry from PersistenceManager

GitOrigin-RevId: 1fc9ffdfa7879d7ab4f0f4683544d09fe8526f3d
2026-02-19 09:05:45 +00:00
Jakob AckermannandBrian Gough 86b29819c2 [document-updater] add missing await (#31034)
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: f0602801ce80ca617027004ad36f71c1e6cdcdd2
2026-01-27 09:06:59 +00:00
Jakob Ackermann 425e7b1e5b [web] enable mongo notablescan in CI (#29501)
* [monorepo] record ERROR/FATAL log messages in junit report

* [web] put SaaS specific code behind feature flag

* [web] use split test cache for getting user assignments

The unit tests needed updating as they did not replicate any of the
 mongo filtering. The acceptance tests cover this logic.

* [web] make better use of existing indexes

* [web] avoid col-scan in tests of notifications module

* [web] remove cleanup of empty feedbacks collection

* [web] add assertion for reason of rejected request in launchpad test

* [web] add missing indexes

* [web] enable mongo notablescan

* [web] make emailNotifications tests compatible with notablescan

GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
2026-01-13 09:06:38 +00:00
Domagoj Kriskovic 0a84cefc96 Revert "Record project notification timestamp in Redis on applyUpdate (#29509)"
This reverts commit faa7a97fa929941cade5d62d2d680c6c3f34cdc8.

GitOrigin-RevId: 31c88ee836fb5e1ab3950da590c28e24b1397edb
2026-01-12 09:06:50 +00:00
Domagoj Kriskovic 8bca2545ee Record project notification timestamp in Redis on applyUpdate (#29509)
* Record project notification timestamp in Redis on applyUpdate

* Refactor project notification timestamp recording in Redis

* Remove tests which are catching errors that are no longer used

GitOrigin-RevId: faa7a97fa929941cade5d62d2d680c6c3f34cdc8
2026-01-12 09:06:07 +00:00
Mathias Jakobsen 82cb0e2839 Merge pull request #30312 from overleaf/mj-mock-web-api-async
[document-updater] Remove callbacks from MockWebApi

GitOrigin-RevId: 0189d5735f3e90160e593fb04043a10b1e922685
2025-12-18 09:05:55 +00:00
Mathias Jakobsen ea4dcd4452 Merge pull request #30311 from overleaf/mj-mock-project-history-async
[document-updater] Remove callbacks from MockProjectHistoryApi

GitOrigin-RevId: b85702bfeeba2dc124880c2b590148514b4e89cb
2025-12-18 09:05:51 +00:00
Mathias Jakobsen 6f67ae4312 Merge pull request #30310 from overleaf/mj-async-mock-docstore-api
[document-updater] Remove callbacks from MockDocstoreApi

GitOrigin-RevId: 86342f293e9bbbcaddd6c44bcca62c301db23670
2025-12-18 09:05:46 +00:00
Brian Gough a45c39389c Merge pull request #29891 from overleaf/bg-docupdater-make-max-unflushed-age-configurable
Make document-updater max unflushed age configurable via settings

GitOrigin-RevId: 3421f2467b7d8459ba8cdb1859e1b0695b0a01f3
2025-12-08 09:05:08 +00:00
729e0f5ac9 move migrations to shared location (#28306)
* fix: correct typedef for Document in helpers.mjs

* add move-migrations codemod

* update migration paths to use shared migrations directory

* move migrations to shared location

* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory

* feat: add migrations tool to workspaces in package.json

* [monorepo] Fix order of docker ignore rules

* [web] remove unused docker ignore file

* [monorepo] replace old references to migrations folder

* [server-ce] copy migrations from new place

* [migrations] Inline web scripts

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] move three web scripts over

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] add missing collection

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] remove lodash dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] avoid mongodb-legacy dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [monorepo] run migrations from tools/migrations

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] simplify migration for adding gitBridge feature to users

* [monorepo] run migrations from tests in all the services

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] fixup running web migrations everywhere

* [monorepo] trigger Jenkins builds on changes to mongo migrations

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] build scripts: update devDependencies before deps scanning

* [monorepo] build scripts: formerly depend on tools/migrations

* [monorepo] run eslint on .mjs files

* [migrations] enable more eslint rules and fix all the errors

* [rake] fix migrations:list task

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
2025-10-16 08:07:37 +00:00
Jakob Ackermann fd0d21fbbd [monorepo] migrate build scripts to Jenkins (#29019)
* [monorepo] migrate build scripts to Jenkins

* [fraud-dash] fix integration with monorepo

GitOrigin-RevId: 6a30ec5ea68a1eae9b663faf9ccdc741ec7f9cf8
2025-10-13 08:07:11 +00:00
Jakob Ackermann dd3c1b686e [document-updater] fix test after other async/await work (#28944)
GitOrigin-RevId: 7338f340924b3355dac39a86c40327a2964c3020
2025-10-09 08:08:02 +00:00
Jakob Ackermann d648c96603 [document-updater] migrate HistoryManager to async/await (#28789)
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.

GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
2025-10-09 08:07:36 +00:00
Tim Down c104aa454e Merge pull request #28845 from overleaf/td-async-await-doc-updater-client
Convert DocUpdateClient in document-updater acceptance tests to async/await

GitOrigin-RevId: 8f2352119f8f1175c2703ed90dbbc483ed039e86
2025-10-08 08:05:26 +00:00
Eric Mc Sween 97b857ceb8 Merge pull request #28311 from overleaf/em-promisify-redis-manager
Promisify RedisManager in document-updater

GitOrigin-RevId: 87c86d3b8904e17e76756789d2ea3e09fab26d02
2025-09-10 08:05:57 +00:00
Eric Mc Sween 2995b29829 Merge pull request #28205 from overleaf/em-promisify-project-manager
Promisify ProjectManager

GitOrigin-RevId: d10807a9a7f11428dd6bfe41210bfd0ce32c37ba
2025-09-03 08:06:42 +00:00
Eric Mc Sween ca9455d7f2 Merge pull request #28182 from overleaf/em-promisify-http-controller
Promisify HttpController in document-updater

GitOrigin-RevId: fb87a38be856d64781401f7391b7f2a2c35c89fa
2025-09-01 08:05:12 +00:00
Domagoj Kriskovic 704e577c23 Add endpoint to retrieve document with history ranges and use it in dsmp API (#27564)
GitOrigin-RevId: 3d2ac33cdc903a07b8ec67f7fb6f723ae9c81a26
2025-08-15 08:05:17 +00:00
Domagoj Kriskovic 49bcfdd575 [ds-mobile-app] Implement reject changes in document updater (#27483)
* [ds-mobile-app] Implement reject changes in document updater

* Return rejected change ids

GitOrigin-RevId: 7915170af9164852ce39d783776106e722aaa925
2025-08-11 08:05:29 +00:00
Jakob Ackermann adf399fb95 [project-history] add support for resync of history-ot ranges (#26475)
* [project-history] add support for resync of history-ot ranges

* [project-history] avoid compressing sharejs and history-ot upgrades

* [document-updater] improve error message of some assertions

... by migrating the assertions like this:
```diff
-stub.calledWith().should.equal(true)
+stub.should.have.been.calledWith()
```
```diff
-stub.called.should.equal(false)
+stub.should.not.have.been.called
```

* [document-updater] move content field in resyncDocContent

* [document-updater] add support for resync of history-ot ranges

GitOrigin-RevId: e6104686a26934a5f25a8f095cbe00c163fbbaa7
2025-06-19 08:05:18 +00:00
Jakob Ackermann 842f6c289f [document-updater] make setDoc aware of tracked deletes in history-ot (#26126)
GitOrigin-RevId: efa1a94f2f435058b553f639e43832454c58591d
2025-06-06 08:05:23 +00:00