* [docstore] use more realistic test stubs for ranges
* [monorepo] define ranges schemas in @overleaf/ranges-tracker
* [ranges-tracker] add legacy orderedRejections field in tracked-changes
* [monorepo] resolve @overleaf/ranges-tracker subpath imports in typescript
* [real-time] validate comment thread ids and tracked change id seeds
* [ranges-tracker] require ObjectId ids and tracked change metadata in ranges
* [monorepo] require ObjectId tracked change ids on the accept routes
* [web] use ObjectId comment thread ids in the DocumentController tests
GitOrigin-RevId: 58b20dec311d85b917ef63f9efbe745a265a2de6
* [editor-core] address zod violation for group_id
* [monorepo] use discriminated union in zod schemas where possible
* [editor-core] v1_source_doc_id is a number
GitOrigin-RevId: 38f591b07cca4e22954bfe30bf855545a6af675d
* [document-updater] adopt strict zod types and extend test coverage
* [saas-e2e] add test for project restore with binary and linked file
GitOrigin-RevId: a8e45bd1e46b6ac8a293da24c8b8ad7b236fd9bd
tid does not exist anywhere else in document-updater at
sha 3d76f4b9bf when
the typo was added. The comment id was read from .t only.
Doc-snapshots are copied the ranges literally without any
normalizing of nested structures.
GitOrigin-RevId: f76fbc930fdf02d9367603b5b1d6414de8d9f559
* [migrations] only apply non-blocking migrations in CI
* [migrations] use "yarn run migrations" instead of "npm run migrations"
GitOrigin-RevId: d9f96db398adf2febcd82333295fb8d5eab741bb
* Record project notification timestamp in Redis on applyUpdate
* Refactor project notification timestamp recording in Redis
* Remove tests which are catching errors that are no longer used
GitOrigin-RevId: faa7a97fa929941cade5d62d2d680c6c3f34cdc8
* fix: correct typedef for Document in helpers.mjs
* add move-migrations codemod
* update migration paths to use shared migrations directory
* move migrations to shared location
* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory
* feat: add migrations tool to workspaces in package.json
* [monorepo] Fix order of docker ignore rules
* [web] remove unused docker ignore file
* [monorepo] replace old references to migrations folder
* [server-ce] copy migrations from new place
* [migrations] Inline web scripts
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] move three web scripts over
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] add missing collection
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] remove lodash dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] avoid mongodb-legacy dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [monorepo] run migrations from tools/migrations
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] simplify migration for adding gitBridge feature to users
* [monorepo] run migrations from tests in all the services
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] fixup running web migrations everywhere
* [monorepo] trigger Jenkins builds on changes to mongo migrations
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] build scripts: update devDependencies before deps scanning
* [monorepo] build scripts: formerly depend on tools/migrations
* [monorepo] run eslint on .mjs files
* [migrations] enable more eslint rules and fix all the errors
* [rake] fix migrations:list task
---------
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.
GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
* [history-ot] rename remaining history-v1-ot references to history-ot
* [web] rename History-v1 OT -> History OT in admin panel
* [web] rename OT Migration -> History OT Migration in admin panel
GitOrigin-RevId: 103ce816d5320d6379d51009cdc08b8a71aa48e6
* [history-v1-ot] initial implementation of using doc-level history-v1-ot
* [web] fix advancing of the otMigrationStage
Use 'nextStage' for the user provided, desired stage when advancing.
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [document-updater] document size check in editor-core
* [history-ot] rename history-v1-ot to history-ot and add types
* [history-ot] apply review feedback
- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
* Revert "[document-updater] add safe rollback point for history-ot (#25283)"
This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
---------
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
- use beforeEach to ensure tests do not interfere with each other
Notably, the 'when the ops come in a single linear order' test suite
had state-changing tests that were dependent on the correct order.
Also, the assigment of 'this.firstOpTimestamp' was in a test.
- consolidate populating project and doc ids
The doc reference in this.update was undefined.
- fix doc reference in updates
There were two misuses of 'doc_id' instead of 'doc'.
- Move mocking of MockWebApi.getDocument to the top and use
sinon.resetHistory() or sinon.restore() for controlling the stub.
- Add another test for simple transforming of updates
See 'when another client is sending a concurrent update'.
GitOrigin-RevId: 61ca8a1b0172920ad6ab1b604a9b9259cebddaad
* [document-updater] fix acceptance test for sending two updates
The Object.create() invocation yields an empty object. The following v
assignment works as expected. The effective update is { v: 43 }.
Processing that fails as no operations were included.
* [document-updater] add endpoint project wide last updated timestamp
* [document-updater] apply review feedback
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
---------
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 81397537bfd85c2077f19669860b1391c15b34a3
* [misc] silence logger when running tests
* [misc] re-enable logging in some tests for scripts
* [misc] make it easy to turn on verbose logging for tests
```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
* Add `unicorn/prefer-node-protocol`
* Fix `unicorn/prefer-node-protocol` ESLint errors
* Run `npm run format:fix`
* Add sandboxed-module sourceTransformers in mocha setups
Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`
* Remove `node:` in the SandboxedModule requires
* Fix new linting errors with `node:`
GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
* added getHistoryOpForAcceptedChange in RangesManager
* rename adjustHistoryUpdatesMetadata to be treated as public
* handle retain op in UpdateTranslator and updateCompressor
* send op to project-history in acceptChanges
* use promises.queueOps
* use ranges in getHistoryOpForAcceptedChange
* using rangesWithChangeRemoved
* acceptChanges acceptance test
* using change.op.hpos
* Revert "using change.op.hpos"
This reverts commit f53333b5099c840ab8fb8bb08df198ad6cfa2d84.
* use getHistoryOpForAcceptedChanges
* fix historyDocLength
* Revert "rename adjustHistoryUpdatesMetadata to be treated as public"
This reverts commit 2ba9443fd040a5c953828584285887c00dc40ea6.
* fix typescript issues
* sort changes before creating history updates
* fix tests
* sinon spy RangesManager.getHistoryUpdatesForAcceptedChanges
* added unit tests
* sort deletes before inserts
* use getDocLength function
* fix docLength calculation
* fix typo
* allow all retains
* fix lint error
* refactor RangesTests
* fix ts error
* fix history_doc_length calculation in RangesManager
* remove retain tracking check from UpdateCompressor
* use makeRanges() properly in tests
* refactor acceptance tests
GitOrigin-RevId: ab12ec53c5f52c20d44827c6037335e048f2edb0
Jakob Ackermann