* fix: correct typedef for Document in helpers.mjs
* add move-migrations codemod
* update migration paths to use shared migrations directory
* move migrations to shared location
* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory
* feat: add migrations tool to workspaces in package.json
* [monorepo] Fix order of docker ignore rules
* [web] remove unused docker ignore file
* [monorepo] replace old references to migrations folder
* [server-ce] copy migrations from new place
* [migrations] Inline web scripts
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] move three web scripts over
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] add missing collection
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] remove lodash dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] avoid mongodb-legacy dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [monorepo] run migrations from tools/migrations
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] simplify migration for adding gitBridge feature to users
* [monorepo] run migrations from tests in all the services
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] fixup running web migrations everywhere
* [monorepo] trigger Jenkins builds on changes to mongo migrations
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] build scripts: update devDependencies before deps scanning
* [monorepo] build scripts: formerly depend on tools/migrations
* [monorepo] run eslint on .mjs files
* [migrations] enable more eslint rules and fix all the errors
* [rake] fix migrations:list task
---------
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.
GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
* [history-ot] rename remaining history-v1-ot references to history-ot
* [web] rename History-v1 OT -> History OT in admin panel
* [web] rename OT Migration -> History OT Migration in admin panel
GitOrigin-RevId: 103ce816d5320d6379d51009cdc08b8a71aa48e6
* [history-v1-ot] initial implementation of using doc-level history-v1-ot
* [web] fix advancing of the otMigrationStage
Use 'nextStage' for the user provided, desired stage when advancing.
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [document-updater] document size check in editor-core
* [history-ot] rename history-v1-ot to history-ot and add types
* [history-ot] apply review feedback
- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
* Revert "[document-updater] add safe rollback point for history-ot (#25283)"
This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
---------
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
- use beforeEach to ensure tests do not interfere with each other
Notably, the 'when the ops come in a single linear order' test suite
had state-changing tests that were dependent on the correct order.
Also, the assigment of 'this.firstOpTimestamp' was in a test.
- consolidate populating project and doc ids
The doc reference in this.update was undefined.
- fix doc reference in updates
There were two misuses of 'doc_id' instead of 'doc'.
- Move mocking of MockWebApi.getDocument to the top and use
sinon.resetHistory() or sinon.restore() for controlling the stub.
- Add another test for simple transforming of updates
See 'when another client is sending a concurrent update'.
GitOrigin-RevId: 61ca8a1b0172920ad6ab1b604a9b9259cebddaad
* [document-updater] fix acceptance test for sending two updates
The Object.create() invocation yields an empty object. The following v
assignment works as expected. The effective update is { v: 43 }.
Processing that fails as no operations were included.
* [document-updater] add endpoint project wide last updated timestamp
* [document-updater] apply review feedback
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
---------
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 81397537bfd85c2077f19669860b1391c15b34a3
* [misc] silence logger when running tests
* [misc] re-enable logging in some tests for scripts
* [misc] make it easy to turn on verbose logging for tests
```
LOG_LEVEL=debug make test_unit
LOG_LEVEL=debug make test_acceptance
```
GitOrigin-RevId: 219bc6d1f9cbdb89ddd7d94742920913ddde4514
* Add `unicorn/prefer-node-protocol`
* Fix `unicorn/prefer-node-protocol` ESLint errors
* Run `npm run format:fix`
* Add sandboxed-module sourceTransformers in mocha setups
Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`
* Remove `node:` in the SandboxedModule requires
* Fix new linting errors with `node:`
GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
* added getHistoryOpForAcceptedChange in RangesManager
* rename adjustHistoryUpdatesMetadata to be treated as public
* handle retain op in UpdateTranslator and updateCompressor
* send op to project-history in acceptChanges
* use promises.queueOps
* use ranges in getHistoryOpForAcceptedChange
* using rangesWithChangeRemoved
* acceptChanges acceptance test
* using change.op.hpos
* Revert "using change.op.hpos"
This reverts commit f53333b5099c840ab8fb8bb08df198ad6cfa2d84.
* use getHistoryOpForAcceptedChanges
* fix historyDocLength
* Revert "rename adjustHistoryUpdatesMetadata to be treated as public"
This reverts commit 2ba9443fd040a5c953828584285887c00dc40ea6.
* fix typescript issues
* sort changes before creating history updates
* fix tests
* sinon spy RangesManager.getHistoryUpdatesForAcceptedChanges
* added unit tests
* sort deletes before inserts
* use getDocLength function
* fix docLength calculation
* fix typo
* allow all retains
* fix lint error
* refactor RangesTests
* fix ts error
* fix history_doc_length calculation in RangesManager
* remove retain tracking check from UpdateCompressor
* use makeRanges() properly in tests
* refactor acceptance tests
GitOrigin-RevId: ab12ec53c5f52c20d44827c6037335e048f2edb0
These changes were previously merged, not deployed, and reverted. This
reverts the revert.
This reverts commit a6b8c6c658b33b6eee78b8b99e43308f32211ae2, reversing
changes made to 93c98921372eed4244d22fce800716cb27eca299.
The project update endpoint accepted updates both in two array params:
docUpdates and fileUpdates, and in a single array: updates. This commit
removes the docUpdates/fileUpdates params now that web uses the updates
param.
Brian Gough