Commit Graph
136 Commits
Author SHA1 Message Date
Anna Claire Fields a63fd554c3 Replace MOCHA_GREP with TEST_NAME_PATTERN (#34404)
GitOrigin-RevId: b1b2d1be3d0725f930d83ef035836482ffb72dcd
2026-09-17 11:45:29 +00:00
Jakob Ackermann 52e5fb3498 [project-history] adopt strict zod types and extend test coverage (#36087)
GitOrigin-RevId: ce1396e87926fc8816afa424c9db2a052c7da5e6
2026-09-17 11:45:20 +00:00
Jakob Ackermann 1b553c0455 [migrations] only apply non-blocking migrations in CI (#34665)
* [migrations] only apply non-blocking migrations in CI

* [migrations] use "yarn run migrations" instead of "npm run migrations"

GitOrigin-RevId: d9f96db398adf2febcd82333295fb8d5eab741bb
2026-06-26 08:08:05 +00:00
Mathias Jakobsen ef05288f81 Merge pull request #34714 from overleaf/mj-tracked-change-merges
[overleaf-editor-core+ranges-tracker] Pick lower timestamp when merging tracking info

GitOrigin-RevId: 5b1c8d7c719003df4506014c381f2a525d0a7ff4
2026-06-19 08:07:27 +00:00
Anna Claire Fields 7035a2e025 Revert "Acf pnp migration master" (#34709)
GitOrigin-RevId: ec7d63e1212e05dd6dfc7a0ddf785c11eca76ad6
2026-06-17 08:07:56 +00:00
Anna Claire Fields 8f07308342 Acf pnp migration master (#34040)
GitOrigin-RevId: 418eab9bc733f27a6e0f8f04a2a8e21c2f5a491f
2026-06-17 08:07:16 +00:00
Jakob Ackermann cb9e834a1f [project-history] return status "423 Locked" when redis lock is taken (#34429)
GitOrigin-RevId: f40230bc98e8ed8a5358e6db5c515cb2a58f9b85
2026-06-16 08:06:38 +00:00
Brian Gough 57d76da263 drop zero-length retain operations in project-history (#34427)
GitOrigin-RevId: ec3069a3ea3b7d487548ba181a2dde9772e308e5
2026-06-11 08:05:33 +00:00
Miguel Serrano 14b04ad4b8 [project-history] Removed request dependency (#32686)
* [project-history] Removed `request` dependency

GitOrigin-RevId: 086bbbf2efeea6026127653a1f68ca6bf0476de6
2026-05-28 08:06:04 +00:00
Malik Glossop 7bf7102b98 Merge pull request #32712 from overleaf/mg-recreate-corrupted-blob
Recover from corrupted blobs during hard resync

GitOrigin-RevId: 7cc764e3bcc8557689c040c8f042991d97f897bc
2026-05-11 08:06:15 +00:00
Mathias JakobsenandJakob Ackermann a478c1a829 [web+project-history] Add lastResyncedAt field to projects and set after resync (#33240)
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: f679a190b88694a46f7816d51eff96446f338dec
2026-04-30 08:06:43 +00:00
Malik Glossop da027f46cf Merge pull request #32530 from overleaf/mg-stuck-sync-repro
Detect and auto-clear stuck sync states

GitOrigin-RevId: 680861a33e42432dab7d40ad421980b707eb6089
2026-04-29 08:05:33 +00:00
Mathias Jakobsen cae558af69 Merge pull request #32779 from overleaf/mj-project-history-bug
[project-history] Fix broken ranges from history resync

GitOrigin-RevId: c839e1d187768b7ce3411d06f78f017ebf7fec6e
2026-04-17 08:05:42 +00:00
Jakob Ackermann 3dc8899334 [web] add admin page for project history failures (#32751)
GitOrigin-RevId: c9564e07239b761851c9a86894f9ef3023dfcf2b
2026-04-15 08:05:57 +00:00
Jakob Ackermann 7faa4c5db4 [project-history] normalize failure messages (#32748)
* [project-history] normalize failure messages

* [project-history] add tests for normalizing failure messages

GitOrigin-RevId: ec15fdba97215047067c10c3fd16ad64696af20b
2026-04-15 08:05:53 +00:00
Eric Mc Sween 77952e5d21 Merge pull request #32628 from overleaf/em-bg-history-bug
Prevent incorrect merging of ops with different hpos offsets

GitOrigin-RevId: d171a93dcb29e952c7105afd81c3b29ca7e9788e
2026-04-08 08:05:03 +00:00
Domagoj KriskovicandCopilot 332588826d [project-history] Fix for invalid range Error (#32198)
* [project-history] Invalid range Error WIP

* Clamp tracked deletes to retention range to prevent negative positions

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
GitOrigin-RevId: 408da6fff99b6c413af3d18fa4c399d88ca5f2a2
2026-04-01 08:06:15 +00:00
Jakob Ackermann ad508b1882 [project-history] migrate simple request usages to fetch-utils (#32539)
* [project-history] remove unused getQueueCounts in tests

* [project-history] migrate simple request usages to fetch-utils

GitOrigin-RevId: 0e299a9d2ea968b87d7f0f2fc1626393ca4e4fdc
2026-04-01 08:05:41 +00:00
Jakob Ackermann 8a67258461 [project-history] remove unused DocumentUpdaterManager (#32537)
GitOrigin-RevId: f2806187e2ae4e768f48a724712daffd4168e4fb
2026-03-31 08:07:43 +00:00
Jakob AckermannandJessica Lawshe 7c70b749d4 [monorepo] remove PII and variables from error messages (#31508)
* [monorepo] remove PII and variables from error messages

Exclusions:
- scripts
- tests
- fuzzing
- SplitTestManager (messages are sent to admin frontend)
- Group setup (we may want an error per unique tuple)
- sharejs (unused types; text type errors are shadowed already)
- history-v1 error messages that are used by the ErrorRecorder
- errors that flag issues with configuration/call signatures

I've used these search terms for finding unwanted error messages:
- new Error(`
- new Error\(\n\s+` (regex search)
- new OError(`
- new OError\(\n\s+` (regex search)

* [web] throw NotFoundError from ProjectLocator

* [github-sync] fix OError.tag call in script

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>

* [templates] revert changes to test client

---------

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>
GitOrigin-RevId: 736857a4fc5d9bfb0f8cb03e0f004eda87e5a220
2026-02-17 09:05:04 +00:00
Rebeka Dekany a648015db8 Centralize prettier configuration to root level (#30501)
* Merge all .prettierignore files into top-level config

* Merge all .prettierrc files into top-level config

* Replace service-specific glob patterns in package.json format scripts with `prettier .`

* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore

* Ignore GitHub templates

* Ignore PUG templates to format them separately with `format:pug`

* Encourage double quotes for YAML, YML files

* Move prettier for PUG source format script to the root

* Move prettier for styles source format script to the root

* Remove prettier for jenkins files from web

* Remove prettier source format script from all services

* Make .prettierrc more readable

* Update format scripts by file type

* Organise `.prettierignore`

* Add `--cache` flag to prettier scripts for faster runs

* Format all files with prettier

* Format all or format services

* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`

* Avoid conlficts with yamllint configuration

* Remove `--cache` flag from prettier scripts

* Update all service Makefiles to use root-level prettier configuration

* Update all Jenkinsfile to use root-level prettier configuration

* Ignore auto-generated files by build_scripts

* Update package-lock.json

* Update root Makefile format targets

* Update SP Jenkinsfile format target

* Update E2E Makefile format script

* Udpate `format_js` to work in both local and CI env

* Add docker-mailtrap to .prettierignore

docker-mailtrap is a third-party git-ignored directory used for testing

* Added Docker env detection to prevent nested Docker spawning

* Ignore handlebars templates

* Add cryptographic files and test output to `.prettierignore`

* Add terraform modules to `.gitignore`

* Remove prettier-plugin-groovy

* Use npx directly instead of Docker for local formatting for faster formatting

* Auto-generate Makefiles

* Revert "Remove prettier-plugin-groovy"

This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.

* Mount monorepo root in RUN_LINT_FORMAT for prettier config access

* Prettier ignores all `node_modules` by default regardless of location

* Show only changed files in format output

* Ignore LICENSE files

* Enable prettier on rendered build_scripts outputs

* Ignoring all the template folders by prettier

* Remove the public/minjs entry since it does not exist

* Remove all non-existent paths

* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files

* Revert "Auto-generate Makefiles"

This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.

* Revert "Use npx directly instead of Docker for local formatting for faster formatting"

This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.

* Ignore dashboard JSON files

* Ignore files generated by bin/update_build_scripts

* Remove unsupported file types from `.prettierignore`

* Ignore test fixture generated files

* Ignore README file types by prettier

* Ignore generate snapshots by prettier

* Allow to format generated bin/update_build_scripts by prettier

* Ensure build script outputs prettier-compatible tsconfig.json

* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double

* Don't read cryptographic files by prettier

* Ignore google verification files by prettier

* Revert npx prettier formatting

* Ignore domain verification files

* Show only changed files in format output

* Make `.github` prettier

* Allow all files to be formatted in jobs by prettier

* Allow server-ce/server-pro files to be formatted by prettier

* Ignore more folders in clsi, filestory, git-bridge by prettier

* Update build script with `RUN_LINTING_CI_MONOREPO`

* Ignore docker-mailtrap and downloads in server-ce by prettier

* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)

* Source format

GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
2026-02-04 09:08:22 +00:00
Domagoj Kriskovic 9bd7171413 [project-history] update getProjectBlobSchema to accept numeric history_id
GitOrigin-RevId: b38d8c9e279e227975a12662d3f0c9e0d21cc1bd
2026-02-02 09:05:33 +00:00
Domagoj Kriskovic 859c21b4dd [project-history] update schemas to accept numeric project IDs
GitOrigin-RevId: 4dac763223f42eb772c30f34e0da0d22464055dd
2026-02-02 09:05:20 +00:00
Domagoj Kriskovic da9da896c2 Revert "Revert "[project-history] migrate from joi to zod (#30187)" (#30681)"
This reverts commit 8d66aa15a29216736755c3f86954a4d40aa20a6c.

GitOrigin-RevId: 9d56dafff3da5202204ead896bed30182cc2aceb
2026-02-02 09:05:15 +00:00
Domagoj Kriskovic da7569e892 Revert "[project-history] migrate from joi to zod (#30187)" (#30681)
This reverts commit f8396a3caadd536edf64c6f971c77f4c941f000a.

GitOrigin-RevId: 8d66aa15a29216736755c3f86954a4d40aa20a6c
2026-01-13 09:07:22 +00:00
Domagoj Kriskovic 6d3dd3601a [project-history] migrate from joi to zod (#30187)
* Add @overleaf/validation-tools in project-history

* use error handler for zod

* Replace joi validations with zod schema

* fix unit tests

* remove Joi validation middleware

* remove optional when there is a default

* fix tests after checking for objectid

* ran build_scripts

* monorepo npm install

GitOrigin-RevId: f8396a3caadd536edf64c6f971c77f4c941f000a
2026-01-13 09:07:00 +00:00
Jakob Ackermann 425e7b1e5b [web] enable mongo notablescan in CI (#29501)
* [monorepo] record ERROR/FATAL log messages in junit report

* [web] put SaaS specific code behind feature flag

* [web] use split test cache for getting user assignments

The unit tests needed updating as they did not replicate any of the
 mongo filtering. The acceptance tests cover this logic.

* [web] make better use of existing indexes

* [web] avoid col-scan in tests of notifications module

* [web] remove cleanup of empty feedbacks collection

* [web] add assertion for reason of rejected request in launchpad test

* [web] add missing indexes

* [web] enable mongo notablescan

* [web] make emailNotifications tests compatible with notablescan

GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
2026-01-13 09:06:38 +00:00
729e0f5ac9 move migrations to shared location (#28306)
* fix: correct typedef for Document in helpers.mjs

* add move-migrations codemod

* update migration paths to use shared migrations directory

* move migrations to shared location

* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory

* feat: add migrations tool to workspaces in package.json

* [monorepo] Fix order of docker ignore rules

* [web] remove unused docker ignore file

* [monorepo] replace old references to migrations folder

* [server-ce] copy migrations from new place

* [migrations] Inline web scripts

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] move three web scripts over

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] add missing collection

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] remove lodash dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] avoid mongodb-legacy dependency

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [monorepo] run migrations from tools/migrations

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [migrations] simplify migration for adding gitBridge feature to users

* [monorepo] run migrations from tests in all the services

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] fixup running web migrations everywhere

* [monorepo] trigger Jenkins builds on changes to mongo migrations

* [migrations] add Jenkins pipeline for linting/formatting

* [monorepo] build scripts: update devDependencies before deps scanning

* [monorepo] build scripts: formerly depend on tools/migrations

* [monorepo] run eslint on .mjs files

* [migrations] enable more eslint rules and fix all the errors

* [rake] fix migrations:list task

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
2025-10-16 08:07:37 +00:00
Jakob Ackermann fd0d21fbbd [monorepo] migrate build scripts to Jenkins (#29019)
* [monorepo] migrate build scripts to Jenkins

* [fraud-dash] fix integration with monorepo

GitOrigin-RevId: 6a30ec5ea68a1eae9b663faf9ccdc741ec7f9cf8
2025-10-13 08:07:11 +00:00
Domagoj Kriskovic 267fc5393a Promisify ProjectHistoryClient, ProjectHistoryApp, SyncTests and SendingUpdatesTests (#28890)
GitOrigin-RevId: 7bf26c6ed1a172c6506449a821d4e43f424a72bd
2025-10-08 08:06:00 +00:00
Domagoj Kriskovic 6d84dfddcc Skip checking status code for fetchJson
GitOrigin-RevId: 1470fdd5aabde3448c5ac5ed8e9203fc509b13e8
2025-10-07 08:05:43 +00:00
Domagoj Kriskovic 62ec3f9644 Promisify ReadingASnapshotTests
GitOrigin-RevId: 281b3a84466175a38b5ebbf9282b4c996e04d221
2025-10-07 08:05:39 +00:00
Domagoj Kriskovic aed47ef01b Promisify LatestSnapshotTests
GitOrigin-RevId: 40b0811e25ac9ee75c295c89cec7ad07570007c4
2025-10-07 08:05:34 +00:00
Domagoj Kriskovic 64d1f160a1 promisify LabelsTests and SummarisedUpdatesTests
GitOrigin-RevId: 32552c8c63ebbd8e73e8c179fcce67b3b955251d
2025-10-07 08:05:30 +00:00
Domagoj Kriskovic 9150fc332c Promisify SummarisedUpdatesTests
GitOrigin-RevId: 65782b0cf6e552425d6fad716f74f8f3f22cf7e9
2025-10-07 08:05:26 +00:00
Domagoj Kriskovic c1bfd2192d promisify GetChangesInChunkSince
GitOrigin-RevId: 82af36d7cb0301e3bf9527b219c67ceb6fc9b34f
2025-10-07 08:05:21 +00:00
Domagoj Kriskovic 54190e1040 promisidy FlushManagerTests
GitOrigin-RevId: fa26499be5e40d6741ea1d012188a0a70f519b51
2025-10-07 08:05:17 +00:00
Domagoj Kriskovic 1c2aa6251e promisify FileTreeDiffTests
GitOrigin-RevId: f7c9c119f28bdbf98c526d767ee06286c33ecfa2
2025-10-07 08:05:12 +00:00
Domagoj Kriskovic 632a563ed2 convert some helper functions in ProjectHistoryClient
GitOrigin-RevId: a73207c4dfbf13cd96456e024adfa6a046002e00
2025-10-03 08:06:31 +00:00
Domagoj Kriskovic 5944d20340 promisify DiscardingUpdatesTests
GitOrigin-RevId: ea64c162bc05d7f30cf95cc397991d7f448c26e2
2025-10-03 08:06:26 +00:00
Domagoj Kriskovic 5e03da3b58 promisify DiffTests
GitOrigin-RevId: 688f0f5f52917483912ee69779381d1ea2469ad8
2025-10-03 08:06:21 +00:00
Domagoj Kriskovic c1645ebe2c promisify RetryTests
GitOrigin-RevId: 8e748a63bd523087d023be5573e8e23ef3bda3f2
2025-10-03 08:06:16 +00:00
Domagoj Kriskovic 3840eaa0d8 promisify DeleteProjectTests
GitOrigin-RevId: a86e6962253bbf4adf5fc9748e7accb9084b31f9
2025-10-03 08:06:11 +00:00
Domagoj Kriskovic a9d805ef9b delete unused helper
GitOrigin-RevId: ff7d96159e8c71b2a5ebed4041ee44f5cf45ceb1
2025-10-03 08:06:06 +00:00
Domagoj Kriskovic 854536fcc9 use async/await HealthCheckTests, promisify helpers
GitOrigin-RevId: 1016296695eb1c09d74d6f50c765438b83df1024
2025-10-03 08:06:01 +00:00
Eric Mc Sween 98069966ba Merge pull request #28672 from overleaf/em-reapply-restore-optimization
Reapply file restore optimization

GitOrigin-RevId: c4479141d890e3e0af746f406a0186bf21f3d918
2025-09-25 08:05:25 +00:00
Domagoj Kriskovic e9b1c63ed7 Revert "File restore optimization - using snapshot timestamp and file paths (#28546)"
This reverts commit 376a53e1f927cb56544e6782b47d80345655038c.

GitOrigin-RevId: 8e4ab1d4042fec8df1b62ab1c5418d873dc6f5eb
2025-09-24 08:06:06 +00:00
Domagoj KriskovicandEric Mc Sween 005eba7502 File restore optimization - using snapshot timestamp and file paths (#28546)
* Use snapshot timestamp, remove request for paths at version

* Add timestamp to core Snapshot object

* Avoid mutating function argument

* Explain assumption about editable files

* snapshot.toRaw() in getContentAtVersion

* fix project-history acceptance test

* fix history_v1 test

* fix web tests

* Include the snapshot timestamp in stored chunks

---------

Co-authored-by: Eric Mc Sween <5454374+emcsween@users.noreply.github.com>
GitOrigin-RevId: 376a53e1f927cb56544e6782b47d80345655038c
2025-09-24 08:06:02 +00:00
Jakob Ackermann adf399fb95 [project-history] add support for resync of history-ot ranges (#26475)
* [project-history] add support for resync of history-ot ranges

* [project-history] avoid compressing sharejs and history-ot upgrades

* [document-updater] improve error message of some assertions

... by migrating the assertions like this:
```diff
-stub.calledWith().should.equal(true)
+stub.should.have.been.calledWith()
```
```diff
-stub.called.should.equal(false)
+stub.should.not.have.been.called
```

* [document-updater] move content field in resyncDocContent

* [document-updater] add support for resync of history-ot ranges

GitOrigin-RevId: e6104686a26934a5f25a8f095cbe00c163fbbaa7
2025-06-19 08:05:18 +00:00
Jakob Ackermann 8b937c91f4 [project-history] fix unit tests after merging multiple PRs (#26442)
GitOrigin-RevId: 9ccbadaa807da2f20010930477724e9370bf103e
2025-06-16 08:06:54 +00:00