Commit Graph
613 Commits
Author SHA1 Message Date
Malik Glossop 863e33af78 Merge pull request #35982 from overleaf/mg-history-ot-comments
HistoryOT - Fix comments being dropped and crashing on detached ranges

GitOrigin-RevId: 0906a55a51a6fda9ac828efcbe72989573161261
2026-09-17 12:03:46 +00:00
Jakob Ackermann 9b44fd075b [monorepo] downgrade ranges ids back to strings for now (#36857)
GitOrigin-RevId: 6c85715bc85adc2f641bfd0e9f3bb6a8f2c839c2
2026-09-17 12:03:33 +00:00
Jakob Ackermann 4e80d65efc [ranges-tracker] add legacy orderedRejections field in tracked-changes (#36565)
* [docstore] use more realistic test stubs for ranges

* [monorepo] define ranges schemas in @overleaf/ranges-tracker

* [ranges-tracker] add legacy orderedRejections field in tracked-changes

* [monorepo] resolve @overleaf/ranges-tracker subpath imports in typescript

* [real-time] validate comment thread ids and tracked change id seeds

* [ranges-tracker] require ObjectId ids and tracked change metadata in ranges

* [monorepo] require ObjectId tracked change ids on the accept routes

* [web] use ObjectId comment thread ids in the DocumentController tests

GitOrigin-RevId: 58b20dec311d85b917ef63f9efbe745a265a2de6
2026-09-17 12:01:19 +00:00
KristinaandJimmy Domagala-Tang 9c005e4e54 [web/document-updater] display preview for accepted/rejected tracked changes (#36240)
* [web] add preview payload to track-change notification types

* [doc-updater] pass accepted/rejected change data through to web for previews

* [web] snapshot change preview onto accepted/rejected notifications

* [web] render resolved change previews instead of accepted/rejected count lines

* [web] add unit coverage for accepted/rejected change preview flow

* fix: notification carries multiple previews per accept/reject batch; extract shared group helper

* fix: one preview per accept/reject action, drop adjacency grouping

* fix: move buildResolvedChangePreviews out of the email builder class

* fix: type error, make tests more explicit

* fix: render resolver below preview

* fix: overflow text, alignment

* fix: test error

* fix: resolved count

* [document-updater/web] send sparse change preview instead of full doc lines

* fix: store sparse preview, hydrate at render, drop CE gate

* fix: advance slice cursor past tracked-delete text in sparse preview

* fix: inline buildChangeGroupPreview

* fix: changePreviewBuilder -> sparseChangePreview

* fix: rm change_contributors

* fix: copilot flagged testing issues

* fix: hydrate and truncate change preview at write time

* fix: avoid materializing full doc text when slicing preview window

* fix: update reject preview test to hydrated shape

* fix: accepted delete removing context

* [document-updater/web] split resolved change previews into one per change cluster

* fix: narrow cluster userIds to string[] for typecheck

* feat: updating to use zod schema for previews

---------

Co-authored-by: Jimmy Domagala-Tang <jimmy.domagala-tang@overleaf.com>
GitOrigin-RevId: 0740713c715a61a220180f353e497b27da6a5b0f
2026-09-17 11:58:14 +00:00
Andrew Rumble 66355e4372 Merge pull request #36361 from overleaf/ar-fix-postdoc-validation-errors
[docstore][document-updater] fix POST doc validation errors

GitOrigin-RevId: 9b5fbc667a3b56b30077ea8de9d81c17253db273
2026-09-17 11:51:12 +00:00
Jakob Ackermann 49969f6e0d [history-v1] adopt strict zod types and extend test coverage (#36089)
GitOrigin-RevId: 7bdffe2c05c7c17be6eb3e4296d8df05d650c53e
2026-09-17 11:49:08 +00:00
Jakob Ackermann 0776357d1a [document-updater] address zod violations (#36147)
* [editor-core] address zod violation for group_id

* [monorepo] use discriminated union in zod schemas where possible

* [editor-core] v1_source_doc_id is a number

GitOrigin-RevId: 38f591b07cca4e22954bfe30bf855545a6af675d
2026-09-17 11:46:42 +00:00
Jakob Ackermann 4e80098424 [document-updater] adopt strict zod types and extend test coverage (#36077)
* [document-updater] adopt strict zod types and extend test coverage

* [saas-e2e] add test for project restore with binary and linked file

GitOrigin-RevId: a8e45bd1e46b6ac8a293da24c8b8ad7b236fd9bd
2026-09-17 11:46:05 +00:00
Jakob Ackermann 81df324bcc [monorepo] prepare for zod everywhere epic (#36027)
GitOrigin-RevId: f082ae7fc7857fa72b2c1823645050bd4f6df305
2026-09-17 11:42:46 +00:00
Jakob Ackermann 8fa62d2864 [document-updater] drop support for tc in briefly (#35864)
GitOrigin-RevId: 00d04d911249b81abc5eb4f9d9dfde4c7f2316b3
2026-09-17 11:39:22 +00:00
Jakob Ackermann 7a0a9c3ce5 [document-updater] implement HTTP endpoints for comments (#35833)
GitOrigin-RevId: 06b4f222a55e21b3def432d6e84d9ab4bf65b64d
2026-09-17 11:39:19 +00:00
Jakob Ackermann 8be65f1f93 [document-updater] add support for flush/restore of tc with history-ot (#35823)
GitOrigin-RevId: a79b0ca97c930ca2e2f3ffeea20defbc6973468c
2026-09-17 11:39:15 +00:00
Jakob Ackermann 00a2765d01 [document-updater] add track-changes support to setDoc (#35820)
GitOrigin-RevId: 5d39dd1945e710ec3f7068830fb2476e59f0f2a5
2026-09-17 11:39:12 +00:00
Jakob Ackermann a77a4782cc [history-v1] add setContent endpoint (#35799)
* [history-v1] add setContent endpoint

* [history-v1] refactor setContent to return a change+v

* [history-v1] make typescript happy

GitOrigin-RevId: e2cb0281367c01aa9cd842f250f2016d591ec0fc
2026-09-17 11:39:09 +00:00
Jakob Ackermann aab08b4c28 [document-updater] remove doc lock from per-doc operations (#35696)
GitOrigin-RevId: c45c1dbf11049c0fab8a7d788006a698ede0d510
2026-09-17 11:34:14 +00:00
Jakob Ackermann bc9d25d385 [monorepo] clean up the per-project pub/sub migration (#35686)
GitOrigin-RevId: 4bbb97d1d2750f21751ef54e206e3e720b88615a
2026-09-17 11:33:51 +00:00
Jakob Ackermann 3565f51a84 [monorepo] prepare for per-project pub/sub for applied ops (#35663)
GitOrigin-RevId: 11d45addf366c3449cad432272ed03d4c8dae49d
2026-09-17 11:33:01 +00:00
Jakob Ackermann 324f011d80 [document-updater] add script for flushing/discarding stuck docs (#35603)
GitOrigin-RevId: 099e1e0a38515b35decbdc608acd75f59de0ec6f
2026-09-17 11:31:56 +00:00
Jakob Ackermann 68352783b0 [monorepo] clean up per-project queue migration (#35573)
GitOrigin-RevId: 988df4cdc7d929efc921e7e99d803c6ce6e52c57
2026-09-17 11:31:52 +00:00
Jakob Ackermann cdba372dfc [monorepo] prepare per-project queues (#35386)
GitOrigin-RevId: ab020603ec04c705d08be466563adac0ce30bc04
2026-09-17 11:26:50 +00:00
Kristina 8741846df8 [doc-updater] fix unbounded Redis key growth in project notifications cron (#35425)
* delete timestamp notification keys for solo projects
* add fallback 7d TTL for keys

GitOrigin-RevId: e0493e74b14b861863afc1d4ef5fa181432ad450
2026-09-17 11:25:41 +00:00
Jakob Ackermann 89f12e6b1f [document-updater] wrap doc locks in project lock (#35416)
* [document-updater] wrap doc locks in project lock

* [document-updater] use BlockingProject as redis key prefix

* [redis-wrapper] update comment

GitOrigin-RevId: 0ce50e76386871e6c29481494febc8d6b37e7df5
2026-09-17 11:24:34 +00:00
Domagoj Kriskovic bf1dd6986f Record project notification timestamp in Redis on applyUpdate (reverted) (#30814)
* Revert "Revert "Record project notification timestamp in Redis on applyUpdate (#29509)""

This reverts commit 31c88ee836fb5e1ab3950da590c28e24b1397edb.

GitOrigin-RevId: fc012324f6035156585fab468aca72b900a9710b
2026-05-22 08:06:26 +00:00
Jakob Ackermann 1f8371e0a3 [document-updater] flush_all: log progress after every 1k projects (#33757)
GitOrigin-RevId: b5b68f6f53bece51234799fb626d0d6a2a5b590c
2026-05-19 08:04:41 +00:00
Jakob Ackermann b62d4814c3 [monorepo] turn throw statements in callback code into callback calls (#33524)
* [eslint-plugin] add rule for throw inside callback code

* [monorepo] enable our custom eslint plugins globally

* [monorepo] fix running make lint from root

* [monorepo] turn throw statements in callback code into callback calls

* [monorepo] add eslint-plugin libraries to all the Dockerfiles

* [monorepo] install eslint-plugin library at the root level

* [linked-url-proxy] add eslint-plugin library into Dockerfile

* [latexqc] add our eslint-plugin to eslint config

GitOrigin-RevId: b05e3ebbefb62370f2422e83880dd3913815270d
2026-05-14 08:05:47 +00:00
Jimmy Domagala-TangandKristina Hjertberg bc2f5ae746 Reject tracked changes notifications (#32917)
* [web] Reject tracked changes notifications

feat: adding new tests

feat: adding rejected changes notifications

feat: adding tests for rejectchanges

feat: updating tests for rejecting notifications;

feat: adding in rejecting user, and improving subject and activity line

fix: moving to a params object instead of positionals for email building

feat: updating to use events triggered from applyUpdate in document-updater

feat: updating to send rejected author ids with rejected change notification instead of change ids

feat: moving rejected author notification determination to updateManager instead of RangesManager, which is used by other paths

feat: only map to author if changes were made

* fix: gate by user status not project status

* fix: unit tests post-rebase

---------

Co-authored-by: Kristina Hjertberg <kristina.hjertberg@overleaf.com>
GitOrigin-RevId: f992e1885c47d1a6cf776740769d6d4763f3cb7c
2026-05-14 08:05:35 +00:00
Domagoj Kriskovic 8f0979d6a2 Trigger reject tracked changes notification from document-updater
GitOrigin-RevId: 9ac47490d052b3058931ca250f4090e6576f56b2
2026-05-07 08:06:57 +00:00
Jimmy Domagala-Tang 994932b8e3 [Web + Doc-Updater] Add track changes accepted notification (#32752)
* feat: update doc manager to return a list of contributors to the accepted change

* feat: add new notification type for accepting a tracked change

* update email with tracked changes accepted

* feat: update tests

* fix: feedback on consistent api and returns

* feat: adding new tests

* feat: self accepted changes shouldnt trigger notification, and using existing changesAccepted hook

* Add better subject and activity list for track change accepted (#33094)

* feat: add better activity list entry and subject header for accepted changes, to match other notifications

* feat: updating tests

* feat: updating accepting_user_id to just user_id

* fix: adding users in emailBuilder test to userCache

GitOrigin-RevId: 6114f77916b5f503b7bbbb5ca8fed99e58edc31b
2026-04-30 08:06:19 +00:00
Brian Gough 15a24db3c9 Merge pull request #31650 from overleaf/bg-handle-timeouts-in-persistence-manager
add missing handling of timeouts in PersistenceManager

GitOrigin-RevId: 7fe74068f3ea647b27103393c3f0b243b4b25fe3
2026-02-20 09:05:50 +00:00
Brian Gough a9c94c4184 Merge pull request #31444 from overleaf/bg-jpa-use-fetch-in-persistence-manager
remove requestretry from PersistenceManager

GitOrigin-RevId: 1fc9ffdfa7879d7ab4f0f4683544d09fe8526f3d
2026-02-19 09:05:45 +00:00
Jakob AckermannandJessica Lawshe 7c70b749d4 [monorepo] remove PII and variables from error messages (#31508)
* [monorepo] remove PII and variables from error messages

Exclusions:
- scripts
- tests
- fuzzing
- SplitTestManager (messages are sent to admin frontend)
- Group setup (we may want an error per unique tuple)
- sharejs (unused types; text type errors are shadowed already)
- history-v1 error messages that are used by the ErrorRecorder
- errors that flag issues with configuration/call signatures

I've used these search terms for finding unwanted error messages:
- new Error(`
- new Error\(\n\s+` (regex search)
- new OError(`
- new OError\(\n\s+` (regex search)

* [web] throw NotFoundError from ProjectLocator

* [github-sync] fix OError.tag call in script

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>

* [templates] revert changes to test client

---------

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>
GitOrigin-RevId: 736857a4fc5d9bfb0f8cb03e0f004eda87e5a220
2026-02-17 09:05:04 +00:00
Rebeka Dekany a648015db8 Centralize prettier configuration to root level (#30501)
* Merge all .prettierignore files into top-level config

* Merge all .prettierrc files into top-level config

* Replace service-specific glob patterns in package.json format scripts with `prettier .`

* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore

* Ignore GitHub templates

* Ignore PUG templates to format them separately with `format:pug`

* Encourage double quotes for YAML, YML files

* Move prettier for PUG source format script to the root

* Move prettier for styles source format script to the root

* Remove prettier for jenkins files from web

* Remove prettier source format script from all services

* Make .prettierrc more readable

* Update format scripts by file type

* Organise `.prettierignore`

* Add `--cache` flag to prettier scripts for faster runs

* Format all files with prettier

* Format all or format services

* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`

* Avoid conlficts with yamllint configuration

* Remove `--cache` flag from prettier scripts

* Update all service Makefiles to use root-level prettier configuration

* Update all Jenkinsfile to use root-level prettier configuration

* Ignore auto-generated files by build_scripts

* Update package-lock.json

* Update root Makefile format targets

* Update SP Jenkinsfile format target

* Update E2E Makefile format script

* Udpate `format_js` to work in both local and CI env

* Add docker-mailtrap to .prettierignore

docker-mailtrap is a third-party git-ignored directory used for testing

* Added Docker env detection to prevent nested Docker spawning

* Ignore handlebars templates

* Add cryptographic files and test output to `.prettierignore`

* Add terraform modules to `.gitignore`

* Remove prettier-plugin-groovy

* Use npx directly instead of Docker for local formatting for faster formatting

* Auto-generate Makefiles

* Revert "Remove prettier-plugin-groovy"

This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.

* Mount monorepo root in RUN_LINT_FORMAT for prettier config access

* Prettier ignores all `node_modules` by default regardless of location

* Show only changed files in format output

* Ignore LICENSE files

* Enable prettier on rendered build_scripts outputs

* Ignoring all the template folders by prettier

* Remove the public/minjs entry since it does not exist

* Remove all non-existent paths

* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files

* Revert "Auto-generate Makefiles"

This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.

* Revert "Use npx directly instead of Docker for local formatting for faster formatting"

This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.

* Ignore dashboard JSON files

* Ignore files generated by bin/update_build_scripts

* Remove unsupported file types from `.prettierignore`

* Ignore test fixture generated files

* Ignore README file types by prettier

* Ignore generate snapshots by prettier

* Allow to format generated bin/update_build_scripts by prettier

* Ensure build script outputs prettier-compatible tsconfig.json

* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double

* Don't read cryptographic files by prettier

* Ignore google verification files by prettier

* Revert npx prettier formatting

* Ignore domain verification files

* Show only changed files in format output

* Make `.github` prettier

* Allow all files to be formatted in jobs by prettier

* Allow server-ce/server-pro files to be formatted by prettier

* Ignore more folders in clsi, filestory, git-bridge by prettier

* Update build script with `RUN_LINTING_CI_MONOREPO`

* Ignore docker-mailtrap and downloads in server-ce by prettier

* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)

* Source format

GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
2026-02-04 09:08:22 +00:00
Jakob AckermannandBrian Gough 86b29819c2 [document-updater] add missing await (#31034)
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: f0602801ce80ca617027004ad36f71c1e6cdcdd2
2026-01-27 09:06:59 +00:00
Andrew Rumble 2cdbdc1291 Merge pull request #30785 from overleaf/ar-add-logs-for-no-projectHistoryId-from-web
[document-updater] add log for missing projectHistoryId from web

GitOrigin-RevId: 750150312e341ef09b6c82b9df467900beb4b784
2026-01-15 14:19:37 +00:00
Andrew Rumble a071cd74bc Merge pull request #30784 from overleaf/revert-30675-ar-jpa-log-missing-history-id-docupdater
Revert "[document-updater] add additional logging when doc is missing projectHistoryId"

GitOrigin-RevId: 725508a8f88ad3354b8b4644f4ee1e91efbecde2
2026-01-15 14:19:24 +00:00
Andrew Rumble 258c6224f4 Merge pull request #30675 from overleaf/ar-jpa-log-missing-history-id-docupdater
[document-updater] add additional logging when doc is missing projectHistoryId

GitOrigin-RevId: fba311cb10e0eeeb0a8165ce4e519c7704a7458f
2026-01-15 14:19:11 +00:00
Domagoj Kriskovic 0a84cefc96 Revert "Record project notification timestamp in Redis on applyUpdate (#29509)"
This reverts commit faa7a97fa929941cade5d62d2d680c6c3f34cdc8.

GitOrigin-RevId: 31c88ee836fb5e1ab3950da590c28e24b1397edb
2026-01-12 09:06:50 +00:00
Domagoj Kriskovic 8bca2545ee Record project notification timestamp in Redis on applyUpdate (#29509)
* Record project notification timestamp in Redis on applyUpdate

* Refactor project notification timestamp recording in Redis

* Remove tests which are catching errors that are no longer used

GitOrigin-RevId: faa7a97fa929941cade5d62d2d680c6c3f34cdc8
2026-01-12 09:06:07 +00:00
Brian Gough a45c39389c Merge pull request #29891 from overleaf/bg-docupdater-make-max-unflushed-age-configurable
Make document-updater max unflushed age configurable via settings

GitOrigin-RevId: 3421f2467b7d8459ba8cdb1859e1b0695b0a01f3
2025-12-08 09:05:08 +00:00
Domagoj Kriskovic c84cfc815a Document updater script for notifying web on project updates (#29490)
* Document updater script for notifying web on project updates

* use lua script for deleting keys

* define jobId when adding to queue

* removeOnFail age

* mongo check if collaborator exists

GitOrigin-RevId: f67a79c890a03ccf44fd84243c9a3f380e7afb43
2025-11-11 09:06:04 +00:00
Jakob Ackermann d648c96603 [document-updater] migrate HistoryManager to async/await (#28789)
shouldFlushHistoryOps has a default value for 'threshold', which keeps
the exports simpler and still lets the unit tests override it.

GitOrigin-RevId: 1c6d4a2778052b5af40e2e338589a230ac2f4646
2025-10-09 08:07:36 +00:00
Jakob Ackermann b0b9733a42 [document-updater] migrate ProjectFlusher to async/await (#28796)
GitOrigin-RevId: 24f61d6c0fab5d65b962cc7031ce0b8c84d5a915
2025-10-09 08:07:31 +00:00
Tim Down c104aa454e Merge pull request #28845 from overleaf/td-async-await-doc-updater-client
Convert DocUpdateClient in document-updater acceptance tests to async/await

GitOrigin-RevId: 8f2352119f8f1175c2703ed90dbbc483ed039e86
2025-10-08 08:05:26 +00:00
Eric Mc Sween 543f3fafcc Merge pull request #28431 from overleaf/em-docupdater-get-ranges
Look in docupdater when obtaining comments for DSMP

GitOrigin-RevId: acaead280f725da8e0c438d5b429fa983c3cad86
2025-09-15 08:05:29 +00:00
Eric Mc Sween 97b857ceb8 Merge pull request #28311 from overleaf/em-promisify-redis-manager
Promisify RedisManager in document-updater

GitOrigin-RevId: 87c86d3b8904e17e76756789d2ea3e09fab26d02
2025-09-10 08:05:57 +00:00
Eric Mc Sween 2995b29829 Merge pull request #28205 from overleaf/em-promisify-project-manager
Promisify ProjectManager

GitOrigin-RevId: d10807a9a7f11428dd6bfe41210bfd0ce32c37ba
2025-09-03 08:06:42 +00:00
Eric Mc Sween ca9455d7f2 Merge pull request #28182 from overleaf/em-promisify-http-controller
Promisify HttpController in document-updater

GitOrigin-RevId: fb87a38be856d64781401f7391b7f2a2c35c89fa
2025-09-01 08:05:12 +00:00
Eric Mc Sween 8f0913fafe Merge pull request #27936 from overleaf/em-unit-tests-mongo
Make Mongo available to unit tests in all services

GitOrigin-RevId: b65bbb69883d5bba31d09802b89f35bdc523fe4d
2025-08-25 08:05:19 +00:00
Domagoj Kriskovic 704e577c23 Add endpoint to retrieve document with history ranges and use it in dsmp API (#27564)
GitOrigin-RevId: 3d2ac33cdc903a07b8ec67f7fb6f723ae9c81a26
2025-08-15 08:05:17 +00:00
Domagoj Kriskovic 49bcfdd575 [ds-mobile-app] Implement reject changes in document updater (#27483)
* [ds-mobile-app] Implement reject changes in document updater

* Return rejected change ids

GitOrigin-RevId: 7915170af9164852ce39d783776106e722aaa925
2025-08-11 08:05:29 +00:00