Commit Graph
94 Commits
Author SHA1 Message Date
Jakob AckermannandClaude Opus 5 7ae8db6b12 [editor-core] extract shared pieces for history-ot in editor (#36828)
* [editor-core] share the pieces the editor, real-time and history-v1 all need

Project-level history-OT has the browser, real-time and history-v1 all reason
about the same changes, so the rules they reason with move into the shared
library rather than being reimplemented on each side.

- `origin/editor_origin.js` plus `EDITOR_ORIGIN_KIND` and its schema entry: the
  origin an editor client's change carries, holding the editorId that scopes a
  change's timestamp.
- `change_identity.js`: how a submission is recognised when it comes back, so a
  resend of a change that already landed is deduplicated rather than applied
  twice.
- `rebase.js`: transforming a change onto concurrent ones, in the same order on
  both sides, which is what lets the server acknowledge a change with only the
  version it landed at.
- `text_file_defaults.js`: the doc-vs-file extension lists, now read from here
  by web's settings instead of being spelled out again.
- `blob_string_length.js`: the string length of a blob, used by history-v1's
  blob store and by web when it writes an uploaded file into history.
- `file_tree_diff.js`: file-tree operations to a diff, used by
  project-history's FileTreeDiffGenerator.
- `Blob.EMPTY_HASH`, so callers can answer for the empty blob without storing
  it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* [project-history] build the file-tree diff with the shared helper

FileTreeDiffGenerator's hand-rolled walk over add/edit/move/delete operations
becomes a call to `buildFileTreeDiff` from overleaf-editor-core, so the editor
and project-history read a file-tree change the same way. Adds the unit tests
the decaffeinated original never had.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* [web] read the editable-file lists from overleaf-editor-core

web's settings spelled out the text extensions and editable filenames that
decide whether a file becomes a doc, and every other service that has to make
that call carried its own copy. The lists now live in overleaf-editor-core and
web builds settings.textExtensions and settings.editableFilenames from them,
keeping the ADDITIONAL_TEXT_EXTENSIONS override for Server CE/Pro on top.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* [history-v1] read a blob's string length from overleaf-editor-core

Two copies of getStringLengthOfFile lived here, the one in recover_zip.js
carrying a TODO asking for exactly this. Both now import the shared
implementation, which the blob store keeps re-exporting for backupArchiver.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: 31948332f416f56848e3b0dfb5142a4c9f93f508
2026-09-17 12:06:17 +00:00
Jakob Ackermann eb36edac6d [project-history] report a resync that is still pending after a flush (#36122)
GitOrigin-RevId: ccd4390048c09f732531d2e02a40336fce88a952
2026-09-17 11:45:55 +00:00
Jakob Ackermann 52e5fb3498 [project-history] adopt strict zod types and extend test coverage (#36087)
GitOrigin-RevId: ce1396e87926fc8816afa424c9db2a052c7da5e6
2026-09-17 11:45:20 +00:00
Mathias Jakobsen ef05288f81 Merge pull request #34714 from overleaf/mj-tracked-change-merges
[overleaf-editor-core+ranges-tracker] Pick lower timestamp when merging tracking info

GitOrigin-RevId: 5b1c8d7c719003df4506014c381f2a525d0a7ff4
2026-06-19 08:07:27 +00:00
Brian Gough 57d76da263 drop zero-length retain operations in project-history (#34427)
GitOrigin-RevId: ec3069a3ea3b7d487548ba181a2dde9772e308e5
2026-06-11 08:05:33 +00:00
Miguel Serrano 14b04ad4b8 [project-history] Removed request dependency (#32686)
* [project-history] Removed `request` dependency

GitOrigin-RevId: 086bbbf2efeea6026127653a1f68ca6bf0476de6
2026-05-28 08:06:04 +00:00
Malik Glossop 7bf7102b98 Merge pull request #32712 from overleaf/mg-recreate-corrupted-blob
Recover from corrupted blobs during hard resync

GitOrigin-RevId: 7cc764e3bcc8557689c040c8f042991d97f897bc
2026-05-11 08:06:15 +00:00
Mathias JakobsenandJakob Ackermann a478c1a829 [web+project-history] Add lastResyncedAt field to projects and set after resync (#33240)
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: f679a190b88694a46f7816d51eff96446f338dec
2026-04-30 08:06:43 +00:00
Malik Glossop da027f46cf Merge pull request #32530 from overleaf/mg-stuck-sync-repro
Detect and auto-clear stuck sync states

GitOrigin-RevId: 680861a33e42432dab7d40ad421980b707eb6089
2026-04-29 08:05:33 +00:00
Mathias Jakobsen cae558af69 Merge pull request #32779 from overleaf/mj-project-history-bug
[project-history] Fix broken ranges from history resync

GitOrigin-RevId: c839e1d187768b7ce3411d06f78f017ebf7fec6e
2026-04-17 08:05:42 +00:00
Eric Mc Sween 77952e5d21 Merge pull request #32628 from overleaf/em-bg-history-bug
Prevent incorrect merging of ops with different hpos offsets

GitOrigin-RevId: d171a93dcb29e952c7105afd81c3b29ca7e9788e
2026-04-08 08:05:03 +00:00
Domagoj KriskovicandCopilot 332588826d [project-history] Fix for invalid range Error (#32198)
* [project-history] Invalid range Error WIP

* Clamp tracked deletes to retention range to prevent negative positions

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
GitOrigin-RevId: 408da6fff99b6c413af3d18fa4c399d88ca5f2a2
2026-04-01 08:06:15 +00:00
Jakob Ackermann 8a67258461 [project-history] remove unused DocumentUpdaterManager (#32537)
GitOrigin-RevId: f2806187e2ae4e768f48a724712daffd4168e4fb
2026-03-31 08:07:43 +00:00
Jakob AckermannandJessica Lawshe 7c70b749d4 [monorepo] remove PII and variables from error messages (#31508)
* [monorepo] remove PII and variables from error messages

Exclusions:
- scripts
- tests
- fuzzing
- SplitTestManager (messages are sent to admin frontend)
- Group setup (we may want an error per unique tuple)
- sharejs (unused types; text type errors are shadowed already)
- history-v1 error messages that are used by the ErrorRecorder
- errors that flag issues with configuration/call signatures

I've used these search terms for finding unwanted error messages:
- new Error(`
- new Error\(\n\s+` (regex search)
- new OError(`
- new OError\(\n\s+` (regex search)

* [web] throw NotFoundError from ProjectLocator

* [github-sync] fix OError.tag call in script

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>

* [templates] revert changes to test client

---------

Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>
GitOrigin-RevId: 736857a4fc5d9bfb0f8cb03e0f004eda87e5a220
2026-02-17 09:05:04 +00:00
Domagoj Kriskovic da9da896c2 Revert "Revert "[project-history] migrate from joi to zod (#30187)" (#30681)"
This reverts commit 8d66aa15a29216736755c3f86954a4d40aa20a6c.

GitOrigin-RevId: 9d56dafff3da5202204ead896bed30182cc2aceb
2026-02-02 09:05:15 +00:00
Domagoj Kriskovic da7569e892 Revert "[project-history] migrate from joi to zod (#30187)" (#30681)
This reverts commit f8396a3caadd536edf64c6f971c77f4c941f000a.

GitOrigin-RevId: 8d66aa15a29216736755c3f86954a4d40aa20a6c
2026-01-13 09:07:22 +00:00
Domagoj Kriskovic 6d3dd3601a [project-history] migrate from joi to zod (#30187)
* Add @overleaf/validation-tools in project-history

* use error handler for zod

* Replace joi validations with zod schema

* fix unit tests

* remove Joi validation middleware

* remove optional when there is a default

* fix tests after checking for objectid

* ran build_scripts

* monorepo npm install

GitOrigin-RevId: f8396a3caadd536edf64c6f971c77f4c941f000a
2026-01-13 09:07:00 +00:00
Jakob Ackermann 8b937c91f4 [project-history] fix unit tests after merging multiple PRs (#26442)
GitOrigin-RevId: 9ccbadaa807da2f20010930477724e9370bf103e
2025-06-16 08:06:54 +00:00
Jakob Ackermann 0aa56fbe2c [project-history] fix callback signature when processing no updates (#26420)
* [project-history] fix tests and cover callback for processing updates

The before setup was hiding that some tests were not doing what the
assertions were expecting.

* [project-history] fix callback signature when processing no updates

GitOrigin-RevId: 4fa14d47b9a1afd998316b0c9024d49760785a47
2025-06-16 08:06:36 +00:00
Jakob Ackermann 6f516b25af [project-history] add metrics for compression of updates (#26307)
* [project-history] add metrics for compression of updates

* [project-history] sample compression metrics

GitOrigin-RevId: 1cd67dc4ec7b44285afb436c62392b464f007f97
2025-06-16 08:06:28 +00:00
Brian Gough 6eada92966 Merge pull request #25406 from overleaf/em-content-hash-validation-resync
Resync project when content hash validation fails

GitOrigin-RevId: ea9b5a78f89c55276fd67835bc262717bc778e92
2025-05-09 08:05:12 +00:00
Eric Mc Sween 35dc7faab6 Merge pull request #24224 from overleaf/em-resync-on-flush-failure
Immediately attempt a resync when a history flush fails

GitOrigin-RevId: 098a0a7edf55c0ed47c48e0a3c080e3562cdceaa
2025-03-18 09:05:36 +00:00
Eric Mc Sween 064baf31ad Merge pull request #23478 from overleaf/em-doc-hash-diff
Fix doc hash on delete + insert combination

GitOrigin-RevId: ecea5f8ab01f24a9064be88611256b502500d91f
2025-02-10 09:05:38 +00:00
Eric Mc Sween 39842c9857 Merge pull request #23250 from overleaf/em-project-history-doc-hash
Propagate the doc hash from project-history to history

GitOrigin-RevId: 341df52df41f7a5f8dbebbce53a47f9d5e1f8175
2025-02-04 09:05:03 +00:00
Jakob Ackermann b0c9176634 [project-history] add resync for project structure only (#23271)
* [project-history] add resync for project structure only

* [project-history] block resyncProjectStructureOnly that update docs

* [project-history] tweak test description

GitOrigin-RevId: ce2749566c36e04dab21c26dd60dd75d93c0d4c0
2025-02-03 09:05:39 +00:00
Jakob Ackermann ba60f885a4 [project-history] fetch most recent chunk once when processing updates (#23261)
GitOrigin-RevId: 08ddd1f953b0dbae0541a60b45fec0f88e0a1d06
2025-01-31 09:05:14 +00:00
Jakob Ackermann 72b4083318 [project-history] forward all the optional file attributes during resync (#23260)
Notably, 'createdBlob' was missing. And others like metadata should not
be added if not set.

GitOrigin-RevId: c4a6746c4824d637f7c54b4c68a7025b60c15ff2
2025-01-31 09:05:09 +00:00
Eric Mc Sween 3e4e9b298f Merge pull request #23089 from overleaf/em-fix-ranges-snapshot
Fix edge cases when restoring comments

GitOrigin-RevId: 6ce2426bfb7233a3c0915bcc8c53bf0588702847
2025-01-27 09:05:06 +00:00
Jakob Ackermann 762266cd87 [project-history] block filestore reads from old queue entries (#23096)
GitOrigin-RevId: 9952cb66e542b17a6a3b5e3b2609d53dc8c371fd
2025-01-27 09:04:54 +00:00
Jakob Ackermann 2640d4a566 [project-history] fixes for checking blob exists (#23051)
* [project-history] skip HEAD on blob without file-hash

* [project-history] only 404s indicate a missing blob

* [project-history] error checking for existing blob is fatal

* [project-history] always skip creating blob if already exists

Ignore createdBlob detail after performing check on blob. It might not
 exist on old operations that have been queued.

* [project-history] fix unit tests

* [project-history] migrate test to new fetch call

GitOrigin-RevId: 457963042e9d836647d3c122516329d4dc9b544f
2025-01-24 09:05:09 +00:00
Andrew Rumble e6a2caa088 Merge pull request #22421 from overleaf/ar-pass-created-blob-for-filetree-uploads
[web] Pass createdBlob through for file creation everywhere

GitOrigin-RevId: 880bebd0a55e351a6a61555a46e80faef22db7cb
2024-12-12 09:05:02 +00:00
Andrew Rumble ad8026038c Merge pull request #22397 from overleaf/revert-22320-revert-22213-ar-avoid-duplicate-blob-writes
Revert "Revert "Avoid duplicate blob writes""

GitOrigin-RevId: f6a0083c0edf9ede14dfb52dbf1809a843aa5ff4
2024-12-10 09:05:07 +00:00
Andrew Rumble ffa28b1a58 Merge pull request #22320 from overleaf/revert-22213-ar-avoid-duplicate-blob-writes
Revert "Avoid duplicate blob writes"

GitOrigin-RevId: 9f86bcea654cd3fa5f66fbdf42080e7f6e2861a7
2024-12-05 09:04:48 +00:00
Andrew Rumble 40b781eb0a Merge pull request #22213 from overleaf/ar-avoid-duplicate-blob-writes
Avoid duplicate blob writes

GitOrigin-RevId: 5e7a472b235bae5ef84389dbbe791189e951908a
2024-12-04 09:05:08 +00:00
Antoine Clausse 7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Jakob Ackermann fab69a16c4 Merge pull request #20979 from overleaf/jpa-handle-error
[project-history] gracefully handle errors when loading Changes

GitOrigin-RevId: 214fadb3c779551ea90f38b0fdf77c58ed5df178
2024-10-15 08:05:15 +00:00
Jakob Ackermann 663f6605d7 Merge pull request #20978 from overleaf/jpa-fix-unit-tests
[project-history] fix unit tests for processing updates

GitOrigin-RevId: ce87296b9b97dcfd7598fbf25b3066b7c145fb6f
2024-10-15 08:05:10 +00:00
Domagoj Kriskovic 0998cc04df [project-history] fix diff view on file revert (#20769)
GitOrigin-RevId: 6c95dab9328ecab636272ad507505f5342ec8709
2024-10-14 11:06:36 +00:00
Miguel Serrano cd6631c105 Fix history label creation for anonymous users (#20200)
* Remove decaffeination artifacts in LabelsTests

* Remove decaffeination artifacts in LabelsManagerTests

* Fix label creation for anonymous users

* Update label creation route in MockProjectHistoryApi tests

* Support both endpoints for backwards compatibility

GitOrigin-RevId: 50ce1ba49388e50f147fb620e0425fea83301c9d
2024-10-14 10:57:33 +00:00
Eric Mc Sween 250acbf33c Merge pull request #20520 from overleaf/em-ts-import
Use new import JSDoc syntax for Typescript annotations

GitOrigin-RevId: 782456d637fc8f2de6163b5d70fabf06c1d74964
2024-09-23 08:04:47 +00:00
Mathias Jakobsen 8245a95b4e Merge pull request #20266 from overleaf/mj-revert-linked-file
[project-history+web] Restore metadata when reverting file

GitOrigin-RevId: dbfa8202a2fe0bf077d8eedb51a2a13b9f1e8a83
2024-09-09 08:04:53 +00:00
Jakob Ackermann 989c48978a Merge pull request #20067 from overleaf/jpa-fix-old-blob-download
[web] project-history expects history-v1 id in blob requests

GitOrigin-RevId: d9170a12fc6070811c188b346dbac32accabbfd7
2024-08-22 14:01:12 +00:00
Jakob Ackermann 577497b655 Merge pull request #19842 from overleaf/jpa-ro-mirror-on-client
[misc] add readonly mirror of full project content on the client

GitOrigin-RevId: 940bd93bfd587f83ca383d10fc44579b38fc3e88
2024-08-22 08:05:13 +00:00
Mathias Jakobsen 654d96ace6 Merge pull request #19743 from overleaf/mj-restore-project
[web+project-history] Add project version reverting

GitOrigin-RevId: 0f77cec730393187d531c0c6561faaa652bebf29
2024-08-21 08:05:08 +00:00
Eric Mc Sween e10478f256 Merge pull request #19867 from overleaf/em-hide-history-resync-details
Hide history resync changes

GitOrigin-RevId: a36876ff637d0a4c34150b21f4b4e623dff0ab03
2024-08-14 08:04:54 +00:00
Jakob Ackermann 86f26043d2 Merge pull request #19847 from overleaf/jpa-rename-history-v1
[misc] rename docker compose service history_v1 to history-v1 in dev-env

GitOrigin-RevId: 0991ee4297bae1ed9e9068af1889966ceeeba66d
2024-08-09 08:04:29 +00:00
Jakob Ackermann d21874b076 Merge pull request #19741 from overleaf/jpa-check-filestore-write
[misc] verify file hash when downloading binary file in project-history

GitOrigin-RevId: 0ef56a0753cdfd55fdea921b3555dea48036766b
2024-08-06 08:04:49 +00:00
Jakob Ackermann e26b6de51b Merge pull request #19740 from overleaf/jpa-linked-file-metadata
[misc] persist linkedFileData in full project history as file metadata

GitOrigin-RevId: f3e8ba947ea34b6796e210a076a248c57188d148
2024-08-06 08:04:44 +00:00
Domagoj Kriskovic 6afb067737 Hide history-resync updates from "All history" (#19435)
* [web] hide history-resync updates from "All history"

* Revert "[web] hide history-resync updates from "All history""

This reverts commit e9d5e7638eabe2baccf36f8c80bb4cd619e383ea.

* filter history resync update in summarizeUpdates

* used isHistoryResyncUpdate for merging

* remove merging for history-resync

* Revert "remove merging for history-resync"

This reverts commit 6ce48bc3f906867a64c0acc12e2dc61c05436c41.

GitOrigin-RevId: 0335eb9c05815fb66188d453a90dd09531504a55
2024-07-19 08:04:21 +00:00
Eric Mc Sween 58ffefc8bb Merge pull request #19298 from overleaf/em-fix-restore-empty-file
Fix file restore when the file is empty

GitOrigin-RevId: be08305c9a41265acfb12046aeb2d003cda92b54
2024-07-15 09:00:51 +00:00