The hash of empty content is a property of the content rather than of anything
stored: every project's empty file has it, whether or not a blob was ever written
under it, and a file may reference it before anything has. Eight blob stores were
each paying a lookup -- a database query, an HTTP request, a read of a cache that
offline mode may be the only thing standing in for -- to be told what the hash
already says, and clsi had already reinvented the answer on its own.
overleaf-editor-core gains BlobStoreBase, which answers getString for that hash
and delegates the rest to the fetchString its subclasses provide. getObject comes
with it: four stores spelled out the same JSON.parse of getString, and a fifth
wrapped it in a log line.
Adopted by history-v1's BlobStore, BackupBlobStore, HashCheckBlobStore and
recover_zip's RecoveryBlobStore, by project-history's BlobStore, by clsi's
BlobStore, and by web's HistoryBlobStore, SimpleBlobStore and FrontendBlobStore.
history-v1 keeps its own answers in getStream, getBlob and getBlobs, which no
other store implements. Writing and copying a blob are untouched, so a project
that wrote empty content still has a blob for it and a read answers the same
either way.
Two paths could not handle a project holding a tracked change or a comment at
all. Loading such a file needs the ranges blob its rangesHash points at, and
hashing one again needs to write it back, so a store missing getObject or
putObject fails outright:
- recover_zip died on `blobStore.getObject is not a function`, making every such
project unrecoverable. The test covering it asserts a tracked delete is absent
from the zip, which the script already asked for and never reached.
- GET /projects/:project_id/latest/hashed_content returned 500 from the same
cause, then would have hit a missing putObject in snapshot.store.
HashCheckBlobStore becomes a class over the base and gains putObject.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: c84f62465beff6cf77ffac26aa772984e1b554cb
* [editor-core] share the pieces the editor, real-time and history-v1 all need
Project-level history-OT has the browser, real-time and history-v1 all reason
about the same changes, so the rules they reason with move into the shared
library rather than being reimplemented on each side.
- `origin/editor_origin.js` plus `EDITOR_ORIGIN_KIND` and its schema entry: the
origin an editor client's change carries, holding the editorId that scopes a
change's timestamp.
- `change_identity.js`: how a submission is recognised when it comes back, so a
resend of a change that already landed is deduplicated rather than applied
twice.
- `rebase.js`: transforming a change onto concurrent ones, in the same order on
both sides, which is what lets the server acknowledge a change with only the
version it landed at.
- `text_file_defaults.js`: the doc-vs-file extension lists, now read from here
by web's settings instead of being spelled out again.
- `blob_string_length.js`: the string length of a blob, used by history-v1's
blob store and by web when it writes an uploaded file into history.
- `file_tree_diff.js`: file-tree operations to a diff, used by
project-history's FileTreeDiffGenerator.
- `Blob.EMPTY_HASH`, so callers can answer for the empty blob without storing
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [project-history] build the file-tree diff with the shared helper
FileTreeDiffGenerator's hand-rolled walk over add/edit/move/delete operations
becomes a call to `buildFileTreeDiff` from overleaf-editor-core, so the editor
and project-history read a file-tree change the same way. Adds the unit tests
the decaffeinated original never had.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [web] read the editable-file lists from overleaf-editor-core
web's settings spelled out the text extensions and editable filenames that
decide whether a file becomes a doc, and every other service that has to make
that call carried its own copy. The lists now live in overleaf-editor-core and
web builds settings.textExtensions and settings.editableFilenames from them,
keeping the ADDITIONAL_TEXT_EXTENSIONS override for Server CE/Pro on top.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [history-v1] read a blob's string length from overleaf-editor-core
Two copies of getStringLengthOfFile lived here, the one in recover_zip.js
carrying a TODO asking for exactly this. Both now import the shared
implementation, which the blob store keeps re-exporting for backupArchiver.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: 31948332f416f56848e3b0dfb5142a4c9f93f508
* Add getDocUpdaterCompatibleRanges utility function
* use history snapshot for file/project restore
* move overleaf-editor-core from devDependencies
GitOrigin-RevId: 62481a5304ada9d931e018418be3c0719bccf1f3
* [history-v1-ot] initial implementation of using doc-level history-v1-ot
* [web] fix advancing of the otMigrationStage
Use 'nextStage' for the user provided, desired stage when advancing.
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [document-updater] document size check in editor-core
* [history-ot] rename history-v1-ot to history-ot and add types
* [history-ot] apply review feedback
- remove extra !!
- merge variable assignment when processing diff-match-match output
- add helper function for getting docstore lines view of StringFileData
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
* Revert "[document-updater] add safe rollback point for history-ot (#25283)"
This reverts commit d7230dd14a379a27d2c6ab03a006463a18979d06
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
---------
Signed-off-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
GitOrigin-RevId: 89c497782adb0427635d50d02263d6f535b12481
* add RestoreFileOrigin in overleaf-editor-core
* support source to be an object
* use sourceOrOrigin as param
* rename to originOrSource so the priority is more clear
* get timestamp from version
* fix test
* include version and min_count in getUpdatesFromHistory
* extractOriginOrSource util function
* fix RestoreManagerTests
GitOrigin-RevId: 0ace05a6ade2794c753a9d0bffb4f858ecc6899a
* [overleaf-editor-core] AddCommentOperation and DeleteCommentOperation
* added add comment op test
* delete comment op test
* import core to escape circle deps
* desctructure in tests
* require directly in builder
* invert of add comment is always delete comment
* no merging on compose
* NoOp if comment is not found
* use comment.clone()
* update test
* change CommentRawData type
* jsdoc assert type
* fix formating
* EditNoOperation
* return other in compose
* use ReturnType
* Revert "use ReturnType"
This reverts commit 2c7e04f1541310e9fc08963170a783a437ed1992.
* transorm add comment operation
* transform delete comment operation
* moved comment.js
* format fix
* fix transform addComment and textoperation
* fix merge
* test more complex test operations
* change to else if
* move range.js
* fix types
* fix AddComment and TextOperation transform
* fixed AddComment-TextOperation trasform, added test
* deletecommentoperation should win
* should not delete comment
* remove unused function, fix type
* fix format
* add resolved for existing comment
* transform EditNoOperation
* fix test description
* change the order of EditNoOperation
* fix DeleteCommentOperation-DeleteCommentOperation transform
* fix types after merging main
* refactor operation types
GitOrigin-RevId: 6f127763a6dc50d4fe3524d9b25dc7526b6b0028
* [overleaf-editor-core] Restructure TextOperation hierachy
Restructures the hierachy of TextOperations to include a superclass
EditOperation. This superclass will later on contain other classes used
for tracked changes and comments.
* [overleaf-editor-core] Update json format of LazyStringFileData
* [history-v1+project-history] Fix TextOperation.fromJSON calls
* [overleaf-editor-core] Change EditOperationBuilder.fromRaw to fromJSON
* [overleaf-editor-core] Update apply and invert functions to accept FileData
* [overleaf-editor-core] Pass missing argument to store method
* [overleaf-editor-core] Remove unused method
* [overleaf-editor-core] User EditOperationTransformer
* [overleaf-editor-core] Clean up JSDoc comments
* [overleaf-editor-core] Add tests for EditOperation
* [overleaf-editor-core] Update JSDoc types
GitOrigin-RevId: 9c22a3a89b8483bdb87b43f329ddbdd887ffed42
