Files
overleaf-cep/libraries/overleaf-editor-core/lib/change_identity.js
T
Jakob AckermannandClaude Opus 5 04871d6049 [editor-core] extract more shared utils (#36954)
* [editor-core] share the blob hash and the doc-vs-file rule

The git blob hash and the rule that decides whether a file is a doc or a binary
file both have to reach the same answer in every service that writes to history,
so they move into overleaf-editor-core rather than being spelled out again on
each side.

blob_utils takes over lib/blob_string_length.js: hashing and the string-length
rule answered two halves of the same question -- what history needs to know
about content before it can store it -- and each carried the same warning about
staying off index.js, which is bundled for the browser. One module states it
once, and blobForFile joins them, describing content on disk as the blob it
amounts to. The hashing functions are flat rather than a namespace of their own:
in a module that also holds the string-length rule, a bare fromStream says
nothing about what it makes. history-v1's blob store and recover_zip follow the
module to its new name.

file_type_detector holds the doc-vs-file rule, Node-only and deep-imported for
the same reason. It takes textExtensions, editableFilenames and maxDocLength as
configuration rather than reading settings: the limit is web's 2 MiB and not
editor-core's own 3 MiB, and the surrogate check covers both halves of a pair
where editor-core's covers the high half.

buildSetContentOperations' test spelled the blob header out for itself, with the
separator as a literal NUL byte, which had git reading the file as binary. It
hashes through blobHashFromString now, so it does not spell the header at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] name the client a change came from on the origin itself

`EditorOrigin` carried an `editorId` that only an editor change could have. What
the id is for -- recognising a change again, so history does not apply a resend
twice and the client does not apply its own change back -- is a property of the
client that submitted it rather than of what it did, so it moves onto `Origin` as
an optional `historyClientId` and `EditorOrigin` goes.

- `Origin` takes `(kind, historyClientId)` and answers `getHistoryClientId()`;
  `dropHistoryClientId()` is what the chunk store uses to strip the id from all
  but a client's latest change, which now leaves the rest of the origin alone --
  a restore that a client submitted stays a restore.
- `toRaw` is built a layer at a time: the base writes the kind and the id, a
  subclass spreads that and adds the fields it owns.
- The named kinds are their own schemas and discriminate on `kind`, so a payload
  claiming one is checked against that shape and the error says what it got
  wrong. The catch-all cannot join them -- its kind is any string -- so it sits
  alongside and is refined to refuse the named kinds, which it used to swallow:
  `{kind: 'restore'}` validated as a kind and nothing else, then read back as a
  restore that recorded nothing about what it restored.
- `lib/origin/*.js` gains `// @ts-check`. Without it the annotations in there
  were decorative: the lazily required subclasses were `any`, so nothing handed
  to them was checked, and `fromRaw` narrows with `in` to drop the catch-all
  variant, which a kind check cannot.

The callers that stamp the id -- real-time's handshake flag, web's applyChange
body -- come with the rest of the branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] recognise a change a writer that is not an editor placed

The counterpart of `isSameEditorChange` for writers that are not an editor
client: an integration that commits a batch on a user's behalf has to recognise
that batch coming back, because a lost response does not say whether the commit
landed and resubmitting one that did applies its removes twice.

It compares the same three fields, and for the same reason -- a rebase rewrites a
change's operations but leaves its origin, authors and timestamp alone -- with
the timestamp compared as a point in time rather than as a string, so that two
spellings of the same instant cannot read as different changes. github-sync is
the first caller, and comes with the rest of the branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] load an editable file's content, or say it has none

Only an editable file has content: any other kind is bytes addressed by hash, and
hollow data records the size of content it does not have. Both answer
`getContent()` with null, so every caller that wants the content of a doc loaded
it, checked the data's class, and reached for its own error when the check failed.
`File#loadEager` does that once and answers the data that always has the content.

`buildSetContentOperations` takes a `ReadWriteBlobStore` -- everything but
looking a blob up by hash, which is all it takes to build operations over content
-- so a caller holding a store that cannot answer `getBlob` can still use it. Its
metadata argument is `FileMetadata` rather than `Object`, which is what the schema
already says a file's metadata is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] say that a pair of operations that do not fit is unprocessable

compose and transform threw a bare Error when the two operations do not describe
the same content. Everything that answers a client for these -- history-v1's
import routes, the git-bridge push, project-history's resync -- decides on
UnprocessableError, so a bare Error reached them as an internal failure: a 500,
which reads as worth retrying, for a pair that fails the same way every time.
A client that resubmits on a 500 resubmits forever.

The guards that cannot be reached through the API keep their bare Error: a
TextOperation composed with something that is not one, and the branch whose
message says it should not happen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* [history-v1] hash blobs with the rule editor-core holds

storage/lib/blob_hash.js was a bluebird-era implementation of the same git blob
hash editor-core holds, exported from the storage barrel and used across the
storage layer, the api and the tests. All of it reads the shared rule now, and
the test that existed only to check the copy is gone -- editor-core's suite
covers the same non-ASCII case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 5a0c1e9a06170a0164c01a26a7bb2d205726d531
2026-09-17 12:07:15 +00:00

142 lines
5.2 KiB
JavaScript

// @ts-check
'use strict'
const { EDITOR_ORIGIN_KIND } = require('./origin')
/**
* @import { RawChange } from "./types"
*/
/**
* Identifying a change an editor client submitted.
*
* history-v1 uses this to recognise a change a client is resending, so as not to
* apply the same operations twice; the client uses it to recognise its own change
* coming back on the change stream, so as not to apply it a second time locally.
* Both sides have to agree exactly — if one of them compares a field the other
* does not, they disagree about what is a duplicate, and a change is either
* applied twice or discarded. Neither reports an error. So the key lives here
* rather than being spelled out in each service.
*
* The fields are the ones a rebase leaves alone: transforming a change rewrites
* its operations, so those cannot identify it, while the origin, the author and
* the timestamp travel through history untouched.
*
* Each rules out a different false match:
*
* - the origin kind keeps other writers out, so a Dropbox sync or a resync is
* never mistaken for a client's change.
* - `historyClientId` separates editors that share an author. Nothing coordinates
* timestamps between two tabs of one user, or between anonymous editors, who
* share an empty author — without it, two clients colliding on a millisecond
* look identical.
* - the author makes a forged `historyClientId` unusable: it arrives from the
* client, but a change belonging to another account can never match.
* - the timestamp separates successive changes from one editor, which the client
* keeps distinct and stable across resends.
*
* @typedef {{historyClientId: string, author: (string|null), timestamp: number}} EditorChangeIdentity
*/
/**
* The identity of a raw change, or null if it is not an identifiable editor
* change — another writer's, or one whose `historyClientId` was dropped when its
* chunk was written.
*
* The timestamp is compared as a point in time rather than as a string, so that
* two spellings of the same instant cannot read as different changes.
*
* @param {RawChange} [raw]
* @return {EditorChangeIdentity | null}
*/
function editorChangeIdentity(raw) {
const origin = raw?.origin
if (!origin || origin.kind !== EDITOR_ORIGIN_KIND) return null
// Absent on a change nothing has to recognise again, and on one whose id was
// dropped when its chunk was written. Neither is identifiable, and an absent id
// must never read as a match.
const { historyClientId } = origin
if (!historyClientId) return null
// real-time stamps exactly one author on every change it forwards, so anything
// else did not come from this path.
const authors = raw.v2Authors
if (!Array.isArray(authors) || authors.length !== 1) return null
const timestamp = new Date(raw.timestamp).getTime()
if (Number.isNaN(timestamp)) return null
return { historyClientId, author: authors[0] ?? null, timestamp }
}
/**
* Build an identity from a client's own record of a change it submitted, for
* comparing against what comes back from history.
*
* The timestamp is still a Date here: the client holds the change it submitted,
* and only the comparison works in milliseconds.
*
* @param {{historyClientId: string, author: string | null, timestamp: Date}} params
* @return {EditorChangeIdentity}
*/
function editorChangeIdentityOf({ historyClientId, author, timestamp }) {
return {
historyClientId,
author: author ?? null,
timestamp: timestamp.getTime(),
}
}
/**
* Whether two identities name the same change. A null identity matches nothing,
* including another null.
*
* @param {EditorChangeIdentity | null} [a]
* @param {EditorChangeIdentity | null} [b]
* @return {boolean}
*/
function isSameEditorChange(a, b) {
if (!a || !b) return false
return (
a.historyClientId === b.historyClientId &&
a.author === b.author &&
a.timestamp === b.timestamp
)
}
/**
* Whether a raw change is one a given writer placed.
*
* The counterpart of the above for writers that are not an editor client: an
* integration that commits a batch on a user's behalf and then has to recognise
* that batch coming back, because a lost response does not say whether the
* commit landed and resubmitting one that did applies its removes twice.
*
* Compares the same three fields, and for the same reason -- a rebase rewrites
* a change's operations but leaves its origin, authors and timestamp alone --
* with the timestamp compared as a point in time rather than as a string, so
* that two spellings of the same instant cannot read as different changes.
*
* @param {RawChange} [raw]
* @param {{originKind: string, author: string, timestamp: Date}} [expected]
* @return {boolean}
*/
function isChangeFrom(raw, expected) {
if (!raw || !expected) return false
if (raw.origin?.kind !== expected.originKind) return false
if (!Array.isArray(raw.v2Authors)) return false
if (!raw.v2Authors.includes(expected.author)) return false
const timestamp = new Date(raw.timestamp).getTime()
if (Number.isNaN(timestamp)) return false
return timestamp === expected.timestamp.getTime()
}
module.exports = {
editorChangeIdentity,
editorChangeIdentityOf,
isSameEditorChange,
isChangeFrom,
}