* [monorepo] reduce writes from chrome in cypress
Chrome puts its shared memory in /dev/shm, and docker's default 64MB is far
too small for it - which is why Cypress passes --disable-dev-shm-usage,
spilling those segments to files on the container's writable layer instead.
Under one spec that is a few hundred MB of write-then-delete churn; under the
CI fan-out it was most of the disk bandwidth on the whole box.
So every service built from dockerfiles/cypress gets shm_size 1gb and every
cypress config drops the flag again: the SaaS e2e service, web's CI and local
component-test services plus its standalone cypress one, and server-ce's e2e
and native cypress services. Both e2e-open variants inherit the setting
through `extends`; the native one is a service of its own, and would be left
running Chrome against 64MB once the flag is gone.
* [monorepo] adopt cypress-pnp-reporter
Materializing the reporter packages copied 267 packages / 3779 files
(59MB) into /tmp on every container start, once per spec. Resolving them
through PnP inside Cypress's Electron process instead drops that to
zero: the entrypoint shell went from 55.6MB of write_bytes to 0.
* [web] serve built assets from a prepared buffer between builds
Serving a chunk cost ~2ms whatever its size, and an editor page pulls ~144
of them, so webpack-dev-server burned 2-3 cores through an e2e run purely
answering requests for output that had not changed since the last build.
It is not bytes: cutting the bundle 2.6x with a cheaper devtool moved CPU
by 8%, and an empty 304 cost nearly as much as the full body. It is the
per-request work - reading the asset out of the in-memory filesystem and
re-hashing the whole body for a strong ETag, on every request.
Three things a build does that this has to respect, each found by tracing
the hooks rather than by reading them:
- `done` fires twice per build, for the compilation and for the manifest,
both with the same hash. A flag would have been cleared by the second
while the first was still running, so the gate is a counter.
- An incremental `done` reports only the assets it rewrote - 16, not 630 -
so asset names accumulate. Resetting them on `invalid` left the
manifest's two behind and nothing was served from here again.
- webpack-dev-middleware holds requests during a rebuild. This sits in
front of it, so it steps aside for that window and drops the cache when
a rebuild starts. Whether a response can be kept is then decided as it
ends rather than as it began: an epoch bumped on `invalid` names the
build its bytes were read from, and only a response from the current
build with no rebuild in flight is stored. A rebuild starting
mid-response can no longer write the previous build's bytes in behind
the cache being dropped, which eviction by asset name could not catch
anyway - an incremental build does not name the assets it left alone.
Measured on the editor entrypoint, 16 concurrent clients:
0.26-0.31s -> 0.045-0.050s of webpack CPU per page load, and 0.0455s
immediately after a rebuild. Same chunk graph, byte-identical output.
GitOrigin-RevId: 08b4ab2d665414069bc01b4763dd64faa9b4e5cb
* [editor-core] share the pieces the editor, real-time and history-v1 all need
Project-level history-OT has the browser, real-time and history-v1 all reason
about the same changes, so the rules they reason with move into the shared
library rather than being reimplemented on each side.
- `origin/editor_origin.js` plus `EDITOR_ORIGIN_KIND` and its schema entry: the
origin an editor client's change carries, holding the editorId that scopes a
change's timestamp.
- `change_identity.js`: how a submission is recognised when it comes back, so a
resend of a change that already landed is deduplicated rather than applied
twice.
- `rebase.js`: transforming a change onto concurrent ones, in the same order on
both sides, which is what lets the server acknowledge a change with only the
version it landed at.
- `text_file_defaults.js`: the doc-vs-file extension lists, now read from here
by web's settings instead of being spelled out again.
- `blob_string_length.js`: the string length of a blob, used by history-v1's
blob store and by web when it writes an uploaded file into history.
- `file_tree_diff.js`: file-tree operations to a diff, used by
project-history's FileTreeDiffGenerator.
- `Blob.EMPTY_HASH`, so callers can answer for the empty blob without storing
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [project-history] build the file-tree diff with the shared helper
FileTreeDiffGenerator's hand-rolled walk over add/edit/move/delete operations
becomes a call to `buildFileTreeDiff` from overleaf-editor-core, so the editor
and project-history read a file-tree change the same way. Adds the unit tests
the decaffeinated original never had.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [web] read the editable-file lists from overleaf-editor-core
web's settings spelled out the text extensions and editable filenames that
decide whether a file becomes a doc, and every other service that has to make
that call carried its own copy. The lists now live in overleaf-editor-core and
web builds settings.textExtensions and settings.editableFilenames from them,
keeping the ADDITIONAL_TEXT_EXTENSIONS override for Server CE/Pro on top.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* [history-v1] read a blob's string length from overleaf-editor-core
Two copies of getStringLengthOfFile lived here, the one in recover_zip.js
carrying a TODO asking for exactly this. Both now import the shared
implementation, which the blob store keeps re-exporting for backupArchiver.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: 31948332f416f56848e3b0dfb5142a4c9f93f508
* [docstore] use more realistic test stubs for ranges
* [monorepo] define ranges schemas in @overleaf/ranges-tracker
* [ranges-tracker] add legacy orderedRejections field in tracked-changes
* [monorepo] resolve @overleaf/ranges-tracker subpath imports in typescript
* [real-time] validate comment thread ids and tracked change id seeds
* [ranges-tracker] require ObjectId ids and tracked change metadata in ranges
* [monorepo] require ObjectId tracked change ids on the accept routes
* [web] use ObjectId comment thread ids in the DocumentController tests
GitOrigin-RevId: 58b20dec311d85b917ef63f9efbe745a265a2de6
* [security] bump cypress 15.12.0 -> 15.20.1 to drop unpatched extract-zip
extract-zip <=2.0.1 has no patched release (GHSA-jmr9-qjv8-65gv). It was a
transitive dependency of cypress, used only to unpack Cypress's own binary.
Cypress 15.20.1 replaces extract-zip with yauzl directly, so upgrading the
parent removes the vulnerable package entirely.
Also bumps the pinned cypress/included Docker base image used by
test_frontend_ct / server-ce e2e / saas-e2e.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* [security] fix cy.get<T> alias typing broken by cypress 15.20.1
Cypress 15.20.1 adds a dedicated Chainable.get overload for alias strings
(get<S>(alias: `@${string}`): Chainable<S>), which now takes priority over
the generic selector overload. cy.get<HTMLInputElement>('@input') therefore
resolved to Chainable<HTMLInputElement> instead of
Chainable<JQuery<HTMLInputElement>>, so .then(element => element.get(0))
no longer type-checked. Pass the JQuery-wrapped type explicitly to match
the new overload.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
GitOrigin-RevId: f4a56a2d5948b5aaf00a07ecb384fc75abf6de53
Fixes GHSA-mwp4-54f8-5fhr (high), GHSA-4xrf-jv44-h6hh (medium),
GHSA-22jq-vg5j-6vgg (medium) in ip-address, a transitive dependency
pinned via a resolutions entry in root package.json.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
GitOrigin-RevId: 1557589d719801cbd0ed95d36a17583fdc215f90
* Migrate services/web/scripts/{plan-prices,translations,ukamf} off npm
Three more of the last standalone npm projects in the repo (own
package-lock.json, no automated Dependabot fix path):
- plan-prices and ukamf fold into the root Yarn workspace (own
package.json, no lockfile), same pattern as jobs/wiki-algolia. ukamf
was missing a "name" field (required for a workspace member) and
relied on moment/lodash/xml2js being available as phantom
dependencies via web's node_modules; under PnP that no longer works,
so they're now declared explicitly, matching the versions already
used by services/web.
- translations stays standalone (own yarn.lock, packageManager field,
node-modules linker) because its Dockerfile builds an isolated
"onesky-client-jail" sandbox image with deliberately restricted
filesystem access — not a good fit for pulling in the whole
monorepo's PnP install. Its Dockerfile now installs via corepack
yarn instead of npm ci.
Updated the plan-prices README and the two dev-manual docs that
referenced running these scripts with plain `node`/`npm install`.
Also trimmed the now-stale "a few helpers still use npm" line from
security-triage's security-context.md (wiki-algolia was already
migrated; this covers the rest, translations still gets its own
yarn.lock note).
checkSanitizeOptions.js surfaces a handful of pre-existing
allowlist/locale mismatches unrelated to this migration (recently
added tags like <sparkle/> and <bold> aren't in sanitize.js's
allowedTags) — flagging but not fixing, out of scope here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Fix stale ukamf install/run instructions per Copilot review
metadata-processor.js/check-certs.js/check-idp-metadata.js's doc
comments and developer-manual's institutional-sso-configuration.md
still told people to `npm install`/`yarn install` inside
scripts/ukamf and run the scripts with plain `node`. Now that ukamf is
a root Yarn workspace member, that install step doesn't do anything
and plain `node` won't resolve PnP-managed deps. Switch both to
`bin/yarn install` (repo root) + `bin/yarn workspace ukamf-scripts
node <script>.js`.
--no-verify: the pre-commit hook's eslint invocation runs from the
repo root, which breaks eslint-plugin-import's packageDir resolution
for services/web/scripts/ukamf's relative packageDir override
(pre-existing, reproduces identically against the unmodified,
already-committed file content -- unrelated to this change). Verified
correctness instead via the actual web-scoped lint:
bin/yarn workspace @overleaf/web exec eslint scripts/ukamf, which
passes clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Align translations Dockerfile's corepack setup with the rest of the repo
Per Anna Claire's review on #36594: use the same corepack bootstrap as
every other service Dockerfile (services/chat, filestore, etc.) --
pin the Yarn version explicitly via `corepack install -g` into
COREPACK_HOME, then disable corepack's network fetch at runtime --
instead of the bare `corepack enable && yarn install --immutable`.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
GitOrigin-RevId: 9d05064de9191ab3eef39a672ca00f0f0f7066bb
Folds jobs/quay-logging into the root Yarn workspace (same pattern as
jobs/wiki-algolia): removes its own package-lock.json, adds it to the
root workspaces array, and updates cloudbuild.yaml to install/run via
corepack yarn instead of npm.
This brings it under yarn-security-update's automated coverage for
Dependabot alerts, instead of needing a manual backup issue.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
GitOrigin-RevId: 5790ed41001c6ea1830d72018ce9af92525097a1
* [web] Remove SubscriptionFormatters, format subscription dates in the frontend
Replace the server-formatted nextPaymentDueAt/nextPaymentDueDate/
trialEndsAtFormatted fields with the raw periodEnd date and format in
the React dashboard (same strings as before, English + UTC). Also
consolidate hand-rolled trial checks onto isInFreeTrial /
SubscriptionHelper.isInTrial.
* [web] Remove the now-unused dateformat dependency
* [web] Address review: payment-dates unit test, UTC dates in acceptance test
Add a dedicated unit test asserting the exact formatted strings from
formatPaymentDate/formatPaymentDateTime, and use explicit UTC timestamps
in the SubscriptionDashboard acceptance fixtures instead of
local-timezone Date constructors.
GitOrigin-RevId: 16ac4f1098b14ef9f1478d22cf39c1357ead7dea
* [editor-core] gracefully handle no-op TextOperation
* [web] avoid sending sync to pdf requests without a ?file
* [validation-tools] log deeper nested violations
* [editor-core] fix round trip of EditNoOperation
* [web] use custom error for sync to pdf w/o file, fix tests
* [validation-tools] include request params in log msg for violation
* Revert "[editor-core] gracefully handle no-op TextOperation"
This reverts commit c1d37261d5db72b37dbf39428c54edcfb5423f41.
* [web] gracefully handle rejected sync to pdf from context menu
GitOrigin-RevId: 21de7dc96b23575b0d67b6eb0c68ed9c2f86292b
* [document-updater] adopt strict zod types and extend test coverage
* [saas-e2e] add test for project restore with binary and linked file
GitOrigin-RevId: a8e45bd1e46b6ac8a293da24c8b8ad7b236fd9bd
Jakob Ackermann