[saas-e2e] reduce disk IO and webpack overhead (#36997)

* [monorepo] reduce writes from chrome in cypress

Chrome puts its shared memory in /dev/shm, and docker's default 64MB is far
too small for it - which is why Cypress passes --disable-dev-shm-usage,
spilling those segments to files on the container's writable layer instead.
Under one spec that is a few hundred MB of write-then-delete churn; under the
CI fan-out it was most of the disk bandwidth on the whole box.

So every service built from dockerfiles/cypress gets shm_size 1gb and every
cypress config drops the flag again: the SaaS e2e service, web's CI and local
component-test services plus its standalone cypress one, and server-ce's e2e
and native cypress services. Both e2e-open variants inherit the setting
through `extends`; the native one is a service of its own, and would be left
running Chrome against 64MB once the flag is gone.

* [monorepo] adopt cypress-pnp-reporter

Materializing the reporter packages copied 267 packages / 3779 files
(59MB) into /tmp on every container start, once per spec. Resolving them
through PnP inside Cypress's Electron process instead drops that to
zero: the entrypoint shell went from 55.6MB of write_bytes to 0.

* [web] serve built assets from a prepared buffer between builds

Serving a chunk cost ~2ms whatever its size, and an editor page pulls ~144
of them, so webpack-dev-server burned 2-3 cores through an e2e run purely
answering requests for output that had not changed since the last build.

It is not bytes: cutting the bundle 2.6x with a cheaper devtool moved CPU
by 8%, and an empty 304 cost nearly as much as the full body. It is the
per-request work - reading the asset out of the in-memory filesystem and
re-hashing the whole body for a strong ETag, on every request.

Three things a build does that this has to respect, each found by tracing
the hooks rather than by reading them:

- `done` fires twice per build, for the compilation and for the manifest,
  both with the same hash. A flag would have been cleared by the second
  while the first was still running, so the gate is a counter.
- An incremental `done` reports only the assets it rewrote - 16, not 630 -
  so asset names accumulate. Resetting them on `invalid` left the
  manifest's two behind and nothing was served from here again.
- webpack-dev-middleware holds requests during a rebuild. This sits in
  front of it, so it steps aside for that window and drops the cache when
  a rebuild starts. Whether a response can be kept is then decided as it
  ends rather than as it began: an epoch bumped on `invalid` names the
  build its bytes were read from, and only a response from the current
  build with no rebuild in flight is stored. A rebuild starting
  mid-response can no longer write the previous build's bytes in behind
  the cache being dropped, which eviction by asset name could not catch
  anyway - an incremental build does not name the assets it left alone.

Measured on the editor entrypoint, 16 concurrent clients:
0.26-0.31s -> 0.045-0.050s of webpack CPU per page load, and 0.0455s
immediately after a rebuild. Same chunk graph, byte-identical output.

GitOrigin-RevId: 08b4ab2d665414069bc01b4763dd64faa9b4e5cb
This commit is contained in:
Jakob Ackermann
2026-09-17 12:07:28 +00:00
committed by Copybot
parent 44910202f3
commit 37e27bb22e
11 changed files with 314 additions and 14 deletions
+18 -5
View File
@@ -7138,6 +7138,21 @@ __metadata:
languageName: node
linkType: hard
"@overleaf/cypress-pnp-reporter@workspace:*, @overleaf/cypress-pnp-reporter@workspace:libraries/cypress-pnp-reporter":
version: 0.0.0-use.local
resolution: "@overleaf/cypress-pnp-reporter@workspace:libraries/cypress-pnp-reporter"
dependencies:
"@types/node": "npm:^24.5.2"
chai: "npm:^4.3.6"
cypress-multi-reporters: "npm:^2.0.5"
eslint: "npm:^10.4.0"
mocha: "npm:^11.1.0"
mocha-junit-reporter: "npm:^2.2.1"
mocha-multi-reporters: "npm:^1.5.1"
typescript: "npm:^5.0.4"
languageName: unknown
linkType: soft
"@overleaf/dictionaries@https://github.com/overleaf/dictionaries/archive/refs/tags/v0.0.3.tar.gz":
version: 0.0.3
resolution: "@overleaf/dictionaries@https://github.com/overleaf/dictionaries/archive/refs/tags/v0.0.3.tar.gz"
@@ -7707,16 +7722,15 @@ __metadata:
"@cypress/code-coverage": "npm:^4.0.2"
"@cypress/grep": "npm:^6.0.2"
"@isomorphic-git/lightning-fs": "npm:^4.6.0"
"@overleaf/cypress-pnp-reporter": "workspace:*"
"@testing-library/cypress": "npm:^10.1.0"
"@types/uuid": "npm:^9.0.8"
adm-zip: "npm:^0.6.0"
cypress: "npm:15.20.1"
cypress-multi-reporters: "npm:^2.0.5"
eslint: "npm:^10.4.0"
isomorphic-git: "npm:^1.33.1"
mailtrap: "npm:^4.3.0"
mocha: "npm:^11.7.5"
mocha-junit-reporter: "npm:^2.2.1"
pdfjs-dist: "npm:^5.1.91"
typescript: "npm:^5.0.4"
uuid: "npm:^9.0.1"
@@ -7728,19 +7742,18 @@ __metadata:
resolution: "@overleaf/server-ce-test@workspace:server-ce/test"
dependencies:
"@isomorphic-git/lightning-fs": "npm:^4.6.0"
"@overleaf/cypress-pnp-reporter": "workspace:*"
"@overleaf/promise-utils": "workspace:*"
"@overleaf/validation-tools": "workspace:*"
"@testing-library/cypress": "npm:10.1.0"
"@types/uuid": "npm:^9.0.8"
adm-zip: "npm:^0.6.0"
cypress: "npm:15.20.1"
cypress-multi-reporters: "npm:^2.0.5"
eslint: "npm:^10.4.0"
express: "npm:4.22.1"
isomorphic-git: "npm:^1.33.1"
js-yaml: "npm:^4.1.1"
mocha: "npm:^11.7.5"
mocha-junit-reporter: "npm:^2.2.1"
pdfjs-dist: "npm:^5.1.91"
typescript: "npm:^5.0.4"
uuid: "npm:^9.0.1"
@@ -7956,6 +7969,7 @@ __metadata:
"@node-saml/passport-saml": "patch:@node-saml/passport-saml@npm%3A5.1.0#~/.yarn/patches/@node-saml-passport-saml-npm-5.1.0-5d7df575fa.patch"
"@overleaf/access-token-encryptor": "workspace:*"
"@overleaf/codemirror-tree-view": "npm:^0.1.3"
"@overleaf/cypress-pnp-reporter": "workspace:*"
"@overleaf/dictionaries": "https://github.com/overleaf/dictionaries/archive/refs/tags/v0.0.3.tar.gz"
"@overleaf/eslint-plugin": "workspace:*"
"@overleaf/fetch-utils": "workspace:*"
@@ -8097,7 +8111,6 @@ __metadata:
csv-parse: "npm:^6.2.1"
csv-stringify: "npm:^6.7.0"
cypress: "npm:15.20.1"
cypress-multi-reporters: "npm:^2.0.5"
cypress-plugin-tab: "npm:^1.0.5"
d3: "npm:^3.5.16"
daterangepicker: "npm:2.1.27"