* using CLSI logic for fetching the project contents and skip the .zip export
* Use unique conversion directory for project-to-docx export to avoid corrupting the shared compile
directory when a compile runs concurrently
* Remove X-Accel-Buffering header — not needed as CLSI does not run behind nginx
* moving log before sending the data
* Return CLSI stream directly instead of buffering to disk on web
Previously convertProjectToDocx wrote the CLSI response to a temp file
on disk, then the controller read it back to stream to the client.
Now the stream is returned directly and piped to the response,
avoiding unnecessary disk I/O on the web server.
* Use href redirect for docx export instead of fetching blob into memory
* making functions and files more generic so they can be used in future for other documents exports as well
* adding export-docx split test
* adding unit tests
* adding cypress E2E test
* format:fix
* renaming the route to download from convert
* adding new icon for export docx button
* format:fix
* remove unused showExportDocumentErrorToast export and adding guard against invalid Content-Length header from CLSI
* format:fix
* refactor(clsi): move promisify(parse) into RequestParser
* refactor: generic conversion endpoint with type as route
param
* refactor: use type→extension map for validated conversion types
* refactor(clsi): remove --standalone flag and fix rejection test
* fixing the href in cypress test
* renaming function
* adding type to Metrics.inc
* fix: rename exportProjectDocument, add WithLock wrapper and metrics type label
* format:fix
* fix: hide docx export from anonymous users and add WithLock wrapper
* format fix
* remove redundant Content-Length validation from DocumentConversionManager
* format:fix
* removing trailing icon
GitOrigin-RevId: e9764fefac2c4b625d23be9e942ea4a8b283c70d
The req.body.rootResourcePath has been shipped three weeks ago, so it's
unlikely to trip up stale editor sessions.
For now, only block node-fetch and log the rest.
GitOrigin-RevId: 541189675f68fdcab09f4b409b4143024a29f94a
* [web] migrate compile of templates/example project to history mode
* [saas-e2e] give clsi-cache more time to process the jobs
* [web] add comment on in-memory project update
Co-authored-by: Brian Gough <briangough@users.noreply.github.com>
* [saas-e2e] add retries to clsi-cache check
* [saas-e2e] remove spurious retry trigger
* [saas-e2e] remove broken assertion
---------
Co-authored-by: Brian Gough <briangough@users.noreply.github.com>
GitOrigin-RevId: 8766662cbe0ab5520c5c79d86fa6f774e012b3da
* [web] gracefully handle broken histories when compiling from history
* [web] trim down schema
GitOrigin-RevId: 97d59b31eb25644d7de1194a45281def6982b130
* [web] enable clsi-cache for all compiles for 20min during outages
* [web] populate clsi-cache when compile from history is enabled
* [web] hide history snapshot from frontend
GitOrigin-RevId: e46c8b5ab401ebdc2407e44eb95439e4d43400da
* [saas-e2e] test gallery templates with binary file
* [rails] add make target for fixing rubocop errors
* [rails] migrate compiles of conversions/submissions to history mode
* [rails] forward version to clsi request
* [rails] trim down compile request
* [saas-e2e] source v1 secrets after make install
GitOrigin-RevId: 65269e1df1051c9f3b4f1813d2e9dcf32a01be50
* [saas-e2e] port history tests to old history
* [web] fix incremental compile from history for old history ids
* [saas-e2e] tweak return type
GitOrigin-RevId: 2c89b570647c292c720cd0d02b6188f66e3e3a69
* [clsi] initial implementation of compile from history
* [clsi] copy changes
* [saas-e2e] extend test case with nested folder
* [saas-e2e] add test case for tracked changes
* [web] fix accumulating changes from multiple chunks
* [web] optimize size check for compile request payload
* [clsi] deduplicate globalBlobs
* [clsi] add validation for request body details
* [clsi] add metrics for compile from history
* [clsi] download binary files concurrently
* [clsi] skip download of empty file blob
* [clsi] break down e2e compile time metric by compileFromHistory
GitOrigin-RevId: 0dadef93e89d8a172c35cb130a1042d9d1bec42a
* [web] increase timeout for downloads from clsi
* [web] detach request timeout from streaming timeout
* [web] align timeout for clsi-cache downloads with clsi downloads
* [web] destroy stream when bailing out early from aborted requests
GitOrigin-RevId: cc4cad9d684214b2eee0f5fc43513e430ceb0977
* [monorepo] switch all output file reads to clsi-nginx
* [clsi-lb] allow gallery download requests
* [terraform] clsi: use nginx.conf from clsi service
* [clsi] fix flakey tests
* [clsi] replace alias with rewrite and root in nginx config
* [k8s] clsi-lb: expose download port on internal service
* [web] add explicit endpoint for downloading all output files
Serve the output.zip endpoint from clsi.
* [clsi] fix regex for latexqc submission ids
Previously, we only handled template submission ids.
GitOrigin-RevId: 6c3b21b01ec41ae767530b14aac31fbe3d640dd5
* [monorepo] remove PII and variables from error messages
Exclusions:
- scripts
- tests
- fuzzing
- SplitTestManager (messages are sent to admin frontend)
- Group setup (we may want an error per unique tuple)
- sharejs (unused types; text type errors are shadowed already)
- history-v1 error messages that are used by the ErrorRecorder
- errors that flag issues with configuration/call signatures
I've used these search terms for finding unwanted error messages:
- new Error(`
- new Error\(\n\s+` (regex search)
- new OError(`
- new OError\(\n\s+` (regex search)
* [web] throw NotFoundError from ProjectLocator
* [github-sync] fix OError.tag call in script
Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>
* [templates] revert changes to test client
---------
Co-authored-by: Jessica Lawshe <jessica.lawshe@overleaf.com>
GitOrigin-RevId: 736857a4fc5d9bfb0f8cb03e0f004eda87e5a220
* [monorepo] fix downloads from non-sharded clsi-cache
* [web] check some compile from cache options server-side
* [web] integrate clsi-cache into download pdf from project dashboard
* [web] remove frontend tests for server-side validation
* [web] remove unused fetch mock
* [web] use helper that adds polyfill for AbortSignal.any()
* [web] upgrade fetch-mock to fix leaking AbortSignal
* [web] do not add an extra timeout to clsi-cache request
The web backend service has a low timeout already.
GitOrigin-RevId: a90984b92f5d4f24005db5a09f2c5d2424436886
* [web] override timeout for a specific template
* [web] fix junit integration for vitest
* [web] consolidate on JUNIT_ROOT_SUITE_NAME
* [web] fix unit test
GitOrigin-RevId: 3ba865e4b9c090c7bf8e07dc3d81e9bcee20d821
Stage timeouts:
- frontend waits 5s
- web/clsi waits 4s
- clsi-cache waits 3s
This should ensure that the frontend can receive a valid response after
any of the backend requests failed.
The circuit breaker will remain closed for TIMEOUT + jitter of 0-3 times
the TIMEOUT of the respective service. This should avoid the bulk of
traffic to fail and occasionally issue retries without hammering the
instances while down.
Also do not try the next backend when the abort signal has expired.
GitOrigin-RevId: d612125616a9e416beff2f4c6d7f30066b5b9d6d
* [web] add clsi-cache prompts
* [web] add new editor variant to segmentation
* [web] add tests for useNewEditorVariant
* [web] adjust start of using clsi-cache in split-test
GitOrigin-RevId: c9c5b1eff2ceefb65ef82516d9074cb971cb4c48
* [clsi] add stats and timings to compile response from clsi-cache
* [clsi] set downloadedFromCache when previously downloaded for synctex
Assumption: every compile will emit an output.log. When the output.log
is missing, but the output.synctex.gz exists, it must have been
downloaded from the cache.
GitOrigin-RevId: 41ea34880931e3c43dda3bc9eb26c0d02054894d
Davinder Singh