Commit Graph

146 Commits

Author SHA1 Message Date
Andrew Rumble 07c827e9fd Merge pull request #29928 from overleaf/ar-last-infrastructure-conversions
[web] last infrastructure conversions

GitOrigin-RevId: ad1aff9b7df0610ed0303157d9e2c8032f32c02b
2025-11-28 09:05:56 +00:00
Andrew Rumble 18f44866e5 Merge pull request #29919 from overleaf/revert-29795-ar-last-infrastructure-conversions
Revert "[web] last infrastructure conversions"

GitOrigin-RevId: 48dc64553012afb5d2db4b2eb9c9898489b7e5ef
2025-11-27 09:05:54 +00:00
Andrew Rumble d748d8d606 Merge pull request #29795 from overleaf/ar-last-infrastructure-conversions
[web] last infrastructure conversions

GitOrigin-RevId: 68aa11625a9bc6d0d5324ecd95bb5ac52af8ee96
2025-11-27 09:05:30 +00:00
Andrew Rumble beb6f6d484 Merge pull request #29597 from overleaf/ar-last-features-esm-conversion
[web] last features esm conversion

GitOrigin-RevId: a35ab995bf654f1cdfe0e0062d8806761ecccf2d
2025-11-21 09:05:36 +00:00
Jakob Ackermann 3586b37491 [web] Remove clsi-cache-prompt/survey, split-tests and events (#29510)
* [web] Remove clsi-cache-prompt/survey and split-tests

* [web] Remove initial 50/50 clsi-cache split-test

* [web] Remove synctex-downloaded-from-cache event

* [web] Remove fallback-to-clsi-cache event

* [saas-e2e] fix tests with clsi-cache enabled

GitOrigin-RevId: b5cf2ab073dc866fe398b81fd5afe46422134c80
2025-11-05 09:06:59 +00:00
Jakob Ackermann a80b69abe5 [web] move free compiles up one bracket (#29411)
GitOrigin-RevId: 613b703d677d91110cd6dcf085e9291008e05f5c
2025-11-03 09:05:23 +00:00
Jakob Ackermann 3a206074f2 [web] move premium compiles up one bracket (#29410)
GitOrigin-RevId: 53423f855b2c215f6cc926b480422b2ce3477a74
2025-10-31 09:06:32 +00:00
Jakob Ackermann f3f6a8a8e0 [web] add clsi-cache prompts (#29281)
* [web] add clsi-cache prompts

* [web] add new editor variant to segmentation

* [web] add tests for useNewEditorVariant

* [web] adjust start of using clsi-cache in split-test

GitOrigin-RevId: c9c5b1eff2ceefb65ef82516d9074cb971cb4c48
2025-10-24 08:05:42 +00:00
Andrew Rumble 3d3be18f57 Features ESM conversion
GitOrigin-RevId: d659326723a90ac0789f4f7acc7f00aa9eaa63e2
2025-10-21 08:05:51 +00:00
Andrew Rumble e909995ce0 Rename features files
GitOrigin-RevId: fc4ce037343b02d002fa07b7cdc283f2d7d4f89b
2025-10-21 08:05:47 +00:00
Jakob Ackermann ad3e8bd65d [web] fix clearing of clsi persistence during migration to new key (#29172)
GitOrigin-RevId: 1d9e6eb87f8f27792fdc866e7dcc29b47e1b2547
2025-10-17 08:06:06 +00:00
Jakob Ackermann 706971ef41 [web] shard compile server persistence by compileBackendClass (#29156)
* [web] shard compile server persistence by compileBackendClass

* [web] make prettier happy

GitOrigin-RevId: d7cf8bde07e7110053d9d7531c007111af5cc46b
2025-10-17 08:06:02 +00:00
MoxAmber 986a728569 Merge pull request #28895 from overleaf/as-promisify-clsiformatchecker
Promisify ClsiFormatChecker and tests

GitOrigin-RevId: 89196708b97ff936660cc50a0b31f549d9723c4b
2025-10-17 08:05:40 +00:00
Jakob Ackermann fbea855690 [web] switch dark compiles to n4 (#29133)
* [clsi-perf] run compiles on any backend with prefix clsi_compile

* [clsi-lb] add n4 backend

* [terraform] clsi: add n4 instances

* [web] switch dark compiles to n4

* [terraform] clsi: remove c4d instances

* [terraform] clsi: scale up n4 instances

GitOrigin-RevId: 8f2d98ef6c3c503366c92c4f148a8dd7394cebd2
2025-10-16 08:07:21 +00:00
Antoine Clausse 33e63d79fc Merge pull request #28584 from overleaf/ac-some-web-esm-migration-5
[web] Convert some Features files to ES modules (part 5)

GitOrigin-RevId: 0cad67f9afe0095e2b066bf2f4d3717c00540dab
2025-10-08 08:06:15 +00:00
Antoine Clausse 46715191e3 Merge pull request #28544 from overleaf/ac-some-web-esm-migration-4
[web] Convert some Features files to ES modules (part 4)

GitOrigin-RevId: cf11a7584e39c4d4de08e2f924240e488a4066c4
2025-09-24 08:05:58 +00:00
Antoine Clausse 71f0b28a84 [web] Convert some Features files to ES modules (part 3) (#28494)
* Rename files to mjs

* Rename test files to mjs

* Update CODEOWNERS

* Update files to ESM

* Update test files to ESM

* Update RestoreManager.test.mjs

* Remove unused `AdminAuthorizationHelper` mock and stub

* Remove unnecessary return

GitOrigin-RevId: 2b9ef126de1d8964afbc6e5641cca36712655866
2025-09-17 08:05:02 +00:00
Andrew Rumble 7f908915aa Merge pull request #28367 from overleaf/ar-migrate-routermjs-to-zod
[web] migrate router.mjs to zod

GitOrigin-RevId: d3fc21a11351f3e2deb5011cd1beeb86286a300b
2025-09-16 08:05:26 +00:00
Jakob Ackermann 16422f972b Merge pull request #28265 from overleaf/jpa-double-compile
[web] add c2d -> c4d double compile test

GitOrigin-RevId: 186cfa867d3db5786c6d4888dfe1ca2f46a5bb0c
2025-09-04 08:05:10 +00:00
Jakob Ackermann 03320bb377 Merge pull request #28264 from overleaf/jpa-synctex
[web] use standard request handling for SyncTeX requests

GitOrigin-RevId: ad5ba1834241d5939675f2533940ade741fc5abf
2025-09-04 08:05:06 +00:00
Jakob Ackermann a85b2b34f5 Merge pull request #28233 from overleaf/jpa-clsi-lb-clear
[web] clear clsi server id/output files in both clsi-lb backends

GitOrigin-RevId: b73ccc2017800d9abbb8f571efeb34f51c9f96c1
2025-09-04 08:04:58 +00:00
Jakob Ackermann bd4130c147 [filestore] remove user files endpoints (#28125)
* [filestore] remove user files endpoints

* [web] remove user files integration for filestore

GitOrigin-RevId: 565fa68a659c07420ee6141d0f276b4e4d2972e0
2025-09-02 08:04:52 +00:00
MoxAmber fc5ae92be1 Merge pull request #27677 from overleaf/as-compile-timeout-enforcement
[web] 10s Compile Timeout - Enforcement Phase

GitOrigin-RevId: 3930eb376cc1293409259e073032218e09d5270e
2025-08-20 08:06:01 +00:00
Jakob Ackermann 29045b9085 [web] download binary files in clsi from filestore via new endpoints (#27505)
* [history-v1] use String.padStart instead of lodash.padStart

* [web] download binary files in clsi from filestore via new endpoints

* [server-ce] tests: Cypress.env() is parsing boolean values

* [server-ce] tests: run history migration as root

GitOrigin-RevId: bdf6c0e542531ccc4b3f13d2ed68ca0d31e580e9
2025-07-31 08:05:28 +00:00
Eric Mc Sween 17d1b0b8d6 Merge pull request #25646 from overleaf/em-ds-mobile-app-compile
Compile endpoint for the DS mobile app

GitOrigin-RevId: 2fd9f4a6e8c2ed4ee868b0c1293f6760b9d113c8
2025-05-21 08:06:36 +00:00
Jakob Ackermann ec1bd69605 [clsi-cache] remove non sharded instances (#25645)
* Revert "[clsi-cache] only use sharding from updated project editor tabs (#25326)"

This reverts commit 1754276bed3186c0536055c983e32476cc90d416.

* [clsi-cache] remove non sharded instances

GitOrigin-RevId: aa3ac46140dfc1722a3350cf7071e5b11af61199
2025-05-16 08:05:02 +00:00
Antoine Clausse bc4c3c4ef8 [web] Promisify ClsiCookieManager and CompileController (reapply again) (#25280)
* Reapply "[web] Promisify ClsiCookieManager and CompileController (reapply and fix)"

This reverts commit 98cb9127ff2b7c7c347c560766f749265d712490.

* Fix: Use query parameters correctly (!!)

* Add unit test on `checkIsLoadSheddingEvent`

* Remove interference between tests: rename to `ClsiCookieManager2` when it's re-sandboxed

* Add test: 'should report "cycle" when other is UP'

GitOrigin-RevId: 3146b149954b908830226cb03b51d9adfa08ec2e
2025-05-07 08:07:07 +00:00
Jakob Ackermann 6881ba956a [clsi-cache] only use sharding from updated project editor tabs (#25326)
GitOrigin-RevId: 1754276bed3186c0536055c983e32476cc90d416
2025-05-07 08:06:39 +00:00
Antoine Clausse 666481d8b2 Merge pull request #25221 from overleaf/revert-25207-ac-promisify-compile-controller-2
Revert "[web] Promisify ClsiCookieManager and CompileController (reapply and fix)"

GitOrigin-RevId: 4495b0fdee22c9c2a7eb39250aef498883389427
2025-05-01 08:06:55 +00:00
Antoine Clausse 61db35ac8f Merge pull request #25207 from overleaf/ac-promisify-compile-controller-2
[web] Promisify ClsiCookieManager and CompileController (reapply and fix)

GitOrigin-RevId: 0737f30c24bf92b33327dc7d0e015ac2cd7d751d
2025-05-01 08:06:47 +00:00
Antoine Clausse d7d60f9d4c Merge pull request #25200 from overleaf/revert-25023-ac-promisify-compile-controller
Revert "[web] Promisify ClsiCookieManager and CompileController"

GitOrigin-RevId: 190ee8d2be23687f092e762c5199a34bcdf37cf9
2025-05-01 08:06:00 +00:00
Antoine Clausse 73476180d4 [web] Promisify ClsiCookieManager and CompileController (#25023)
* Refactor `ClsiCookieManager` to use async/await

* Refactor `CompileController` to use async/await

* Fix tests: CompileControllerTests.js

* Fix tests: ClsiCookieManagerTests.js

* Fixup: keep old object key (`serverId`->`clsiServerId`)

* Prefix non-express methods with an underscore

* Use async versions of `rclient` methods

* Fix: `canContinue` must be `false` when rate limited (!!)

* Remove unused `ClsiCookieManager.getCookieJar`

* Remove bad comments

* Fix linting

* Replace `request` by `@overleaf/fetch-utils`

* Replace `callsFake` by `resolves`

* Update `catch` block after request->fetch update: check status code

* Re-add timeout of 30s (was removed in 3df75ab5ccc8a63bd69a6a6e6052ef0451b76220)

* `npm run format:fix`

* Don't throw on 4xx errors; keep functionality similar to the current implementation using `request`.

GitOrigin-RevId: ef161f6c252d13f407e9cf28029b62061d6b992f
2025-05-01 08:05:42 +00:00
Andrew Rumble 2ad9f36706 Promisify tests
GitOrigin-RevId: 6f413f4c5ef8d034b4e94afacdf2d7b43c3a8830
2025-04-29 08:05:18 +00:00
Jakob Ackermann f0edc7ba00 [web] update the projects lastUpdated timestamp when changing file-tree (#24867)
* [misc] freeze time before any other unit test setup steps

Freezing it after other work (notably sandboxed-module imports) will
result in flaky tests.

* [web] update the projects lastUpdated timestamp when changing file-tree

GitOrigin-RevId: b82b2ff74dc31886f3c4bd300375117eead6e0cd
2025-04-16 08:05:14 +00:00
Jakob Ackermann b538d56591 [clsi-cache] backend (#24388)
* [clsi-cache] initial revision of the clsi-cache service

* [clsi] send output files to clsi-cache and import from clsi-cache

* [web] pass editorId to clsi

* [web] clear clsi-cache when clearing clsi cache

* [web] add split-tests for controlling clsi-cache rollout

* [web] populate clsi-cache when cloning/creating project from template

* [clsi-cache] produce less noise when populating cache hits 404

* [clsi-cache] push docker image to AR

* [clsi-cache] push docker image to AR

* [clsi-cache] allow compileGroup in job payload

* [clsi-cache] set X-Zone header from latest endpoint

* [clsi-cache] use method POST for /enqueue endpoint

* [web] populate clsi-cache in zone b with template data

* [clsi-cache] limit number of editors per project/user folder to 10

* [web] clone: populate the clsi-cache unless the TeXLive release changed

* [clsi-cache] keep user folder when clearing cache as anonymous user

* [clsi] download old output.tar.gz when synctex finds empty compile dir

* [web] fix lint

* [clsi-cache] multi-zonal lookup of single build output

* [clsi-cache] add more validation and limits

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [clsi] do not include clsi-cache tar-ball in output.zip

* [clsi-cache] fix reference after remaining constant

Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>

* [web] consolidate validation of filename into ClsiCacheHandler

* [clsi-cache] extend metrics and event tracking

- break down most of the clsi metrics by label
  - compile=initial - new compile dir without previous output files
  - compile=recompile - recompile in existing compile dir
  - compile=from-cache - compile using previous clsi-cache
- extend segmentation on compile-result-backend event
  - isInitialCompile=true - found new compile dir at start of request
  - restoredClsiCache=true - restored compile dir from clsi-cache

* [clsi] rename metrics labels for download of clsi-cache

This is in preparation for synctex changes.

* [clsi] use constant for limit of entries in output.tar.gz

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [clsi-cache] fix cloning of project cache

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 4901a65497af13be1549af7f38ceee3188fcf881
2025-04-10 08:05:17 +00:00
Jakob Ackermann 13bf214a3c [web] generate clsi buildId ahead of fetching project content (#24337)
* [web] generate clsi buildId ahead of fetching project content

The buildIds timestamp component will be used for cache invalidation.

* [clsi] strict validation for buildId

* [clsi] validate buildId parameter

GitOrigin-RevId: 88e8b2d48e78fa137b6dca7f2e6b93bbcf88a777
2025-03-24 10:46:02 +00:00
Jakob Ackermann 195a12a3dc [web] fix download of output.pdf when creating Server Pro template (#23988)
GitOrigin-RevId: 899b9bfd54af8d3f345393068663d769ec3a869b
2025-03-03 09:05:26 +00:00
Brian Gough 2ce82fabab Merge pull request #22533 from overleaf/ar-only-use-history-for-blobs-when-enabled
[web] only use history for blobs when enabled

GitOrigin-RevId: 010983e9b29657d4c594e03945dca5700577bf0a
2025-01-10 09:05:32 +00:00
Brian Gough bb1072a3a4 Merge pull request #22219 from overleaf/jpa-handle-missing-hash
[web] gracefully handle missing hash when compiling

GitOrigin-RevId: a9b121716319117ead84b7fed9a67098f63fd9a7
2024-11-29 09:05:43 +00:00
Jakob Ackermann d19c5e236f Merge pull request #22208 from overleaf/jpa-clsi-hash
[misc] clsi: read files from history-v1 with fallback to filestore

GitOrigin-RevId: c54bb128780198c14e7a63818f39fad62ce65d4e
2024-11-29 09:05:39 +00:00
Andrew Rumble d1a58e6b77 Merge pull request #18538 from overleaf/ar-add-download-all-link-in-web
[web] add download all link for output files

GitOrigin-RevId: 3d574d75d53e577cb0f8fd3caa4f757d9e1b7889
2024-06-03 08:04:07 +00:00
Andrew Rumble 9c3d9ef590 Merge pull request #17935 from overleaf/ar-refactor-compile-async
[web] make CompileManager async

GitOrigin-RevId: 617bde1f429fa9aafc7d4bf4ec628b2a22386b19
2024-04-30 08:04:43 +00:00
Jakob Ackermann a540754f6e Merge pull request #18116 from overleaf/jpa-bulk-replace-localhost
[misc] bulk replace localhost with 127.0.0.1

GitOrigin-RevId: d238f3635302e8ff5500d611108c4d1bef216726
2024-04-26 08:04:39 +00:00
Antoine Clausse 616bd0df16 Merge pull request #17925 from overleaf/jpa-ac-20s-timeout-migration-logging-tweaks
[web] timeout 20s split-test teardown cleanups (remove override, remove tests, ...)

GitOrigin-RevId: 8d8c44539cf45d0f5142f84cf8372cecda3bf77a
2024-04-17 08:04:30 +00:00
Antoine Clausse d35204033f Merge pull request #17909 from overleaf/ac-tear-down-compile-timeout-tests-2
[web]  Remove split-tests `compile-backend-class*` and `compile-timeout-20s*` (attempt 2)

GitOrigin-RevId: 5658f2977d3e7089eec5bbe7a33eee81c153e41d
2024-04-16 08:04:08 +00:00
Antoine Clausse 491bc2628d Merge pull request #17899 from overleaf/revert-17700-ac-tear-down-compile-timeout-tests
Revert "[web] Remove split-tests `compile-backend-class*` and `compile-timeout-20s*`"

GitOrigin-RevId: d5070ced06adbd435e782a44b7ef767e395bd6a0
2024-04-15 08:04:37 +00:00
Antoine Clausse 2dd10c7fee [web] Remove split-tests compile-backend-class* and compile-timeout-20s* (#17700)
* Remove split-tests of `compile-timeout-20s` and `compile-timeout-20s-existing-users`

* Remove `NEW_COMPILE_TIMEOUT_ENFORCED_CUTOFF` variables

* Revert timeout override `60` -> `20`

* Update settings.overrides.saas.js: `compileTimeout: 20`

* Remove `compile-backend-class-n2d`

* Remove `force_new_compile_timeout`

* Remove `showNewCompileTimeoutUI`

* Remove `compileTimeChanging`

* Simplify code by removing segmentation object

* Remove `CompileTimeoutChangingSoon`

* Remove `user.features.compileTimeout = '20 (with 10s prompt)'`

* Remove `CompileTimeWarning`

* Remove `TimeoutUpgradePrompt` (old)

* Remove `compile-backend-class`

* Remove unused translations

* Update tests

* Fix: Show `CompileTimeout` even if `!window.ExposedSettings.enableSubscriptions`

* Create script to migrate users to 20s compileTimeout

* migration script: exclude `compileTimeout: 20` from the match

* migration script: use `batchedUpdate`

* Remove `showFasterCompilesFeedbackUI` and `FasterCompilesFeedback`

Helped-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Remove `_getCompileBackendClassDetails`, simplify definition of `limits` object

* Remove `Settings.apis.clsi.defaultBackendClass`

* Remove unnecessary second scan of the whole user collection in dry mode

* Override `timeout` to 20 for users having `compileGroup === 'standard' && compileTimeout <= 60`

* Remove second `logCount`: re-run the script in dry-mode if you want to see that count

* Use secondary readPreference when counting users

* Fix script setup and exit 0

* Fix: Remove `user.` from query path!

* Add acceptance test on script migration_compile_timeout_60s_to_20s.js

GitOrigin-RevId: 3cb65130e6d7fbd9c54005f4c213066d0473e9d8
2024-04-15 08:04:24 +00:00
Davinder Singh 5c5a01777c Making getAnalyticsIdFromMongoUser private to UserAnalyticsIdCache (#16125)
* Making getAnalyticsIdFromMongoUser private to UserAnalyticsIdCache

* renaming userAnalyticsIdCache to UserAnalyticsIdCache

* deleting function _getAnalyticsIdFromMongoUser

* renaming userAnalyticsIdCache to UserAnalyticsIdCache part 2

* format:fix

* adding upperCamelCase

* capital case first letter

GitOrigin-RevId: 2e9c18c544b8cffb53838aed56e1ef16979606a5
2024-01-08 09:05:31 +00:00
Thomas 3cd0cefd30 Gradual rollout of new compile timeout to existing users (#15478)
* Rollout compile timeout to existing users

* Update copy for compile timeout messages

GitOrigin-RevId: bada02efb3fc506ae6e35bdb7871a58ef4295cbe
2023-10-31 09:04:11 +00:00
Jakob Ackermann 347da4e6bb Merge pull request #15247 from overleaf/jpa-submission-backend-class
[web] use n2d instances for compiling submissions

GitOrigin-RevId: 57eb27af230b994c3816c0476673ea558226ac1e
2023-10-25 08:04:03 +00:00