Commit Graph

931 Commits

Author SHA1 Message Date
Jakob Ackermann
5fe9d3d6e9 [monorepo] migrate monorepo checks to Jenkins (#28765)
GitOrigin-RevId: ebc3db95cebe99226f8de0f66c1830eb5d78e26c
2025-10-01 08:06:28 +00:00
Jakob Ackermann
e05ec0321b [server-pro] extend hotfix 5.5.5: avoid creating user_files directory (#28738)
* [WIP] restore 100_make_overleaf_data_dirs.sh

* [WIP] patch for hotfix 5.5.5

* Revert "[WIP] patch for hotfix 5.5.5"

This reverts commit cc83a526ecd86ed7b1a0e061b074d2fb15d4d672.

* Revert "[WIP] restore 100_make_overleaf_data_dirs.sh"

This reverts commit e9b0597fe8451e9ef1deaf7ed31790f7d7fc996e.

* [server-pro] extend hotfix 5.5.5: avoid creating user_files directory

* [server-ce] run binary files migration in CI

GitOrigin-RevId: ac5bb86df92b441201c69dfac6b302058b501d8e
2025-09-29 08:07:01 +00:00
Jakob Ackermann
a51ba7a7d5 [server-ce] tests: add junit integration for Jenkins (#28732)
GitOrigin-RevId: b4c96ad71a02bf57c184a8031f475e3d1548c58a
2025-09-29 08:06:46 +00:00
Jakob Ackermann
2db902bdd2 [monorepo] remove debug logging for mongo containers (#28716)
GitOrigin-RevId: 5034cadca5933c6c29c190aa171f52b71216ec6d
2025-09-29 08:06:37 +00:00
Miguel Serrano
dc1203e2bd [CE/SP] Rebuild base image (#28727)
Update epoch ENV to trigger a rebuild

GitOrigin-RevId: 5c46547cc1699e3ad916477107a7c035984ac423
2025-09-29 08:06:32 +00:00
Jakob Ackermann
3b232ca419 [server-pro] add caching to docker build (#27887)
GitOrigin-RevId: 26a5f8b66f710afec3aedb5e55d205095d6e0a7d
2025-09-29 08:05:41 +00:00
Rebeka Dekany
3b5ea89a1c Update loading spinner status (#28177)
* Update test for the loading spinner component

* Create a story for the loading spinner component

* Move role and use CSS for spacing instead

* Add a classname prop

* Reuse LoadingSpinner

* Use OLSpinner instead of Spinner

* Use data-testid since status role was moved

* Wait for journals to load

* Use `isLoading` prop instead and fix the button's height

* Use `isLoading` prop instead

* Use LoadingSpinner instead and remove spacing

* Update test for the loading spinner component

* Use `isLoading` prop instead

* Add aria-describedby to layout button for processing state

* Replace `spinner` to `ol-spinner`

* Scope status

* Remove redundant `div.loading`

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 8f43b991f8f458b2abd5a4661913ac9b972d892a
2025-09-26 08:05:46 +00:00
Rebeka Dekany
a8734e191e Update learn wiki e2e tests for current navigation structure (#28685)
GitOrigin-RevId: d7d87bf712e0ed399c5fe3a7108db5c47d641ce8
2025-09-25 08:05:49 +00:00
Jakob Ackermann
a6e9a5c7e9 [monorepo] run prettier on Jenkinsfiles (#28565)
* [monorepo] consolidate .editorconfig files at the root

* [monorepo] run prettier on Jenkinsfiles

* [saas-e2e] increase timeout for XeTeX compile

GitOrigin-RevId: 48aa82f7c81611899837753ae92c7732998d4ca5
2025-09-19 08:07:49 +00:00
Jakob Ackermann
66934586b1 [monorepo] reset Jenkins pipeline vars (#28573)
GitOrigin-RevId: 9523895a16262fc7d9b8eecc716e72cdb6cb5d1a
2025-09-19 08:07:44 +00:00
Jakob Ackermann
74746a096c [monorepo] rebuild lezer output as part of 'make install' (#28545)
* [monorepo] rebuild lezer output as part of 'make install'

* [monorepo] run 'make install' with high concurrency in Jenkins

* [monorepo] rebuild lezer parser output once

* [monorepo] headless npm run

* [web] update docs for lezer

GitOrigin-RevId: f385bb8a9cdab1a6bfc28a8843b5128ceec364ec
2025-09-18 08:06:02 +00:00
Miguel Serrano
f1dc6ec155 Upgrade aws-sdk to v3 (re-attempt) (#28410)
* Upgrade `aws-sdk` to v3

Also vendored nodemailer-ses-transport

* Moved default region to CE settings

* Ensure timeout is added to `requestHandler` and `region` is populated

* fix unsigned header in signed URL request

The x-amz-acl header is not needed when using the signed request,
as ACL are already defined when creating the signed URL in
PutObjectCommand constructor.

* Add default AWS region for latexqc service

* remove unnecessary region in compose files

* Use AWS_REGION=us-west-2 for history-v1

* prevent retries uploading streams with PutObjectCommand

* Remove AWS SDK JS message suppression

GitOrigin-RevId: 6fda6f02160023ffed76143397bbd965f86a9509
2025-09-18 08:05:22 +00:00
Jakob Ackermann
1fe750910e [monorepo] add 3 retries to all the Jenkins pipelines (#28516)
GitOrigin-RevId: d52cb329481586daddade00cecc91875e9ae0de6
2025-09-17 08:05:38 +00:00
Jakob Ackermann
a13fca15d3 [saas-e2e] harden recompiling (#28514)
* [saas-e2e] harden recompiling

* [server-ce] import changes from SaaS e2e tests

GitOrigin-RevId: 9b86ed864457b068b8dded94a7e12658a047aac2
2025-09-17 08:05:34 +00:00
Antoine Clausse
cfcfa6cb74 Remove outdated overrides (sha.js, form-data) (#28258)
* Update `isomorphic-git` and remove `sha.js` overrides

It was fixed in https://github.com/isomorphic-git/isomorphic-git/pull/2190

* `bin/npm update @cypress/request` and remove override

Bumps `form-data` to `~4.0.4` which is safe per https://github.com/overleaf/internal/security/dependabot/1533

Now it has the safe versions (2.5.5 and 4.0.4):
```
$ npm ls form-data

overleaf@ /Users/aclausse/Code/internal
├─┬ @overleaf/analytics@ -> ./services/analytics
│ └─┬ request@2.88.2 overridden
│   └── form-data@2.5.5 overridden
├─┬ @overleaf/fetch-utils@0.1.0 -> ./libraries/fetch-utils
│ └─┬ @types/node-fetch@2.6.11
│   └── form-data@4.0.4 overridden
├─┬ @overleaf/metrics@4.2.0 -> ./libraries/metrics
│ └─┬ @google-cloud/profiler@6.0.3
│   └─┬ @google-cloud/common@5.0.2
│     └─┬ retry-request@7.0.2 overridden
│       └─┬ @types/request@2.48.12 overridden
│         └── form-data@2.5.5 overridden
├─┬ @overleaf/saas-e2e@ -> ./tools/saas-e2e
│ └─┬ mailtrap@3.4.0
│   └─┬ axios@1.8.4
│     └── form-data@4.0.4 deduped
├─┬ @overleaf/tpdsworker@ -> ./services/tpdsworker
│ └─┬ chai-http@4.4.0
│   └─┬ superagent@8.1.2
│     └── form-data@4.0.4 deduped
├─┬ @overleaf/web@ -> ./services/web
│ └─┬ jsdom@19.0.0
│   └── form-data@4.0.4 deduped
├─┬ latexqc@0.0.1 -> ./services/latexqc
│ └─┬ vitest@3.1.2
│   └─┬ jsdom@20.0.3
│     └── form-data@4.0.4 deduped
└─┬ overleaf-editor@1.0.0 -> ./services/history-v1
  └─┬ swagger-tools@0.10.4 overridden
    ├─┬ json-refs@3.0.15
    │ └─┬ path-loader@1.0.12
    │   └─┬ superagent@7.1.6 overridden
    │     └── form-data@4.0.4 deduped
    └─┬ superagent@3.8.3 overridden
      └── form-data@2.5.5 overridden
```

* `bin/npm update @types/request` and remove override

Bumps `form-data` to `2.5.5` which is safe per https://github.com/overleaf/internal/security/dependabot/1537

* Remove `form-data` overrides that aren't necessary

```
$ npm ls form-data

overleaf@ /Users/aclausse/Code/internal
├─┬ @overleaf/analytics@ -> ./services/analytics
│ └─┬ request@2.88.2 overridden
│   └── form-data@2.5.5 invalid: "~2.3.2" from node_modules/request
├─┬ @overleaf/fetch-utils@0.1.0 -> ./libraries/fetch-utils
│ └─┬ @types/node-fetch@2.6.11
│   └── form-data@4.0.4
├─┬ @overleaf/metrics@4.2.0 -> ./libraries/metrics
│ └─┬ @google-cloud/profiler@6.0.3
│   └─┬ @google-cloud/common@5.0.2
│     └─┬ retry-request@7.0.2
│       └─┬ @types/request@2.48.13
│         └── form-data@2.5.5
├─┬ @overleaf/saas-e2e@ -> ./tools/saas-e2e
│ ├─┬ cypress@13.13.2
│ │ └─┬ @cypress/request@3.0.9
│ │   └── form-data@4.0.4 deduped
│ └─┬ mailtrap@3.4.0
│   └─┬ axios@1.8.4
│     └── form-data@4.0.4 deduped
├─┬ @overleaf/tpdsworker@ -> ./services/tpdsworker
│ └─┬ chai-http@4.4.0
│   └─┬ superagent@8.1.2
│     └── form-data@4.0.4 deduped
├─┬ @overleaf/web@ -> ./services/web
│ └─┬ jsdom@19.0.0
│   └── form-data@4.0.4 deduped
├─┬ latexqc@0.0.1 -> ./services/latexqc
│ └─┬ vitest@3.1.2
│   └─┬ jsdom@20.0.3
│     └── form-data@4.0.4 deduped
└─┬ overleaf-editor@1.0.0 -> ./services/history-v1
  └─┬ swagger-tools@0.10.4 overridden
    ├─┬ json-refs@3.0.15
    │ └─┬ path-loader@1.0.12
    │   └─┬ superagent@7.1.6
    │     └── form-data@4.0.4 deduped
    └─┬ superagent@3.8.3
      └── form-data@2.5.5
```

* Remove 2025-08-07-form-data-cypress.json

* Reapply the form-data override in request@2.88.2

GitOrigin-RevId: 89143de1a53226dc43fc474db443fc7d7698908a
2025-09-05 08:05:17 +00:00
Andrew Rumble
8f37ed1113 Merge pull request #27050 from overleaf/renovate-definitelytyped
[Platform] Update definitelyTyped

GitOrigin-RevId: 8ef840733942a909af6e6b1fee3e3fc01e0dc61d
2025-09-04 08:04:53 +00:00
Rebeka Dekany
97d8f73749 Update selectors to improve CE test stability (#28096)
* Update selectors to improve test stability

* Update selectors to improve test stability

* Use plain string matchers

* Fix test

* [monorepo] use plain string matchers everywhere

* [web] remove Kb/ prefix from title of learn wiki links

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 12e13c39822795338a3bee20236454f9948e6221
2025-09-02 08:04:56 +00:00
Jakob Ackermann
bd4130c147 [filestore] remove user files endpoints (#28125)
* [filestore] remove user files endpoints

* [web] remove user files integration for filestore

GitOrigin-RevId: 565fa68a659c07420ee6141d0f276b4e4d2972e0
2025-09-02 08:04:52 +00:00
Jakob Ackermann
2036947981 [server-pro] test: harden reconfiguring (#28193)
When reconfiguring fails, the config snapshot in the frontend is not
updated. When the next test uses the previous config, it will not
attempt to restore that config again, and instead use the other config.
Reset the config before making changes to ensure all tests get their
desired config, even when reconfiguring fails.

GitOrigin-RevId: 0939e709cf93da820a86fef2599940669138d728
2025-09-01 08:05:17 +00:00
Miguel Serrano
8948e40b22 Revert "Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade" (#28151)
This reverts commit 4989ae920d8b7fd9e79623947b7c40bcc2e56d92.

GitOrigin-RevId: 541d95bfeaa7ce820e8af67f646f013fe4fe5d21
2025-08-28 08:05:52 +00:00
Miguel Serrano
39381d808a Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade
Upgrade `aws-sdk` to v3

GitOrigin-RevId: 4989ae920d8b7fd9e79623947b7c40bcc2e56d92
2025-08-28 08:05:36 +00:00
Rebeka Dekany
74533fc401 Replace placeholders with labels (#26811)
* Replace placeholders for labels

* Remove redundant aria-label and update labels with `.visually-hidden`

* Change "Invite more members" to a label

* Fix helper text for group members and fix responsiveness

* Change error message to an error notification

* Use label and helper text instead of a placeholder

* Remove redundant label and use the placeholder text instead for the admin user searchbar

* Extract translations

GitOrigin-RevId: a504fda9779da82920b57c7b8aad38a8b027d09a
2025-08-26 08:05:02 +00:00
renovate[bot]
7dc7b322a8 [Platform] Update dependency isomorphic-git to ^1.33.0 from ^1.25.10 (#28073)
* [Platform] Update dependency isomorphic-git to ^1.33.0 from ^1.25.10

* Override sha.js version

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
GitOrigin-RevId: 56bf1ef4b6fa7043ea3609e46a21c36a651905b0
2025-08-22 08:06:11 +00:00
Jakob Ackermann
f9c011473b [web] print mongo logs on acceptance/unit test failure (#28034)
* [web] print mongo logs on acceptance/unit test failure

* [web] turn on logging for mongo again

* [monorepo] improve printing of mongo logs

Co-authored-by: Gernot Schulz <gernot.schulz@overleaf.com>

* [monorepo] fix waiting for mongo startup

---------

Co-authored-by: Gernot Schulz <gernot.schulz@overleaf.com>
GitOrigin-RevId: 713ffacd1f80b134ef3cb84a8f281660ef54e5d6
2025-08-22 08:06:06 +00:00
Andrew Rumble
6190005f9b [monorepo] Upgrade to node 22.18.0 (#27977)
* Upgrade to node 22.18.0

* Switch projects-data fixture to strip-types compatible form

GitOrigin-RevId: 4a7ec56f4a82395d2be8870526df0c679de38ab5
2025-08-22 08:05:17 +00:00
Brian Gough
f5dbbadf79 add option to disable link sharing (#27626)
* add option to remove link-sharing from backend

* restrict make link-sharing in the frontend based on capability

* extend e2e project-sharing tests to cover OVERLEAF_DISABLE_LINK_SHARING=true

* throw an error when link sharing is disabled in TokenAccessHandler

* throw errors when attempting to add users to projects with link sharing disabled

* Update server-ce/test/project-sharing.spec.ts

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* add tests for existing access when link sharing is disabled

* update tests to specify access restrictions for read-only and read-write link shared projects

* [web] block access to legacy public project with link-sharing disabled

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 5f194dbcb790e973e427c58a3a4a738a5dd74cb4
2025-08-20 08:05:33 +00:00
Jakob Ackermann
2c9ce619dd [monorepo] document jenkins pipelines (#27954)
* [saas-e2e] reset database when db-migration files were removed

* [monorepo] document jenkins pipelines

* [monorepo] trim docs for jenkins pipelines

GitOrigin-RevId: 783ad94f05b36f4057e3ac6cc8b8c40f850e5d2f
2025-08-19 08:06:02 +00:00
Jakob Ackermann
56f41c77ff [history-v1] add migrations record after full binary file migration (#27932)
* [history-v1] add migrations record after full binary file migration

* [server-pro] add hotfix 5.5.5

* [server-ce] test: build hotfix 5.5.5 and use it in tests

GitOrigin-RevId: fb84e5710c59f466a3305de5f32f78e0ac9ce15d
2025-08-19 08:05:54 +00:00
Jakob Ackermann
21f0490456 [server-ce] force cleanup of docker images (#27944)
GitOrigin-RevId: d515f191c660e382c16497544a6d13e434a3c9e2
2025-08-18 08:05:21 +00:00
Jakob Ackermann
e9bbc113a0 [server-pro] trust local nginx proxy running inside Server Pro/CE (#27719)
GitOrigin-RevId: ed9bb136effb1602aa0239b107c116fd6385ebbc
2025-08-15 08:05:26 +00:00
Jakob Ackermann
b6b6bf82bd [monorepo] try harder at timing out Jenkins builds (#27897)
GitOrigin-RevId: 9d9b675d4603da761b725ab0470ecc7af4141ef0
2025-08-14 08:06:49 +00:00
Jakob Ackermann
df383265c3 [saas-e2e] adjust project name for new SpamSafe limits (#27875)
* [saas-e2e] adjust project name for new SpamSafe limits

* [saas-e2e] stay below 6 digits for spam safe project name

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 77c039312c01b33aeb3b57661ec6ca5b88a0a36b
2025-08-14 08:06:02 +00:00
Jakob Ackermann
99feb94d4b [monorepo] add workaround for flaky mongo db init (#27833)
* [monorepo] add workaround for flaky mongo db init

Upstream issue: https://github.com/docker-library/mongo/issues/730
Upstream PR: https://github.com/docker-library/mongo/pull/731

* [server-ce] revert entrypoint changes

* [monorepo] fix typo in path

* [server-ce] run E2E tests from internal monorepo in cloudbuild

* [monorepo] avoid repeated changes to mongo entrypoint

GitOrigin-RevId: 617db9caee1600c8a506a7a147723a306e57ded3
2025-08-14 08:05:58 +00:00
Jakob Ackermann
d055f01c00 [web] add config option for custom maintenance message (#27618)
GitOrigin-RevId: e01eebecce780af6e9e3f8b82321cb7776d414e2
2025-08-13 08:06:12 +00:00
Jakob Ackermann
836e8b5144 [monorepo] extend cleanup in Jenkins VMs (#27793)
* [monorepo] extend docker compose cleanup

* [web] cleanup docker images

Only retain the branch builds for dependencies.

* [server-ce] jenkins: remove build specific image

Retain the branch image as cache.

* [monorepo] jenkins: prune untagged docker images and docker build cache

* [monorepo] jenkins: remove temporary files from monorepo

* [monorepo] jenkins: remove containers from sandboxed compiles

GitOrigin-RevId: c847015986a060c5c84c39cd06236fd73bbf6f3a
2025-08-12 08:06:27 +00:00
Jakob Ackermann
ac9c20e8d2 [monorepo] tweaks for Jenkins/CI (#27761)
* [packer] jenkins-worker: install htop and lsof

* [web] trim tailing space in Jenkinsfile

* [web] cleanup after finishing Jenkins pipeline

* [server-ce] test: cleanup after finishing Jenkins pipeline

* [saas-e2e] add timeout to waiting for rail/web startup

* [monorepo] add 60s timeout for waiting on mongo to startup

* [saas-e2e] cleanup after finishing Jenkins pipeline

GitOrigin-RevId: 9954ab17b4db6a95c444e8e0b55cebb8d5daae59
2025-08-11 08:06:31 +00:00
Brian Gough
1a8a14a731 Merge pull request #27702 from overleaf/bg-update-form-data-in-cypress
update form-data in cypress

GitOrigin-RevId: 90b48e5c77420e83ce1b0ff7dbe25b3e4a3a87ad
2025-08-08 08:07:42 +00:00
Brian Gough
7ca70eb563 Merge pull request #27652 from overleaf/jpa-latest-to-main
[server-pro] run-e2e-tests: tag local images as "main"

GitOrigin-RevId: b32522bd278a44a551898cab32b8a698b04ab8a6
2025-08-07 08:05:11 +00:00
Jakob Ackermann
16daea27ad [server-ce] tests: optimize sharding (#27635)
* [server-ce] tests: optimize sharding

* [server-ce] tests: remove shard CE_CUSTOM_2

* [server-ce] tests: move project sharing tests onto a new shard

Previously they were on their own CE_CUSTOM_1 shard.

GitOrigin-RevId: de256e04f5956b4651a5dc4ab47cdb5972723c5d
2025-08-06 08:04:46 +00:00
Jakob Ackermann
6da84cdcf6 [server-ce] tests: debug failing mongo startup (#27634)
GitOrigin-RevId: 5b1c272a7e0059d448347d20999eef7ad368bfd5
2025-08-05 08:06:39 +00:00
Jakob Ackermann
48ff696a69 [monorepo] fail fast when parallel build steps fail in Jenkins (#27632)
GitOrigin-RevId: 6362139cc1cd72e00f3eb5900ba94c810c476d82
2025-08-05 08:06:34 +00:00
Jakob Ackermann
98e82e9f88 [server-ce] tests: add jenkins integration (#27600)
* [server-ce] remove unused CE_CUSTOM_3 shard

* [server-ce] tests: add jenkins integration

* [server-ce] tests: log failed host-admin requests in CI

* [server-ce] tests: attempt at reducing flakiness in Cloud Build

GitOrigin-RevId: 8d999efe1ccf041dca148bc8710dddb8561614ab
2025-08-05 08:05:55 +00:00
Jakob Ackermann
7b726fb1ac Merge pull request #27599 from overleaf/jpa-flaky-test
[server-ce] tests: work around overflow of OVERLEAF_APP_NAME

GitOrigin-RevId: da12b8874273319d2b624f10b257b123a3ea44cc
2025-08-05 08:05:29 +00:00
Brian Gough
8785c5c265 Merge pull request #25168 from overleaf/bg-history-redis-add-tls-to-sp-ce
add SP/CE redis TLS configuration options for history-v1 for future use

GitOrigin-RevId: 08d51b604f900559f999457d0c57d5e31021a27b
2025-08-04 08:05:54 +00:00
Jakob Ackermann
3e8fa183c9 Merge pull request #27522 from overleaf/jpa-reenable-tests
[server-ce] tests: enable disabled tests again

GitOrigin-RevId: 447b95bfc6b369f574a518ea46a264445efc5985
2025-08-04 08:05:13 +00:00
Jakob Ackermann
dc3207726e Merge pull request #27578 from overleaf/jpa-fix-saas-e2e
[saas-e2e] fix all the broken tests

GitOrigin-RevId: 38e16abc5e2c9392b0d9e585b6bef27a51c386f5
2025-08-04 08:05:04 +00:00
Jakob Ackermann
b01459508c [packer] rework jenkins worker image (#27573)
- Use Makefile for coordinating all the independent tasks
- Pull down lots of other docker images, notably all the images for
  running Server Pro E2E tests
- Run copybara to populate it's caches
- Build Server CE and Server Pro images to populate npm caches
- Build all the monorepo images for SaaS E2E tests
- Restore the latest database backup into the CI local dev-env

GitOrigin-RevId: 821b036d1afaf2f00372dc6d0f82a51b4c3b7cc4
2025-08-04 08:04:59 +00:00
Jakob Ackermann
dbb2deca5f [server-ce] tests: split prefetching (#27566)
GitOrigin-RevId: de0bfb0d3923749b8844a8812d0a45d71ae4fa0e
2025-08-01 08:07:01 +00:00
Jakob Ackermann
092c8aaf32 [server-pro] add hotfix 5.5.4 (#27514)
GitOrigin-RevId: f992fce918d2617017fa5af5be3edd9e54a5919d
2025-07-31 08:05:33 +00:00
Jakob Ackermann
29045b9085 [web] download binary files in clsi from filestore via new endpoints (#27505)
* [history-v1] use String.padStart instead of lodash.padStart

* [web] download binary files in clsi from filestore via new endpoints

* [server-ce] tests: Cypress.env() is parsing boolean values

* [server-ce] tests: run history migration as root

GitOrigin-RevId: bdf6c0e542531ccc4b3f13d2ed68ca0d31e580e9
2025-07-31 08:05:28 +00:00