Commit Graph

655 Commits

Author SHA1 Message Date
Jakob Ackermann
81b7121408 [clsi] initial implementation of compile from history (#31883)
* [clsi] initial implementation of compile from history

* [clsi] copy changes

* [saas-e2e] extend test case with nested folder

* [saas-e2e] add test case for tracked changes

* [web] fix accumulating changes from multiple chunks

* [web] optimize size check for compile request payload

* [clsi] deduplicate globalBlobs

* [clsi] add validation for request body details

* [clsi] add metrics for compile from history

* [clsi] download binary files concurrently

* [clsi] skip download of empty file blob

* [clsi] break down e2e compile time metric by compileFromHistory

GitOrigin-RevId: 0dadef93e89d8a172c35cb130a1042d9d1bec42a
2026-03-06 09:12:07 +00:00
Simon Gardner
89e8044a25 Further improvements to Recurly -> Stripe Customer Upsert script
* handle billingInfo / account discrepancies

* fix rate limiting behaviour of fetchRecurlyActiveSubscription

* make force-invalid-tax aware of stripe tax validation errors

* clear taxInfoPending if vatNumber is now valid or not present on second runs

* re-instate pre-validation of tax ids in dry-run mode

* handle tax exemption

* fix tax pre-validation tests

* fix comment typos

* remove redundant hadConflict

* re-use normalizeName function

* throw if more than 1 active recurly subscription when resolving conflict

---------

Co-authored-by: Simon Gardner <simon.gardner@overleaf.com>
GitOrigin-RevId: a88cad87887bd9e043fb457091e999c1d555f546
2026-03-06 09:10:28 +00:00
Jessica Lawshe
0c9382e36a Merge pull request #31543 from overleaf/jel-unconfirmed-emails-removed-v1
[web] Script to remove v1 affiliation for unconfirmed emails removed in web

GitOrigin-RevId: c698297c57bfeb6bd3743978d772673efe6fc3ad
2026-03-06 09:09:12 +00:00
Andrew Rumble
de75ea6d3b Merge pull request #31818 from overleaf/ar-actually-dedupe
[monorepo] run npm dedupe

GitOrigin-RevId: 6133acd7b28b536b777962024fd318c9c4d6c091
2026-03-06 09:09:07 +00:00
Simon Gardner
fcac73e27f Improvements to Recurly -> Stripe customer upsert script (#31539)
* migrate Recurly ccEmails to Stripe additional_emails.cc

* improve tax error reporting

* remove redundant call to recurly.getBillingInfo

GitOrigin-RevId: fa26fd5312d2b7ac5734fc78118ede6e1cfa17c8
2026-03-06 09:08:51 +00:00
Kristina
aa2ce6f675 [web] handle duplicate customer accounts better (#31677)
* [web] extract card comparison function

* [web] throw if more than 1 duplicate customer account

* [web] attempt to pick the pre-existing account

* [web] overwrite email of duplicate account

* Update services/web/scripts/helpers/migrate_recurly_customers_to_stripe.helpers.mjs

* [web] handle setting being undefined

* [web] allow other account to be reused if migrating account is using paypal

* [web] allow matching for customers with no payment method

GitOrigin-RevId: 7d8eeb4734a20c09ce3e0d73095e28696e212549
2026-03-06 09:07:40 +00:00
Kristina
820adf568e [web] add support for migrating paypal subscriptions (#31607)
* support Paypal for migration
* remove sensitive payment info in logging
* add `resolveStripeCustomer`

GitOrigin-RevId: a8fc7de4a4bf3971c5221a0dec3fc279d8d2f67d
2026-02-20 09:06:43 +00:00
Jimmy Domagala-Tang
8e5c207f6b Remove Writefull.enabled (#31476)
* feat: migrate from aiErrorAssist naming for disabling AI features to aiFeatures.enabled to avoid confusion

feat: keep aiErrorAssistant as setting on user object until migration is run

* feat: migrate writefull.enabled unset to instead use promotionSet false

* feat: remove wf.enabled in favor for aiFeatures.enabled for display, and writefull.promotionSet for determining if we should autoload or autocreate

GitOrigin-RevId: 2b7a5b8a430a804f6c9804cc926cb5c057e34df5
2026-02-20 09:06:08 +00:00
Kristina
8872d851ec [web] rm unnecessary scripts (#31631)
GitOrigin-RevId: c6388685bd9cd43d8d22e38f25db4ee579def808
2026-02-19 09:06:45 +00:00
Jimmy Domagala-Tang
97c4c550d7 Migrate user.writefull.enabled -> user.writefull.promotionSet (#31475)
* feat: migrate from aiErrorAssist naming for disabling AI features to aiFeatures.enabled to avoid confusion

feat: keep aiErrorAssistant as setting on user object until migration is run

* feat: migrate writefull.enabled unset to instead use promotionSet false

* Update tools/migrations/20260213102825_swap_writefull_enabled_for_initialized.mjs

Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>

* feat: addind backfill of ai features as a migration as well to help with dev environment setup

---------

Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: fd3f0fc07c91678cd7fd1e0d5237221694b3027b
2026-02-18 09:05:38 +00:00
Kristina
85066c6cb3 [web] various migration script improvements (#31538)
* make warnings more eye catching in clean up dry run
* only increment processedCount after processing
* always rm recurlyAccountCode
* check if existing Stripe customer already has a subscription
* ensure no other customer accounts share the user id
* set recurly_to_stripe_migration_status to cancelled in cancel script
* add script for updating metadata of canceled migrations

GitOrigin-RevId: 3331de480e99774679ff2649b90d41e981a8fdef
2026-02-18 09:05:34 +00:00
Jimmy Domagala-Tang
551d7b3908 feat: migrate from aiErrorAssist naming for disabling AI features to aiFeatures.enabled to avoid confusion (#31273)
feat: keep aiErrorAssistant as setting on user object until migration is run
GitOrigin-RevId: df53914163566065d0b91b8d130476fbcdf1ea96
2026-02-13 09:05:57 +00:00
Kristina
8ecb575d06 [web] apply runtime improvements to migration cleanup scripts (#31488)
GitOrigin-RevId: bfd7a34ecc5ab0b89a0aebae3f7bec995d490b31
2026-02-12 09:05:56 +00:00
Kristina
21ccf42ae8 [web] add script to bulk cancel subscription schedules (#31469)
GitOrigin-RevId: 1b037eb1335dd14a302bf478c3a4b71e219b074e
2026-02-12 09:05:19 +00:00
Kristina
d6cda7f770 [web] throw an error if multiple active subscriptions found (#31458)
GitOrigin-RevId: e0c2fe9ab73b0f8800f419ac1a3ff78adce29c5c
2026-02-11 09:06:47 +00:00
Kristina
f8d74187e1 [web] prefer mongo email when sending data to customerio (#31378)
GitOrigin-RevId: 33e137239d505e7feb6f6e1b2a00d7b5e60d0880
2026-02-10 09:06:08 +00:00
Kristina
a184a0c32f [web] handle pending changes in finalization script (#31342)
GitOrigin-RevId: 3f17c32d897e818d0ae1b0e33e66350de8513a8a
2026-02-10 09:06:03 +00:00
Kristina
1f5fec628b [web] apply runtime improvements to the finalization script (#31360)
* extract RateLimiter
* remove unnecessary remapping and wrappers

GitOrigin-RevId: fda1cdefa15f2f3fa9a042346a5ba4243897b90a
2026-02-10 09:05:58 +00:00
Antoine Clausse
9e41d9afdb Add script to create Stripe prices from CSV (#31144)
* Extract currency conversion and rate limiting functions to helpers

* Add script to create prices from CSV for Stripe integration

* Add tests for create prices from CSV script

* Add usage documentation for create_prices_from_csv script

* Add `planCode` metadata

* Temp: Plans CSV example

* Revert "Temp: Plans CSV example"

This reverts commit 810b1ed67052f7a1a0deb20b70f14507a282fcf1.

* Various improvements to price and product creation script

---------

Co-authored-by: Tim Down <158919+timdown@users.noreply.github.com>
GitOrigin-RevId: c015c6dd904db3143781581db4210cef282a4070
2026-02-10 09:05:28 +00:00
Simon Gardner
a5f9e6ecd6 web: fix: BOM parsing and rate limiting bugs in Recurly -> Stripe customer upsert script (#31398)
* fix: add check for BOM to csv parser

* fix bug in throttling that allowed 'stampeding herds'

* fix theoretical error handling bug in throttle()

GitOrigin-RevId: cfe5da63281590b03ababef370a2fb1a316f2dbf
2026-02-09 09:07:43 +00:00
Simon Gardner
4d338efe56 updates to recurly -> stripe customer upsert script
GitOrigin-RevId: dfe277807f9282804fed5a9cdf86e654a719de96
2026-02-06 09:05:15 +00:00
Kristina
4decfd343d [web] update finalization scripts for phase 2 (#31293)
* handle group quantities & add-ons when detecting changes
* include analyticsId in output
* flag users in customer-io
* rely on customer metadata for tax info needing updating
* rollback customer.io data in rollback script
* refactor how we handle errors when migrating to avoid losing data for output

GitOrigin-RevId: f77430b0b366217ac85b72dde92e9364dc879023
2026-02-06 09:05:07 +00:00
Rebeka Dekany
a648015db8 Centralize prettier configuration to root level (#30501)
* Merge all .prettierignore files into top-level config

* Merge all .prettierrc files into top-level config

* Replace service-specific glob patterns in package.json format scripts with `prettier .`

* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore

* Ignore GitHub templates

* Ignore PUG templates to format them separately with `format:pug`

* Encourage double quotes for YAML, YML files

* Move prettier for PUG source format script to the root

* Move prettier for styles source format script to the root

* Remove prettier for jenkins files from web

* Remove prettier source format script from all services

* Make .prettierrc more readable

* Update format scripts by file type

* Organise `.prettierignore`

* Add `--cache` flag to prettier scripts for faster runs

* Format all files with prettier

* Format all or format services

* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`

* Avoid conlficts with yamllint configuration

* Remove `--cache` flag from prettier scripts

* Update all service Makefiles to use root-level prettier configuration

* Update all Jenkinsfile to use root-level prettier configuration

* Ignore auto-generated files by build_scripts

* Update package-lock.json

* Update root Makefile format targets

* Update SP Jenkinsfile format target

* Update E2E Makefile format script

* Udpate `format_js` to work in both local and CI env

* Add docker-mailtrap to .prettierignore

docker-mailtrap is a third-party git-ignored directory used for testing

* Added Docker env detection to prevent nested Docker spawning

* Ignore handlebars templates

* Add cryptographic files and test output to `.prettierignore`

* Add terraform modules to `.gitignore`

* Remove prettier-plugin-groovy

* Use npx directly instead of Docker for local formatting for faster formatting

* Auto-generate Makefiles

* Revert "Remove prettier-plugin-groovy"

This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.

* Mount monorepo root in RUN_LINT_FORMAT for prettier config access

* Prettier ignores all `node_modules` by default regardless of location

* Show only changed files in format output

* Ignore LICENSE files

* Enable prettier on rendered build_scripts outputs

* Ignoring all the template folders by prettier

* Remove the public/minjs entry since it does not exist

* Remove all non-existent paths

* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files

* Revert "Auto-generate Makefiles"

This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.

* Revert "Use npx directly instead of Docker for local formatting for faster formatting"

This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.

* Ignore dashboard JSON files

* Ignore files generated by bin/update_build_scripts

* Remove unsupported file types from `.prettierignore`

* Ignore test fixture generated files

* Ignore README file types by prettier

* Ignore generate snapshots by prettier

* Allow to format generated bin/update_build_scripts by prettier

* Ensure build script outputs prettier-compatible tsconfig.json

* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double

* Don't read cryptographic files by prettier

* Ignore google verification files by prettier

* Revert npx prettier formatting

* Ignore domain verification files

* Show only changed files in format output

* Make `.github` prettier

* Allow all files to be formatted in jobs by prettier

* Allow server-ce/server-pro files to be formatted by prettier

* Ignore more folders in clsi, filestory, git-bridge by prettier

* Update build script with `RUN_LINTING_CI_MONOREPO`

* Ignore docker-mailtrap and downloads in server-ce by prettier

* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)

* Source format

GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
2026-02-04 09:08:22 +00:00
Kristina
0cabc81d46 Merge pull request #31218 from overleaf/kh-update-assistant-prices
* update Stripe AI assist prices
* add soft archive option to the archiving prices script

GitOrigin-RevId: 3f0b66cf227e31e03fb3337b3cb4c1b6a82bd1db
2026-02-04 09:06:34 +00:00
Jakob Ackermann
28a7aa5e19 [web] deactivate_projects: spread archiving jobs over the cron interval (#31212)
* [web] deactivate_projects: spread archiving jobs over the cron interval

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [web] deactivate_projects: stop processing after graceful shutdown

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [web] deactivate_projects: exit w/ code 1 when aborting due to max-time

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [web] deactivate_projects: adjust bail out for timeout

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [web] deactivate_projects: handle small number of projects better

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* add comment about use of --maxTime option

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: c62c66725233d391fd2c8d86ce95275cea88f36c
2026-02-04 09:05:59 +00:00
Kristina
3000a9108d [web] fix source on analytics mapping in finalization script (#31143)
* fix source on analytics mapping in finalization script
* add script to redo mappings for finalized subscriptions

GitOrigin-RevId: a62af6c6c7ae11f9bea62ed643c8dfa7595ba99b
2026-01-30 09:06:17 +00:00
Kristina
d6fbed2a74 [web] check end state before terminating (#31136)
* check and log unexpected end states before terminating Recurly subscriptions
* update finalize and rollback scripts to only postpone active subscriptions

GitOrigin-RevId: 7fe6ffa56cb8ddf19133eb0cb59e39fd783430b7
2026-01-30 09:06:12 +00:00
Anna Claire Fields
e80ce42896 Upgrade Node.js from 22.18 to 24.13 (#30509)
GitOrigin-RevId: c2405a0583e68c61351f52b32ba28faec994fbc0
2026-01-30 09:05:28 +00:00
Kristina
ac1a61cafa [web] remap customer.metadata.recurlyAccountCode in migration scripts (#31097)
* remap customer.metadata.recurlyAccountCode to customer.metadata.userId
* verify recurlyAccountCode is what we expect it to be before setting

GitOrigin-RevId: 8ab1d5e311dba34bb8c15d01096f6c31273a7506
2026-01-29 09:06:08 +00:00
Simon Gardner
08c1de479f [web] Only report existing stripe fields where the update will change the values
GitOrigin-RevId: 665cc99a4acec1a12812b4874542512e11d25ad4
2026-01-28 09:06:23 +00:00
Kristina
c876bf2c5f [web] set default payment method based on recurly data (#31025)
GitOrigin-RevId: ad7ffe400748ace66aa7e4775eedb738028c8a0c
2026-01-27 09:06:45 +00:00
Simon Gardner
60bb53bbfa if stripe fields present, warn and log both records.
GitOrigin-RevId: 4ae0a6859b01f25bce8391a35b6789ea73ae344c
2026-01-27 09:06:33 +00:00
Simon Gardner
83971b4a8a [web] set default payment method on customers
GitOrigin-RevId: 3603b80f11b70493c04e145d976c540ea718512a
2026-01-27 09:06:28 +00:00
Simon Gardner
b517df6672 Merge pull request #30958 from overleaf/slg-recurly-stripe-migration-30684
add script for migrating customer metadata from recurly to stripe

GitOrigin-RevId: 019413eda20cef2e09c9cc278a8806fa244fe019
2026-01-27 09:06:03 +00:00
Kristina
49591a5190 [web] add scripts to finalize recurly -> stripe migration (#30925)
GitOrigin-RevId: 2149aa516a00b18927fea46e9241496b74478152
2026-01-26 09:06:33 +00:00
Olzhas Askar
f95b471797 Merge pull request #30938 from overleaf/oa-recurly-updates
[web] Check upcoming Recurly schedule changes

GitOrigin-RevId: 355ad6f12ff1f0605e9c2f007d2d66a26c0b5810
2026-01-23 09:06:39 +00:00
Miguel Serrano
d161cf6bbc [web] Add --append to scripts/update_institution_user_saml_ids (#30851)
* [Add `--append` to `scripts/update_institution_user_saml_ids`
* Added a `--search-regexp` argument

GitOrigin-RevId: e78cd8b61365145d18176a0ee1a4f98ba2cabda6
2026-01-23 09:05:56 +00:00
Andrew Rumble
c0a4c597f0 Merge pull request #30902 from overleaf/ar-fix-deleted-project-cleanup
[monorepo] use nested deletedProjectId

GitOrigin-RevId: a74f1beed86610c01fe6f5f991f3a1d9c057252a
2026-01-22 09:06:18 +00:00
Domagoj Kriskovic
6b575c9235 [web] remove scriptRunner from a cron job script (#30871)
* [web] remove scriptRunner from a cron job script

* disable eslint check

* fix logger params, add error description

GitOrigin-RevId: 60cfc32fc62d08283222bf7cdbe846b6c1d4796f
2026-01-22 09:06:13 +00:00
Jakob Ackermann
4be9ecbe14 [e2e] enable --notablescan for Server Pro and SaaS E2E tests (#30839)
* [e2e] enable --notablescan for Server Pro and SaaS E2E tests

* [server-ce] remove --notablescan override when downgrading

GitOrigin-RevId: 41eb6060e066b8a531d9e815802f1b4464bf5102
2026-01-20 09:06:51 +00:00
Jakob Ackermann
914362611e Revert "[e2e] enable --notablescan for Server Pro and SaaS E2E tests (#30677)" (#30833)
This reverts commit f08812899647bc8e4bc0c0d14d3250eb0b9f5c59.

GitOrigin-RevId: 7484cc5d5de6af0a5e4083fed4e8cacd9e1e643f
2026-01-20 09:05:49 +00:00
Jakob Ackermann
3786471135 [e2e] enable --notablescan for Server Pro and SaaS E2E tests (#30677)
* [e2e] enable --notablescan for Server Pro and SaaS E2E tests

* [web] fix query for expiring projects of E2E test users

* [web] check for --notablescan in e2e test setup

GitOrigin-RevId: f08812899647bc8e4bc0c0d14d3250eb0b9f5c59
2026-01-19 09:07:18 +00:00
Anna Claire Fields
ff8f77d85c replace staff access with role based sytem (#30004)
- remove references to staff access

GitOrigin-RevId: 5d7df3ae8bc78aa02b65ec0dac0a323520c3df15
2026-01-19 09:06:59 +00:00
Miguel Serrano
422b474a17 [web] update_institution_user_saml_ids.mjs script (#30691)
The update_institution_user_saml_ids.mjs script updates `samlIdentifiers.externalUserId`
for users with a specific `samlIdentifiers.providerId`, performing a string replacement.

GitOrigin-RevId: 9d9897b17a528ea2ef3abf60d3b5e6679bf11943
2026-01-14 10:17:57 +00:00
Olzhas Askar
3f7555e5a0 Merge pull request #30674 from overleaf/oa-import-coupons
[web] Simplify and allow multiple promotion codes

GitOrigin-RevId: 6ac1115bcdf4298579df33b3896ee0b786720bac
2026-01-14 09:05:55 +00:00
Kristina
29f5026dda [web] fix import products script when tax code is null (#30605)
GitOrigin-RevId: 32ca603376db7f186283514067178d27038a865d
2026-01-12 09:07:16 +00:00
Liangjun Song
2706c91b25 Merge pull request #30285 from overleaf/ls-skip-webhook-event-for-admin-user-id-removal
Skip webhook event for admin user id removal

GitOrigin-RevId: f5b3b3543bf4efcbf0d99caa6325259132823c77
2025-12-12 09:06:36 +00:00
Kristina
c6fc921c11 [web] add script for updating prices for existing Stripe subscriptions (#30216)
GitOrigin-RevId: 8f20a6ba9c7a737bb304d8698c4fbcb06b6ddc8c
2025-12-12 09:06:32 +00:00
Andrew Rumble
ef4341a541 Merge pull request #30279 from overleaf/ar/add-esm-import-eslint-rules
[monorepo] add esm import eslint rules

GitOrigin-RevId: 5637bb1107aa034701cae2e37a85cc3d8d8036d2
2025-12-12 09:06:19 +00:00
Olzhas Askar
38b4ff72cd Replace learn with learnwiki (#30259)
* feat: replace learn with learnwiki

* revert: for sharelatex

* Change `MW_SERVER` env variable to `learnwiki`

---------

Co-authored-by: Daniel Kontsek <daniel.kontsek@overleaf.com>
GitOrigin-RevId: 97cdf034ed864d1620b5fd8673f0a7fe64056e01
2025-12-12 09:06:03 +00:00