The hash of empty content is a property of the content rather than of anything
stored: every project's empty file has it, whether or not a blob was ever written
under it, and a file may reference it before anything has. Eight blob stores were
each paying a lookup -- a database query, an HTTP request, a read of a cache that
offline mode may be the only thing standing in for -- to be told what the hash
already says, and clsi had already reinvented the answer on its own.
overleaf-editor-core gains BlobStoreBase, which answers getString for that hash
and delegates the rest to the fetchString its subclasses provide. getObject comes
with it: four stores spelled out the same JSON.parse of getString, and a fifth
wrapped it in a log line.
Adopted by history-v1's BlobStore, BackupBlobStore, HashCheckBlobStore and
recover_zip's RecoveryBlobStore, by project-history's BlobStore, by clsi's
BlobStore, and by web's HistoryBlobStore, SimpleBlobStore and FrontendBlobStore.
history-v1 keeps its own answers in getStream, getBlob and getBlobs, which no
other store implements. Writing and copying a blob are untouched, so a project
that wrote empty content still has a blob for it and a read answers the same
either way.
Two paths could not handle a project holding a tracked change or a comment at
all. Loading such a file needs the ranges blob its rangesHash points at, and
hashing one again needs to write it back, so a store missing getObject or
putObject fails outright:
- recover_zip died on `blobStore.getObject is not a function`, making every such
project unrecoverable. The test covering it asserts a tracked delete is absent
from the zip, which the script already asked for and never reached.
- GET /projects/:project_id/latest/hashed_content returned 500 from the same
cause, then would have hit a missing putObject in snapshot.store.
HashCheckBlobStore becomes a class over the base and gains putObject.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: c84f62465beff6cf77ffac26aa772984e1b554cb
Two of the B2B request schemas flagged in #36577 reject bodies we don't
want to reject:
- POST /manage/groups/:id/settings/sso: the group settings frontend
spreads the SSO config document it fetched from the GET route back into
the POST body, so the request carries `_id`, `__v`, `validated`,
`enabled` and `useSettingsUKAMF` that the handler recomputes or ignores.
- POST /manage/groups/:id/invites: an internal script sends an unsupported
`expiresAt` alongside the email; the handler only reads `email`.
Swap z.strictObject() for z.object() on both, with a comment on each
explaining why it's deliberately non-strict.
GitOrigin-RevId: 5ff94ded579bdda07efc59d9ceb1d495964a912d
Stop rejecting unrecognized request fields in the Dropbox, GitHub-sync, git-bridge and reference-manager APIs
GitOrigin-RevId: 5efdee90830b2a48172f839da8e59ef938746f65
* [web] read a linked project file's source from history
Behind the `linked-file-from-history` flag, a linked `project_file` import reads
the source out of the source project's latest history snapshot rather than out of
its file tree in mongo, where docs and files are two kinds of thing.
- `ProjectFileAgent` gains `_importFromHistory` alongside the existing
`_importFromProject`, and the flag picks between them. The snapshot only holds
what has reached history, so the source project is flushed through
document-updater and project-history first. A file that is editable is loaded
eagerly, which applies the edit operations sitting on top of its blob; one that
is not keeps its hash and is streamed. The file is promisified on the way.
- `History/HistoryOTContentManager.mjs` reads the latest chunk and rolls its
changes into a snapshot, retrying the read. `History/HistoryBlobStore.mjs` is
the blob store overleaf-editor-core asks for while loading a file.
- `LinkedFilesController` resolves the flag, including from the referer: the
import endpoints have no query string of their own. Every agent takes the
assignment so that the ones that do not read it still pass it along.
- `linked-file-from-history.spec.ts` drives the flag from the editor url, over
three imports from another project: a file uploaded to the source project and
then refreshed from it, an edit the source project has only made in its editor,
and a binary file. Both sources produce the same file, so which one was read is
asserted in the acceptance tests rather than here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [saas-e2e] add test coverage for tracked-delete in linked file source
* Apply suggestions from code review
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: 3907d6d399e4b5315e68bf833663c06bc72f4011
* feat: gating labs experiments behind premium compile feature
* feat: move filtering to the labs module, and only propogate experiments that user can opt into on the FE
GitOrigin-RevId: 984aa0339c7dc8d61f0f6830780a599508306d5e
* [docstore] use more realistic test stubs for ranges
* [monorepo] define ranges schemas in @overleaf/ranges-tracker
* [ranges-tracker] add legacy orderedRejections field in tracked-changes
* [monorepo] resolve @overleaf/ranges-tracker subpath imports in typescript
* [real-time] validate comment thread ids and tracked change id seeds
* [ranges-tracker] require ObjectId ids and tracked change metadata in ranges
* [monorepo] require ObjectId tracked change ids on the accept routes
* [web] use ObjectId comment thread ids in the DocumentController tests
GitOrigin-RevId: 58b20dec311d85b917ef63f9efbe745a265a2de6
* feat: adding 2025 and 2026 incremental compiles options
* feat: remove dmtcp from compiler list and autoselect checkpointing version with labs inclusion
* feat: updating seccomp profile for checkpointing
* feat: disable non-premium dmtcp compiles, and align checkpoint version detection between clsi and web
* feat: adding in reminder of checkpoint experiment, and using session based split test retrieval
* Increase Packer build.sh timeout from 1h15m to 1h30m as build takes approx 1h
* fix: moving checkpointing check from tag name based to hasCheckpointing flag set as part of allowed_images, and removing uneeded seccomp rule
* feat: move variant/ image selection to CLSI, and have web control only what options to set for a compile. removing uneeded extra variants from webs list of allowed images
* feat: adding compile timer as a query string as well
---------
Co-authored-by: Tim Gibbon <tim.gibbon@overleaf.com>
GitOrigin-RevId: cdbdf76a62dcd5a585925f18d64b6cb9cd6ed373
* upsert: migrate US/CA PayPal without a payment method when flagged
getPaymentMethod no longer throws for a PayPal billing agreement destined for
Stripe US/CA when the account is flagged ignore_payment_method; instead it
returns null so the customer is migrated without a default payment method
(the send_invoice paymentMethodPending flow). Unflagged accounts still throw.
* finalize: expect the plan-code remap for legacy-plan subscriptions
When a subscription carries legacy_remap=1 metadata, compare the Stripe product's planCode against the recorded target plan code instead of the raw Recurly plan code, so the intentional remap is not misread as drift (the price is preserved, so the amount comparison is unchanged). Clear the legacy_remap metadata during post-migration cleanup.
* finalize: expect the assistant add-on reprice
When a subscription carries assist_addon_repriced=1 metadata, compare the assistant item against the recorded target amount (Stripe minor units) instead of the raw Recurly amount, so the intended reprice is not misread as drift. Clear the reprice metadata during post-migration cleanup.
* create_coupons: create migration coupons by name with amount_off, currency and product scope
Create the pipeline's per-region coupon_specs_<region>.csv coupons by name (idempotent, auto-paginated), supporting amount_off + currency (fixed) or percent_off, duration, and applies_to.products, with no promotion code. Region comes from the existing --region flag. The existing name-based percent-off + promotion-code path is unchanged.
* finalize: schedule the pending change as a phase 2
For a subscription flagged pending_change_scheduled, compare drift against the current Recurly state (not the pending change), then in Step 7 release the toolkit schedule and create a fresh two-phase subscription schedule (current until renewal; pending state from renewal with proration disabled and end_behavior release), clearing the scheduling metadata once the schedule exists.
* finalize: flag address-pending migrations and push the needs_to_update_address trait
When a migrated subscription carries metadata.addressPending, append -address-pending to the finalization status and push needs_to_update_address to customer.io (alongside the existing tax-info-pending handling), so the customer can be driven to add a billing address. The flag stays on the subscription so the customer.updated webhook can enable automatic_tax and clear it once the address resolves.
* webhook: heal address-pending migrations when the tax location resolves
On customer.updated (address/tax change), for a customer whose location is no longer unrecognized_location, enable automatic_tax on their migrated subscriptions still flagged metadata.addressPending and clear the flag. Enabling automatic_tax is rejected by Stripe while the location is still invalid, so the update is a safe idempotent no-op until the address is fixed; the cancellation state is never touched.
* cron: cancel unresolved address-pending subscriptions before they renew untaxed
Backstop for the address-pending cohort: searches the address-pending subscriptions in a region and, for any about to renew (within a configurable lead window) whose customer location is still unrecognized_location, cancels it at period end so it never renews untaxed. Skips subscriptions that are already cancelling and those whose location has resolved (healed by the webhook). Dry-run unless --commit.
* upsert: set tax.validate_location=auto on customer update
Conform PAN-copied customers to the account's default tax-location validation. Uses 'auto' rather than 'immediately' (which throws for country-only addresses and would break the address-pending migration flow).
* finalize-migration: send payment method info to customer.io
Add payment_method (card/paypal/none) and needs_to_add_payment_method
traits to the stripe_migration customer.io payload so the migration
email campaign can personalise per payment method without enumerating
the full cross-join. The payment method is read from the already-fetched
customer/subscription (default_payment_method now expanded on that fetch),
so no extra Stripe request is made. Also report a -payment-method-pending
status suffix in the CSV output.
* upsert: flag customers whose address Stripe cannot resolve
After pushing the Recurly address, read customer.tax.automatic_tax; when it is 'unrecognized_location', record address_unrecognized in the output so the subscription is migrated untaxed (address-pending) instead of failing at the billing-migration toolkit. This is the authoritative, region-agnostic detector (covers US/CA country-only and broken non-US addresses, e.g. India with a postal but no state/city). Adds isCustomerAddressUnrecognized helper + unit tests.
* Close reactivation-without-valid-address loophole
* webhook: detect newly-set address/tax with !== undefined
Review follow-up: the address-pending heal guard used a truthy check, which
would miss an address newly set from empty. Harmonise with _billingChanges,
which already uses !== undefined.
* upsert: set tax.validate_location via customerParams
Review follow-up: move the tax.validate_location='auto' directive into the
assembled customerParams instead of overriding it inline at the update call,
so it also appears in the dry-run output.
* create_coupons: guard migration coupons against ambiguous discount specs
Review follow-up: throw a clear per-coupon error if a migration coupon spec
has neither or both of amount_off/percent_off, catching data-processing
issues instead of silently sending percent_off=NaN.
* Rely on presence of customer.tax
* webhook: cover the address-pending heal path in unit tests
The customer.updated fixtures carried no `tax` hash and the mocked region
client had no `stripe` property, so the one test that reached the heal branch
threw and was swallowed by the caller's error handler. The heal logic had no
real coverage as a result.
Add the missing fixture fields and tests covering the heal itself, the
still-unrecognized early return, the skip rules for unflagged and cancelled
subscriptions, the Stripe rejection path that leaves the flag set, and the
missing-tax guard.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
GitOrigin-RevId: 8162e3e0f01e136d3aa560c22e98f43e36da4eb5
* [web] add preview payload to track-change notification types
* [doc-updater] pass accepted/rejected change data through to web for previews
* [web] snapshot change preview onto accepted/rejected notifications
* [web] render resolved change previews instead of accepted/rejected count lines
* [web] add unit coverage for accepted/rejected change preview flow
* fix: notification carries multiple previews per accept/reject batch; extract shared group helper
* fix: one preview per accept/reject action, drop adjacency grouping
* fix: move buildResolvedChangePreviews out of the email builder class
* fix: type error, make tests more explicit
* fix: render resolver below preview
* fix: overflow text, alignment
* fix: test error
* fix: resolved count
* [document-updater/web] send sparse change preview instead of full doc lines
* fix: store sparse preview, hydrate at render, drop CE gate
* fix: advance slice cursor past tracked-delete text in sparse preview
* fix: inline buildChangeGroupPreview
* fix: changePreviewBuilder -> sparseChangePreview
* fix: rm change_contributors
* fix: copilot flagged testing issues
* fix: hydrate and truncate change preview at write time
* fix: avoid materializing full doc text when slicing preview window
* fix: update reject preview test to hydrated shape
* fix: accepted delete removing context
* [document-updater/web] split resolved change previews into one per change cluster
* fix: narrow cluster userIds to string[] for typecheck
* feat: updating to use zod schema for previews
---------
Co-authored-by: Jimmy Domagala-Tang <jimmy.domagala-tang@overleaf.com>
GitOrigin-RevId: 0740713c715a61a220180f353e497b27da6a5b0f
* [web] Remove SubscriptionFormatters, format subscription dates in the frontend
Replace the server-formatted nextPaymentDueAt/nextPaymentDueDate/
trialEndsAtFormatted fields with the raw periodEnd date and format in
the React dashboard (same strings as before, English + UTC). Also
consolidate hand-rolled trial checks onto isInFreeTrial /
SubscriptionHelper.isInTrial.
* [web] Remove the now-unused dateformat dependency
* [web] Address review: payment-dates unit test, UTC dates in acceptance test
Add a dedicated unit test asserting the exact formatted strings from
formatPaymentDate/formatPaymentDateTime, and use explicit UTC timestamps
in the SubscriptionDashboard acceptance fixtures instead of
local-timezone Date constructors.
GitOrigin-RevId: 16ac4f1098b14ef9f1478d22cf39c1357ead7dea
* Fix negative AI quota countdown after reset period elapses
Backend (`FeatureUsageRateLimiter`, `TokenUsageRateLimiter`): once the
24h period had lapsed, `remainingUsage`/`remainingTokens` reset to the
full allowance but `resetDate` kept pointing at the old `periodStart +
24h` (i.e. in the past). Advance to `now + 24h` in that case.
Frontend (`NewPlanPaywall`, `ErrorGetPremium`): the "wait X hours and Y
minutes" copy computed `restoreTime - now` without clamping, so a past
`restoreTime` produced negative hours/minutes (users saw "wait -118
hours and -35 minutes"). Clamp to 0.
* Address PR review on quota reset fix
- Keep `resetDate` tied to the stored `periodStart + PERIOD` rather
than jumping to `now + PERIOD` when the period has lapsed. The
moving-target variant unnecessarily reinterpreted a stable field
(existing consumers such as `use-ai-access-gate` check whether the
reset date is in the past).
- Clamp `remainingUsage` / `remainingTokens` at 0 so a concurrent
over-usage never leaks a negative count.
- Align the boundary check with `_checkRateLimit` / `checkUsage`
(inclusive `<=`).
- Cover the period boundary and the clamp in unit tests for both
`FeatureUsageRateLimiter` and `WorkbenchRateLimiter`.
* Truncate test fixtures to whole seconds
`resetDate` is serialized with `Date.toString()`, which has second
resolution, so fixtures with a millisecond component failed the
boundary assertions.
* Revert code changes; keep tests that document the bug
The fixes masked the symptom (negative countdown) without addressing
the underlying issue, and it is not fully understood yet. Revert the
rate limiters and the writefull paywall components to their original
behaviour, and instead pin the current behaviour in tests:
- `it.fails` tests document that `remainingUsage`/`remainingTokens`
go negative when usage exceeds the allowance — these flip red when
the bug is fixed, at which point they should become plain `it`.
- The lapsed-period tests assert (with explanatory comments) that
`resetDate` keeps pointing at the end of the lapsed period, which is
the server-side ingredient of the negative-countdown support ticket.
* Reapply quota fixes
Restores the post-review state of the rate limiters (clamp remaining
counts at 0, inclusive period boundary, resetDate tied to periodStart)
and the writefull paywall components (clamp countdown at 0).
Tests intentionally untouched in this commit; the two `it.fails` tests
now need flipping to plain `it` in a follow-up.
* Flip bug-documenting tests to plain assertions
The clamp fixes are reapplied, so the `it.fails` tests now assert the
fixed behaviour directly. Trim the explanatory comments.
* Set resetDate to `Date.now() + PERIOD_IN_MILLISECONDS` instead of the expired/stale date.
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
* Update lapsed-period tests for the fresh resetDate
`resetDate` is now `Date.now() + PERIOD` once the period has lapsed, so
assert an approximately-fresh date instead of the stale period end.
---------
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: ad39627ab9e5ceccb2a61443b2b7d39c68600594
* feat: adding in check for if the project could have had a png optimized as part of the compile-result-backend event
* feat: updating using new compile metrics to filter for only eligible project compiles
* move projectHasUnconvertedPngs to stats
* add e2e test for projectHasUnconvertedPngs
* add unit tests for compile-with-optimizable-pngs analytics event
---------
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 97e9d184420c34fc1d3e9f6cfe63f6b916f0202f
* feat: cleaning up ai quota leftovers
fix: removing more straggler invos
* fix: featureUpdater still need analyticsId in the user projection
* feat: updating cypress tests to also check for fair usage text
GitOrigin-RevId: 09c1167106c144084c111beea56867f6aa4a0906
* Rename user-facing Papers reference manager text to ReadCube
* Rename Papers component and files name to Readcube
* Update locales from Papers to ReadCube
* Rename ID name from papers to readcube
* Fix a chinese locale to use ReadCube
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
GitOrigin-RevId: ee89e3f5da5bcf9987c0ee3e558b9ae1dab9372c
David Rotermund