Revert "Library search: make backend search tokens consistent with frontend (follow-up to #34386)"
GitOrigin-RevId: 83328f2ee4a9c3f668325fa4de1ca652e7c718bf
* Migration and index setup script
* Helper script to clear linked library
* Pull users collections
* Implementing get items from users zotero library
* Initial zotero sync client
* Pulling users overleaf collection
* Pulling users overleaf collection
* Initial sync pull from Zotero to Library
* Sync and import workflow from Zotero into Library
* Using user getter and setter for updating user library info
* Using new collection for library sync states
* Adding tests for new sync functionality
* Harden Zotero sync route
* Simplify Zotero sync state read and skip accounting
Fold the ensure+get double round-trip in sync-state discovery into one
findOneAndUpdate, and derive itemsSkipped from items.length - itemsInserted
instead of a running counter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Redact URL-encoded token secret from OAuth1 request logs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fixing type failure
* Rename upsertZoteroReferences to importZoteroReferences
The method is insert-only; the bulkWrite upsert just makes re-runs
idempotent rather than updating, so the name now reflects that.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Validate librarySyncStates reads with zod
Replace the `as` / `as unknown as` casts in getLibrarySyncState and
ensureLibrarySyncState with a zod schema parse, matching
LibraryReferenceRepository. Types are now derived from the schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Defer Zotero sync watermark until pagination exists
Don't record lastSyncedVersion after a single-page sync: committing the
page-1 version before pagination (#34286) lands would let incremental
sync (#34282) skip items beyond the first page. Drops the commitSyncState
call and toVersion from SyncResult; the version envelope is left for the
follow-ups. commitSyncState stays in the repository for #34282.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Extract OAuth1Transport from OAuth1Handler
Lift the OAuth1 client/config registry, getAccessToken, redactSecret and
classifyOAuth1Error into oauth1/OAuth1Transport.js, plus a signedGet
helper for provider modules to reuse. OAuth1Handler now imports them; its
method bodies are unchanged. Test mocks move to esmock's global scope so
they reach the transport, with Errors pinned for instanceof checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Move Zotero collections/items into a per-provider ZoteroProvider
Carve the Zotero-only operations out of the generic OAuth1 handler into
providers/ZoteroProvider.js, dispatched via providers/ProviderRegistry.js.
ReferenceProviderHandler now resolves the provider from the registry and
throws UnsupportedOperationError (mapped to 501) for providers without the
operation, instead of bolting Zotero-only methods onto OAuth1Handler.
Zotero's collectionsUri/itemsTopUri are read from settings by ZoteroProvider
rather than copied into the shared OAuth1 registry. Removes getCollections/
getUsersItemsChanges (and their tests) from OAuth1Handler.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add ZoteroProvider unit tests and unsupported-provider coverage
Cover ZoteroProvider collections/items parsing, header envelope and
malformed-payload rejection, and assert ReferenceProviderHandler throws
UnsupportedOperationError for providers without a sync implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Make addSyncedCollection idempotent
Only push a synced collection when its key isn't already recorded, so a
repeat sync can't create duplicate syncedCollections entries regardless of
the caller-side guard. Adds a sequential test asserting no duplicate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Hardcode Zotero library endpoint paths, derive host from site
Move the collections/items-top endpoints out of settings and into
ZoteroProvider as path constants, taking only the host from the
provider's `site`. Aligns Zotero `site` with the other providers
(API host) and avoids adding per-endpoint config points.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Make Zotero sync routes explicit, 404 unknown providers
Hardcode the collections/items routes to /zotero rather than a generic
/:ref_provider, so a non-Zotero request 404s on the missing route
instead of returning a misleading 501. Drop the UnsupportedOperationError
501 branch, reverting OAuth2 unsupported ops to their prior 500.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
GitOrigin-RevId: 50e16a9c46f995c2e9db6eed50ef81c3a71ea71d
* [web] always provide an analyticsId when registering new users
* [web] add recordEventForMongoUser and use it during login
* [web] use recordEventForMongoUser/recordEventForSession where possible
* [web] use session cache for analyticsId when refreshing features
* [web] add setUserPropertyForMongoUser and use it where possible
* [web] use setUserPropertyForSessionInBackground where possible
* [web] use mongo user for analyticsId when setting utm user properties
* [web] admin-panel: user updated user when sending events
* [web] enforce labsProgram being populated on user object
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
* [web] revert FeaturesUpdater.refreshFeatures change
* [web] labs: move session maintenance into handler
* [web] analyticsMiddleware: ensure that all users have labsProgram field
* [web] migrate a few more events to recordEventForSession
* [migrations] mark back-fill of db.users.analyticsId as required
* [web] setUserPropertyForSessionInBackground: throw early
---------
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
GitOrigin-RevId: 19c26bc36abd8c11f0a8f458d25abfb7cbda0bb7
* [web] throw early in UserCreator.createNewUser with missing analyticsId
* [migrations] rerun back-fill of db.users.analyticsId
GitOrigin-RevId: eabf16326192fca4dbea00ad643cee4315c426c6
* add script for uploading a file to a project
* fix default source name in script
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* reject folders as an upload destination
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* validate destination path when normalizing
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
GitOrigin-RevId: 4a6ce79652074475fc2065156718a23156e8b0bd
* Remove double spaces
* Enforce curly apostrophes
* Enforce French typographic spaces in fr.json
Add vitest test enforcing French spacing rules per
https://typographisme.net/post/Les-espaces-typographiques-et-le-web:
- NNBSP (U+202F) before ? ! ;
- NBSP (U+00A0) before :
- NBSP (U+00A0) around guillemets « »
Fix 38 fr.json values that were using NBSP (or no space) before ? ! ;.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Extend French typography test to numbers and percent signs
Add rules per https://typographisme.net/post/Les-espaces-typographiques-et-le-web:
- NNBSP (U+202F) between digit groups (thousands separator)
- NBSP (U+00A0) between a number and a following word/unit
- NBSP (U+00A0) before %
Fix 9 fr.json values where a regular space was used instead of NBSP between
a number and a following word (e.g. "1 résultat", "10 jetons", "7 jours").
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Use point médian for inclusive writing in fr.json
Replace parenthetical gender/plural markers like "sûr(e)", "abonné(e)",
"licence(s)" with the point médian "·" (e.g. "sûr·e", "abonné·e",
"licence·s") per modern French écriture inclusive.
Add a vitest rule rejecting "word(suffix)" patterns where suffix is 1-3
letters and there is no space before the opening parenthesis.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Simplify inclusive-writing check to "(e)" substring match
The previous regex (matchAll without the /g flag) would have thrown at
runtime. Drop matchAll entirely and use includes('(e)').
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Add point médian to participles in fr.json
Apply inclusive writing (·e) to past participles agreeing with the
addressed user ("affilié·e", "invité·e", "connecté·e", "abonné·e", etc.).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Use inclusive forms in fr.json
* Change "de Overleaf" to "d’Overleaf"
* Fix tests
* Check for consecutive spaces, not just double spaces
* Update comment about guillemets
* Add ESLint rules for locale JSON conventions
Add three rules to @overleaf/eslint-plugin that run against locale JSON
files via @eslint/json:
- no-consecutive-spaces-in-locales
- no-straight-apostrophes-in-locales
- french-typography-in-locales
Wire them up in services/web/eslint.config.mjs against `locales/*.json`
(and `locales/fr.json` for the French-typography rule).
* Add `@eslint/json` so JSON files can be parsed
* Scope JS/TS-specific config to JS/TS files only so that built-in rules aren't applied to JSON files
* Disable `no-irregular-whitespace` in french-typography-in-locales.js
* Make the rules fixable
* Improve regexes: user `\s` instead of custom matchers
* Add sorted-keys-in-locales ESLint rule
Mirrors scripts/translations/sort.js as a fixable ESLint rule:
require top-level keys in locale JSON files to be sorted alphabetically.
The autofix re-serializes the file with sorted keys.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Add no-unused-locale-keys ESLint rule
Mirrors scripts/translations/cleanupUnusedLocales.js as a fixable ESLint
rule applied only to en.json. Shells out to `find` + `cat` once per
process (cached) to grep all candidate source files for key references,
then reports any en.json key that isn't found. The autofix re-serializes
en.json without the unused keys.
Hardcoded ALWAYS_USED set covers keys built at runtime (month names,
third-party-datastore notifications) that the regex scan can't see.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Add locale-variables-match-en ESLint rule
Mirrors scripts/translations/checkVariables.js as an ESLint rule applied
to non-en locale JSON files. Reads en.json (cached) and reports any
member whose `__name__` placeholders differ from the base.
GLOBALS (e.g. __appName__) are allowed as extras; IGNORE_NESTING_FOR
preserves the script's per-key exceptions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Replace some double linebreak (@overleaf/no-consecutive-spaces-in-locales)
* Add no-orphan-locale-keys ESLint rule
Detect translation keys in non-en locale files that aren't present in
en.json. Mirrors the `--sync-non-en` flag of cleanupUnusedLocales.js.
The autofix re-serializes the locale file without the orphan keys,
sorted alphabetically.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Add tests for the locale JSON ESLint rules
Cover the four single-file rules:
- no-consecutive-spaces-in-locales
- no-straight-apostrophes-in-locales
- sorted-keys-in-locales
- french-typography-in-locales (including the inclusive-writing
suggestion for "(e)")
The three cross-file rules (no-unused-locale-keys, no-orphan-locale-keys,
locale-variables-match-en) read from disk and aren't covered here; they
need integration tests via fixtures.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Drop bin/lint_locales — checks are now covered by ESLint rules
The four checks in bin/lint_locales are duplicated by the new
@overleaf/eslint-plugin rules:
- sort.js --check → sorted-keys-in-locales
- cleanupUnusedLocales.js --check → no-unused-locale-keys
+ no-orphan-locale-keys
- checkVariables.js → locale-variables-match-en
- single-quote grep → no-straight-apostrophes-in-locales
`yarn lint` runs the ESLint rules against locales/*.json, so the
lint_locales script and Makefile target are redundant.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Remove Locales.test.mjs
* Delete scripts replaced by @overleaf/eslint-plugin rules
Remove the three locale scripts now superseded by ESLint rules:
- scripts/translations/sort.js → sorted-keys-in-locales
- scripts/translations/cleanupUnusedLocales.js → no-unused-locale-keys
+ no-orphan-locale-keys
- scripts/translations/checkVariables.js → locale-variables-match-en
Also drop the matching Makefile targets (sort_locales,
cleanup_unused_locales), update AGENTS.md / copilot-instructions.md to
point at `yarn lint --fix` instead, and prune the deleted entries from
the esm-check-migration allowlist.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Delete redundant scripts/translations/sort-locale-file.js
Single-file sort utility — superseded by `yarn lint --fix path/to/file.json`
via the sorted-keys-in-locales ESLint rule. No callers in repo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Update comments and messages
* Add test file pattern to ESLint configuration
* Reorder "Backend specific rules" so `no-extraneous-dependencies` doesn't fail on test files
* Make french-typography fixes idempotent and catch edge cases
Apply Copilot's review feedback on PR #34221:
- autoFix() now uses `\\s*`/`\\s+` instead of `\\s?`/`\\s` so it
collapses any run of mixed whitespace into the correct NBSP/NNBSP,
making the fix idempotent and order-independent.
- The `:` lint regex now captures optional whitespace and reports
when it isn't NBSP (catches `"Liste :"` with a regular space).
- The `%` lint regex now matches the missing-space case (`"10%"`)
via `\\s*` instead of requiring one char between digit and `%`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Remove sort-locale-file.js from pre-commit
* Revert no-unused-locale-keys ESLint rule; Restore lint_locales scripts
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
GitOrigin-RevId: 8351fab074e6e7f77920b70237df1cd1c781e053
* Allow admin access to user PATs
* Tests for new screen in admin panel
* Adding error for invalid token and way to parse error for OAuth 2
* Git bridge handles expired PAT
* Script for alerting on close to expiry and expired git tokens
* Refactoring and simplifying
* Updating email templates to match agreed docs
* tweak to email subject to include Overleaf
* Allowing dry run in scripts and general tidy up
* removing redundant tests and dry running script
* Fixing CI errors
* Adding new tab to admin test expectation
* Address PR feedback on oauth2-server changes
- Replace ad-hoc overleafErrorCode prop with a TokenExpiredError subclass
- Collapse listTokens/listTokensForAdmin into a single hook
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Adding cron definitions for alerting on expiring git pat
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
GitOrigin-RevId: 69b9fd901a201592a580c69abe7bd7d603e85d3a
* Initial working version of library search
draft fetch allowing optional search param
draft debounce search
draft search bar
draft using for search
draft search params
draft data index creation
draft prefix-regex search
draft add fields only on search
draft index setup
draft search tests
draft search tests for extra params
draft using correct display value from bib entry for tokenization
* Library search handles diacritics
* Library styling and refreshing table data without
reloading table
* Updating mongo search query and creating migration
scripts for existing data
* Using Mongo query for sorting results
* Moving copied files into shared directory
* Addressing review comments
* Pulling changes from bibtex-search-token for consistency with migration
* Fixing lint
* Using mongo collation for handling case and diacritics in search queries
* Boosting citation keys with check for tokens
* Removing double foldLatinDigraphs call
* Matching figma designs for Library search component
* Adding cursor for paginated Library search results
* Re-fixing flash after searching library
* Unit test for cursor search
* Using same cursor object for search and get all results
* Data migration moved to manual script
GitOrigin-RevId: b7e6a1f07f775c8450dd97e7269cab3b68ca0eb3
* [docstore] add useSecondary flag to projectHasRanges
The rev-check for unarchiving always consults with the primary.
Two extra changes:
- Add a projection argument to peekDoc in order to skip lines download
from projectHasRanges.
- Add one retry to peekDoc to reduce chances of surfacing a rev-check
violation.
* [web] resync_projects: use the secondaries for all reads
* [web] add default value for useSecondary
* [docstore] add default value for useSecondary
* [k8s] docstore: set MONGO_HAS_SECONDARIES=true
GitOrigin-RevId: f15ec4fdc1cabe74c1eab87bec85f28d6f7a587d
* [history-v1] add endpoint for downloading latest zip
* [web] address review feedback
* [web] tests: do not overwrite db.projects.overleaf, extend it
* [web] set includeReferer flag from downloading zip
GitOrigin-RevId: e63e549f004230086f82eccf03b43fd62bde6071
* [github] code spaces: add extra-split-tests.json
* [github] address review feedback
* [github] add missing bind mount
* [monorepo] mount split test configs with :ro
* [monorepo] only mount backup/split-tests into web
GitOrigin-RevId: 6334a44599a2c8bb79bf1ad698e656c8bee3992b
Migrates the Overleaf monorepo package manager from npm (v11) to Yarn 4 (v4.9.1) using node-modules linker mode.
GitOrigin-RevId: 50d32ab01955c15e29679eff9e9e9cfb897fab2d
* [web] Add `update_group_policy` script
The script allows to update a policy for a subscription, or update
multiple policies via a file containing a list of subscription ids
GitOrigin-RevId: f3b3a6666eb62bb93d13c4c75d60b99f21f23db4
* [github] code spaces: add more standard users with a common password
* [github] code spaces: automatically share dev-env with Overleaf org
* [github] code spaces: merge split tests rather than replacing them
* [github] code spaces: ts-check the setup script in web
GitOrigin-RevId: 1a86ac7e0304d47e68290352f58a0e193eed1d77
* [web] serve site manifest as application/json
* [monorepo] bail out early from dev-env-setup when inside devcontainer
* [monorepo] move utils for waiting for services into shared bash script
* [monorepo] move domain related env-vars into dedicated .env file
The .env file will be overridden in GitHub Code Spaces.
* [monorepo] remove unused initial_mongo_setup Make target
It was previously used for the mongo replica set setup.
That setup is a docker init script now.
* [monorepo] add flag for silencing docker build output
* [monorepo] add integration for GitHub Code Spaces
* [web] make prettier happy
* [web] restore no use-credentials for manifest outside code spaces
* [monorepo] run make install as part of post-start again
GitOrigin-RevId: 8c2cc3b969b7e3e8f9c5dac778192c9412cbf4df
Jakob Ackermann