Revert "Library search: make backend search tokens consistent with frontend (follow-up to #34386)"
GitOrigin-RevId: 83328f2ee4a9c3f668325fa4de1ca652e7c718bf
* Migration and index setup script
* Helper script to clear linked library
* Pull users collections
* Implementing get items from users zotero library
* Initial zotero sync client
* Pulling users overleaf collection
* Pulling users overleaf collection
* Initial sync pull from Zotero to Library
* Sync and import workflow from Zotero into Library
* Using user getter and setter for updating user library info
* Using new collection for library sync states
* Adding tests for new sync functionality
* Harden Zotero sync route
* Simplify Zotero sync state read and skip accounting
Fold the ensure+get double round-trip in sync-state discovery into one
findOneAndUpdate, and derive itemsSkipped from items.length - itemsInserted
instead of a running counter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Redact URL-encoded token secret from OAuth1 request logs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fixing type failure
* Rename upsertZoteroReferences to importZoteroReferences
The method is insert-only; the bulkWrite upsert just makes re-runs
idempotent rather than updating, so the name now reflects that.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Validate librarySyncStates reads with zod
Replace the `as` / `as unknown as` casts in getLibrarySyncState and
ensureLibrarySyncState with a zod schema parse, matching
LibraryReferenceRepository. Types are now derived from the schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Defer Zotero sync watermark until pagination exists
Don't record lastSyncedVersion after a single-page sync: committing the
page-1 version before pagination (#34286) lands would let incremental
sync (#34282) skip items beyond the first page. Drops the commitSyncState
call and toVersion from SyncResult; the version envelope is left for the
follow-ups. commitSyncState stays in the repository for #34282.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Extract OAuth1Transport from OAuth1Handler
Lift the OAuth1 client/config registry, getAccessToken, redactSecret and
classifyOAuth1Error into oauth1/OAuth1Transport.js, plus a signedGet
helper for provider modules to reuse. OAuth1Handler now imports them; its
method bodies are unchanged. Test mocks move to esmock's global scope so
they reach the transport, with Errors pinned for instanceof checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Move Zotero collections/items into a per-provider ZoteroProvider
Carve the Zotero-only operations out of the generic OAuth1 handler into
providers/ZoteroProvider.js, dispatched via providers/ProviderRegistry.js.
ReferenceProviderHandler now resolves the provider from the registry and
throws UnsupportedOperationError (mapped to 501) for providers without the
operation, instead of bolting Zotero-only methods onto OAuth1Handler.
Zotero's collectionsUri/itemsTopUri are read from settings by ZoteroProvider
rather than copied into the shared OAuth1 registry. Removes getCollections/
getUsersItemsChanges (and their tests) from OAuth1Handler.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add ZoteroProvider unit tests and unsupported-provider coverage
Cover ZoteroProvider collections/items parsing, header envelope and
malformed-payload rejection, and assert ReferenceProviderHandler throws
UnsupportedOperationError for providers without a sync implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Make addSyncedCollection idempotent
Only push a synced collection when its key isn't already recorded, so a
repeat sync can't create duplicate syncedCollections entries regardless of
the caller-side guard. Adds a sequential test asserting no duplicate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Hardcode Zotero library endpoint paths, derive host from site
Move the collections/items-top endpoints out of settings and into
ZoteroProvider as path constants, taking only the host from the
provider's `site`. Aligns Zotero `site` with the other providers
(API host) and avoids adding per-endpoint config points.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Make Zotero sync routes explicit, 404 unknown providers
Hardcode the collections/items routes to /zotero rather than a generic
/:ref_provider, so a non-Zotero request 404s on the missing route
instead of returning a misleading 501. Drop the UnsupportedOperationError
501 branch, reverting OAuth2 unsupported ops to their prior 500.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
GitOrigin-RevId: 50e16a9c46f995c2e9db6eed50ef81c3a71ea71d
* [web] always provide an analyticsId when registering new users
* [web] add recordEventForMongoUser and use it during login
* [web] use recordEventForMongoUser/recordEventForSession where possible
* [web] use session cache for analyticsId when refreshing features
* [web] add setUserPropertyForMongoUser and use it where possible
* [web] use setUserPropertyForSessionInBackground where possible
* [web] use mongo user for analyticsId when setting utm user properties
* [web] admin-panel: user updated user when sending events
* [web] enforce labsProgram being populated on user object
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
* [web] revert FeaturesUpdater.refreshFeatures change
* [web] labs: move session maintenance into handler
* [web] analyticsMiddleware: ensure that all users have labsProgram field
* [web] migrate a few more events to recordEventForSession
* [migrations] mark back-fill of db.users.analyticsId as required
* [web] setUserPropertyForSessionInBackground: throw early
---------
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
GitOrigin-RevId: 19c26bc36abd8c11f0a8f458d25abfb7cbda0bb7
* [web] throw early in UserCreator.createNewUser with missing analyticsId
* [migrations] rerun back-fill of db.users.analyticsId
GitOrigin-RevId: eabf16326192fca4dbea00ad643cee4315c426c6
* [web] add migration for back-filling db.users.labsProgram=false
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
* [web] include user.labsProgram=false in session
---------
Co-authored-by: Andrew Rumble <andrew.rumble@overleaf.com>
GitOrigin-RevId: 0b3322007f6a36d2999d646869dc84f2c6ad4d2d
* [web] Fix Stripe Subscription ID being saved to wrong field in admin panel
* add migration for unsetting recurlySubscription_id
GitOrigin-RevId: ccae470897690a8cdc8a8bce883983808a963d24
registerExternalAuthAdmin() now generates a random password on admin registration.
A migration clears the password for existing installs only in CE/SP
GitOrigin-RevId: 94a82d35dc8cd46915c31fb24f477c19367025eb
* Initial working version of library search
draft fetch allowing optional search param
draft debounce search
draft search bar
draft using for search
draft search params
draft data index creation
draft prefix-regex search
draft add fields only on search
draft index setup
draft search tests
draft search tests for extra params
draft using correct display value from bib entry for tokenization
* Library search handles diacritics
* Library styling and refreshing table data without
reloading table
* Updating mongo search query and creating migration
scripts for existing data
* Using Mongo query for sorting results
* Moving copied files into shared directory
* Addressing review comments
* Pulling changes from bibtex-search-token for consistency with migration
* Fixing lint
* Using mongo collation for handling case and diacritics in search queries
* Boosting citation keys with check for tokens
* Removing double foldLatinDigraphs call
* Matching figma designs for Library search component
* Adding cursor for paginated Library search results
* Re-fixing flash after searching library
* Unit test for cursor search
* Using same cursor object for search and get all results
* Data migration moved to manual script
GitOrigin-RevId: b7e6a1f07f775c8450dd97e7269cab3b68ca0eb3
* [web] cleanup archived split-test assignments from user record on login
Co-authored-by: Anna Claire Fields <anna.fields@overleaf.com>
* [migrations] purge archived split tests from all users
Co-authored-by: Anna Claire Fields <anna.fields@overleaf.com>
* [web] add missing mock and update snapshot test
* [web] gracefully access db.users.splitTests
---------
Co-authored-by: Anna Claire Fields <anna.fields@overleaf.com>
GitOrigin-RevId: bd185074a402556d7b7c812208cf834dd52b27a5
Migrates the Overleaf monorepo package manager from npm (v11) to Yarn 4 (v4.9.1) using node-modules linker mode.
GitOrigin-RevId: 50d32ab01955c15e29679eff9e9e9cfb897fab2d
* feat: migrate from aiErrorAssist naming for disabling AI features to aiFeatures.enabled to avoid confusion
feat: keep aiErrorAssistant as setting on user object until migration is run
* feat: migrate writefull.enabled unset to instead use promotionSet false
* Update tools/migrations/20260213102825_swap_writefull_enabled_for_initialized.mjs
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
* feat: addind backfill of ai features as a migration as well to help with dev environment setup
---------
Co-authored-by: Mathias Jakobsen <mathias.jakobsen@overleaf.com>
GitOrigin-RevId: fd3f0fc07c91678cd7fd1e0d5237221694b3027b
* [monorepo] enable caching for eslint/prettier/stylelint
* [monorepo] speed up prettier by swapping --list-different for --check
--list-different will print each file that it processes. We have a lot
of files in the monorepo. Using --check only prints mismatching files.
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
* [monorepo] explicitly configure prettier cache-location
This is the default location. Prettier will only discover that location
if the top level node_modules folder is writable, which is not the case
in CI. We create the .cache folder outside of docker, writable to node
inside docker.
The proper fix would be in prettier, to only check for write access in
the cache folder. Something to raise/upstream another day.
* [monorepo] run top-level format/format_fix in a single container
With the cache in place, it is much faster to use a single container.
As there is a single shared cache file, concurrent processes may see a
partially (re-)written cache file and bail out.
- all in a single container: 24s
- previous with -j4: 41s
- previous with -j8: failed due to corrupted cache file
---------
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
GitOrigin-RevId: 7850a3a980ae6c836393d97fe56a6316ffc3fa18
* Merge all .prettierignore files into top-level config
* Merge all .prettierrc files into top-level config
* Replace service-specific glob patterns in package.json format scripts with `prettier .`
* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore
* Ignore GitHub templates
* Ignore PUG templates to format them separately with `format:pug`
* Encourage double quotes for YAML, YML files
* Move prettier for PUG source format script to the root
* Move prettier for styles source format script to the root
* Remove prettier for jenkins files from web
* Remove prettier source format script from all services
* Make .prettierrc more readable
* Update format scripts by file type
* Organise `.prettierignore`
* Add `--cache` flag to prettier scripts for faster runs
* Format all files with prettier
* Format all or format services
* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`
* Avoid conlficts with yamllint configuration
* Remove `--cache` flag from prettier scripts
* Update all service Makefiles to use root-level prettier configuration
* Update all Jenkinsfile to use root-level prettier configuration
* Ignore auto-generated files by build_scripts
* Update package-lock.json
* Update root Makefile format targets
* Update SP Jenkinsfile format target
* Update E2E Makefile format script
* Udpate `format_js` to work in both local and CI env
* Add docker-mailtrap to .prettierignore
docker-mailtrap is a third-party git-ignored directory used for testing
* Added Docker env detection to prevent nested Docker spawning
* Ignore handlebars templates
* Add cryptographic files and test output to `.prettierignore`
* Add terraform modules to `.gitignore`
* Remove prettier-plugin-groovy
* Use npx directly instead of Docker for local formatting for faster formatting
* Auto-generate Makefiles
* Revert "Remove prettier-plugin-groovy"
This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.
* Mount monorepo root in RUN_LINT_FORMAT for prettier config access
* Prettier ignores all `node_modules` by default regardless of location
* Show only changed files in format output
* Ignore LICENSE files
* Enable prettier on rendered build_scripts outputs
* Ignoring all the template folders by prettier
* Remove the public/minjs entry since it does not exist
* Remove all non-existent paths
* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files
* Revert "Auto-generate Makefiles"
This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.
* Revert "Use npx directly instead of Docker for local formatting for faster formatting"
This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.
* Ignore dashboard JSON files
* Ignore files generated by bin/update_build_scripts
* Remove unsupported file types from `.prettierignore`
* Ignore test fixture generated files
* Ignore README file types by prettier
* Ignore generate snapshots by prettier
* Allow to format generated bin/update_build_scripts by prettier
* Ensure build script outputs prettier-compatible tsconfig.json
* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double
* Don't read cryptographic files by prettier
* Ignore google verification files by prettier
* Revert npx prettier formatting
* Ignore domain verification files
* Show only changed files in format output
* Make `.github` prettier
* Allow all files to be formatted in jobs by prettier
* Allow server-ce/server-pro files to be formatted by prettier
* Ignore more folders in clsi, filestory, git-bridge by prettier
* Update build script with `RUN_LINTING_CI_MONOREPO`
* Ignore docker-mailtrap and downloads in server-ce by prettier
* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)
* Source format
GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
* [monorepo] record ERROR/FATAL log messages in junit report
* [web] put SaaS specific code behind feature flag
* [web] use split test cache for getting user assignments
The unit tests needed updating as they did not replicate any of the
mongo filtering. The acceptance tests cover this logic.
* [web] make better use of existing indexes
* [web] avoid col-scan in tests of notifications module
* [web] remove cleanup of empty feedbacks collection
* [web] add assertion for reason of rejected request in launchpad test
* [web] add missing indexes
* [web] enable mongo notablescan
* [web] make emailNotifications tests compatible with notablescan
GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
* Implement notifications preferences model
* NotificationPreferences class
* remove index performance tests
* move to unit tests
* use zod
* saving multiple preference values
* set unique:true
* throw error if preferences can't be parsed
* use z.infer
* strict schema when saving preferences
* fix: update notifPreferenceKey type to only contain key values
---------
Co-authored-by: Domagoj Kriskovic <dom.kriskovic@overleaf.com>
GitOrigin-RevId: 20cf3799c91e6a890ab6217667594d181237c791
* Add `project-created` audit log only for managed users
* Include project audit logs in group audit logs
* Added details column in Group Audit Logs UI
GitOrigin-RevId: 96c7a31b37270912df1629e27d905b692f28da46
* [migrations] make old migrations compatible with --notablescan in mongo
Re-running the migration for db.users.emails.reversedHostname on all
Server Pro/CE users will not be harmfull.
* [monorepo] enable mongo notablescan for all services but web
GitOrigin-RevId: e2a710c9de08de9ac2aa9df1f3082d5f1c33ea05
* Revert "Revert "[web] Add User logs to Group Audit Logs view (#29155)" (#29479)"
This reverts commit 40a1516ab9cec690d0487a0a870b9fab17598d60.
* Fix `managedUsersEnabled` flag in frontend
GitOrigin-RevId: ae3edf5bcbc01ec46bc18028e758d3364072c307
Eric Mc Sween