agw/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2026-11-15 06:33:55 +01:00
Code Issues Packages Projects Releases Wiki Activity

Centralize prettier configuration to root level (#30501)

Browse Source 
* Merge all .prettierignore files into top-level config

* Merge all .prettierrc files into top-level config

* Replace service-specific glob patterns in package.json format scripts with `prettier .`

* Add template files with Jinja2, Go template, envsubst, and Handlebars syntax to .prettierignore

* Ignore GitHub templates

* Ignore PUG templates to format them separately with `format:pug`

* Encourage double quotes for YAML, YML files

* Move prettier for PUG source format script to the root

* Move prettier for styles source format script to the root

* Remove prettier for jenkins files from web

* Remove prettier source format script from all services

* Make .prettierrc more readable

* Update format scripts by file type

* Organise `.prettierignore`

* Add `--cache` flag to prettier scripts for faster runs

* Format all files with prettier

* Format all or format services

* Remove `format`/`format:fix` scripts from services since now it runs from root `package.json`

* Avoid conlficts with yamllint configuration

* Remove `--cache` flag from prettier scripts

* Update all service Makefiles to use root-level prettier configuration

* Update all Jenkinsfile to use root-level prettier configuration

* Ignore auto-generated files by build_scripts

* Update package-lock.json

* Update root Makefile format targets

* Update SP Jenkinsfile format target

* Update E2E Makefile format script

* Udpate `format_js` to work in both local and CI env

* Add docker-mailtrap to .prettierignore

docker-mailtrap is a third-party git-ignored directory used for testing

* Added Docker env detection to prevent nested Docker spawning

* Ignore handlebars templates

* Add cryptographic files and test output to `.prettierignore`

* Add terraform modules to `.gitignore`

* Remove prettier-plugin-groovy

* Use npx directly instead of Docker for local formatting for faster formatting

* Auto-generate Makefiles

* Revert "Remove prettier-plugin-groovy"

This reverts commit 194a33589a2e1e4d2225d10c67e9f025e4222025.

* Mount monorepo root in RUN_LINT_FORMAT for prettier config access

* Prettier ignores all `node_modules` by default regardless of location

* Show only changed files in format output

* Ignore LICENSE files

* Enable prettier on rendered build_scripts outputs

* Ignoring all the template folders by prettier

* Remove the public/minjs entry since it does not exist

* Remove all non-existent paths

* Sync `.prettierignore` with ignored files by `.gitignore` and `.dockerignore` files

* Revert "Auto-generate Makefiles"

This reverts commit c0233e490de1bc95fe437219d65e0b66e0331ec9.

* Revert "Use npx directly instead of Docker for local formatting for faster formatting"

This reverts commit 1d2b2cf1a6c6974c76885852a90dd55e84167e41.

* Ignore dashboard JSON files

* Ignore files generated by bin/update_build_scripts

* Remove unsupported file types from `.prettierignore`

* Ignore test fixture generated files

* Ignore README file types by prettier

* Ignore generate snapshots by prettier

* Allow to format generated bin/update_build_scripts by prettier

* Ensure build script outputs prettier-compatible tsconfig.json

* Fix build script output to match prettier formatting
- Fix Jinja2 whitespace in docker-compose templates
- Change YAML quotes from single to double

* Don't read cryptographic files by prettier

* Ignore google verification files by prettier

* Revert npx prettier formatting

* Ignore domain verification files

* Show only changed files in format output

* Make `.github` prettier

* Allow all files to be formatted in jobs by prettier

* Allow server-ce/server-pro files to be formatted by prettier

* Ignore more folders in clsi, filestory, git-bridge by prettier

* Update build script with `RUN_LINTING_CI_MONOREPO`

* Ignore docker-mailtrap and downloads in server-ce by prettier

* Restore prettier configs and prettierignore for V1 since it has its own prettier (an older version)

* Source format

GitOrigin-RevId: 637adc3cc422d1f20c86d6ebc8ec514d60758287
This commit is contained in:
Rebeka Dekany
2026-02-04 09:08:22 +00:00
committed by Copybot
parent 9b10da6eaf
commit a648015db8
141 changed files with 770 additions and 992 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/chat/Makefile
+5 -6
View File
@@ -52,11 +52,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -77,13 +76,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/chat
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/chat
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/chat
lint:
$(RUN_LINTING) lint
services/chat/README.md
+2 -4
View File
@@ -1,10 +1,8 @@
overleaf/chat
===============
# overleaf/chat
The backend API that powers the chat service in Overleaf
License
-------
## License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the `LICENSE` file.
services/chat/chat.yaml
+56 -56
View File
@@ -3,12 +3,12 @@ x-stoplight:
id: okoe8mh50pjec
info:
title: chat
version: '1.0'
version: "1.0"
servers:
- url: 'http://chat:3010'
- url: "http://chat:3010"
x-exegesis-controller: messagesController
paths:
'/project/{projectId}/messages':
"/project/{projectId}/messages":
parameters:
- schema:
type: string
@@ -19,14 +19,14 @@ paths:
summary: Get Global messages
tags: []
responses:
'201':
"201":
description: OK
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/Message'
$ref: "#/components/schemas/Message"
operationId: getGlobalMessages
description: Get global messages for the project with Project ID provided
parameters:
@@ -42,21 +42,21 @@ paths:
summary: Send Global message
operationId: sendGlobalMessage
responses:
'201':
"201":
description: OK
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Message'
$ref: "#/components/schemas/Message"
examples:
example-1:
value:
user_id: string
content: string
description: 'UserID and Content of the message to be posted. '
description: "UserID and Content of the message to be posted. "
description: Send global message for the project with Project ID provided
'/project/{projectId}/messages/{messageId}':
"/project/{projectId}/messages/{messageId}":
parameters:
- schema:
type: string
@@ -72,13 +72,13 @@ paths:
summary: Get single global message
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/Message'
'404':
$ref: "#/components/schemas/Message"
"404":
description: Message not found
operationId: getGlobalMessage
description: Get a single global message by message ID for the project with Project ID provided
@@ -86,10 +86,10 @@ paths:
summary: Delete global message
operationId: deleteGlobalMessage
responses:
'204':
"204":
description: No Content
description: 'Delete global message'
'/project/{projectId}/thread/{threadId}/messages':
description: "Delete global message"
"/project/{projectId}/thread/{threadId}/messages":
parameters:
- schema:
type: string
@@ -105,19 +105,19 @@ paths:
summary: Send message
operationId: sendMessage
responses:
'201':
"201":
description: Created
description: Add a message to the thread with thread ID provided from the Project with Project ID provided.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Message'
$ref: "#/components/schemas/Message"
description: |-
JSON object with :
- user_id: Id of the user
- content: Content of the message
'/project/{projectId}/threads':
"/project/{projectId}/threads":
parameters:
- schema:
type: string
@@ -128,20 +128,20 @@ paths:
summary: Get Threads
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/Thread'
$ref: "#/components/schemas/Thread"
examples: {}
'404':
"404":
description: Not Found
operationId: getThreads
description: Get the list of threads for the project with Project ID provided
'/project/{projectId}/thread/{threadId}/messages/{messageId}/edit':
"/project/{projectId}/thread/{threadId}/messages/{messageId}/edit":
parameters:
- schema:
type: string
@@ -162,9 +162,9 @@ paths:
summary: Edit message
operationId: editMessage
responses:
'204':
"204":
description: No Content
'404':
"404":
description: Not Found
requestBody:
content:
@@ -186,7 +186,7 @@ paths:
- user_id: Id of the user (optional)
description: |
Update message with Message ID provided from the Thread ID and Project ID provided
'/project/{projectId}/messages/{messageId}/edit':
"/project/{projectId}/messages/{messageId}/edit":
parameters:
- schema:
type: string
@@ -202,9 +202,9 @@ paths:
summary: Edit global message
operationId: editGlobalMessage
responses:
'204':
"204":
description: No Content
'404':
"404":
description: Not Found
requestBody:
content:
@@ -226,7 +226,7 @@ paths:
- user_id: Id of the user (optional)
description: |
Update global message with Message ID provided from the Project ID provided
'/project/{projectId}/thread/{threadId}/messages/{messageId}':
"/project/{projectId}/thread/{threadId}/messages/{messageId}":
parameters:
- schema:
type: string
@@ -247,13 +247,13 @@ paths:
summary: Get thread message
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/Message'
'404':
$ref: "#/components/schemas/Message"
"404":
description: Message not found
operationId: getThreadMessage
description: Get a specific message by message ID from the thread with Thread ID and Project ID provided
@@ -261,10 +261,10 @@ paths:
summary: Delete message
operationId: deleteMessage
responses:
'204':
"204":
description: No Content
description: 'Delete message with Message ID provided, from the Thread with ThreadID and ProjectID provided'
'/project/{projectId}/thread/{threadId}/user/{userId}/messages/{messageId}':
description: "Delete message with Message ID provided, from the Thread with ThreadID and ProjectID provided"
"/project/{projectId}/thread/{threadId}/user/{userId}/messages/{messageId}":
parameters:
- schema:
type: string
@@ -290,9 +290,9 @@ paths:
summary: Delete message written by a given user
operationId: deleteUserMessage
responses:
'204':
"204":
description: No Content
'/project/{projectId}/thread/{threadId}/resolve':
"/project/{projectId}/thread/{threadId}/resolve":
parameters:
- schema:
type: string
@@ -308,7 +308,7 @@ paths:
summary: Resolve Thread
operationId: resolveThread
responses:
'204':
"204":
description: No Content
requestBody:
content:
@@ -324,7 +324,7 @@ paths:
JSON object with :
- user_id: Id of the user.
description: Mark Thread with ThreadID and ProjectID provided owned by the user with UserID provided as resolved.
'/project/{projectId}/thread/{threadId}/reopen':
"/project/{projectId}/thread/{threadId}/reopen":
parameters:
- schema:
type: string
@@ -340,12 +340,12 @@ paths:
summary: Reopen Thread
operationId: reopenThread
responses:
'204':
"204":
description: No Content
description: |-
Reopen Thread with ThreadID and ProjectID provided.
i.e unmark it as resolved.
'/project/{projectId}/thread/{threadId}':
"/project/{projectId}/thread/{threadId}":
parameters:
- schema:
type: string
@@ -361,13 +361,13 @@ paths:
summary: Get Thread
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/Thread'
'404':
$ref: "#/components/schemas/Thread"
"404":
description: Thread not found
operationId: getThread
description: Get a thread with ThreadID and ProjectID provided
@@ -375,10 +375,10 @@ paths:
summary: Delete thread
operationId: deleteThread
responses:
'204':
"204":
description: No Content
description: Delete thread with ThreadID and ProjectID provided
'/project/{projectId}/resolved-thread-ids':
"/project/{projectId}/resolved-thread-ids":
parameters:
- schema:
type: string
@@ -389,9 +389,9 @@ paths:
summary: Get resolved thread ids
operationId: getResolvedThreadIds
responses:
'200':
"200":
description: Resolved thread ids
'/project/{projectId}':
"/project/{projectId}":
parameters:
- schema:
type: string
@@ -402,15 +402,15 @@ paths:
summary: Destroy project
operationId: destroyProject
responses:
'204':
"204":
description: No Content
description: 'Delete all threads from Project with Project ID provided, and all messages in those threads.'
description: "Delete all threads from Project with Project ID provided, and all messages in those threads."
/status:
get:
summary: Check status
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
@@ -423,7 +423,7 @@ paths:
summary: Check status
tags: []
responses:
'200':
"200":
description: OK
content:
application/json:
@@ -432,7 +432,7 @@ paths:
description: chat is alive
operationId: getStatus
description: Check that the Chat service is alive
'/project/{projectId}/duplicate-comment-threads':
"/project/{projectId}/duplicate-comment-threads":
parameters:
- schema:
type: string
@@ -453,7 +453,7 @@ paths:
items:
type: string
responses:
'200':
"200":
content:
application/json:
schema:
@@ -463,7 +463,7 @@ paths:
type: object
description: Mapping of old thread ids to their duplicated thread ids
description: Duplicate a list of comment threads
'/project/{projectId}/generate-thread-data':
"/project/{projectId}/generate-thread-data":
parameters:
- schema:
type: string
@@ -484,7 +484,7 @@ paths:
items:
type: string
responses:
'200':
"200":
content:
application/json:
schema:
@@ -514,4 +514,4 @@ components:
id: 0ppt3jw4h5bua
type: array
items:
$ref: '#/components/schemas/Message'
$ref: "#/components/schemas/Message"
services/chat/docker-compose.ci.yml
+1 -2
View File
@@ -20,7 +20,6 @@ services:
mongo:
condition: service_started
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -41,7 +40,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -49,6 +47,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
mongo:
image: mongo:8.0.11
command: --replSet overleaf --notablescan
services/chat/docker-compose.yml
-1
View File
@@ -62,4 +62,3 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
services/chat/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:acceptance:_run": "mocha --recursive --timeout 15000 --exit $@ test/acceptance/js",
"test:unit:_run": "mocha --recursive --exit $@ test/unit/js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/clsi/Makefile
+5 -6
View File
@@ -52,11 +52,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -77,13 +76,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/clsi
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/clsi
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/clsi
lint:
$(RUN_LINTING) lint
services/clsi/docker-compose.ci.yml
-2
View File
@@ -16,7 +16,6 @@ services:
NODE_OPTIONS: "--unhandled-rejections=strict"
VITEST_NO_CACHE: true
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -43,7 +42,6 @@ services:
- /var/run/docker.sock:/var/run/docker.sock
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
services/clsi/docker-compose.yml
-1
View File
@@ -51,4 +51,3 @@ services:
SANDBOXED_COMPILES_HOST_DIR_COMPILES: $PWD/compiles
SANDBOXED_COMPILES_HOST_DIR_OUTPUT: $PWD/output
command: npm run --silent test:acceptance
services/clsi/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:unit": "npm run test:unit:_run",
"nodemon": "node --watch app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/clsi/test/acceptance/fixtures/examples/draft_legacy/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"draft": true
}
}
services/clsi/test/acceptance/fixtures/examples/draft_mode/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"draft": true
}
}
services/clsi/test/acceptance/fixtures/examples/feynmp/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "latex"
"compiler": "latex"
}
services/clsi/test/acceptance/fixtures/examples/fontawesome_xelatex/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "xelatex"
"compiler": "xelatex"
}
services/clsi/test/acceptance/fixtures/examples/latex_compiler/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "latex"
"compiler": "latex"
}
services/clsi/test/acceptance/fixtures/examples/lualatex_compiler/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "lualatex"
"compiler": "lualatex"
}
services/clsi/test/acceptance/fixtures/examples/tikz_feynman/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "lualatex"
"compiler": "lualatex"
}
services/clsi/test/acceptance/fixtures/examples/xelatex_compiler/options.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
"compiler": "xelatex"
"compiler": "xelatex"
}
services/contacts/Makefile
+5 -6
View File
@@ -52,11 +52,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/contacts/reports:/overleaf/services/contacts/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/contacts/reports:/overleaf/services/contacts/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -77,13 +76,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/contacts
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/contacts
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/contacts
lint:
$(RUN_LINTING) lint
services/contacts/README.md
+2 -4
View File
@@ -1,10 +1,8 @@
overleaf/contacts
===================
# overleaf/contacts
An API for tracking contacts of a user
License
-------
## License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the `LICENSE` file.
services/contacts/docker-compose.ci.yml
+1 -2
View File
@@ -20,7 +20,6 @@ services:
mongo:
condition: service_started
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -41,7 +40,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -49,6 +47,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
mongo:
image: mongo:8.0.11
command: --replSet overleaf --notablescan
services/contacts/docker-compose.yml
-1
View File
@@ -62,4 +62,3 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
services/contacts/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:unit": "npm run test:unit:_run -- --grep=$MOCHA_GREP",
"nodemon": "node --watch app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/docstore/Makefile
+5 -6
View File
@@ -54,11 +54,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -79,13 +78,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/docstore
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/docstore
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/docstore
lint:
$(RUN_LINTING) lint
services/docstore/README.md
+2 -4
View File
@@ -1,10 +1,8 @@
overleaf/docstore
===================
# overleaf/docstore
A CRUD API for storing and updating text documents in projects
License
-------
## License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the `LICENSE` file.
services/docstore/docker-compose.ci.yml
+2 -2
View File
@@ -22,7 +22,6 @@ services:
mongo:
condition: service_started
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -48,7 +47,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -56,6 +54,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
mongo:
image: mongo:8.0.11
command: --replSet overleaf --notablescan
@@ -70,6 +69,7 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
gcs:
image: fsouza/fake-gcs-server:1.52.3
command: ["--port=9090", "--scheme=http", "--external-url=http://gcs:9090"]
services/docstore/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:unit": "npm run test:unit:_run",
"nodemon": "node --watch app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/document-updater/Makefile
+5 -6
View File
@@ -55,11 +55,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -80,13 +79,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/document-updater
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/document-updater
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/document-updater
lint:
$(RUN_LINTING) lint
services/document-updater/README.md
+2 -5
View File
@@ -1,12 +1,9 @@
overleaf/document-updater
===========================
# overleaf/document-updater
An API for applying incoming updates to documents in real-time.
License
-------
## License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the `LICENSE` file.
Copyright (c) Overleaf, 2014-2019.
services/document-updater/app/js/sharejs/README.md
-1
View File
@@ -3,4 +3,3 @@ This folder contains a modified version of the ShareJS source code, forked from
The original CoffeeScript code has been decaffeinated to JavaScript, and further modified. Some folders have been removed. See https://github.com/josephg/ShareJS/blob/v0.5.0/src/types/README.md for the original README.
The original code, and the current modified code in this directory, are published under the MIT license.
services/document-updater/app/js/sharejs/types/README.md
-4
View File
@@ -3,7 +3,6 @@ This directory contains all the operational transform code. Each file defines a
Most of the types in here are for testing or demonstration. The only types which are sent to the webclient
are `text` and `json`.
# An OT type
All OT types have the following fields:
@@ -18,7 +17,6 @@ Given op1, op2, `apply(s, op2, transform(op1, op2, 'left')) == apply(s, op1, tra
Transform and apply must never modify their arguments.
Optional properties:
`tp2`: _(bool)_ True if the transform function supports TP2. This allows p2p architectures to work.
@@ -29,14 +27,12 @@ Optional properties:
`normalize(op) -> op`: Fix up an op to make it valid. Eg, remove skips of size zero.
`api`: _(object)_ Set of helper methods which will be mixed in to the client document object for manipulating documents. See below.
# Examples
`count` and `simple` are two trivial OT type definitions if you want to take a look. JSON defines
the ot-for-JSON type (see the wiki for documentation) and all the text types define different text
implementations. (I still have no idea which one I like the most, and they're fun to write!)
# API
Types can also define API functions. These methods are mixed into the client's Doc object when a document is created.
services/document-updater/docker-compose.ci.yml
+1 -2
View File
@@ -27,7 +27,6 @@ services:
redis_test:
condition: service_healthy
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -55,7 +54,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -63,6 +61,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
redis_test:
image: redis:7.4.3
healthcheck:
services/document-updater/docker-compose.yml
-1
View File
@@ -83,4 +83,3 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
services/document-updater/package.json
-2
View File
@@ -12,8 +12,6 @@
"nodemon": "node --watch app.js",
"benchmark:apply": "node benchmarks/apply",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/filestore/.prettierignore
-1
View File
@@ -1 +0,0 @@
node_modules
services/filestore/Makefile
+5 -6
View File
@@ -52,11 +52,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -77,13 +76,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/filestore
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/filestore
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/filestore
lint:
$(RUN_LINTING) lint
services/filestore/README.md
+10 -12
View File
@@ -1,21 +1,19 @@
overleaf/filestore
====================
# overleaf/filestore
An API for CRUD operations on binary files stored in S3
filestore acts as a proxy between the CLSIs and (currently) Amazon S3 storage, presenting a RESTful HTTP interface to the CLSIs on port 3009 by default. Urls are mapped to node functions in https://github.com/overleaf/filestore/blob/master/app.coffee . URLs are of the form:
* `/project/:project_id/file/:file_id`
* `/template/:template_id/v/:version/:format`
* `/project/:project_id/public/:public_file_id`
* `/project/:project_id/size`
* `/bucket/:bucket/key/*`
* `/shutdown`
* `/status` - returns HTTP 200 `filestore is up` or HTTP 503 when shutting down
* `/health_check`
- `/project/:project_id/file/:file_id`
- `/template/:template_id/v/:version/:format`
- `/project/:project_id/public/:public_file_id`
- `/project/:project_id/size`
- `/bucket/:bucket/key/*`
- `/shutdown`
- `/status` - returns HTTP 200 `filestore is up` or HTTP 503 when shutting down
- `/health_check`
License
-------
## License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the `LICENSE` file.
services/filestore/docker-compose.ci.yml
+5 -5
View File
@@ -20,7 +20,6 @@ services:
NODE_OPTIONS: "--unhandled-rejections=strict"
VITEST_NO_CACHE: true
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -29,7 +28,7 @@ services:
MONGO_HOST: mongo
POSTGRES_HOST: postgres
AWS_S3_ENDPOINT: https://minio:9000
AWS_S3_PATH_STYLE: 'true'
AWS_S3_PATH_STYLE: "true"
DELETE_OBJECTS_MD5_FALLBACK: true
AWS_ACCESS_KEY_ID: OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY
@@ -64,7 +63,6 @@ services:
user: node
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -72,6 +70,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
certs:
build:
dockerfile_inline: |
@@ -83,7 +82,7 @@ services:
working_dir: /certs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
if [ ! -f private.key ] || [ ! -f public.crt ]; then
/certgen -host minio
@@ -115,7 +114,7 @@ services:
- minio-certs:/root/.mc/certs/CAs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
sleep 1
mc alias set s3 https://minio:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD \
@@ -218,6 +217,7 @@ services:
mc admin policy create s3 overleaf-filestore policy-filestore.json
mc admin policy attach s3 overleaf-filestore \
--user=OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID
gcs:
image: fsouza/fake-gcs-server:1.52.3
command: ["--port=9090", "--scheme=http", "--external-url=http://gcs:9090"]
services/filestore/docker-compose.yml
+4 -3
View File
@@ -41,7 +41,7 @@ services:
MONGO_CONNECTION_STRING: mongodb://mongo/test-overleaf
POSTGRES_HOST: postgres
AWS_S3_ENDPOINT: https://minio:9000
AWS_S3_PATH_STYLE: 'true'
AWS_S3_PATH_STYLE: "true"
DELETE_OBJECTS_MD5_FALLBACK: true
AWS_ACCESS_KEY_ID: OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY
@@ -85,7 +85,7 @@ services:
working_dir: /certs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
if [ ! -f private.key ] || [ ! -f public.crt ]; then
/certgen -host minio
@@ -117,7 +117,7 @@ services:
- minio-certs:/root/.mc/certs/CAs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
sleep 1
mc alias set s3 https://minio:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD \
@@ -220,6 +220,7 @@ services:
mc admin policy create s3 overleaf-filestore policy-filestore.json
mc admin policy attach s3 overleaf-filestore \
--user=OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID
gcs:
image: fsouza/fake-gcs-server:1.52.3
command: ["--port=9090", "--scheme=http", "--external-url=http://gcs:9090"]
services/filestore/package.json
-2
View File
@@ -12,8 +12,6 @@
"start": "node app.js",
"nodemon": "node --watch app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"test:acceptance:_run": "mocha --recursive --timeout 15000 --exit $@ test/acceptance/js",
"test:unit:_run": "vitest --config ./vitest.config.unit.cjs",
"lint:fix": "eslint --fix .",
services/git-bridge/README.md
+3 -4
View File
@@ -3,7 +3,7 @@
## Docker
The `Dockerfile` contains all the requirements for building and running the
writelatex-git-bridge.
writelatex-git-bridge.
```bash
# build the image
@@ -17,8 +17,8 @@ docker run -v `pwd`/conf/local.json:/conf/runtime.json writelatex-git-bridge
### Required packages
* `maven` (for building, running tests and packaging)
* `jdk-8` (for compiling and running)
- `maven` (for building, running tests and packaging)
- `jdk-8` (for compiling and running)
### Commands
@@ -114,7 +114,6 @@ The configuration file is in `.json` format.
You have to restart the server for configuration changes to take effect.
## Creating OAuth app
In dev-env, run the following command in mongo to create the oauth application
services/git-bridge/conf/example_config.json
+28 -28
View File
@@ -1,30 +1,30 @@
{
"port": 8080,
"bindIp": "127.0.0.1",
"idleTimeout": 30000,
"rootGitDirectory": "/tmp/wlgb",
"allowedCorsOrigins": "https://localhost",
"apiBaseUrl": "https://localhost/api/v0",
"postbackBaseUrl": "https://localhost",
"serviceName": "Overleaf",
"oauth2Server": "https://localhost",
"repoStore": {
"maxFileNum": 2000,
"maxFileSize": 52428800
},
"swapStore": {
"type": "s3",
"awsAccessKey": "asdf",
"awsSecret": "asdf",
"s3BucketName": "com.overleaf.testbucket",
"awsRegion": "us-east-1"
},
"swapJob": {
"minProjects": 50,
"lowGiB": 128,
"highGiB": 256,
"intervalMillis": 3600000,
"compressionMethod": "gzip"
},
"sqliteHeapLimitBytes": 512000000
"port": 8080,
"bindIp": "127.0.0.1",
"idleTimeout": 30000,
"rootGitDirectory": "/tmp/wlgb",
"allowedCorsOrigins": "https://localhost",
"apiBaseUrl": "https://localhost/api/v0",
"postbackBaseUrl": "https://localhost",
"serviceName": "Overleaf",
"oauth2Server": "https://localhost",
"repoStore": {
"maxFileNum": 2000,
"maxFileSize": 52428800
},
"swapStore": {
"type": "s3",
"awsAccessKey": "asdf",
"awsSecret": "asdf",
"s3BucketName": "com.overleaf.testbucket",
"awsRegion": "us-east-1"
},
"swapJob": {
"minProjects": 50,
"lowGiB": 128,
"highGiB": 256,
"intervalMillis": 3600000,
"compressionMethod": "gzip"
},
"sqliteHeapLimitBytes": 512000000
}
services/history-v1/Makefile
+5 -6
View File
@@ -57,11 +57,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -82,13 +81,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/history-v1
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/history-v1
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/history-v1
lint:
$(RUN_LINTING) lint
services/history-v1/README.md
+2
View File
@@ -3,11 +3,13 @@
The history service uses knex to manage PostgreSQL migrations.
To create a new migrations, run:
```
npx knex migrate:make migration_name
```
To apply migrations, run:
```
npx knex migrate:latest
```
services/history-v1/config/custom-environment-variables.json
+4 -4
View File
@@ -59,10 +59,10 @@
"zipTimeoutMs": "ZIP_STORE_ZIP_TIMEOUT_MS"
},
"backupStore": {
"chunksBucket":"BACKUP_OVERLEAF_EDITOR_CHUNKS_BUCKET",
"deksBucket":"BACKUP_OVERLEAF_EDITOR_DEKS_BUCKET",
"globalBlobsBucket":"BACKUP_OVERLEAF_EDITOR_GLOBAL_BLOBS_BUCKET",
"projectBlobsBucket":"BACKUP_OVERLEAF_EDITOR_PROJECT_BLOBS_BUCKET"
"chunksBucket": "BACKUP_OVERLEAF_EDITOR_CHUNKS_BUCKET",
"deksBucket": "BACKUP_OVERLEAF_EDITOR_DEKS_BUCKET",
"globalBlobsBucket": "BACKUP_OVERLEAF_EDITOR_GLOBAL_BLOBS_BUCKET",
"projectBlobsBucket": "BACKUP_OVERLEAF_EDITOR_PROJECT_BLOBS_BUCKET"
},
"healthCheckBlobs": "HEALTH_CHECK_BLOBS",
"healthCheckProjects": "HEALTH_CHECK_PROJECTS",
services/history-v1/config/development.json
+4 -4
View File
@@ -24,10 +24,10 @@
"bucket": "overleaf-development-zips"
},
"backupStore": {
"chunksBucket":"overleaf-development-history-chunks",
"deksBucket":"overleaf-development-history-deks",
"globalBlobsBucket":"overleaf-development-history-global-blobs",
"projectBlobsBucket":"overleaf-development-history-project-blobs"
"chunksBucket": "overleaf-development-history-chunks",
"deksBucket": "overleaf-development-history-deks",
"globalBlobsBucket": "overleaf-development-history-global-blobs",
"projectBlobsBucket": "overleaf-development-history-project-blobs"
},
"backupPersistor": {
"keyEncryptionKeys": "[{\"key\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\",\"salt\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"}]",
services/history-v1/config/test.json
+4 -4
View File
@@ -22,10 +22,10 @@
"bucket": "overleaf-test-zips"
},
"backupStore": {
"chunksBucket":"overleaf-test-history-chunks",
"deksBucket":"overleaf-test-history-deks",
"globalBlobsBucket":"overleaf-test-history-global-blobs",
"projectBlobsBucket":"overleaf-test-history-project-blobs"
"chunksBucket": "overleaf-test-history-chunks",
"deksBucket": "overleaf-test-history-deks",
"globalBlobsBucket": "overleaf-test-history-global-blobs",
"projectBlobsBucket": "overleaf-test-history-project-blobs"
},
"backupPersistor": {
"keyEncryptionKeys": "[{\"key\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\",\"salt\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"}]",
services/history-v1/docker-compose.ci.yml
+6 -5
View File
@@ -30,7 +30,6 @@ services:
redis_test:
condition: service_healthy
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -44,7 +43,7 @@ services:
MONGO_HOST: mongo
POSTGRES_HOST: postgres
AWS_S3_ENDPOINT: https://minio:9000
AWS_S3_PATH_STYLE: 'true'
AWS_S3_PATH_STYLE: "true"
DELETE_OBJECTS_MD5_FALLBACK: true
AWS_ACCESS_KEY_ID: OVERLEAF_HISTORY_S3_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY
@@ -79,7 +78,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -87,6 +85,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
redis_test:
image: redis:7.4.3
healthcheck:
@@ -108,6 +107,7 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
postgres:
image: postgres:10
environment:
@@ -132,7 +132,7 @@ services:
working_dir: /certs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
if [ ! -f private.key ] || [ ! -f public.crt ]; then
/certgen -host minio
@@ -164,7 +164,7 @@ services:
- minio-certs:/root/.mc/certs/CAs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
sleep 1
mc alias set s3 https://minio:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD \
@@ -267,6 +267,7 @@ services:
mc admin policy create s3 overleaf-history policy-history.json
mc admin policy attach s3 overleaf-history \
--user=OVERLEAF_HISTORY_S3_ACCESS_KEY_ID
gcs:
image: fsouza/fake-gcs-server:1.52.3
command: ["--port=9090", "--scheme=http", "--external-url=http://gcs:9090"]
services/history-v1/docker-compose.yml
+4 -3
View File
@@ -60,7 +60,7 @@ services:
MONGO_CONNECTION_STRING: mongodb://mongo/test-overleaf
POSTGRES_HOST: postgres
AWS_S3_ENDPOINT: https://minio:9000
AWS_S3_PATH_STYLE: 'true'
AWS_S3_PATH_STYLE: "true"
DELETE_OBJECTS_MD5_FALLBACK: true
AWS_ACCESS_KEY_ID: OVERLEAF_HISTORY_S3_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY
@@ -138,7 +138,7 @@ services:
working_dir: /certs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
if [ ! -f private.key ] || [ ! -f public.crt ]; then
/certgen -host minio
@@ -170,7 +170,7 @@ services:
- minio-certs:/root/.mc/certs/CAs
entrypoint: sh
command:
- '-cex'
- "-cex"
- |
sleep 1
mc alias set s3 https://minio:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD \
@@ -273,6 +273,7 @@ services:
mc admin policy create s3 overleaf-history policy-history.json
mc admin policy attach s3 overleaf-history \
--user=OVERLEAF_HISTORY_S3_ACCESS_KEY_ID
gcs:
image: fsouza/fake-gcs-server:1.52.3
command: ["--port=9090", "--scheme=http", "--external-url=http://gcs:9090"]
services/history-v1/package.json
-2
View File
@@ -64,8 +64,6 @@
"start": "node app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"lint:fix": "eslint --fix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"test:unit": "npm run test:unit:_run -- --grep=$MOCHA_GREP",
"test:acceptance": "npm run test:acceptance:_run -- --grep=$MOCHA_GREP",
"test:unit:_run": "mocha --recursive --exit $@ test/unit/js",
services/history-v1/storage/scripts/global-blobs-db-cleanup/README.md
+4 -4
View File
@@ -1,9 +1,9 @@
Scripts in this directory were used when we cleaned up the global blobs table,
ensuring that it only contained global blobs. The scripts are meant to be run in this order:
* `01-create-blob-hashes-table.sql`
* `02-set-global-flag.sql`
* `03-create-global-blobs-table.sql`
* `04-swap-global-blob-tables.sql`
- `01-create-blob-hashes-table.sql`
- `02-set-global-flag.sql`
- `03-create-global-blobs-table.sql`
- `04-swap-global-blob-tables.sql`
The `rollback.sql` can be run to reverse the effect of `03-swap-global-blob-tables.sql`.
services/notifications/Makefile
+5 -6
View File
@@ -53,11 +53,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -78,13 +77,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/notifications
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/notifications
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/notifications
lint:
$(RUN_LINTING) lint
services/notifications/README.md
+5 -8
View File
@@ -1,20 +1,17 @@
overleaf/notifications
===============
# overleaf/notifications
An API for managing user notifications in Overleaf
database indexes
================
# database indexes
For notification expiry to work, a TTL index on `notifications.expires` must be created:
```javascript
db.notifications.createIndex({expires: 1}, {expireAfterSeconds: 10})
db.notifications.createIndex({ expires: 1 }, { expireAfterSeconds: 10 })
```
License
=======
# License
The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3.
Copyright (c) Overleaf, 2016–2019.
services/notifications/docker-compose.ci.yml
+1 -2
View File
@@ -22,7 +22,6 @@ services:
mongo:
condition: service_started
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -44,7 +43,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -52,6 +50,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
mongo:
image: mongo:8.0.11
command: --replSet overleaf --notablescan
services/notifications/docker-compose.yml
-1
View File
@@ -64,4 +64,3 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
services/notifications/package.json
-2
View File
@@ -11,8 +11,6 @@
"test:unit:_run": "vitest --config ./vitest.config.unit.cjs",
"test:unit": "npm run test:unit:_run",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit",
"nodemon": "node --watch app.ts"
services/project-history/Makefile
+5 -6
View File
@@ -56,11 +56,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/app/js/types.ts:/overleaf/services/document-updater/app/js/types.ts --volume $(MONOREPO)/services/project-history/reports:/overleaf/services/project-history/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/app/js/types.ts:/overleaf/services/document-updater/app/js/types.ts --volume $(MONOREPO)/services/project-history/reports:/overleaf/services/project-history/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -81,13 +80,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/project-history
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/project-history
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/project-history
lint:
$(RUN_LINTING) lint
services/project-history/README.md
+5 -10
View File
@@ -1,10 +1,8 @@
@overleaf/project-history
==========================
# @overleaf/project-history
An API for converting raw editor updates into a compressed and browseable history.
Running project-history
-----------------------
## Running project-history
The app runs natively using npm and Node on the local system:
@@ -13,8 +11,7 @@ npm install
npm run start
```
Unit Tests
----------
## Unit Tests
The test suites run in Docker.
@@ -33,8 +30,7 @@ During development it is often useful to only run a subset of tests, which can b
make test_unit MOCHA_ARGS='--grep=AuthorizationManager'
```
Acceptance Tests
----------------
## Acceptance Tests
Acceptance tests are run against a live service, which runs in the `acceptance_test` container defined in `docker-compose.tests.yml`.
@@ -61,8 +57,7 @@ During development it is often useful to only run a subset of tests, which can b
make test_acceptance_run MOCHA_ARGS='--grep=AuthorizationManager'
```
Makefile and npm scripts
------------------------
## Makefile and npm scripts
The commands used to compile the app and tests, to run the mocha tests, and to run the app are all in `package.json`. These commands call out to `coffee`, `mocha`, etc which are available to `npm` in the local `node_modules/.bin` directory, using the local versions. Normally, these commands should not be run directly, but instead run in docker via make.
services/project-history/docker-compose.ci.yml
+1 -2
View File
@@ -27,7 +27,6 @@ services:
redis_test:
condition: service_healthy
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -55,7 +54,6 @@ services:
entrypoint: /overleaf/bin/shared/wait_for_it mongo:27017 --timeout=60 --
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -63,6 +61,7 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
redis_test:
image: redis:7.4.3
healthcheck:
services/project-history/docker-compose.yml
-1
View File
@@ -83,4 +83,3 @@ services:
# replica set. This override is not needed when running the setup after
# starting up mongo.
- mongo:127.0.0.1
services/project-history/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:acceptance:_run": "mocha --loader=esmock --recursive --timeout 15000 --exit $@ test/acceptance/js",
"test:unit:_run": "mocha --loader=esmock --recursive --exit $@ test/unit/js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/project-history/test/acceptance/fixtures/chunks/0-3.json
+6 -19
View File
@@ -21,17 +21,11 @@
"operations": [
{
"pathname": "main.tex",
"textOperation": [
1912,
"Hello world",
18726
]
"textOperation": [1912, "Hello world", 18726]
}
],
"timestamp": "2017-12-04T10:23:35.633Z",
"authors": [
31
]
"authors": [31]
},
{
"operations": [
@@ -41,24 +35,17 @@
}
],
"timestamp": "2017-12-04T10:27:26.874Z",
"authors": [
31
]
"authors": [31]
},
{
"operations": [
{
"pathname": "foo.tex",
"textOperation": [
26,
"\n\nFour five six"
]
"textOperation": [26, "\n\nFour five six"]
}
],
"timestamp": "2017-12-04T10:28:33.724Z",
"authors": [
31
]
"authors": [31]
}
]
},
@@ -71,4 +58,4 @@
"name": "James"
}
]
}
}
services/project-history/test/acceptance/fixtures/chunks/4-6.json
+6 -19
View File
@@ -21,32 +21,21 @@
"operations": [
{
"pathname": "foo.tex",
"textOperation": [
41,
"\n\nSeven eight nince"
]
"textOperation": [41, "\n\nSeven eight nince"]
}
],
"timestamp": "2017-12-04T10:29:17.786Z",
"authors": [
31
]
"authors": [31]
},
{
"operations": [
{
"pathname": "foo.tex",
"textOperation": [
58,
-1,
1
]
"textOperation": [58, -1, 1]
}
],
"timestamp": "2017-12-04T10:29:22.905Z",
"authors": [
31
]
"authors": [31]
},
{
"operations": [
@@ -56,9 +45,7 @@
}
],
"timestamp": "2017-12-04T10:29:26.120Z",
"authors": [
31
]
"authors": [31]
}
]
},
@@ -71,4 +58,4 @@
"name": "James"
}
]
}
}
services/project-history/test/acceptance/fixtures/chunks/7-8.json
+5 -17
View File
@@ -21,33 +21,21 @@
"operations": [
{
"pathname": "main.tex",
"textOperation": [
1923,
" also updated",
18726
]
"textOperation": [1923, " also updated", 18726]
}
],
"timestamp": "2017-12-04T10:32:47.277Z",
"authors": [
31
]
"authors": [31]
},
{
"operations": [
{
"pathname": "bar.tex",
"textOperation": [
28,
-15,
16
]
"textOperation": [28, -15, 16]
}
],
"timestamp": "2017-12-04T10:32:52.877Z",
"v2Authors": [
"5a5637efdac84e81b71014c4"
]
"v2Authors": ["5a5637efdac84e81b71014c4"]
}
]
},
@@ -60,4 +48,4 @@
"name": "James"
}
]
}
}
services/real-time/Makefile
+5 -6
View File
@@ -51,11 +51,10 @@ clean:
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(HERE) node:24.13.0 npm run --silent
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/real-time/reports:/overleaf/services/real-time/reports $(IMAGE_CI) npm run --silent
# Same but from the top of the monorepo
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) node:24.13.0 npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/real-time/reports:/overleaf/services/real-time/reports -w /overleaf $(IMAGE_CI) npm run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -76,13 +75,13 @@ shellcheck_fix:
done
format:
$(RUN_LINTING) format
$(RUN_LINTING_MONOREPO) format -- services/real-time
format_ci:
$(RUN_LINTING_CI) format
$(RUN_LINTING_CI_MONOREPO) format -- services/real-time
format_fix:
$(RUN_LINTING) format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/real-time
lint:
$(RUN_LINTING) lint
services/real-time/docker-compose.ci.yml
+1 -3
View File
@@ -25,7 +25,6 @@ services:
redis_test:
condition: service_healthy
test_acceptance:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -49,7 +48,6 @@ services:
user: node
command: npm run test:acceptance
tar:
build: .
image: ci/$PROJECT_NAME:$BRANCH_NAME-$BUILD_NUMBER
@@ -57,10 +55,10 @@ services:
- ./:/tmp/build/
command: tar -czf /tmp/build/build.tar.gz --exclude=build.tar.gz --exclude-vcs .
user: root
redis_test:
image: redis:7.4.3
healthcheck:
test: ping="$$(redis-cli ping)" && [ "$$ping" = 'PONG' ]
interval: 1s
retries: 20
services/real-time/docker-compose.yml
-1
View File
@@ -59,4 +59,3 @@ services:
test: ping=$$(redis-cli ping) && [ "$$ping" = 'PONG' ]
interval: 1s
retries: 20
services/real-time/package.json
-2
View File
@@ -12,8 +12,6 @@
"test:unit": "npm run test:unit:_run",
"nodemon": "node --watch app.js",
"lint": "eslint --max-warnings 0 --format unix .",
"format": "prettier --list-different $PWD/'**/{*.*js,*.ts}'",
"format:fix": "prettier --write $PWD/'**/{*.*js,*.ts}'",
"lint:fix": "eslint --fix .",
"types:check": "tsc --noEmit"
},
services/web/.prettierignore
-40
View File
@@ -1,40 +0,0 @@
# NOTE: changing paths may require updating them in the Makefile too.
data/
scripts/translations/.cache/
node_modules
frontend/js/vendor
modules/**/frontend/js/vendor
public/js
public/minjs
frontend/stylesheets/modules/metrics/nvd3.scss
frontend/js/features/source-editor/lezer-latex/latex.mjs
frontend/js/features/source-editor/lezer-latex/latex.terms.mjs
frontend/js/features/source-editor/lezer-bibtex/bibtex.mjs
frontend/js/features/source-editor/lezer-bibtex/bibtex.terms.mjs
frontend/js/features/source-editor/hunspell/wasm/hunspell.mjs
# complex pages
app/views/project/editor.pug
app/views/project/editor/**
modules/open-in-overleaf/app/views/documentation.pug
modules/references-search/app/views/project/editor/**
modules/rich-text/app/views/toolbar.pug
# loops
app/views/referal/bonus.pug
modules/templates/app/views/tag.pug
# expressions that could not be formatted correctly
app/views/_mixins/faq_search.pug
app/views/external/home/v2.pug
app/views/project/token/access.pug
app/views/user/primaryEmailCheck.pug
app/views/user/restricted.pug
modules/admin-panel/app/views/project/show.pug
modules/templates/app/views/project/editor/_left-menu.pug
modules/two-factor-authentication/app/views/_mixins.pug
# minified files
app/views/_google_analytics.pug
app/views/_customer_io.pug
frontend/js/shared/lottie/**
services/web/.prettierrc
-23
View File
@@ -1,23 +0,0 @@
{
"arrowParens": "avoid",
"jsxSingleQuote": false,
"pugAttributeSeparator": "as-needed",
"pugBracketSpacing": false,
"pugClassNotation": "as-is",
"pugIdNotation": "as-is",
"pugSortAttributesBeginning": ["name", "data-type"],
"plugins": ["@prettier/plugin-pug", "prettier-plugin-groovy"],
"semi": false,
"singleQuote": true,
"trailingComma": "es5",
"tabWidth": 2,
"useTabs": false,
"overrides": [
{
"files": "*.pug",
"options": {
"useTabs": true
}
}
]
}
services/web/Makefile
+25 -18
View File
@@ -431,9 +431,16 @@ ci:
#
# Lint & format
#
# Run the linting commands in the scope of the monorepo.
# Eslint and prettier (plus some configs) are on the root.
RUN_LINTING_MONOREPO = docker run --rm -v $(MONOREPO):$(MONOREPO) -w $(MONOREPO) {{node_image}}:{{node_version}} npm run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/web/data/reports:/overleaf/services/web/data/reports -w /overleaf $(IMAGE_CI) npm run --silent
ORG_PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN_LINT_FORMAT ?= \
docker run --rm --env BRANCH_NAME --env CI --env COMMIT_SHA --env MONOREPO --volume ${PWD}/data/reports:/overleaf/services/web/data/reports ${IMAGE_CI}
docker run --rm --env BRANCH_NAME --env CI --env COMMIT_SHA --env MONOREPO --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume ${PWD}/data/reports:/overleaf/services/web/data/reports ${IMAGE_CI}
NODE_MODULES_PATH := ${PATH}:${PWD}/node_modules/.bin:/overleaf/services/web/node_modules/.bin
WITH_NODE_MODULES_PATH = \
@@ -505,34 +512,34 @@ lint_in_docker:
format: format_js
format_js:
npm run --silent format
$(RUN_LINTING_MONOREPO) format -- services/web
format: format_styles
format_styles:
npm run --silent format:styles
format: format_pug
format_pug:
npm run --silent format:pug
format: format_jenkins
format_jenkins:
npm run --silent format:jenkins
format_ci:
$(RUN_LINTING_CI_MONOREPO) format -- services/web
format_fix:
npm run --silent format:fix
$(RUN_LINTING_MONOREPO) format:fix -- services/web
format_styles:
$(RUN_LINTING_MONOREPO) format:styles
format_styles_fix:
npm run --silent format:styles:fix
$(RUN_LINTING_MONOREPO) format:styles:fix
format_pug:
$(RUN_LINTING_MONOREPO) format:pug
format_pug_fix:
npm run --silent format:pug:fix
$(RUN_LINTING_MONOREPO) format:pug:fix
format_jenkins:
$(RUN_LINTING_MONOREPO) format:jenkins
format_jenkins_fix:
npm run --silent format:jenkins:fix
$(RUN_LINTING_MONOREPO) format:jenkins:fix
format_in_docker:
$(RUN_LINT_FORMAT) make format -j2 --output-sync
$(RUN_LINT_FORMAT) npm run --prefix /overleaf format -- services/web
SHELLCHECK_OPTS = \
--shell=bash \
services/web/README.md
+9 -9
View File
@@ -1,12 +1,10 @@
overleaf/web
==============
# overleaf/web
overleaf/web is the front-end web service of the open-source web-based collaborative LaTeX editor,
[Overleaf](https://www.overleaf.com).
It serves all the HTML pages, CSS and javascript to the client. overleaf/web also contains
a lot of logic around creating and editing projects, and account management.
The rest of the Overleaf stack, along with information about contributing can be found in the
[overleaf/overleaf](https://github.com/overleaf/overleaf) repository.
@@ -22,11 +20,13 @@ $ npm run start
### Running Tests
To run all tests run:
```
make test
```
To run both unit and acceptance tests for a module run:
```
make test_module MODULE=saas-authentication
```
@@ -50,11 +50,13 @@ make test_unit MOCHA_GREP='AuthorizationManager'
```
To run only the unit tests for a single module do:
```
make test_unit_module MODULE=saas-authentication
```
Module tests can also use a MOCHA_GREP argument:
```
make test_unit_module MODULE=saas-authentication MOCHA_GREP=SSO
```
@@ -86,23 +88,22 @@ make test_acceptance_run MOCHA_GREP='AuthorizationManager'
```
To run only the acceptance tests for a single module do:
```
make test_acceptance_module MODULE=saas-authentication
```
Module tests can also use a MOCHA_GREP argument:
```
make test_acceptance_module MODULE=saas-authentication MOCHA_GREP=SSO
```
Routes
------
## Routes
Run `bin/routes` to print out all routes in the project.
License and Credits
-------------------
## License and Credits
This project is licensed under the [AGPLv3 license](http://www.gnu.org/licenses/agpl-3.0.html)
@@ -127,4 +128,3 @@ of these icons are used within Overleaf inside the `public/img/silk` and
We gratefully acknowledge [IconShock](http://www.iconshock.com) for use of the icons
in the `public/img/iconshock` directory found via
[findicons.com](http://findicons.com/icon/498089/height?id=526085#)
services/web/cypress/support/component-index.html
+5 -5
View File
@@ -1,12 +1,12 @@
<!DOCTYPE html>
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width,initial-scale=1.0" />
<title>Components App</title>
</head>
<body>
<div data-cy-root></div>
</body>
</html>
</html>
services/web/docker-compose.ci.yml
+11 -12
View File
@@ -2,7 +2,6 @@ volumes:
data:
services:
test_unit:
build:
context: ../..
@@ -49,7 +48,7 @@ services:
OVERLEAF_CONFIG:
REDIS_HOST: redis_test
extra_hosts:
- 'www.overleaf.test:127.0.0.1'
- "www.overleaf.test:127.0.0.1"
volumes:
- ../../bin/shared/wait_for_it:/overleaf/bin/shared/wait_for_it
- ./data/reports:/overleaf/services/web/data/reports
@@ -151,15 +150,15 @@ services:
# REF: https://github.com/moby/moby/issues/8231#issuecomment-63877553
# REF: https://github.com/moby/moby/issues/8231#issuecomment-63871343
command:
- 'bash'
- '-c'
- 'ulimit -n 1024 && exec bash /run.sh'
- "bash"
- "-c"
- "ulimit -n 1024 && exec bash /run.sh"
saml:
image: us-east1-docker.pkg.dev/overleaf-ops/ol-docker/saml-test
logging:
driver: none
environment:
SAML_BASE_URL_PATH: 'http://saml/simplesaml/'
SAML_TEST_SP_ENTITY_ID: 'overleaf-test-saml'
SAML_TEST_SP_LOCATION: 'http://www.overleaf.test:23000/saml/callback'
image: us-east1-docker.pkg.dev/overleaf-ops/ol-docker/saml-test
logging:
driver: none
environment:
SAML_BASE_URL_PATH: "http://saml/simplesaml/"
SAML_TEST_SP_ENTITY_ID: "overleaf-test-saml"
SAML_TEST_SP_LOCATION: "http://www.overleaf.test:23000/saml/callback"
services/web/docker-compose.yml
+9 -10
View File
@@ -2,7 +2,6 @@ volumes:
data:
services:
test_unit:
image: node:24.13.0
volumes:
@@ -56,7 +55,7 @@ services:
# OVERLEAF_ALLOW_ANONYMOUS_READ_AND_WRITE_SHARING: 'true'
extra_hosts:
- 'www.overleaf.test:127.0.0.1'
- "www.overleaf.test:127.0.0.1"
depends_on:
- redis_test
- mongo
@@ -143,13 +142,13 @@ services:
# REF: https://github.com/moby/moby/issues/8231#issuecomment-63877553
# REF: https://github.com/moby/moby/issues/8231#issuecomment-63871343
command:
- 'bash'
- '-c'
- 'ulimit -n 1024 && exec bash /run.sh'
- "bash"
- "-c"
- "ulimit -n 1024 && exec bash /run.sh"
saml:
image: us-east1-docker.pkg.dev/overleaf-ops/ol-docker/saml-test
environment:
SAML_BASE_URL_PATH: 'http://saml/simplesaml/'
SAML_TEST_SP_ENTITY_ID: 'overleaf-test-saml'
SAML_TEST_SP_LOCATION: 'http://www.overleaf.test:23000/saml/callback'
image: us-east1-docker.pkg.dev/overleaf-ops/ol-docker/saml-test
environment:
SAML_BASE_URL_PATH: "http://saml/simplesaml/"
SAML_TEST_SP_ENTITY_ID: "overleaf-test-saml"
SAML_TEST_SP_LOCATION: "http://www.overleaf.test:23000/saml/callback"
services/web/frontend/fonts/README.md
+12 -12
View File
@@ -1,14 +1,14 @@
This directory contains fonts used by the Overleaf web application.
* [DM Mono](https://github.com/googlefonts/dm-mono)
* [Font Awesome](https://fontawesome.com/v4/)
* [Inter](https://rsms.me/inter/)
* [Lato](https://www.latofonts.com/)
* [Material Symbols](https://github.com/google/material-design-icons)
* [Merriweather](https://github.com/SorkinType/Merriweather)
* [Noto Sans](https://notofonts.github.io/#latin-greek-cyrillic)
* [Noto Serif](https://notofonts.github.io/#latin-greek-cyrillic)
* [OpenDyslexic Mono](https://github.com/antijingoist/opendyslexic/tree/master)
* [Open Sans](https://github.com/googlefonts/opensans)
* [Source Code Pro](https://github.com/adobe-fonts/source-code-pro)
* [STIX Two Math](https://github.com/stipub/stixfonts/)
- [DM Mono](https://github.com/googlefonts/dm-mono)
- [Font Awesome](https://fontawesome.com/v4/)
- [Inter](https://rsms.me/inter/)
- [Lato](https://www.latofonts.com/)
- [Material Symbols](https://github.com/google/material-design-icons)
- [Merriweather](https://github.com/SorkinType/Merriweather)
- [Noto Sans](https://notofonts.github.io/#latin-greek-cyrillic)
- [Noto Serif](https://notofonts.github.io/#latin-greek-cyrillic)
- [OpenDyslexic Mono](https://github.com/antijingoist/opendyslexic/tree/master)
- [Open Sans](https://github.com/googlefonts/opensans)
- [Source Code Pro](https://github.com/adobe-fonts/source-code-pro)
- [STIX Two Math](https://github.com/stipub/stixfonts/)
services/web/frontend/js/features/settings/data/countries-list.ts
+1 -1
View File
@@ -253,4 +253,4 @@ const countries = <const>[
]
export default countries
export type CountryCode = typeof countries[number]['code']
export type CountryCode = (typeof countries)[number]['code']
services/web/frontend/js/features/source-editor/hunspell/wasm/README.md
+3 -3
View File
@@ -2,8 +2,8 @@
The files in this directory are:
* `hunspell.wasm`: [Hunspell](https://github.com/hunspell/hunspell) compiled to WebAssembly using Emscripten, via the [build.sh](../build.sh) script.
* `hunspell.mjs`: a JavaScript wrapper for the WebAssembly module, generated by Emscripten.
* `hunspell.d.ts`: manually-created types for the exports from the JavaScript module.
- `hunspell.wasm`: [Hunspell](https://github.com/hunspell/hunspell) compiled to WebAssembly using Emscripten, via the [build.sh](../build.sh) script.
- `hunspell.mjs`: a JavaScript wrapper for the WebAssembly module, generated by Emscripten.
- `hunspell.d.ts`: manually-created types for the exports from the JavaScript module.
Note: To speed up compilation on ARM architecture (e.g. Apple M1), add `-arm64` to the Docker image tag in `Dockerfile`.
services/web/frontend/js/features/source-editor/lezer-latex/README.md
+2 -8
View File
@@ -4,7 +4,6 @@ Lezer-LaTeX is a LaTeX parser implemented with [lezer](https://lezer.codemirror.
The parser is written in a "grammar" file, (and a "tokens" file with custom tokenizer logic) which is then compiled by `@lezer/generator` into a parser module and a "terms" module. The parser module is then loaded by the CodeMirror 6 in the web frontend codebase.
## Important files
- Source files:
@@ -23,7 +22,6 @@ The parser is written in a "grammar" file, (and a "tokens" file with custom toke
- Webpack plugins:
- `web/webpack-plugins/lezer-grammar-compiler.js`: A webpack plugin that calls the generator as part of the webpack build. In dev, it will automatically re-build the parser when the grammar file changes.
## NPM tasks
- `lezer-latex:generate`: Generate the parser files from the grammar
@@ -33,7 +31,6 @@ The parser is written in a "grammar" file, (and a "tokens" file with custom toke
- `lezer-latex:run`: Run the parser against a file
- (Calls `lezer-latex/run.js`)
### Generating the parser
From the monorepo root:
@@ -46,7 +43,6 @@ make install
bin/npm -w services/web run 'lezer-latex:generate'
```
## Tests
Unit tests for the parser live in `web/test/unit/src/LezerLatex`. There are three kinds of test, in three subdirectories:
@@ -57,11 +53,10 @@ Unit tests for the parser live in `web/test/unit/src/LezerLatex`. There are thre
These tests run as part of `test_frontend`. You can run these tests alone by invoking:
``` sh
```sh
make test_unit MOCHA_GREP='lezer-latex'
```
## Trying the parser
While developing the parser, you can run it against a file by calling the `lezer-latex:run` task. There are
@@ -69,13 +64,12 @@ some example files in the test suite, at `web/test/unit/src/LezerLatex/examples/
For example:
``` sh
```sh
bin/npm -w services/web run 'lezer-latex:run' web/test/unit/src/LezerLatex/examples/amsmath.tex
```
If you omit the file path, the default file (`examples/demo.tex`) will be run.
## Integration into web
The web frontend imports the parser (from `latex.mjs`), in `frontend/js/features/source-editor/languages/latex/index.ts`.
services/web/frontend/js/features/subscription/data/add-on-codes.ts
+13 -12
View File
@@ -1,20 +1,20 @@
import { PaidSubscription } from "../../../../../types/subscription/dashboard/subscription"
import { PendingPaymentProviderPlan } from "../../../../../types/subscription/plan"
import { PaidSubscription } from '../../../../../types/subscription/dashboard/subscription'
import { PendingPaymentProviderPlan } from '../../../../../types/subscription/plan'
export const AI_ASSIST_STANDALONE_MONTHLY_PLAN_CODE = 'assistant'
export const AI_ADD_ON_CODE = 'assistant'
// we dont want translations on plan or add-on names
export const ADD_ON_NAME = "AI Assist"
export const ADD_ON_NAME = 'AI Assist'
export const AI_ASSIST_STANDALONE_ANNUAL_PLAN_CODE = 'assistant-annual'
export function isStandaloneAiPlanCode(planCode?: string) {
return planCode === AI_ASSIST_STANDALONE_MONTHLY_PLAN_CODE || planCode === AI_ASSIST_STANDALONE_ANNUAL_PLAN_CODE
return (
planCode === AI_ASSIST_STANDALONE_MONTHLY_PLAN_CODE ||
planCode === AI_ASSIST_STANDALONE_ANNUAL_PLAN_CODE
)
}
export function hasPendingAiAddonCancellation(subscription: PaidSubscription){
export function hasPendingAiAddonCancellation(subscription: PaidSubscription) {
const pendingPlan = subscription.pendingPlan as PendingPaymentProviderPlan
const hasAiAddon = subscription.addOns?.some(
@@ -22,11 +22,12 @@ export function hasPendingAiAddonCancellation(subscription: PaidSubscription){
)
// cancellation of entire plan counts as removing the add-on
if(hasAiAddon && !pendingPlan){
if (hasAiAddon && !pendingPlan) {
return true
}
return hasAiAddon &&
return (
hasAiAddon &&
!pendingPlan.addOns?.some(addOn => addOn.code === AI_ADD_ON_CODE)
}
)
}
services/web/frontend/stories/docs/cloud-builds.mdx
+1 -2
View File
@@ -1,8 +1,7 @@
import { Meta } from '@storybook/blocks';
import { Meta } from '@storybook/blocks'
<Meta title="Storybook Guideline / Storybook builds" />
# Cloud Builds
Storybook builds can be used to share your development work. They can be found at [https://storybook.dev-overleaf.com/](https://storybook.dev-overleaf.com/).
services/web/frontend/stories/docs/feature-flags.mdx
+3 -2
View File
@@ -1,4 +1,4 @@
import { Canvas, Controls, Meta } from '@storybook/blocks';
import { Canvas, Controls, Meta } from '@storybook/blocks'
import * as FormatCurrency from './format-currency.stories'
<Meta title="Storybook Guideline / Feature flags" />
@@ -8,7 +8,7 @@ import * as FormatCurrency from './format-currency.stories'
You can wrap your story with the `withSplitTests` utility to add split test variants to your Storybook stories, so you can toggle feature flags directly in the Storybook UI. See [withSplitTests on GitHub](https://github.com/overleaf/internal/blob/main/services/web/.storybook/utils/with-split-tests.tsx).
1. Define your split test argTypes:
Add your split test configurations to the `splitTestsArgTypes` object.
Add your split test configurations to the `splitTestsArgTypes` object.
```js
export const splitTestsArgTypes = {
@@ -19,6 +19,7 @@ export const splitTestsArgTypes = {
},
}
```
2. Wrap your story with `withSplitTests`.
Import `withSplitTests` and `Meta` from '@storybook/react' in your stories.
services/web/frontend/stories/docs/foundations.mdx
+74 -52
View File
@@ -1,19 +1,35 @@
import { Meta, ColorItem, ColorPalette, Title, Typeset } from '@storybook/blocks';
import colors from '../../stylesheets/foundations/tokens/colors.json';
import typography from '../../stylesheets/foundations/tokens/typography.json';
import borderRadius from '../../stylesheets/foundations/tokens/borderRadius.json';
import spacing from '../../stylesheets/foundations/tokens/spacing.json';
import {
Meta,
ColorItem,
ColorPalette,
Title,
Typeset,
} from '@storybook/blocks'
import colors from '../../stylesheets/foundations/tokens/colors.json'
import typography from '../../stylesheets/foundations/tokens/typography.json'
import borderRadius from '../../stylesheets/foundations/tokens/borderRadius.json'
import spacing from '../../stylesheets/foundations/tokens/spacing.json'
<Meta title="Storybook Guideline / Foundations" />
# Foundations
Foundations in UX design are the basic rules and core elements, like colors, fonts, and spacing, that ensure consistency across the design.
<p>
These palettes are generated from our token files. The tokens were exported from Figma and split into separate files to categorise: <a href="https://github.com/overleaf/internal/tree/main/services/web/frontend/stylesheets/bootstrap-5/foundations/tokens" target="_blank" rel="noopener noreferrer">Token files on GitHub</a>
These palettes are generated from our token files. The tokens were exported
from Figma and split into separate files to categorise:{' '}
<a
href="https://github.com/overleaf/internal/tree/main/services/web/frontend/stylesheets/bootstrap-5/foundations/tokens"
target="_blank"
rel="noopener noreferrer"
>
Token files on GitHub
</a>
</p>
**Table of Contents**
- [Primitive colors](#primitive-colors)
- [Background colors](#background-colors)
- [Content colors](#content-colors)
@@ -26,17 +42,14 @@ Foundations in UX design are the basic rules and core elements, like colors, fon
- [Spacing](#spacing)
# Colors
<br/>
<br />
## Primitive colors
<ColorPalette>
{Object.entries(colors.PrimitiveColor).map(([name, color]) => (
<ColorItem
key={name}
title={name}
colors={[color.$value]}
/>
<ColorItem key={name} title={name} colors={[color.$value]} />
))}
</ColorPalette>
@@ -126,12 +139,15 @@ Colors used for specific UI elements like color pickers.
</ColorPalette>
# Typography
<br/>
<br />
## Font weight
<br/>
<br />
{Object.entries(typography.Fontweight).map(([name, typography]) => (
<div key={name}>
<strong>{name}</strong>
<Typeset
@@ -143,30 +159,33 @@ Colors used for specific UI elements like color pickers.
))}
## Font size
<br/>
<br />
{Object.entries(typography['Fontsize']).map(([name, typography]) => {
const mixins = typography.$mixin ? [].concat(typography.$mixin) : [];
const mixins = typography.$mixin ? [].concat(typography.$mixin) : [];
return (
<div key={name} style={{ marginBottom: '2rem' }}>
<strong>
{name}
{mixins.map(mixin => (
<span key={mixin}> / @mixin {mixin}</span>
))}
</strong>
<Typeset
fontSizes={[`${typography.$value}px`]}
fontWeight={500}
sampleText="The quick brown fox jumps over the lazy dog"
/>
</div>
);
return (
<div key={name} style={{ marginBottom: '2rem' }}>
<strong>
{name}
{mixins.map(mixin => (
<span key={mixin}> / @mixin {mixin}</span>
))}
</strong>
<Typeset
fontSizes={[`${typography.$value}px`]}
fontWeight={500}
sampleText="The quick brown fox jumps over the lazy dog"
/>
</div>
);
})}
## Border radius
<br/>
<br />
<table style={{ width: '100%' }}>
<thead>
@@ -177,30 +196,33 @@ Colors used for specific UI elements like color pickers.
</tr>
</thead>
<tbody>
{Object.entries(borderRadius['BorderRadius']).map(([name, borderRadius]) => (
<tr key={name}>
<td>
<strong>{name}</strong>
</td>
<td style={{ textAlign: 'center' }}>{borderRadius.$value}px</td>
<td style={{ textAlign: 'center', padding: '0.5rem' }}>
<div
style={{
width: '80px',
height: '80px',
borderRadius: `${borderRadius.$value}px`,
backgroundColor: '#098842',
display: 'inline-block',
}}
/>
</td>
</tr>
))}
{Object.entries(borderRadius['BorderRadius']).map(
([name, borderRadius]) => (
<tr key={name}>
<td>
<strong>{name}</strong>
</td>
<td style={{ textAlign: 'center' }}>{borderRadius.$value}px</td>
<td style={{ textAlign: 'center', padding: '0.5rem' }}>
<div
style={{
width: '80px',
height: '80px',
borderRadius: `${borderRadius.$value}px`,
backgroundColor: '#098842',
display: 'inline-block',
}}
/>
</td>
</tr>
)
)}
</tbody>
</table>
## Spacing
<br/>
<br />
<table style={{ width: '100%' }}>
<thead>
services/web/locales/README.md
+1 -2
View File
@@ -1,5 +1,4 @@
Locales
=======
# Locales
These files are not to be edited by hand.
services/web/package.json
+1 -10
View File
@@ -25,14 +25,6 @@
"webpack": "webpack serve --config webpack.config.dev.js",
"webpack:production": "webpack --config webpack.config.prod.js",
"webpack:profile": "webpack --config webpack.config.prod.js --profile --json > stats.json",
"format": "prettier --list-different $PWD/'**/*.{js,jsx,mjs,ts,tsx,json}'",
"format:fix": "prettier --write $PWD/'**/*.{js,jsx,mjs,ts,tsx,json}'",
"format:styles": "prettier --list-different $PWD/'**/*.{css,scss}'",
"format:styles:fix": "prettier --write $PWD/'**/*.{css,scss}'",
"format:pug": "prettier --list-different $PWD/'**/*.pug'",
"format:pug:fix": "prettier --write $PWD/'**/*.pug'",
"format:jenkins": "prettier --list-different $PWD/'**/Jenkinsfile'",
"format:jenkins:fix": "prettier --write $PWD/'**/Jenkinsfile'",
"lint": "eslint --max-warnings 0 --format unix --ext .js,.jsx,.mjs,.ts,.tsx .",
"lint:fix": "eslint --fix --ext .js,.jsx,.mjs,.ts,.tsx .",
"lint:styles": "stylelint '**/*.scss'",
@@ -200,6 +192,7 @@
"zod-validation-error": "^4.0.1"
},
"devDependencies": {
"5to6-codemod": "^1.8.0",
"@ai-sdk/react": "^3.0.2",
"@babel/core": "^7.28.5",
"@babel/plugin-proposal-decorators": "^7.28.0",
@@ -231,7 +224,6 @@
"@pollyjs/adapter-node-http": "^6.0.6",
"@pollyjs/core": "^6.0.6",
"@pollyjs/persister-fs": "^6.0.6",
"@prettier/plugin-pug": "^3.4.0",
"@replit/codemirror-emacs": "overleaf/codemirror-emacs#4394c03858f27053f8768258e9493866e06e938e",
"@replit/codemirror-indentation-markers": "overleaf/codemirror-indentation-markers#371ce3b56f453a392eb0d3b85ab019c185c68e1f",
"@replit/codemirror-vim": "overleaf/codemirror-vim#1bef138382d948018f3f9b8a4d7a70ab61774e4b",
@@ -285,7 +277,6 @@
"@writefull/core": "^1.27.27",
"@writefull/ui": "^1.27.27",
"@writefull/utils": "^1.27.27",
"5to6-codemod": "^1.8.0",
"abort-controller": "^3.0.0",
"acorn": "^7.1.1",
"acorn-walk": "^7.1.1",
services/web/scripts/delete-orphaned-docs/README.md
+11
View File
@@ -12,15 +12,19 @@ and copying the files to your local machine.
Run the following doc export command to export all doc ids and their associated
project ids in batches of 10,000,000.
```
mongoexport --uri $READ_ONLY_MONGO_CONNECTION_STRING --collection docs --fields '_id,project_id' --skip 0 --limit 10000000 --type=csv --out docs.00000000.csv
```
This will produce files like:
```
_id,project_id
ObjectId(5babb6f864c952737a9a4c32),ObjectId(5b98bba5e2f38b7c88f6a625)
ObjectId(4eecaffcbffa66588e000007),ObjectId(4eecaffcbffa66588e00000d)
```
Concatenate these into a single file: `cat docs.*csv > all-docs-doc_id-project_id.csv`
For object ids the script will accept either plain hex strings or the `ObjectId(...)`
@@ -29,35 +33,42 @@ format used by mongoexport.
### Exporting Projects
Export project ids from all `projects` and `deletedProjects`
```
mongoexport --uri $READ_ONLY_MONGO_CONNECTION_STRING --collection projects --fields '_id' --type=csv --out projects.csv
mongoexport --uri $READ_ONLY_MONGO_CONNECTION_STRING --collection deletedProjects --fields 'project._id' --type=csv --out deleted-projects.csv
```
Concatenate these: `cat projects.csv deleted-projects.csv > all-projects-project_id.csv`
## Processing Exported Data
### Create a unique sorted list of project ids from docs
```
cut -d, -f 2 all-docs-doc_id-project_id.csv | sort | uniq > all-docs-project_ids.sorted.uniq.csv
```
### Create a unique sorted list of projects ids from projects
```
sort all-projects-project_id.csv | uniq > all-projects-project_id.sorted.uniq.csv
```
### Create list of project ids in docs but not in projects
```
comm --check-order -23 all-docs-project_ids.sorted.uniq.csv all-projects-project_id.sorted.uniq.csv > orphaned-doc-project_ids.csv
```
### Create list of docs ids with project ids not in projects
```
grep -F -f orphaned-doc-project_ids.csv all-docs-doc_id-project_id.csv > orphaned-doc-doc_id-project_id.csv
```
## Run doc deleter
```
node delete-orphaned-docs orphaned-doc-doc_id-project_id.csv
```
services/web/scripts/learn/checkSanitize/README.md
+2 -2
View File
@@ -7,10 +7,10 @@ node scripts/learn/checkSanitize/index.mjs https://LEARN_WIKI
## Bulk export
There is a bulk export for media wiki pages, but it produces different
html escaping compared to the regular parse API we use in web.
html escaping compared to the regular parse API we use in web.
The bulk export does not escape all the placeholder HTML-like elements,
like `<project-id` or `<document goes here>`.
like `<project-id` or `<document goes here>`.
## Example output
services/web/scripts/sync-user-entitlements/README.md
+2
View File
@@ -14,6 +14,7 @@ Follow the directions in `google-ops/README.md` for exporting data from mongo
and copying the files to your local machine.
Run the following user export command.
```
mongoexport --uri $READ_ONLY_MONGO_CONNECTION_STRING --collection users --fields '_id,email,emails,samlIdentifiers' --query '{"samlIdentifiers.providerId": {"$exists": 1}}' --out user-entitlements.json
```
@@ -26,6 +27,7 @@ is never transferred or stored insecurely and that it is deleted ASAP**
Connect to postgres by running `heroku psql -a electric-leaf-4093`
Run the following v2_user_universities export comand.
```
\copy (select uu.user_id, uu.email, uu.cached_entitlement, ud.university_id from v2_user_universities uu LEFT JOIN university_domains ud ON uu.university_domain_id = ud.id WHERE uu.removed_at IS NULL) to 'cached-entitlements.csv' with csv;
```
services/web/test/smoke/src/README.md
+8 -7
View File
@@ -1,12 +1,12 @@
# SmokeTests
For the SmokeTests we implemented a Mini-Framework that is tailored for our
tooling, specifically OError, and does not need a large runner, such as mocha.
tooling, specifically OError, and does not need a large runner, such as mocha.
The SmokeTests are separated into individual `steps`.
Each `step` can have a `run` function and a `cleanup` function.
The former will run in sequence with the other steps, the later in reverse
order from the finish, or the last failure.
order from the finish, or the last failure.
```js
async function run(ctx) {
@@ -19,21 +19,22 @@ module.exports = { cleanup, run }
```
Steps will get called with a context object with common helpers and details:
- `request` a promisified `request` module with defaults for `baseUrl`,
`timeout` and internals for cookie handling.
`timeout` and internals for cookie handling.
- `assertHasStatusCode` a helper for asserting response status codes, pass
a response and desired status code. It will throw with OError context set.
a response and desired status code. It will throw with OError context set.
- `getCsrfTokenFor` a helper for retrieving CSRF tokens, pass an endpoint.
- `processWithTimeout` a helper for awaiting Promises with a timeout, pass
`{ work: Promise.resolve(), timeout: 42, message: 'foo timedout' }`
`{ work: Promise.resolve(), timeout: 42, message: 'foo timedout' }`
- `stats` an object for performance tracking.
- `timeout` the step timeout
Steps should handle timeouts locally to ensure appropriate cleanup of timed out
actions.
actions.
Steps may pass values along to the next steps in returning an object with the
desired fields from the `run` or `cleanup` function.
desired fields from the `run` or `cleanup` function.
The returned values will overwrite existing details in the `ctx`.
Alpha-numeric sorting of step filenames determines the processing sequence.