Files
Jakob AckermannandClaude Opus 5 04871d6049 [editor-core] extract more shared utils (#36954)
* [editor-core] share the blob hash and the doc-vs-file rule

The git blob hash and the rule that decides whether a file is a doc or a binary
file both have to reach the same answer in every service that writes to history,
so they move into overleaf-editor-core rather than being spelled out again on
each side.

blob_utils takes over lib/blob_string_length.js: hashing and the string-length
rule answered two halves of the same question -- what history needs to know
about content before it can store it -- and each carried the same warning about
staying off index.js, which is bundled for the browser. One module states it
once, and blobForFile joins them, describing content on disk as the blob it
amounts to. The hashing functions are flat rather than a namespace of their own:
in a module that also holds the string-length rule, a bare fromStream says
nothing about what it makes. history-v1's blob store and recover_zip follow the
module to its new name.

file_type_detector holds the doc-vs-file rule, Node-only and deep-imported for
the same reason. It takes textExtensions, editableFilenames and maxDocLength as
configuration rather than reading settings: the limit is web's 2 MiB and not
editor-core's own 3 MiB, and the surrogate check covers both halves of a pair
where editor-core's covers the high half.

buildSetContentOperations' test spelled the blob header out for itself, with the
separator as a literal NUL byte, which had git reading the file as binary. It
hashes through blobHashFromString now, so it does not spell the header at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] name the client a change came from on the origin itself

`EditorOrigin` carried an `editorId` that only an editor change could have. What
the id is for -- recognising a change again, so history does not apply a resend
twice and the client does not apply its own change back -- is a property of the
client that submitted it rather than of what it did, so it moves onto `Origin` as
an optional `historyClientId` and `EditorOrigin` goes.

- `Origin` takes `(kind, historyClientId)` and answers `getHistoryClientId()`;
  `dropHistoryClientId()` is what the chunk store uses to strip the id from all
  but a client's latest change, which now leaves the rest of the origin alone --
  a restore that a client submitted stays a restore.
- `toRaw` is built a layer at a time: the base writes the kind and the id, a
  subclass spreads that and adds the fields it owns.
- The named kinds are their own schemas and discriminate on `kind`, so a payload
  claiming one is checked against that shape and the error says what it got
  wrong. The catch-all cannot join them -- its kind is any string -- so it sits
  alongside and is refined to refuse the named kinds, which it used to swallow:
  `{kind: 'restore'}` validated as a kind and nothing else, then read back as a
  restore that recorded nothing about what it restored.
- `lib/origin/*.js` gains `// @ts-check`. Without it the annotations in there
  were decorative: the lazily required subclasses were `any`, so nothing handed
  to them was checked, and `fromRaw` narrows with `in` to drop the catch-all
  variant, which a kind check cannot.

The callers that stamp the id -- real-time's handshake flag, web's applyChange
body -- come with the rest of the branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] recognise a change a writer that is not an editor placed

The counterpart of `isSameEditorChange` for writers that are not an editor
client: an integration that commits a batch on a user's behalf has to recognise
that batch coming back, because a lost response does not say whether the commit
landed and resubmitting one that did applies its removes twice.

It compares the same three fields, and for the same reason -- a rebase rewrites a
change's operations but leaves its origin, authors and timestamp alone -- with
the timestamp compared as a point in time rather than as a string, so that two
spellings of the same instant cannot read as different changes. github-sync is
the first caller, and comes with the rest of the branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] load an editable file's content, or say it has none

Only an editable file has content: any other kind is bytes addressed by hash, and
hollow data records the size of content it does not have. Both answer
`getContent()` with null, so every caller that wants the content of a doc loaded
it, checked the data's class, and reached for its own error when the check failed.
`File#loadEager` does that once and answers the data that always has the content.

`buildSetContentOperations` takes a `ReadWriteBlobStore` -- everything but
looking a blob up by hash, which is all it takes to build operations over content
-- so a caller holding a store that cannot answer `getBlob` can still use it. Its
metadata argument is `FileMetadata` rather than `Object`, which is what the schema
already says a file's metadata is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [editor-core] say that a pair of operations that do not fit is unprocessable

compose and transform threw a bare Error when the two operations do not describe
the same content. Everything that answers a client for these -- history-v1's
import routes, the git-bridge push, project-history's resync -- decides on
UnprocessableError, so a bare Error reached them as an internal failure: a 500,
which reads as worth retrying, for a pair that fails the same way every time.
A client that resubmits on a 500 resubmits forever.

The guards that cannot be reached through the API keep their bare Error: a
TextOperation composed with something that is not one, and the branch whose
message says it should not happen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* [history-v1] hash blobs with the rule editor-core holds

storage/lib/blob_hash.js was a bluebird-era implementation of the same git blob
hash editor-core holds, exported from the storage barrel and used across the
storage layer, the api and the tests. All of it reads the shared rule now, and
the test that existed only to check the copy is gone -- editor-core's suite
covers the same non-ASCII case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 5a0c1e9a06170a0164c01a26a7bb2d205726d531
2026-09-17 12:07:15 +00:00

818 lines
26 KiB
JavaScript

'use strict'
const { expect } = require('chai')
const fs = require('node:fs')
const { Readable } = require('node:stream')
const HTTPStatus = require('http-status')
const fetch = require('node-fetch')
const sinon = require('sinon')
const cleanup = require('../storage/support/cleanup')
const fixtures = require('../storage/support/fixtures')
const testFiles = require('../storage/support/test_files')
const {
zipStore,
BlobStore,
persistChanges,
redisBuffer,
} = require('../../../../storage')
const {
blobHashFromStream,
blobHashFromString,
} = require('overleaf-editor-core/lib/blob_utils')
const { expectHttpError } = require('./support/expect_response')
const testServer = require('./support/test_server')
const { createEmptyProject } = require('./support/test_projects')
const {
File,
Snapshot,
Change,
AddFileOperation,
EditFileOperation,
TextOperation,
Operation,
} = require('overleaf-editor-core')
const testProjects = require('./support/test_projects')
const { ObjectId } = require('mongodb')
describe('project controller', function () {
beforeEach(cleanup.everything)
beforeEach(fixtures.create)
describe('initializeProject', function () {
it('can initialize a new project', async function () {
const projectId = await testProjects.createEmptyProject()
expect(projectId).to.be.a('string')
})
})
describe('createZip', function () {
let importSnapshot
let createZip
before(function () {
importSnapshot =
testServer.basicAuthClient.apis.ProjectImport.importSnapshot1
createZip = testServer.basicAuthClient.apis.Project.createZip
})
beforeEach(function () {
// Don't start the work in the background in this test --- it is flaky.
sinon.stub(zipStore, 'storeZip').resolves()
})
afterEach(function () {
zipStore.storeZip.restore()
})
it('creates a URL to a zip file', async function () {
// Create a test blob.
const testProjectId = fixtures.docs.uninitializedProject.id
const response = await fetch(
testServer.url(
`/api/projects/${testProjectId}/blobs/${testFiles.HELLO_TXT_HASH}`
),
{
method: 'PUT',
body: fs.createReadStream(testFiles.path('hello.txt')),
headers: {
Authorization: testServer.basicAuthHeader,
},
}
)
expect(response.ok).to.be.true
// Import a project with the test blob.
const testFilePathname = 'hello.txt'
const testSnapshot = new Snapshot()
testSnapshot.addFile(
testFilePathname,
File.fromHash(testFiles.HELLO_TXT_HASH)
)
const importResponse = await importSnapshot({
project_id: testProjectId,
snapshot: testSnapshot.toRaw(),
})
expect(importResponse.obj.projectId).to.equal(testProjectId)
const createZipResponse = await createZip({
project_id: testProjectId,
version: 0,
})
expect(createZipResponse.status).to.equal(HTTPStatus.OK)
const zipInfo = createZipResponse.obj
expect(zipInfo.zipUrl).to.match(
/^http:\/\/gcs:9090\/download\/storage\/v1\/b\/overleaf-test-zips/
)
expect(zipStore.storeZip.calledOnce).to.be.true
})
})
describe('getLatestZip', function () {
it('returns a zip of the latest snapshot', async function () {
const projectId = fixtures.docs.uninitializedProject.id
const uploadResponse = await fetch(
testServer.url(
`/api/projects/${projectId}/blobs/${testFiles.HELLO_TXT_HASH}`
),
{
method: 'PUT',
body: fs.createReadStream(testFiles.path('hello.txt')),
headers: { Authorization: testServer.basicAuthHeader },
}
)
expect(uploadResponse.ok).to.be.true
const snapshot = new Snapshot()
snapshot.addFile('hello.txt', File.fromHash(testFiles.HELLO_TXT_HASH))
const importResponse =
await testServer.basicAuthClient.apis.ProjectImport.importSnapshot1({
project_id: projectId,
snapshot: snapshot.toRaw(),
})
expect(importResponse.obj.projectId).to.equal(projectId)
const downloadClient =
await testServer.createClientForDownloadZip(projectId)
const zipResponse = await downloadClient.apis.Project.getLatestZip({
project_id: projectId,
})
expect(zipResponse.status).to.equal(HTTPStatus.OK)
expect(zipResponse.headers['x-history-version']).to.equal('0')
expect(zipResponse.headers['content-type']).to.equal(
'application/octet-stream'
)
expect(zipResponse.headers['content-disposition']).to.equal(
'attachment; filename=project.zip'
)
const testFile = File.fromHash(testFiles.HELLO_TXT_HASH)
const testChange = new Change(
[Operation.addFile('main.tex', testFile)],
new Date()
)
const importchangesResponse =
await testServer.basicAuthClient.apis.ProjectImport.importChanges1({
project_id: projectId,
end_version: 0,
changes: [testChange.toRaw()],
})
expect(importchangesResponse.status).to.equal(HTTPStatus.CREATED)
expect(importchangesResponse.obj).to.deep.equal({ resyncNeeded: false })
const zipResponse2 = await downloadClient.apis.Project.getLatestZip({
project_id: projectId,
})
expect(zipResponse2.status).to.equal(HTTPStatus.OK)
expect(zipResponse2.headers['x-history-version']).to.equal('1')
})
it('returns 404 for an unknown project', async function () {
const unknownProjectId = new ObjectId().toString()
const downloadClient =
await testServer.createClientForDownloadZip(unknownProjectId)
await expectHttpError(
downloadClient.apis.Project.getLatestZip({
project_id: unknownProjectId,
}),
HTTPStatus.NOT_FOUND
)
})
})
describe('blob stats', function () {
let populatedPostgresProjectId,
populatedMongoProjectId,
emptyPostgresProjectId,
emptyMongoProjectId
async function populateProject(projectId) {
const files = {
[testFiles.GRAPH_PNG_HASH]: testFiles.path('graph.png'),
[testFiles.HELLO_TXT_HASH]: testFiles.path('hello.txt'),
}
for (const [hash, path] of Object.entries(files)) {
const response = await fetch(
testServer.url(`/api/projects/${projectId}/blobs/${hash}`),
{
method: 'PUT',
body: fs.createReadStream(path),
headers: {
Authorization: testServer.basicAuthHeader,
},
}
)
expect(response.status).to.equal(201)
}
}
beforeEach(async function () {
emptyPostgresProjectId = await testProjects.createEmptyProject()
emptyMongoProjectId = await testProjects.createEmptyProject(
new ObjectId().toString()
)
populatedPostgresProjectId = await testProjects.createEmptyProject()
await populateProject(populatedPostgresProjectId)
populatedMongoProjectId = await testProjects.createEmptyProject(
new ObjectId().toString()
)
await populateProject(populatedMongoProjectId)
})
describe('getProjectBlobsStats', function () {
it('handles empty postgres project', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getProjectBlobsStats({
body: { projectIds: [emptyPostgresProjectId] },
})
expect(body).to.deep.equal([
{
projectId: emptyPostgresProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
},
])
})
it('handles populated postgres project', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getProjectBlobsStats({
body: { projectIds: [populatedPostgresProjectId] },
})
expect(body).to.deep.equal([
{
projectId: populatedPostgresProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes:
testFiles.HELLO_TXT_BYTE_LENGTH + testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 1,
},
])
})
it('handles empty mongo project', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getProjectBlobsStats({
body: { projectIds: [emptyMongoProjectId] },
})
expect(body).to.deep.equal([
{
projectId: emptyMongoProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
},
])
})
it('handles populated mongo project', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getProjectBlobsStats({
body: { projectIds: [populatedMongoProjectId] },
})
expect(body).to.deep.equal([
{
projectId: populatedMongoProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes:
testFiles.HELLO_TXT_BYTE_LENGTH + testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 1,
},
])
})
it('handles batch of projects', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getProjectBlobsStats({
body: {
projectIds: [
populatedPostgresProjectId,
populatedMongoProjectId,
emptyPostgresProjectId,
emptyMongoProjectId,
],
},
})
expect(body).to.deep.equal([
{
projectId: populatedPostgresProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes:
testFiles.HELLO_TXT_BYTE_LENGTH + testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 1,
},
{
projectId: populatedMongoProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes:
testFiles.HELLO_TXT_BYTE_LENGTH + testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 1,
},
{
projectId: emptyPostgresProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
},
{
projectId: emptyMongoProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
},
])
})
})
describe('getBlobStats', function () {
it('handles empty list of hashes', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: { blobHashes: [] },
})
expect(body).to.deep.equal({
projectId: populatedPostgresProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
})
})
it('handles a mix of text and binary blobs', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: {
blobHashes: [testFiles.HELLO_TXT_HASH, testFiles.GRAPH_PNG_HASH],
},
})
expect(body).to.deep.equal({
projectId: populatedPostgresProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes:
testFiles.HELLO_TXT_BYTE_LENGTH + testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 1,
})
})
it('handles only text blobs', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: {
blobHashes: [testFiles.HELLO_TXT_HASH],
},
})
expect(body).to.deep.equal({
projectId: populatedPostgresProjectId,
textBlobBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
binaryBlobBytes: 0,
totalBytes: testFiles.HELLO_TXT_BYTE_LENGTH,
nTextBlobs: 1,
nBinaryBlobs: 0,
})
})
it('handles only binary blobs', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: {
blobHashes: [testFiles.GRAPH_PNG_HASH],
},
})
expect(body).to.deep.equal({
projectId: populatedPostgresProjectId,
textBlobBytes: 0,
binaryBlobBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
totalBytes: testFiles.GRAPH_PNG_BYTE_LENGTH,
nTextBlobs: 0,
nBinaryBlobs: 1,
})
})
it('handles non-existent blobs', async function () {
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: {
blobHashes: [testFiles.STRING_AB_HASH],
},
})
expect(body).to.deep.equal({
projectId: populatedPostgresProjectId,
textBlobBytes: 0,
binaryBlobBytes: 0,
totalBytes: 0,
nTextBlobs: 0,
nBinaryBlobs: 0,
})
})
it('throws an error for bad hashes', async function () {
await expectHttpError(
testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: populatedPostgresProjectId,
body: {
blobHashes: ['non-existent-hash'],
},
}),
HTTPStatus.UNPROCESSABLE_ENTITY
)
})
it('handles a request with a large number of blobs', async function () {
const projectId = await testProjects.createEmptyProject()
const blobHashes = []
let expectedTextBytes = 0
let expectedBinaryBytes = 0
const nTextBlobs = 10
const nBinaryBlobs = 10
for (let i = 0; i < nTextBlobs; i++) {
const content = `text blob ${i}`
const hash = blobHashFromString(content)
blobHashes.push(hash)
expectedTextBytes += content.length
const res = await fetch(
testServer.url(`/api/projects/${projectId}/blobs/${hash}`),
{
method: 'PUT',
body: content,
headers: { Authorization: testServer.basicAuthHeader },
}
)
expect(res.status).to.equal(HTTPStatus.CREATED)
}
for (let i = 0; i < nBinaryBlobs; i++) {
const content = Buffer.from([0, i, i + 1, i + 2])
const hash = await blobHashFromStream(
content.length,
Readable.from(content)
)
blobHashes.push(hash)
expectedBinaryBytes += content.length
const res = await fetch(
testServer.url(`/api/projects/${projectId}/blobs/${hash}`),
{
method: 'PUT',
body: content,
headers: {
Authorization: testServer.basicAuthHeader,
'Content-Type': 'application/octet-stream',
},
}
)
expect(res.status).to.equal(HTTPStatus.CREATED)
}
const { body } =
await testServer.basicAuthClient.apis.Project.getBlobStats({
project_id: projectId,
body: { blobHashes },
})
expect(body).to.deep.equal({
projectId,
textBlobBytes: expectedTextBytes,
binaryBlobBytes: expectedBinaryBytes,
totalBytes: expectedTextBytes + expectedBinaryBytes,
nTextBlobs,
nBinaryBlobs,
})
})
})
})
// eslint-disable-next-line mocha/no-pending-tests
describe.skip('getLatestContent', function () {
// TODO: remove this endpoint entirely, see
// https://github.com/overleaf/write_latex/pull/5120#discussion_r244291862
})
describe('project with changes', function () {
let projectId
beforeEach(async function () {
// used to provide a limit which forces us to persist all of the changes.
const farFuture = new Date()
farFuture.setTime(farFuture.getTime() + 7 * 24 * 3600 * 1000)
const limits = {
minChangeTimestamp: farFuture,
maxChangeTimestamp: farFuture,
}
const changes = [
new Change(
[new AddFileOperation('test.tex', File.fromString('ab'))],
new Date(),
[]
),
new Change(
[new AddFileOperation('other.tex', File.fromString('hello'))],
new Date(),
[]
),
]
projectId = await createEmptyProject()
await persistChanges(projectId, changes, limits, 0)
})
describe('getLatestHashedContent', function () {
it('returns a snapshot', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getLatestHashedContent({
project_id: projectId,
})
expect(response.status).to.equal(HTTPStatus.OK)
const snapshot = Snapshot.fromRaw(response.obj)
expect(snapshot.countFiles()).to.equal(2)
expect(snapshot.getFile('test.tex').getHash()).to.equal(
testFiles.STRING_AB_HASH
)
})
})
describe('getChanges', function () {
it('returns all changes when not given a limit', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
})
expect(response.status).to.equal(HTTPStatus.OK)
const { changes, hasMore } = response.obj
expect(changes.length).to.equal(2)
const filenames = changes
.flatMap(change => change.operations)
.map(operation => operation.pathname)
expect(filenames).to.deep.equal(['test.tex', 'other.tex'])
expect(hasMore).to.be.false
})
it('returns only requested changes', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
since: 1,
})
expect(response.status).to.equal(HTTPStatus.OK)
const { changes, hasMore } = response.obj
expect(changes.length).to.equal(1)
const filenames = changes
.flatMap(change => change.operations)
.map(operation => operation.pathname)
expect(filenames).to.deep.equal(['other.tex'])
expect(hasMore).to.be.false
})
it('rejects negative versions', async function () {
await expect(
testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
since: -1,
})
).to.be.rejectedWith('request failed with status 400')
})
it('rejects out of bounds versions', async function () {
await expect(
testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
since: 20,
})
).to.be.rejectedWith('request failed with status 400')
})
})
describe('project with many chunks', function () {
let projectId, changes
beforeEach(async function () {
// used to provide a limit which forces us to persist all of the changes.
const farFuture = new Date()
farFuture.setTime(farFuture.getTime() + 7 * 24 * 3600 * 1000)
const limits = {
minChangeTimestamp: farFuture,
maxChangeTimestamp: farFuture,
maxChunkChanges: 5,
}
projectId = await createEmptyProject()
const blobStore = new BlobStore(projectId)
const blob = await blobStore.putString('')
changes = [
new Change(
[new AddFileOperation('test.tex', File.createLazyFromBlobs(blob))],
new Date(),
[]
),
]
for (let i = 0; i < 20; i++) {
const textOperation = new TextOperation()
textOperation.retain(i)
textOperation.insert('x')
changes.push(
new Change(
[new EditFileOperation('test.tex', textOperation)],
new Date(),
[]
)
)
}
await persistChanges(projectId, changes, limits, 0)
})
it('returns the first chunk when not given a limit', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
})
expect(response.status).to.equal(HTTPStatus.OK)
expect(response.obj).to.deep.equal({
changes: changes.slice(0, 5).map(c => c.toRaw()),
hasMore: true,
})
})
it('returns only requested changes', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
since: 12,
})
expect(response.status).to.equal(HTTPStatus.OK)
expect(response.obj).to.deep.equal({
changes: changes.slice(12, 15).map(c => c.toRaw()),
hasMore: true,
})
})
it('returns changes in the latest chunk', async function () {
const response =
await testServer.basicAuthClient.apis.Project.getChanges({
project_id: projectId,
since: 20,
})
expect(response.status).to.equal(HTTPStatus.OK)
expect(response.obj).to.deep.equal({
changes: changes.slice(20).map(c => c.toRaw()),
hasMore: false,
})
})
})
})
describe('getLatestHistoryRaw', function () {
it('should handles read', async function () {
const projectId = fixtures.docs.initializedProject.id
const response =
await testServer.pseudoJwtBasicAuthClient.apis.Project.getLatestHistoryRaw(
{
project_id: projectId,
readOnly: 'true',
}
)
expect(response.body).to.deep.equal({
startVersion: 0,
endVersion: 1,
endTimestamp: '2032-01-01T00:00:00.000Z',
})
})
})
describe('deleteProject', function () {
it('deletes the project chunks', async function () {
const projectId = fixtures.docs.initializedProject.id
const historyResponse =
await testServer.pseudoJwtBasicAuthClient.apis.Project.getLatestHistory(
{
project_id: projectId,
}
)
expect(historyResponse.status).to.equal(HTTPStatus.OK)
expect(historyResponse.body).to.have.property('chunk')
const deleteResponse =
await testServer.basicAuthClient.apis.Project.deleteProject({
project_id: projectId,
})
expect(deleteResponse.status).to.equal(HTTPStatus.NO_CONTENT)
await expectHttpError(
testServer.pseudoJwtBasicAuthClient.apis.Project.getLatestHistory({
project_id: projectId,
}),
HTTPStatus.NOT_FOUND
)
})
it('deletes the project blobs', async function () {
const projectId = fixtures.docs.initializedProject.id
const token = testServer.createTokenForProject(projectId)
const authHeaders = { Authorization: `Bearer ${token}` }
const hash = testFiles.HELLO_TXT_HASH
const fileContents = await fs.promises.readFile(
testFiles.path('hello.txt')
)
const blobUrl = testServer.url(`/api/projects/${projectId}/blobs/${hash}`)
const response1 = await fetch(blobUrl, {
method: 'PUT',
headers: authHeaders,
body: fileContents,
})
expect(response1.ok).to.be.true
const response2 = await fetch(blobUrl, { headers: authHeaders })
const payload = await response2.text()
expect(payload).to.equal(fileContents.toString())
const deleteResponse =
await testServer.basicAuthClient.apis.Project.deleteProject({
project_id: projectId,
})
expect(deleteResponse.status).to.equal(HTTPStatus.NO_CONTENT)
const response3 = await fetch(blobUrl, { headers: authHeaders })
expect(response3.status).to.equal(HTTPStatus.NOT_FOUND)
})
it('deletes the project from the redis buffer', async function () {
const projectId = await createEmptyProject()
const blobStore = new BlobStore(projectId)
const blob = await blobStore.putString('this is a test')
const snapshot = new Snapshot()
const change = new Change(
[new AddFileOperation('test.tex', File.createLazyFromBlobs(blob))],
new Date(),
[]
)
await redisBuffer.queueChanges(projectId, snapshot, 0, [change])
const changesBefore = await redisBuffer.getNonPersistedChanges(
projectId,
0
)
expect(changesBefore.length).to.equal(1)
const deleteResponse =
await testServer.basicAuthClient.apis.Project.deleteProject({
project_id: projectId,
})
expect(deleteResponse.status).to.equal(HTTPStatus.NO_CONTENT)
const changesAfter = await redisBuffer.getNonPersistedChanges(
projectId,
0
)
expect(changesAfter.length).to.equal(0)
const finalState = await redisBuffer.getState(projectId)
expect(finalState).to.deep.equal({
changes: [],
expireTime: null,
headSnapshot: null,
headVersion: null,
persistTime: null,
persistedVersion: null,
})
})
it('deletes an empty project from the redis buffer', async function () {
const projectId = await createEmptyProject()
const deleteResponse =
await testServer.basicAuthClient.apis.Project.deleteProject({
project_id: projectId,
})
expect(deleteResponse.status).to.equal(HTTPStatus.NO_CONTENT)
const changesAfter = await redisBuffer.getNonPersistedChanges(
projectId,
0
)
expect(changesAfter.length).to.equal(0)
})
})
})