feat(work): Create the "Work" window, move TaskQueue, Nylas sync workers
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-28 07:39:40 +08:00
|
|
|
_ = require 'underscore'
|
|
|
|
fs = require 'fs'
|
|
|
|
path = require 'path'
|
2015-12-18 03:46:05 +08:00
|
|
|
{NylasAPI,
|
|
|
|
Thread,
|
|
|
|
DatabaseStore,
|
|
|
|
DatabaseTransaction,
|
|
|
|
Actions} = require 'nylas-exports'
|
feat(mail-rules): Per-account mail rules filter incoming, existing mail
Summary:
Originally, this was going to be a totally independent package, but
I wasn't able to isolate the functionality and get it tied in to
the delta-stream consumption. Here's how it currently works:
- The preferences package has a new tab which allows you to edit
mail filters. Filters are saved in a new core store, and a new
stock component (ScenarioEditor) renders the editor. The editor
takes a set of templates that define a value space, and outputs
a valid set of values.
- A new MailFilterProcessor takes messages and creates tasks to
apply the actions from the MailFiltersStore.
- The worker-sync package now uses the MailFilterProcessor to
apply filters /before/ it calls didPassivelyReceiveNewModels,
so filtrs are applied before any notifications are created.
- A new task, ReprocessMailFiltersTask allows you to run filters
on all of your existing mail. It leverages the existing TaskQueue
architecture to: a) resume where it left off if you quit midway,
b) be queryable (for status) from all windows and c) cancelable.
The TaskQueue is a bit strange because it runs performLocal and
performRemote very differently, and I had to use `performRemote`.
(todo refactor soon.)
This diff also changes the EditableList a bit to behave like a
controlled component and render focused / unfocused states.
Test Plan: Run tests, only for actual filter processing atm.
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D2379
2015-12-23 15:19:32 +08:00
|
|
|
|
2016-11-29 09:33:14 +08:00
|
|
|
NylasSyncWorkerPool = require('../lib/nylas-sync-worker-pool').default
|
feat(mail-rules): Per-account mail rules filter incoming, existing mail
Summary:
Originally, this was going to be a totally independent package, but
I wasn't able to isolate the functionality and get it tied in to
the delta-stream consumption. Here's how it currently works:
- The preferences package has a new tab which allows you to edit
mail filters. Filters are saved in a new core store, and a new
stock component (ScenarioEditor) renders the editor. The editor
takes a set of templates that define a value space, and outputs
a valid set of values.
- A new MailFilterProcessor takes messages and creates tasks to
apply the actions from the MailFiltersStore.
- The worker-sync package now uses the MailFilterProcessor to
apply filters /before/ it calls didPassivelyReceiveNewModels,
so filtrs are applied before any notifications are created.
- A new task, ReprocessMailFiltersTask allows you to run filters
on all of your existing mail. It leverages the existing TaskQueue
architecture to: a) resume where it left off if you quit midway,
b) be queryable (for status) from all windows and c) cancelable.
The TaskQueue is a bit strange because it runs performLocal and
performRemote very differently, and I had to use `performRemote`.
(todo refactor soon.)
This diff also changes the EditableList a bit to behave like a
controlled component and render focused / unfocused states.
Test Plan: Run tests, only for actual filter processing atm.
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D2379
2015-12-23 15:19:32 +08:00
|
|
|
NylasSyncWorkerPool = new NylasSyncWorkerPool()
|
feat(work): Create the "Work" window, move TaskQueue, Nylas sync workers
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-28 07:39:40 +08:00
|
|
|
|
|
|
|
fixturesPath = path.resolve(__dirname, 'fixtures')
|
|
|
|
|
|
|
|
describe "NylasSyncWorkerPool", ->
|
|
|
|
|
|
|
|
describe "handleDeltas", ->
|
|
|
|
beforeEach ->
|
|
|
|
@sampleDeltas = JSON.parse(fs.readFileSync("#{fixturesPath}/sample.json"))
|
|
|
|
@sampleClustered = JSON.parse(fs.readFileSync("#{fixturesPath}/sample-clustered.json"))
|
|
|
|
|
|
|
|
it "should immediately fire the received raw deltas event", ->
|
|
|
|
spyOn(Actions, 'longPollReceivedRawDeltas')
|
|
|
|
spyOn(NylasSyncWorkerPool, '_clusterDeltas').andReturn({create: {}, modify: {}, destroy: []})
|
|
|
|
NylasSyncWorkerPool._handleDeltas(@sampleDeltas)
|
|
|
|
expect(Actions.longPollReceivedRawDeltas).toHaveBeenCalled()
|
|
|
|
|
|
|
|
it "should call helper methods for all creates first, then modifications, then destroys", ->
|
|
|
|
spyOn(Actions, 'longPollProcessedDeltas')
|
|
|
|
|
|
|
|
handleDeltaDeletionPromises = []
|
|
|
|
resolveDeltaDeletionPromises = ->
|
|
|
|
fn() for fn in handleDeltaDeletionPromises
|
|
|
|
handleDeltaDeletionPromises = []
|
|
|
|
|
|
|
|
spyOn(NylasSyncWorkerPool, '_handleDeltaDeletion').andCallFake ->
|
|
|
|
new Promise (resolve, reject) ->
|
|
|
|
handleDeltaDeletionPromises.push(resolve)
|
|
|
|
|
|
|
|
handleModelResponsePromises = []
|
|
|
|
resolveModelResponsePromises = ->
|
|
|
|
fn() for fn in handleModelResponsePromises
|
|
|
|
handleModelResponsePromises = []
|
|
|
|
|
|
|
|
spyOn(NylasAPI, '_handleModelResponse').andCallFake ->
|
|
|
|
new Promise (resolve, reject) ->
|
|
|
|
handleModelResponsePromises.push(resolve)
|
|
|
|
|
|
|
|
spyOn(NylasSyncWorkerPool, '_clusterDeltas').andReturn(JSON.parse(JSON.stringify(@sampleClustered)))
|
|
|
|
NylasSyncWorkerPool._handleDeltas(@sampleDeltas)
|
|
|
|
|
|
|
|
createTypes = Object.keys(@sampleClustered['create'])
|
|
|
|
expect(NylasAPI._handleModelResponse.calls.length).toEqual(createTypes.length)
|
|
|
|
expect(NylasAPI._handleModelResponse.calls[0].args[0]).toEqual(_.values(@sampleClustered['create'][createTypes[0]]))
|
|
|
|
expect(NylasSyncWorkerPool._handleDeltaDeletion.calls.length).toEqual(0)
|
|
|
|
|
|
|
|
NylasAPI._handleModelResponse.reset()
|
|
|
|
resolveModelResponsePromises()
|
|
|
|
advanceClock()
|
|
|
|
|
|
|
|
modifyTypes = Object.keys(@sampleClustered['modify'])
|
|
|
|
expect(NylasAPI._handleModelResponse.calls.length).toEqual(modifyTypes.length)
|
|
|
|
expect(NylasAPI._handleModelResponse.calls[0].args[0]).toEqual(_.values(@sampleClustered['modify'][modifyTypes[0]]))
|
|
|
|
expect(NylasSyncWorkerPool._handleDeltaDeletion.calls.length).toEqual(0)
|
|
|
|
|
|
|
|
NylasAPI._handleModelResponse.reset()
|
|
|
|
resolveModelResponsePromises()
|
|
|
|
advanceClock()
|
|
|
|
|
|
|
|
destroyCount = @sampleClustered['destroy'].length
|
|
|
|
expect(NylasSyncWorkerPool._handleDeltaDeletion.calls.length).toEqual(destroyCount)
|
|
|
|
expect(NylasSyncWorkerPool._handleDeltaDeletion.calls[0].args[0]).toEqual(@sampleClustered['destroy'][0])
|
|
|
|
|
|
|
|
expect(Actions.longPollProcessedDeltas).not.toHaveBeenCalled()
|
|
|
|
|
|
|
|
resolveDeltaDeletionPromises()
|
|
|
|
advanceClock()
|
|
|
|
|
|
|
|
expect(Actions.longPollProcessedDeltas).toHaveBeenCalled()
|
|
|
|
|
|
|
|
describe "clusterDeltas", ->
|
|
|
|
beforeEach ->
|
|
|
|
@sampleDeltas = JSON.parse(fs.readFileSync("#{fixturesPath}/sample.json"))
|
|
|
|
@expectedClustered = JSON.parse(fs.readFileSync("#{fixturesPath}/sample-clustered.json"))
|
|
|
|
|
|
|
|
it "should collect create/modify events into a hash by model type", ->
|
|
|
|
{create, modify} = NylasSyncWorkerPool._clusterDeltas(@sampleDeltas)
|
|
|
|
expect(create).toEqual(@expectedClustered.create)
|
|
|
|
expect(modify).toEqual(@expectedClustered.modify)
|
|
|
|
|
|
|
|
it "should collect destroys into an array", ->
|
|
|
|
{destroy} = NylasSyncWorkerPool._clusterDeltas(@sampleDeltas)
|
|
|
|
expect(destroy).toEqual(@expectedClustered.destroy)
|
|
|
|
|
|
|
|
describe "handleDeltaDeletion", ->
|
|
|
|
beforeEach ->
|
|
|
|
@thread = new Thread(id: 'idhere')
|
|
|
|
@delta =
|
|
|
|
"cursor": "bb95ddzqtr2gpmvgrng73t6ih",
|
|
|
|
"object": "thread",
|
|
|
|
"event": "delete",
|
|
|
|
"id": @thread.id,
|
|
|
|
"timestamp": "2015-08-26T17:36:45.297Z"
|
|
|
|
|
2015-12-18 03:46:05 +08:00
|
|
|
spyOn(DatabaseTransaction.prototype, 'unpersistModel')
|
|
|
|
|
feat(work): Create the "Work" window, move TaskQueue, Nylas sync workers
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-28 07:39:40 +08:00
|
|
|
it "should resolve if the object cannot be found", ->
|
|
|
|
spyOn(DatabaseStore, 'find').andCallFake (klass, id) =>
|
|
|
|
return Promise.resolve(null)
|
|
|
|
waitsForPromise =>
|
|
|
|
NylasSyncWorkerPool._handleDeltaDeletion(@delta)
|
|
|
|
runs =>
|
|
|
|
expect(DatabaseStore.find).toHaveBeenCalledWith(Thread, 'idhere')
|
2015-12-18 03:46:05 +08:00
|
|
|
expect(DatabaseTransaction.prototype.unpersistModel).not.toHaveBeenCalled()
|
feat(work): Create the "Work" window, move TaskQueue, Nylas sync workers
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-28 07:39:40 +08:00
|
|
|
|
|
|
|
it "should call unpersistModel if the object exists", ->
|
|
|
|
spyOn(DatabaseStore, 'find').andCallFake (klass, id) =>
|
|
|
|
return Promise.resolve(@thread)
|
|
|
|
waitsForPromise =>
|
|
|
|
NylasSyncWorkerPool._handleDeltaDeletion(@delta)
|
|
|
|
runs =>
|
|
|
|
expect(DatabaseStore.find).toHaveBeenCalledWith(Thread, 'idhere')
|
2015-12-18 03:46:05 +08:00
|
|
|
expect(DatabaseTransaction.prototype.unpersistModel).toHaveBeenCalledWith(@thread)
|
feat(work): Create the "Work" window, move TaskQueue, Nylas sync workers
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-28 07:39:40 +08:00
|
|
|
|
2015-09-03 03:22:20 +08:00
|
|
|
describe "handleModelResponse", ->
|
2015-10-02 12:39:44 +08:00
|
|
|
# SEE spec/nylas-api-spec.coffee
|