More spec fixes
replace process.nextTick with setTimeout(fn, 0) for specs
Also added an unspy in the afterEach
Temporarily disable specs
fix(spec): start fixing specs
Summary:
This is the WIP fix to our spec runner.
Several tests have been completely commented out that will require
substantially more work to fix. These have been added to our sprint
backlog.
Other tests have been fixed to update to new APIs or to deal with genuine
bugs that were introduced without our knowing!
The most common non-trivial change relates to observing the `NylasAPI` and
`NylasAPIRequest`. We used to observe the arguments to `makeRequest`.
Unfortunately `NylasAPIRequest.run` is argumentless. Instead you can do:
`NylasAPIRequest.prototype.run.mostRecentCall.object.options` to get the
`options` passed into the object. the `.object` property grabs the context
of the spy when it was last called.
Fixing these tests uncovered several concerning issues with our test
runner. I spent a while tracking down why our participant-text-field-spec
was failling every so often. I chose that spec because it was the first
spec to likely fail, thereby requiring looking at the least number of
preceding files. I tried binary searching, turning on and off, several
files beforehand only to realize that the failure rate was not determined
by a particular preceding test, but rather the existing and quantity of
preceding tests, AND the number of console.log statements I had. There is
some processor-dependent race condition going on that needs further
investigation.
I also discovered an issue with the file-download-spec. We were getting
errors about it accessing a file, which was very suspicious given the code
stubs out all fs access. This was caused due to a spec that called an
async function outside ot a `waitsForPromise` block or a `waitsFor` block.
The test completed, the spies were cleaned up, but the downstream async
chain was still running. By the time the async chain finished the runner
was already working on the next spec and the spies had been restored
(causing the real fs access to run).
Juan had an idea to kill the specs once one fails to prevent cascading
failures. I'll implement this in the next diff update
Test Plan: npm test
Reviewers: juan, halla, jackie
Differential Revision: https://phab.nylas.com/D3501
Disable other specs
Disable more broken specs
All specs turned off till passing state
Use async-safe versions of spec functions
Add async test spec
Remove unused package code
Remove canary spec
This reverts commit ee5609bdb0.
Updates to nylas sync worker to support multiple cursors
Convert NylasSyncWorker to es6
Convert NylasSyncWorkerPool to es6
Extract into deltaProcessor
Update names to NylasSyncWorker state
Working on spec fixes
More spec fixes
Delta stream refactor fixes
- Threads: N1 always asks for the 'expanded' view, so K2 just always returns
the expanded view. N1 no longer needs to specify this.
- Messages: N1 only needed a 'count' view to display the initial sync progress
We've removed this progress bar and the corresponding request.
Summary:
Fixes a test that was broken due to my unfamiliarity with
CoffeeScript :-/
Test Plan: Tests
Reviewers: bengotow
Reviewed By: bengotow
Subscribers: juan
Differential Revision: https://phab.nylas.com/D3302
Summary:
This should help us avoid the thundering herd problem if we have some
kind of API outage affecting a wide number of clients.
Test Plan: Tests
Reviewers: bengotow
Reviewed By: bengotow
Subscribers: juan
Differential Revision: https://phab.nylas.com/D3297
- Completely remove nylas-long-connection.coffee file and consolidate all logic into
nylas-long-connection.es6 and worker-sync/delta-streaming-connection.es6:
- Separate it into DeltaStreamingConnection which is specific to the worker-sync
package for handling deltas
- NylasLongConnection is a "reusable" persistent connection to our API
which is also used by the search package
- This is a basically a fixed revision of https://phab.nylas.com/D2875,
see difff for more info
- Makes it so delta streaming does not retry so aggressively on 403s, which
happens whenever a user’s trial has expired
Summary:
- Remove nylas-long-connection.coffee duplicate code from sync worker package
and replace with delta-streaming-connection which uses NylasAPI.longConnection
- Replacement for XHR will come in another diff -- still need to clean up and
fix some edge cases
Test Plan:
- TODO! Will add test coverage for NylasLongConnection and
DeltaStreamingConnection
Reviewers: evan, bengotow
Reviewed By: evan, bengotow
Differential Revision: https://phab.nylas.com/D2875
Summary:
This diff includes several updates:
- Update sqlite version to use FTS5
- Adds new methods to DatabaseStore to create and update Search Indexes
- Currently indexing subject, participants, and thread message bodies or snippets if the body is not available
- Update initial sync to fetch 5000 message bodies per account
- Adds a new SearchMatcher
- Add new thread-search-index package to run in the worker window to
init and keep thread search index up to date
- Converts Thread to ES6
TODO:
- Remove/Update suggestions dropdown
- Add tests
Test Plan: - TODO
Reviewers: evan, bengotow
Reviewed By: evan, bengotow
Differential Revision: https://phab.nylas.com/D2826
Summary:
The TaskQueue does it's own throttling and has it's own processQueue retry timeout, no need for longPollConnected
Remove dead code (OfflineError)
Rename long connection state to status so we don't ask for `state.state`
Remove long poll actions related to online/offline in favor of exposing connection state through NylasSyncStatusStore
Consoliate notifications and account-error-heaer into a single package and organize files into sidebar vs. header.
Update the DeveloperBarStore to query the sync status store for long poll statuses
Test Plan: All existing tests pass
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D2835
Summary:
Snooze should wait for categories on all accounts
Fix authPlugin to rembmer `plugin+accountId`, not pluginId, add specs
categories() returned [], categories(acctId) returned {}
dry up sync worker, fetch metadata before anything else
Test Plan: Run tests
Reviewers: drew, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D2693
Summary:
Previously we were storing sync cursors in config.cson. They were by
far the most frequently updated piece of data in config. To make things worse,
all the writing was happening in the worker window - the main window was just
seeing the changes on disk and reloading.
I believe there's an edge case which causes the main window to read the config
file when it's mid-write and empty. This causes the accounts array to become
empty in the AccountStore and lots of downstream issues. It's also then possible
for the main window to write a config change of it's own and empty the file
permanently.
Test Plan: A few new tests to make sure this is backwards compatible.
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D2642
Summary:
Until now, we've been hiding transactions beneath the surface. When you call persistModel, you're implicitly creating a transaction.
You could explicitly create them with `atomically`..., but there were several critical problems that are fixed in this diff:
- Calling persistModel / unpersistModel within a transaction could cause the DatabaseStore to trigger. This could result in other parts of the app making queries /during/
the transaction, potentially before the COMMIT occurred and saved the changes. The new, explicit inTransaction syntax holds all changes until after COMMIT and then triggers.
- Calling atomically and then calling persistModel inside that resulted in us having to check whether a transaction was present and was gross.
- Many parts of the code ran extensive logic inside a promise chained within `atomically`:
BAD:
```
DatabaseStore.atomically =>
DatabaseStore.persistModel(draft) =>
GoMakeANetworkRequestThatReturnsAPromise
```
OVERWHELMINGLY BETTER:
```
DatabaseStore.inTransaction (t) =>
t.persistModel(draft)
.then =>
GoMakeANetworkRequestThatReturnsAPromise
```
Having explicit transactions also puts us on equal footing with Sequelize and other ORMs. Note that you /have/ to call DatabaseStore.inTransaction (t) =>. There is no other way to access the methods that let you alter the database. :-)
Other changes:
- This diff removes Message.labels and the Message-Labels table. We weren't using Message-level labels anywhere, and the table could grow very large.
- This diff changes the page size during initial sync from 250 => 200 in an effort to make transactions a bit faster.
Test Plan: Run tests!
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D2353
Summary:
Add concept of "final" to Query, clean up internals
Tiny bug fixes
RxJs Observables!
WIP
Test Plan: Run tests
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D2319
Summary:
Previously, when an error was encountered during initial mailbox sync we just started it
over after a retry delay. Recent API uptime issues mean that this was happening often and lots of
people were seeing sync retry many times. This is bad because the app is less performant while
it's syncing mail, and also generates unnecessary load as the app re-fetches threads it already has.
In this diff, there are new specs and functionality in nylas-sync-worker to start fetching
where we left off. This is typically going to be OK because the default sort ordering of the
threads endpoint is newest->oldest, so if new items have arrived since we started fetching
and page boundaries have changed, we'll get duplicate data rather than missing data. Connceting
to the streaming API as soon as we start the sync also ensures that we roll up any changes to
data we've already paginated over.
Test Plan: Run tests
Reviewers: drew, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D2132
Summary:
Move sync workers and Edgehill token checks to work window
Move the task queue and database setup to the work window
Move ContactStore background refresh to work window
Store the task queue in the database
WIP
The TaskQueue now puts tasks in the database instead of in a file, which also means it can be observed
Move all delta sync and initial sync to a package, make NylasSyncStore which exposes read-only sync state
DraftStore no longer reads task status. Once you set the "sending" bit on a draft, it never gets unset. But that's fine actually.
If your package lists windowTypes, you *only* get loaded in those windowTypes. If you specify no windowTypes, you get loaded in the root window.
This means that onboarding, worker-ui, worker-sync, etc. no longer get loaded into the main window
ActivitySidebar has a special little store that observes the task queue since it's no longer in the window
Move "toggle component regions" / "toggle react remote" to the Developer menu
Move sync worker specs, update draft store specs to not rely on TaskQueue at all
Test Plan: Run existing tests, all pass
Reviewers: dillon, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D1936
2015-08-27 16:39:40 -07:00
Renamed from spec-nylas/nylas-sync-worker-spec.coffee (Browse further)
