Summary:
Given that we perform syncback actions optimistically, i.e. we save the changes to the database before they are synced, some messages might not have an IMAP UID if they haven't been synced in the loop.
When we encounter messages without uid, we could just skip them, but this would mean that we end up moving only a subset of the messages we intended to move, or not move any at all, while thinking that we /did/ move everything, which might cause the thread to bounceback later on when we sync the messages that were missing.
The permanent fix is for syncback actions to not succeed until their changes have been synced, which is coming up in a separate diff
Test Plan: manual
Reviewers: evan, spang, khamidou, mark
Reviewed By: mark
Subscribers: mark
Differential Revision: https://phab.nylas.com/D3789
Summary: It is better to notify that a task failed rather than having a thread or group of threads randomly bounce back.
Test Plan: manual
Reviewers: evan, spang, mark, halla
Reviewed By: mark, halla
Subscribers: mark
Differential Revision: https://phab.nylas.com/D3790
Summary:
In JavaScript, null <= null is truthy. So if you set fetchedmax but not
fetchedmin and minUID, and then highestmodseq doesn't increment, we will
incorrectly skip updates on All Mail after the first batch of Gmail
inbox UID downloads, only making further progress on sync if the user
receives a new mail.
This patch tightens isSyncComplete() to require that all variables used
in the math comparisons are set.
Test Plan: manual 😢
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3787
Summary:
Summary of the changes:
- renamed some files to es6 to be able to use babel
- only try to create a db folder when we're using sqlite
- drop use of redis
- create a skeleton folder for workers.
Test Plan: Works on staging.
Reviewers: juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3784
Summary:
Users who only had a custom imap account could not open the window to
add a custom imap account the first time the auth was opened, rendering
the app useless for them.
Github issue #3185 and T7691
Test Plan: manual
Reviewers: halla, spang, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3783
Summary:
These windows don't need to receive IPC until they're being used for something
(i.e. after transitioning from hot window to composer, popout, etc)
Test Plan:
Run locally w/ hot windows shown, make sure that hot window doesn't
receive IPC
Reviewers: evan, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3780
Summary: See diff title
Test Plan: Run locally, check in htop for new titles
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3778
Summary:
Adds a new `npm run test-window` that will launch specs in a window so you
can use the debugger
The spec window wouldn't close because `onbeforeunload` was unnecessarily
preventing close. This circumvents this in spec mode.
Most significantly I discovered we can't use the synchronous timer for the
promise scheduler anymore. Suppose you do:
```
it('should error', async () => {
try {
await doSomething()
throw new Error("doSomething should have thrown!")
} catch (err) {
expect(err.message).toMatch(/my message/)
}
})
```
The way async/await is transpiled, when `doSomething` throws, the error
will propagate all the way back up to the uncaughtPromiseException handler
before the `catch` gets called and registered. The transpilation method
assumes that when the function gets executed it can synchrously advance
beyond the call before the `then` or `catch` resolve. When the promise
scheduler is synchronous this doesn't happen.
I chose to use `setTimeout` instead of `process.nextTick` or
`setImmediate` as the promise scheduler. `setTimeout` seems to work better
with Chrome's async function call stacks. `nextTick` and `setImmediate`,
being Node methods, skip Chrome's async watchers.
Test Plan:
I talked with Juan about these changes, in an upcoming diff he will be
testing these in the context of our broader test suite.
Reviewers: mark, khamidou, halla, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3779
Summary:
Previously we just assumed that we were in the "charging" state and wouldn't
update that until the state changed. This would cause us to throttle even if
the app was opened while plugged in. Now we don't do that.
Test Plan: Run locally, verify that we no longer throttle
Reviewers: spang, evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3782
Summary: This bumps the spellcheck and restores the specs
Test Plan:
Manually open a composer, type some text and ensure it underlines them as
misspelled. Also restore test coverage
Reviewers: mark, halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3776
Summary:
Previously, when updating message attributes during folder sync, we
would fetch local db messages only by folderImapUID. This was incorrect,
because messages in different folders could have the same uid, so our
query would return messages from other folders, causing us to
incorrectly update their attributes.
This commit ensures we fetch local messages in the current folder
Test Plan: manual
Reviewers: evan, halla, mark, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3777
We now correctly `await` for the `setFolders` and `setLabels`
operations, and make sure that the thread is always saved.
Apparently the `thread.id` check wasn't working as expected and we were skipping a
thread save, which only became apparent when we awaited for `setFolders`
and `setLabels`. Now, we always save the thread (like we did before this
method got updated)
Summary:
When I tested if we could use labels in place of the flag, I had the
Drafts folder selected, not All Mail, and the label did not show up.
Turns out that if you have All Mail selected, all draft messages will be
correctly tagged with the Draft label.
With this patch, we should correctly sync isDraft on all supported
providers and suppress these messages from the app until if or when we
decide to implement draft sync.
Thanks Brandon Long from the Gmail team for the help in getting this right.
Test Plan: manual
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3775
Summary:
Otherwise accounts with `Sent` and an `Outbox` will both get the `sent`
role, causing inconsistencies in sync
Addresses T7682
Test Plan: manual
Reviewers: khamidou, spang
Reviewed By: khamidou, spang
Differential Revision: https://phab.nylas.com/D3773
Summary:
Sometimes you have to perform a query that needs to do a table scan. For those
times you don't want to silence entire classes of queries (e.g. by table name),
you can now silence individual queries.
Test Plan: Run locally, verify that noisy stuff disappears from logs
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3763
Summary:
Now that message processing is serialized, this is unnecessary. I
was debugging something else and watched the code step through the
exact same db query twice in a row.
Test Plan: manual
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3764
Summary:
Now that we don't do strict validation of certificates for non-major IMAP
providers this shouldn't come up as much, but when it does we're gonna
want a better error message to help support out.
I am not 100% sure there aren't other socket errors that should be fatal,
but this was the one I could figure out by test authing against a server
with a self-signed cert and grepping around the node socket source code.
Test Plan: manual
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3774
Summary:
This removes `refreshHealthOfAccounts`, which I originally found because I
was trying to refactor it out of the IdentityStore. We need it gone from
the IdentityStore so that store can trigger frequently as things like the
usageLimits constantly update on it.
This also fixes T7520 where we'd get `ECONNREFUSED 127.0.0.1:2578` on
launch unnecessarily.
The concept of refreshing the health of accounts by occassionally polling
the /accounts endpoint is obsolete. This depends on
D3769 which entirely removes the /accounts endpoint
from K2.
Account health is pushed to us by `Actions.updateAccount` which is fired
directly from the main local-sync sync loop.
This also removes `refreshAccounts` and `Actions.refreshAllDeltaConnections`. These were fired by the Identity Store whenever it updated. We no longer need the Identity Store to tell us to reset the delta connections with our local-sync accounts. The delta connections will either be reset by our offline notification button, or when adding or removing an account. The Identity Store was a redundant mechanisms
Test Plan:
Manually verify things work with an existing account. Add a new account
and ensure sync starts. Remove an account and ensure it gets cleaned up.
Reviewers: halla, khamidou, mark, juan
Reviewed By: juan
Maniphest Tasks: T7520
Differential Revision: https://phab.nylas.com/D3770
Summary:
It turns out we don't ever use the /account API endpoint.
The GET /accounts endpoint was designed to query all accounts connected to
a system for old K2, but we don't use that anymore. The environment
variable protecting the endpoint isn't set anywhere.
We used to `GET /account` from the N1 AccountStore to attempt to refresh
the health of the accounts. The accompianing diff to this one makes that
obsolete. We never need to query for the account health since the sync
loop pushes it to us through `Actions.updateAccount`.
We also never `DELETE /account` because our local-sync runs a function
called `ensureK2Consistency` that compares its DB against the Nylas Mail
`AccountStore`.
This file has always been a huge source of confusion to me and a massive
red herring for anyone trying to understand how the account system work.
The accompyaning diff has more comments explaining the existing system
Depends on https://phab.nylas.com/D3770
Test Plan:
Manually boot N1. Ensure existing account works. Add a new account. Remove
an account. Open the developer tools and check that all the tabs still
work.
Lots of grepping through the code base.
Reviewers: halla, mark, juan, khamidou
Reviewed By: juan, khamidou
Differential Revision: https://phab.nylas.com/D3769
Summary:
This diff is a rename. No logical changes.
We used to have a set of files called `nylas-sync-worker`. In Old N1 these
used to have a lot of logic to slowly sync mail from our API. Since we
exclusively use local-sync via a soon-to-be-obsolesced delta stream, these
files really just manage the delta streaming connection.
It's incredibly confusing to have a set of files called worker-sync when
there's a sync-worker already in the codebase. This renames everything to
refer to them as account sync workers.
The reason I wanted this rename is because the IdentityStore on initial
launch triggers and fires a fairly scary-sounding
`Actions.refreshAllSyncWorkers()`. In reality all it does is
`Actions.refreshAllDeltaConnections()`. I'm also tired of staring at files
for a full minute before realizing that this is not the sync worker I was
looking for.
Test Plan:
After rename, booted the app and ensured that deltas were coming through
for new mail and no errors were being thrown
Reviewers: halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3767
Summary:
This will solve T7579 when saving messages to the sent folder. I
attempted to clean up the references code but decided it was better left for a
new diff, so added a bunch of TODO's in this diff
Test Plan: manual
Reviewers: halla, spang, evan
Reviewed By: spang, evan
Differential Revision: https://phab.nylas.com/D3766
Summary:
Unfortunately, many IMAP hosts outside the major ones do not have
certificates issued by a certificate authority, and it is very confusing
to folks to have their account auth not work. This patch relaxes our
certificate requirements for IMAP hosts outside the major providers.
It's cool that node 6 has secure TLS settings by default!
Fixes: T7673
Test Plan: manual
Reviewers: mark, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3771
