Summary:
This will prevent us from showing error messages to the user when we
can automatically recover from the error
Test Plan: manual-- throw error from syncback task, check expected results
Reviewers: evan, mark, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3812
Summary:
There are 2 types of IMAP errors that need to be treated as retryable. See code
comments as to why.
Test Plan: manual
Reviewers: khamidou, evan, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3811
We added new presets and plugins to the k2 .babelrc but didn't add the
respective dependencies in our package.json, so the build was failing.
Add the required dependencies, and make sure that `build-n1-cloud.js`
uses the correct babel presets and plugins
Summary: This was broken because babel could not compile the `super` keyword
Test Plan: manual
Reviewers: evan, mark, halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3804
Summary:
EnsureMessageInSentFolder also needs to sync the sent folder to fetch
the uid of the newly moved message (like `MoveThreadToFolder` does)
Test Plan: manual
Reviewers: halla, mark, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3803
Summary:
In `FetchNewMessagesInFolder`, sometimes we haven't synced anything in the folder
we are trying to fetch new messages in. Previously this would just throw
an error, now we properly check if we have a fetchedmax, and if not just
run a normal fetch.
Also, when the target folder box was already open, we were not fetching the /latest/ box status to check the latest uidnext value, so we would skip fetching new messages when in fact there were new messages to fetch
(This can happen for example when moving a sent message to the Sent
folder before we've started syncing the Sent folder)
Test Plan: manual
Reviewers: halla, mark, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3802
Summary: see title
Test Plan: manual, deploy to staging, check that it works
Reviewers: evan, spang, tomasz, khamidou
Reviewed By: tomasz
Differential Revision: https://phab.nylas.com/D3800
Summary:
Syncback tasks that move messages to a different folder used to leave
those messages without a uid because it was unknown at the moment, and
it would only be discovered later on in the sync loop.
This had the unwanted effect of not allowing you to perform more than 1
syncback action on the same thread back to back (e.g. undoing an archive, or
sending and archiving immediately)
This commit makes it so that `SetThreadFolderAndLabels` and
`MoveThreadToFolder` runs a new sync task to fetch the new uids for the
moved messages.
We do this via a new sync task, `FetchNewMessagesInFolder` which /only/
fetches new messages in a folder (no fetching old messages or attribute
updates). This is a first step to cleaning up the gigantic
`FetchMessagesInFolder` task into smaller parts-- but that will come in
a separate diff. For now we want to fix the immediate problem.
See D3788 and D3789 for more details
Test Plan:
manually move threads around, undo moving threads, reply on a thread
and immediately archive
Reviewers: khamidou, mark, spang, halla, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3798
Summary:
updateMessagesFromThread requires `messages` to be an array only when
recompute is falsy
Test Plan: manual
Reviewers: halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3797
Summary:
If there were any uidvalidity changes after we've completely synced a folder, we
would completely ignore them and not attempt to sync the folder.
Also, we weren't saving the latest uidvalidity from the box to the folder
syncState as soon as we recovered, we only saved it until after fetching
messages. This meant that if the operation was interrupted before updating
syncState.uidvalidity, we would always think that we were in a state of uid invalidity
Recovering from uidvalidity was also broken because we weren't resetting the
fetched ranges, and unnecessarily setting the folderId to null, which meant that
we would never restore the uids of messages we had already fetched.
Test Plan: manual
Reviewers: evan, khamidou, mark, halla
Reviewed By: mark
Differential Revision: https://phab.nylas.com/D3796
Summary:
If we attempt to operate on a box that is no longer open, we should make
the error retryable so that we re-open the correct box and continue
syncing instead of showing the scary red box to users
Addresses T7680
Test Plan: manual
Reviewers: evan, spang, halla, mark
Reviewed By: halla, mark
Subscribers: mark
Differential Revision: https://phab.nylas.com/D3792
Summary:
Given that we perform syncback actions optimistically, i.e. we save the changes to the database before they are synced, some messages might not have an IMAP UID if they haven't been synced in the loop.
When we encounter messages without uid, we could just skip them, but this would mean that we end up moving only a subset of the messages we intended to move, or not move any at all, while thinking that we /did/ move everything, which might cause the thread to bounceback later on when we sync the messages that were missing.
The permanent fix is for syncback actions to not succeed until their changes have been synced, which is coming up in a separate diff
Test Plan: manual
Reviewers: evan, spang, khamidou, mark
Reviewed By: mark
Subscribers: mark
Differential Revision: https://phab.nylas.com/D3789
Summary:
In JavaScript, null <= null is truthy. So if you set fetchedmax but not
fetchedmin and minUID, and then highestmodseq doesn't increment, we will
incorrectly skip updates on All Mail after the first batch of Gmail
inbox UID downloads, only making further progress on sync if the user
receives a new mail.
This patch tightens isSyncComplete() to require that all variables used
in the math comparisons are set.
Test Plan: manual 😢
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3787
Summary:
Summary of the changes:
- renamed some files to es6 to be able to use babel
- only try to create a db folder when we're using sqlite
- drop use of redis
- create a skeleton folder for workers.
Test Plan: Works on staging.
Reviewers: juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3784
Summary:
Previously we just assumed that we were in the "charging" state and wouldn't
update that until the state changed. This would cause us to throttle even if
the app was opened while plugged in. Now we don't do that.
Test Plan: Run locally, verify that we no longer throttle
Reviewers: spang, evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3782
Summary:
Previously, when updating message attributes during folder sync, we
would fetch local db messages only by folderImapUID. This was incorrect,
because messages in different folders could have the same uid, so our
query would return messages from other folders, causing us to
incorrectly update their attributes.
This commit ensures we fetch local messages in the current folder
Test Plan: manual
Reviewers: evan, halla, mark, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3777
We now correctly `await` for the `setFolders` and `setLabels`
operations, and make sure that the thread is always saved.
Apparently the `thread.id` check wasn't working as expected and we were skipping a
thread save, which only became apparent when we awaited for `setFolders`
and `setLabels`. Now, we always save the thread (like we did before this
method got updated)
Summary:
When I tested if we could use labels in place of the flag, I had the
Drafts folder selected, not All Mail, and the label did not show up.
Turns out that if you have All Mail selected, all draft messages will be
correctly tagged with the Draft label.
With this patch, we should correctly sync isDraft on all supported
providers and suppress these messages from the app until if or when we
decide to implement draft sync.
Thanks Brandon Long from the Gmail team for the help in getting this right.
Test Plan: manual
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3775
Summary:
Otherwise accounts with `Sent` and an `Outbox` will both get the `sent`
role, causing inconsistencies in sync
Addresses T7682
Test Plan: manual
Reviewers: khamidou, spang
Reviewed By: khamidou, spang
Differential Revision: https://phab.nylas.com/D3773
Summary:
Now that message processing is serialized, this is unnecessary. I
was debugging something else and watched the code step through the
exact same db query twice in a row.
Test Plan: manual
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3764
Summary:
Now that we don't do strict validation of certificates for non-major IMAP
providers this shouldn't come up as much, but when it does we're gonna
want a better error message to help support out.
I am not 100% sure there aren't other socket errors that should be fatal,
but this was the one I could figure out by test authing against a server
with a self-signed cert and grepping around the node socket source code.
Test Plan: manual
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3774
Summary:
It turns out we don't ever use the /account API endpoint.
The GET /accounts endpoint was designed to query all accounts connected to
a system for old K2, but we don't use that anymore. The environment
variable protecting the endpoint isn't set anywhere.
We used to `GET /account` from the N1 AccountStore to attempt to refresh
the health of the accounts. The accompianing diff to this one makes that
obsolete. We never need to query for the account health since the sync
loop pushes it to us through `Actions.updateAccount`.
We also never `DELETE /account` because our local-sync runs a function
called `ensureK2Consistency` that compares its DB against the Nylas Mail
`AccountStore`.
This file has always been a huge source of confusion to me and a massive
red herring for anyone trying to understand how the account system work.
The accompyaning diff has more comments explaining the existing system
Depends on https://phab.nylas.com/D3770
Test Plan:
Manually boot N1. Ensure existing account works. Add a new account. Remove
an account. Open the developer tools and check that all the tabs still
work.
Lots of grepping through the code base.
Reviewers: halla, mark, juan, khamidou
Reviewed By: juan, khamidou
Differential Revision: https://phab.nylas.com/D3769
Summary:
This will solve T7579 when saving messages to the sent folder. I
attempted to clean up the references code but decided it was better left for a
new diff, so added a bunch of TODO's in this diff
Test Plan: manual
Reviewers: halla, spang, evan
Reviewed By: spang, evan
Differential Revision: https://phab.nylas.com/D3766
Summary:
Unfortunately, many IMAP hosts outside the major ones do not have
certificates issued by a certificate authority, and it is very confusing
to folks to have their account auth not work. This patch relaxes our
certificate requirements for IMAP hosts outside the major providers.
It's cool that node 6 has secure TLS settings by default!
Fixes: T7673
Test Plan: manual
Reviewers: mark, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3771
Summary:
We've been syncing drafts messages but not the drafts flag in K2, making
them appear in Edgehill as regular old messages.
This commit makes K2 sync the drafts flag, and also correctly label
folders called "Drafts" with the 'drafts' role.
Because 2-way syncing of drafts is very complex and error-prone since
you need to add new drafts and delete the old ones on every update, and
we reaally don't want to do things like create multiplying draft copies
or accidentally lose a draft someone started composing elsewhere, we
simply exclude messages marked as drafts from being serialized to
Edgehill through the delta stream for now. This removes the confusing
behaviour and also sets a better stage for completing drafts sync later.
Eventually we will also want to add functionality to allow users to
select their drafts folder, but for now this code does the right thing
in many more cases.
While investigating this behaviour, I also discovered a bug we've never
seen before where Gmail isn't applying the \Draft flag to draft
messages, no matter which folder we fetch them from. :-/ This is very
unfortunate and there's no way for us to work around it other than to
fetch messages in the Drafts folder and manually apply the flag locally,
since "drafts" is not a label in Gmail, only another IMAP folder. Brandon
Long from the Gmail team says that this is because they've had
problems with clients which sync drafts, so the Gmail web client and
mobile apps do not set the \Draft flag on drafts. (I don't get how this
solves their problem, but okay.) Let's solve the issue on Gmail if it
comes up by user demand—should be relatively straightforward to
implement, but it adds sync work & complexity.
Fixes T7593
Test Plan: manual
Reviewers: halla, juan
Reviewed By: juan
Maniphest Tasks: T7593
Differential Revision: https://phab.nylas.com/D3749
