This reverts commit 0b3e3d2f39.
Interrupting sync by closing connection causes errors downstream when
`syncNow` is called elsewhere. Instead of interrupting by closing the
connection, we will post a patch to interrupt the sync loop properly
Summary:
- Ensure delete deltas make it through to N1
- Don't fail if we can't find a category that needs to be deleted
Test Plan: local
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3593
Summary:
- Adds `File` objects onto the `Message`s so N1 sees attachments
- Ensures `File` is eagerly loaded for all messages
- Base 64 streams attachments through the local /download endpoint
- ExtractFile only uses disposition type attachment when extracting
attachments
- Makes sure we save existing messages when processing them
Test Plan: manual :(
Reviewers: juan, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3595
Summary: I've found a pretty annoying bug --- N1 would stop syncing all accounts after the Internet connection dropped. It seems that deep inside node-imap or NodeJS itself, connections aren't timing out the right way. To work around this, this diff unilaterally restarts the sync every `nextSyncIn` milliseconds.
Test Plan: Tested manually by cutting internet access and checking that K2 recovered.
Reviewers: evan, juan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3573
Summary:
If you sent an email to yourself it would not show up in your inbox. This
is because sent messages would never get a lastMessageReceived timestamp.
Since we order the inbox by lastMessageReceived, setting that to null to
on sent mail would mean it never shows up in the thread list.
Also fixed an assertion bug in SFDC that requires transactions to return a
promise.
Finally added extra debug interfaces that will show more info if the delta
stream detects an inconsistency
Test Plan: manual
Reviewers: juan, halla, jackie
Reviewed By: jackie
Differential Revision: https://phab.nylas.com/D3552
Summary:
This replaces the API delta stream with a direct in-memory one
Addresses T7300
Test Plan: manual
Reviewers: jackie, halla, juan
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3548
Summary:
Associated N1 Diff: D3545
This commit ensures that authh notifications are showed when the
underlying sync worker fails and are cleared when an account is
successfully reconnected
To achieve this, we manually keep track and update syncStates where
appropriate via `Actions.updateAccount`, given that we have access to
N1's version of the account directly from local-sync.
Initially I was considering account delta stream to the cloud-api and the local-api, but that
just complicated things more than it helped.
This commit also fixes a bug with refreshing the gmail token in which we
we were only attempting a token refresh upon restarting the app
This addresses: T7346, T7305, T7335
Test Plan: Manual
Reviewers: halla, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3544
Summary:
This is a refactor of the auth APIs to use async/await. Gmail Auth is
pretty confusing and I wanted to make it cleaner to read and easier to
use. This is also part of the general API upgrade to modern ES6
This also fixes the Gmail auth error we saw at showcase
Test Plan: manual
Reviewers: halla, jackie, mark, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3535
Summary:
See https://github.com/mscdex/node-imap/issues/585 for details.
This issue was causing us constantly run the sync loop without pauses,
i.e. every next sync loop was scheduled immediately.
Currently, when we receive a new `'mail'`event, we trigger a new sync loop. Previously, when this happened while a sync loop was already in progress we would just ignore the event. However, my recent patch keeps track of how many times we tried to start a sync loop while one was already in progress. If the number of times this happens is > 0, it will schedule the next sync loop immediately (as opposed to waiting a constant amount seconds before the next loop).
The problem is that this new logic is making the sync worker always schedule the next sync loop immediately (without pausing for a few seconds). This is due to the following chain of events (assume we are just syncing `all` and `trash` folders):
This commit is a temporary workaround to this problem.
Test Plan: manual
Reviewers: evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3537
Summary:
Associated N1 Diff: D3530
This commit converts multi-send from a 3 step process into a 2 step
process
The first step creates the base message and sends a message per
recipient, each with its customized message body for tracking.
The second step reconciles all sent messages, specifically removing any
sent messages created by gmail, and saving the correct message to the
sent folder
This commit also ensures that we run the send tasks immediately by
ensuring we restart the sync loop if its already running
Test Plan: Manual
Reviewers: evan, jackie, halla
Reviewed By: jackie, halla
Differential Revision: https://phab.nylas.com/D3529
Summary:
Adds support for office 365
Depends on D3532
Test Plan: manual
Reviewers: jackie, halla, mark, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3533
Summary:
This is a small patch but it's pretty complex, because of the numbers of moving parts. Gmail has two types of tokens, access and refresh tokens. Access tokens have a limited shelf life of one hour. After that they expire and you need to use your refresh token to get a new one.
We've decided to do the access token generation on the server, because we don't feel comfortable giving our users both our Google client id and secret. To do that, I've added an endpoint, `/gmail/auth/refresh` which returns a valid access token as well as an expiration date for the token.
The only place where we handle token expiration is in the sync workers. Before trying opening a new connection we check if our access token is expired. If yes, we get a new one from the API. If there's an issue doing this, we notify N1 using `NylasAPIHelpers.handleAuthenticationFailure`.
There's a second patch for N1 with tiny related fixes.
Test Plan: Tested manually. Will need to test more in the real world.
Reviewers: evan, jackie, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3522
Summary:
Adds babel to K2
Creates a simple build script so it'll run on prod.
Test Plan: manual
Reviewers: jackie, halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3527
Summary:
Move sendmail-client and errors to isomorphic-core, given that they will
probably be used by cloud-workers (plugin backends) and cloud-api
Depends on D3510
Test Plan: Manual
Reviewers: halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3512
Summary:
This diff solves a few separate issues from T7313, T7316, T7282, and it refactors
the send code a little bit.
Initially, the problem that led to this diff was generating message ids that
wouldn't collide (which was causing errors in the message-processor). Collisions
in ids were being caused by messages that contained the exact same participants,
subject and date (most likely due bots or scripts sending emails in quick
succession)
To prevent collisions this commit adds the `message-id` header as part of the
database message id, and ensures that we set it correctly before sending, and
that it remains consistent through send, multi-send, and the sync loop.
During the refactor and review, I removed some code that assumed that we were
syncing drafts (which we aren't), and also fixes a few other known and
unknown issues around sending, message creation, and tracking, like assigning
the correct date header (we were previously assigning the draft creation date
from within N1), fixing the tracking regex, among other smaller bugs/typos.
Will address inline TODOs in a separate diff
Test Plan: TODO!!! I will add tests in another diff
Reviewers: evan, halla, jackie, khamidou
Reviewed By: halla, jackie
Differential Revision: https://phab.nylas.com/D3507
Summary: See title
Test Plan: Run locally
Reviewers: juan, evan
Reviewed By: juan, evan
Maniphest Tasks: T7281
Differential Revision: https://phab.nylas.com/D3498
Summary: See title
Test Plan: Ran it locally
Reviewers: khamidou, juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3496
Summary:
This commit fixes the following bugs in message parsing:
- we were unilaterally decoding MIME bodies as UTF-8; instead, decode according
to the charset data in the mimepart header
- '7bit' content-transfer-encoding means us-ascii, NOT utf-7
- only interpret valid content-transfer-encodings (previously we were trying
to treat various charsets as transfer-encodings)
- clearer naming: s/values/parsedMessage/
- unify snippet cleanup between plaintext & stripped HTML (merging
whitespace etc.)
Test Plan: units tests coming
Reviewers: juan
Differential Revision: https://phab.nylas.com/D3491
Summary:
Fixes T7283. We weren't creating deltas for contacts because we were inserting contacts using `UPSERT`, which requires us to add another sequelize hook. Unfortunately, support for this hook is only available on sequelize 4.x.
I didn't want to upgrade our sequelize version right now and cause of bunch of mysterious failures, so I've simply changed the `UPSERT` to be either an `INSERT`or `UPDATE`. We didn't really need it anyway.
Test Plan: Checked that N1 was receiving contact deltas.
Reviewers: bengotow
Reviewed By: bengotow
Maniphest Tasks: T7283
Differential Revision: https://phab.nylas.com/D3481
Also renames JSONType() -> buildJSONColumnOptions() and
JSONARRAYType() -> buildJSONARRAYColumnOptions() to prevent passing
those return values in as just the type value instead of the entire
options object.
- Don’t need functions in delta.js which must be called to return promsies. Fun of promsies is that you don’t need to care when they’re built to attach a .then.
- Make boundary between route handler and delta stream builder more explicit, don’t do query parsing in helpers, always reply from handler.
- Remove pushJSON extension to outputStream which never actually received JSON.
- Remove `takeUntil` - disposing of the downstream observable should dispose of all the merged/upstream observables
- Rename inflate => stringify since the returned value is a string not an object.
- Remove support for delta streams with no cursors. Don’t think this was supposed to be a feature.
- Add accountId to Transaction models
- Make database hooks shared in isomorphic core
This also ensures that we can use the accountId as part of the ids for
other models, like labels and folders
Before this commit, labels and folders with the same name for different
accounts would have the same id, screwing things up in n1
