Summary:
When multiple accounts are syncing, it's very hard to scan the local
sync logs because it is unclear to which account the logs belong to,
and it makes debugging hard.
This commit makes it so that all logs from local-sync include the
account info, with the account email prefixed at the beginning of each
log line (this allows filtering), and color coded by account.
Test Plan: manual
Reviewers: mark, spang, khamidou, evan, halla
Reviewed By: evan, halla
Differential Revision: https://phab.nylas.com/D3851
Summary:
It's slow, so only do it if we've never seen the Account before. This
fixes jank that would happen somewhat randomly. It was especially noticeable
if the user had a lot of Accounts (and therefore a lot of passwords to
load).
Test Plan: Run locally, verify that we only load new Accounts
Reviewers: spang, evan, juan
Reviewed By: evan, juan
Maniphest Tasks: T7766
Differential Revision: https://phab.nylas.com/D3848
Summary:
We don't want to overwhelm a user with a bunch of bings and bongs when
they open their laptop after a long weekend. This diff takes a
relatively simple approach by debouncing the notification sounds every 5
seconds.
Test Plan: Run locally, make sure we still get notified but not too much.
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3841
Summary:
On each sync loop, we increment the socketTimeout based on how many times we've
seen socket timeouts in a row. The max socket timeout is 10m
Test Plan: manual
Reviewers: evan, spang, mark
Reviewed By: mark
Differential Revision: https://phab.nylas.com/D3843
Summary:
This diff adds support for database migration to our cloud API. It's partially inspired by Halla's local-sync migration diff (D3809). You can run a migration by calling "node-babel scripts/migrate-db up|down" or by calling "npm script upgrade-db|downgrade-db".
Note that for simplicity reasons we assume that we're only writing migrations for our MySQL database – people developing locally may have to blow up there dbs whenever there's a schema change, though in practice `ALTER TABLE ADD COLUMN`statements work the same on both dbs.
Test Plan: Tested locally. Will run the metadata migration on staging.
Reviewers: evan, spang, halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3840
Summary:
This commit converts list-tabular to JS, and in the process re-adds shouldComponentUpdate which had been previously removed (D3837).
This time, shouldComponentUpdate will correctly check if actual data to render has changed, as opposed to checking if `itemPropsGenerator` had changed.
Test Plan: manual
Reviewers: halla, spang, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3839
Summary:
In most cases (and especially so on Gmail and in the inbox on generic
IMAP), messages with higher UIDs are newer---and even if they aren't the
newest possible messages in other generic IMAP folders, they are the
most recent messages that have been moved to that folder.
Our previous batching strategy unfortunately resulted in us downloading
the lowest UID in each batch first, which was especially confusing when
connecting a new account and having the first message pop up on the
screen be a message from hours or days ago.
This patch changes the batching strategy in three ways:
1. Within a batch, we process downloaded messages from highest UID to
lowest UID.
2. We download batches in order of the ones containing the highest UIDs
first.
3. We group together more UIDs within a single batch by ignoring charset
and transfer-encoding on parts and grouping only by MIME part IDs (which
is the only thing you have to pass to the IMAP FETCH command---no idea
why we included this extraneous part data before, probably just
convenience.)
Example old grouping:
batch key: '[{"id":"2","transferEncoding":"QUOTED-PRINTABLE","charset":"UTF-8","mimeType":"text/html"}]'
batch UIDs: [356416,356418,356420,356423,356432,356433,356435,356436,356437,356442,356444]
batch key: '[{"id":"2","transferEncoding":"QUOTED-PRINTABLE","charset":"Windows-1252","mimeType":"text/html"}]'
batch UIDs: [353777]
In the new strategy, all of these messages will be downloaded with the
same FETCH command, reducing IMAP round trips before message processing
begins.
Fixes T7770
Test Plan: manual - connect a new account and see that most recent message downloads first
Reviewers: mark, evan, juan
Reviewed By: juan
Maniphest Tasks: T7770
Differential Revision: https://phab.nylas.com/D3838
Summary:
We recently added a `shouldComponentUpdate` method to ListTabular to
improve scrolling performance. However, this was preventing ListTabular
from updating the selected state of its items when we selected new
threads via cmd or shift keys.
This occurred because instead of passing data as props for each of ListTabular's
items, we are passing a function that calculates the props for each
item, so when we diffed old props and new props in shouldComponentUpdate,
given that the function reference never changes, we thought that the props
hadn't changed and prevented a re render, when in fact the props for each item
generated by that function might actually change.
Unfortunately, we can't use shouldComponentUpdate in this component
given how it is structured
Test Plan: manual
Reviewers: evan, spang, halla
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3837
Summary:
This is a WIP
Depends on D3799 on billing.nylas.com
This adds a `FeatureUsageStore` which determines whether a feature can be
used or not. It also allows us to record "using" a feature.
Feature Usage is ultimately backed by the Nylas Identity and cached
locally in the Identity object. Since feature usage is attached to the
Nylas Identity, we move the whole Identity object (except for the ID) to
the database.
This includes a migration (with tests!) to move the Nylas Identity from
the config into the Database. We still, however, need the Nylas ID to stay
in the config so it can be synchronously accessed by the /browser process
on bootup when determining what windows to show. It's also convenient to
know what the Nylas ID is by looking at the config. There's logic (with
tests!) to make sure these stay in sync. If you delete the Nylas ID from
the config, it'll be the same as logging you out.
The schema for the feature usage can be found in more detail on D3799. By
the time it reaches Nylas Mail, the Nylas ID object has a `feature_usage`
attribute that has each feature (keyed by the feature name) and
information about the plans attached to it. The schema Nylas Mail sees
looks like:
```
"feature_usage": {
"snooze": {
quota: 10,
peroid: 'monthly',
used_in_period: 8,
feature_limit_name: 'Snooze Group A',
},
}
```
See D3799 for more info about how these are generated.
One final change that's in here is how Stores are loaded. Most of our
core stores are loaded at require time, but now things like the
IdentityStore need to do asynchronous things on activation. In reality
most of our stores do this and it's a miracle it hasn't caused more
problems! Now when stores activate we optionally look for an `activate`
method and `await` for it. This was necessary so downstream classes (like
the Onboarding Store), see a fully initialized IdentityStore by the time
it's time to use them
Test Plan: New tests!
Reviewers: khamidou, juan, halla
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3808
Summary:
`isSending` was an artifact from the cloud sync engine that was used to
double check that the sending process for a multi-send draft had been
initiated. I don't believe the intermediate steps are API calls anymore,
and we've had the relevant code commented out for awhile. Time to kill it!
I've revived the double-sending tests in N1 to ensure we still have sufficient
checks against sending the draft again while the first call is still sending.
See D3834
Test Plan: N1 unit tests
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3835
Summary:
We need to wait for the `DraftEditingSession` to set it's draft before we can
pass it to the composer. I've modified these tests to properly wait for the
draft promise. Additionally, I removed the `DraftStore.isSendingDraft()` stub
so we run the actual logic for determing if a draft can be sent. We had some
tests with this stub, but then we had other tests that made sure it worked
without the stub, and I don't see why we don't just run the tests without the
stub to begin with. I added a stub to `@session.ensureCorrectAccount()`
to get around issues with committing the draft's change set.
Test Plan: Ran the suite!
Reviewers: evan, juan
Reviewed By: evan, juan
Differential Revision: https://phab.nylas.com/D3834
Summary:
This code is dead and has confused me grepping around the codebase before.
If for some unexpected reason we need these routes back in the future, we
can always extract them from version control. For now the routes we aren't
using are a distraction.
Test Plan: been using Nylas Mail with this local patch all week
Reviewers: evan, halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3833
Summary:
Previously, when moving to trash or spam, we were not correctly removing
the inbox category. This produced slightly different results because the
generated task was slightly different.
Also, in ChangeMailTask we were removing the locks on the models before
the tasks had actually succeeded-- these locks allowed us to ignore
deltas on those models while the task was in progress, so removing them
before it completed caused threads to flicker when performing actions
This depends on D3829 which also properly addresses
archiving/deleting/moving to spam threads that contain sent messages
Addresses T7757 and #2706
Test Plan: unit
Reviewers: evan, halla, spang
Reviewed By: evan, halla, spang
Differential Revision: https://phab.nylas.com/D3830
Summary:
We can't try to set or remove the sent label on gmail accounts because
the operation will silently fail and cause the threads to later bounce
back.
This occurred when trying to delete or archive a thread that contained a
sent message, and we incorrectly tried to overwrite or remove all of the
labels on messages, without regard for sent.
This was causing https://github.com/nylas/nylas-mail/issues/2706 and
sending and archiving to immediately bounce back.
Addresses T7757
Test Plan: unit tests
Reviewers: halla, evan, spang
Reviewed By: halla, evan, spang
Differential Revision: https://phab.nylas.com/D3829
Summary:
Previously, we attempted to immeditely retry syncback tasks when a
RetryableError was encountered. However, if the error was an
IMAPConnection error, we would keep retrying with a broken connection,
which would keep failing.
The correct way to retry is to wait for the next sync loop, since at the
beginning of each loop we ensure that we are correctly connected to
imap.
To achieve this this commit simply marks a failed task as NEW if it
encoutners a RetryableError and we haven't retried too many times. To
keep track of the number of retries, we save a new field in the `props`
field of the request.
Test Plan: manual
Reviewers: evan, halla, mark, spang
Reviewed By: spang
Subscribers: khamidou
Differential Revision: https://phab.nylas.com/D3831
Summary: This is a pretty small diff – it changes the snooze-store to save metadata for the individual messages affected instead of for the whole thread. We need this to have snoozing work without running an actual sync of the whole mailbox.
Test Plan: WIP.
Reviewers: evan, halla
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3815
Summary:
I've decided to break my snooze patches in multiple parts to make it easier to review. This diff contains all the code related to running workers in the cloud. Cloud workers all inherit from the `Worker` class which defines a bunch of useful things like error handling.
What's left to do:
- spawn workers based on the plugin type
- add monitoring (I'm going to add a simple HTTP endpoint for that)
- writing a migration for the local sync db and the prod metadata db.
Test Plan: Tested manually.
Reviewers: halla, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3814
Summary:
We're having some strange issues regarding corruption of the config
settings at runtime. This diff refactors the code to be more
straightforward and to be louder when things don't go as expected.
Test Plan: Run locally
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3828
Summary: Clean up k2's test database after each spec. Depends on D3826
Test Plan: unit
Reviewers: evan, halla, spang
Reviewed By: halla, spang
Differential Revision: https://phab.nylas.com/D3827
Summary: Make them map the same directory structure of `src`
Test Plan: unit
Reviewers: evan, halla, spang
Reviewed By: halla, spang
Differential Revision: https://phab.nylas.com/D3826
Summary: Just a quick fix to remove the warning in the console
Test Plan: manual
Reviewers: mark
Reviewed By: mark
Differential Revision: https://phab.nylas.com/D3823
Summary: Move config to the database
Test Plan:
Launch app before commit.
Launch on commit.
Confirm all settings and accounts stay the same.
Console.logged progress
Reviewers: mark, halla, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3820
Summary:
Sister diff to D3817
syncState on folders may contain arbitrarily long arrays of UIDs
(particularly, failedUIDs). If we serialize this JSON column to
edgehill.db, we can end up serializing very large objects when
persisting the local task queue. When the queue contains many tasks,
this can balloon the JSON blob to megabytes, causing the main window and
the worker window to become unresponsive.
The UI doesn't need to know about IMAP bookkeeping internals, so
serialize the sync progress instead of the sync state. This has the
advantages that (1) we don't need to worry about future keys added
to the syncState being large and (2) when we add Exchange support
we already have an abstraction for sync progress.
Test Plan: manual
Reviewers: juan, mark, halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3819
Summary:
syncState on folders may contain arbitrarily long arrays of UIDs
(particularly, failedUIDs). If we serialize this JSON column to
edgehill.db, we can end up serializing very large objects when
persisting the local task queue. When the queue contains many tasks,
this can balloon the JSON blob to megabytes, causing the main window and
the worker window to become unresponsive.
The UI doesn't need to know about IMAP bookkeeping internals, so
serialize the sync progress instead of the sync state. This has the
advantages that (1) we don't need to worry about future keys added
to the syncState being large and (2) when we add Exchange support
we already have an abstraction for sync progress.
Test Plan: manual
Reviewers: juan, mark, halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3817
