Commit graph

5778 commits

Author SHA1 Message Date
Juan Tejada
d75a5ee243 [client-app] Add better logging when restarting stuck sync worker
Summary: see title

Test Plan: manual

Reviewers: halla, mark, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4358
2017-04-04 21:37:55 -07:00
Juan Tejada
f8d84a76e2 [iso-core] Remove unused vars from IMAPConnectionPool
Summary: see title

Test Plan: manual

Reviewers: mark, halla, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4357
2017-04-04 21:34:20 -07:00
Juan Tejada
2949ecffd5 [client-sync] 🎨 sync worker
Summary: Just renaming some stuff

Test Plan: manual

Reviewers: mark, spang, evan, halla

Reviewed By: spang, evan, halla

Differential Revision: https://phab.nylas.com/D4342
2017-04-04 17:53:13 -07:00
Halla Moore
5c8915f144 [client-app] Fix the unapply transformation for overlaid components
Summary:
The unapply transformation was incorrectly passing the props parameter
to buildAnchorTag() as a string, even though it expects an object.
buildAnchorTag() goes on to stringify this parameter, which causes extra
surrounding quotations and escape slashes in front of all the other
quotations. The unapply transform is applied on several re-renders,
which causes the number of escape characters to unboundedly increase.
Firstly, this was causing inline images to not appear properly in the
draft, and secondly, the large number of escape characters was making
the draft body large enough to make the app unresponsive.

This generally wasn't an issue because the unapply transformation is
only applied when there has been an apply transformation, and this is
triggered by the SyncbackDraftTask. This task was previously only queued
upon send, in which case the user never re-opens the draft. Now,
enabling send-later on a draft will queue the task, and these issues
appear if the user re-opens the draft for editing.

This diff makes it so that the unapply transform passes in the props as
an object, like the buildAnchorTag function expects.

Test Plan: manual

Reviewers: evan, juan

Reviewed By: evan, juan

Differential Revision: https://phab.nylas.com/D4348
2017-04-04 16:43:05 -07:00
Halla Moore
2b39af5aba [client-app] Allow the packageMigrationManager to re-enable packages
Summary:
Remove packages from the disabled package list when they have been
migrated to be enabled by default.

Also, the last migration would not have worked properly anyways because
the daily channel was already on 1.0.56, and we check for greater-than
rather than equal-to. Bump that version to match the next update.

Test Plan: manual

Reviewers: juan, evan

Reviewed By: juan, evan

Differential Revision: https://phab.nylas.com/D4335
2017-04-04 16:36:39 -07:00
Juan Tejada
7d99567855 [client-app] Make sure to fetch latest update url every time we check for updates
Summary:
Previously, is you signed out of your NylasID, you would not pick up new
updates because we never updated the autoupdater url to use the new id.

Test Plan: manual

Reviewers: mark, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4346
2017-04-04 16:30:52 -07:00
Juan Tejada
d1b81b6afe [client-app] Prevent from making any requests when NylasID isn't present
Summary:
Sometimes, when logging out of your NylasID and restarting the app, we
would continue making some requests from the worker window that required a
NylasID. This would make the app enter a restart loop and become
completely unresponsive because when we made a request without a
NylasID, we would force the user to log out and restart the app, and
then we would again make the requests without the id, ad infinitum.

To fix this, we make sure we have a NylasID before making any requests
that require it

Test Plan: manual

Reviewers: halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4344
2017-04-04 16:26:32 -07:00
Christine Spang
2095f0907e [scripts] Batch upload each row
Makes the script like 4x faster.
2017-04-04 15:24:01 -07:00
Mark Hahnenberg
00e1df8491 [client-app] Add script to run benchmarks once per day at specified time
Summary: See title

Test Plan: Run locally

Reviewers: juan, evan, spang

Reviewed By: spang

Differential Revision: https://phab.nylas.com/D4345
2017-04-04 15:07:08 -07:00
Christine Spang
bd933f1bc0 [scripts] Add script to upload benchmark data to Google Sheets
Test Plan: ran locally (need client secret for this)

Reviewers: mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4343
2017-04-04 15:04:03 -07:00
Mark Hahnenberg
4eef086e3e [client-app] Fix benchmark script
Summary:
We weren't changing into the directory of the git repository, so running
the script from anywhere but inside the git repo didn't work. We also
weren't pulling before checking for the latest commit. Now we do both of
these things.

Test Plan: Run locally

Reviewers: spang, juan, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4339
2017-04-04 13:28:46 -07:00
Juan Tejada
a461490e41 [client-sync] Ensure we use refreshed access token for mail listener conn
Summary:
Before this commit, we would just establish a single mail listener imap
connection and never check if we needed to re-connect it due to an
expired access token. Even though we correctly refreshed the access
token at the beggining of each sync loop (hidden under
`ensureSMTPConnection`), we would never re-establish the mail listener
connection with the new access token.

This would cause the app to enter an `Invalid Credentials` error loop in
the sync loop, preventing from syncing any mail at all (T8064)

Test Plan: manual

Reviewers: spang, halla, mark, evan

Reviewed By: mark, evan

Differential Revision: https://phab.nylas.com/D4338
2017-04-04 13:27:43 -07:00
Juan Tejada
7dec807b28 [client-sync] Fix mail listener imap connection
Summary:
Given that we keep the imap mail listener conenction open outside of the
imap connection pool, it might close itself after a while or after
putting your computer to sleep.

Previously, if we had the connection object we just assumed it was
connected and proceeded with sync, and if it wasn't, we would fall into
an error loop in the sync worker preventing it from syncing at all (T8065)

To fix this, we just call `connect` every time to ensure the connection is open.
(This was the case before we introduced the connection pool)

Test Plan: manual

Reviewers: mark, spang, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4334
2017-04-04 13:27:04 -07:00
Mark Hahnenberg
f629818075 [client-app] Remove sudo from benchmark-initial-sync.sh
Summary:
We created the process so we can also kill it with no problems. This
makes it easier to automatically run benchmarks without human
intervention.

Test Plan: Run locally

Reviewers: spang, evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4332
2017-04-04 12:49:00 -07:00
Mark Hahnenberg
e06f729275 [client-app] Limit search to focused perspective
Summary:
Previously we would always search all mail. Now, if the user has focused
a particular folder we will limit our search to that folder. The inbox
is an exception--it will always search all mail unless the user
explicitly uses an "in:" clause.

Test Plan: Run locally, verify that searching folders returns the correct results.

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4328
2017-04-04 12:48:04 -07:00
Evan Morikawa
f6bc2c4e5f bump(version): 2.0.0 2017-04-04 09:05:53 -07:00
Nylas Coffee Machine
dfde1b9672 bump(version): 1.0.56 2017-04-04 09:03:34 -07:00
Mark Hahnenberg
7818f42d42 [client-app] Fix sqlite build for older versions of clang
Summary: See title

Test Plan: Run locally on older version of clang

Reviewers: juan, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4329
2017-04-03 17:12:02 -07:00
Halla Moore
7079b21b05 [client-sync] Don't assign duplicate roles in folder fetch
Summary:
Check that there aren't any other categories with a role before we try
to assign it to the current category.

Addresses T7835

Test Plan: manual

Reviewers: juan, evan, mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4298
2017-04-03 17:04:25 -07:00
Evan Morikawa
2f082f8beb [client-app] re-setup IdentityStore in new window
Summary:
Before this patch, the IdentityStore would initialize in our empty hot
window. However, hot windows don't receive any `action-bridge-message`s,
which include DB updates. Since the hot window loads first, it was with a
stale verison of the Identity. The main window fetches a fresh identity,
but that fresh update failed to get to new composers because the hot
window wasn't listening to changes to the DB.

This makes it such that the IdentityStore properly boots up when the
window props change.

Test Plan: manual

Reviewers: halla, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4327
2017-04-03 16:44:01 -07:00
Evan Morikawa
368201f334 [client-app] add feature limits to reminders and send later
Summary:
This adds feature limit modals (graphics pending) for reminders and send
later

Test Plan: manual

Reviewers: juan, halla

Reviewed By: halla

Differential Revision: https://phab.nylas.com/D4322
2017-04-03 15:38:47 -07:00
Evan Morikawa
11684c6f9e [client-app] enable snooze, send later, reminders on 1.0.56
Summary: This enables packages on 1.0.56

Test Plan: manual

Reviewers: juan, halla

Reviewed By: juan, halla

Differential Revision: https://phab.nylas.com/D4320
2017-04-03 15:38:31 -07:00
Juan Tejada
b01976ff10 [client-app] Remove rogue scripts-tmp folder 2017-04-03 14:07:29 -07:00
Juan Tejada
020fd79db1 [client-app] Update changelog 2017-03-31 17:41:22 -07:00
Juan Tejada
be237d076e [client-app] Remove unecessary db setup for mail rules
Summary: This shaves off ~150ms trying to issue this query

Test Plan: manual

Reviewers: evan, spang, mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4317
2017-03-31 17:38:22 -07:00
Nylas Coffee Machine
7e0194e6c3 bump(version): 1.0.55 2017-03-31 16:14:55 -07:00
Evan Morikawa
96685bed01 [client-app] don't override existing bodies on K2
Summary:
Bodies that already exist on K2 shouldn't be overridden.
See the comment for how this caused open/link tracking to fail in certain
cases when a lot of messages were being sent

Test Plan: manual

Reviewers: halla, mark, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4319
2017-03-31 16:13:52 -07:00
Halla Moore
1864a2d48a [client-sync] Add comment about reason for try/catch block 2017-03-31 15:34:52 -07:00
Mark Hahnenberg
70bb602246 [client-sync] Split Contact ranking query
Summary:
We were using a join for the contact ranking query and for whatever
reason on large databases this was extremely slow in SQLite. This diff
splits the query into first finding the sent Folder/Label and then
searching for non-draft Messages in that category.

Test Plan: Run locally, verify the query is faster

Reviewers: juan, evan, spang

Reviewed By: spang

Differential Revision: https://phab.nylas.com/D4315
2017-03-31 15:29:20 -07:00
Juan Tejada
c4763d4bd5 [client-app] Update changelog 2017-03-31 15:01:10 -07:00
Mark Hahnenberg
f6a0ead259 [client-app] Remove getUnindexedAccounts
Summary: It was really slow, and we no longer need it.

Test Plan: Run locally, verify that we no longer do that super slow query

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4312
2017-03-31 14:49:31 -07:00
Evan Morikawa
e6b399ea27 [client-app] remove the ANALYZE in favor of pragma.optimize
Summary:
Removes some VERY long running `ANLAYZE` queries. Was taking up to 50
seconds on my 9GB database on every boot
https://sqlite.org/pragma.html#pragma_optimize

Test Plan:
I tested to make sure the app still quits quickly. It does. The SQLite
docs also say this should be fast.

Reviewers: halla, spang, mark, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4314
2017-03-31 14:45:14 -07:00
Nylas Coffee Machine
328ea428be bump(version): 1.0.54 2017-03-31 12:22:21 -07:00
Mark Hahnenberg
0205ac3f61 [client-sync] Simplify Contact ranking query
Summary:
We were using a couple of left outer joins to avoid checking whether we
should be joining on Labels or Folders. We can greatly simplify the query
by just checking which we should use and issuing the correct inner join.

Test Plan: Run locally, verify contact ranking still works.

Reviewers: spang, juan, evan

Reviewed By: juan, evan

Differential Revision: https://phab.nylas.com/D4310
2017-03-31 12:19:58 -07:00
Evan Morikawa
4deef8b90c [client-app] delay building new hot window to improve win perf
Summary:
This improves window open times by about 70ms
We would spend a very long time in the backend browser process building a
new hot window when we didn't need to do that until much later

Test Plan: manual

Reviewers: halla, mark, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4311
2017-03-31 12:17:34 -07:00
Juan Tejada
590b17ae6f [client-app] Don't imeout removing sync worker when removing an account
Summary:
Most of the times when removing a sync worker (i.e. when removing an
account), we would see database errors in the console. This happened
because more often than not we would interrupt in the middle of message
processing, but `processMessage` is not interruptible, which means that
the worker would be interrupted right after processing its current
message. However, if `processMessage` would take more than 500ms (the
current timeout for stopping the worker), we would destroy the database
before processing was done and it would throw a bunch of errors.

To fix this, we just don't set a timeout when removing the worker as a consequence
of removing an account. However, when we are removing the worker when we
detect that it is stuck, we set a time out of 5 seconds.

Test Plan: manually test removing accounts, verify that it doesn't error

Reviewers: mark, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4308
2017-03-31 12:09:20 -07:00
Juan Tejada
9181914453 [client-app] Ensure sync worker can never continue running after being destroyed
Summary:
Previously, it was possible for the sync worker to continue running after being
interrupted, e.g. it would break out of `performSync` and then try to run
`onSyncCompleted`. This is fine if we were just interrupting to restart the loop,
but when we stop it we don't want it to continue running anything at all.

This also refactors the syncworker to have a single `destroy` method, which sets
a `destroyed` flag and uses that one exclusively instead of the `stopped` flag.

Test Plan: manually check it works

Reviewers: spang, halla, mark, evan

Reviewed By: mark, evan

Differential Revision: https://phab.nylas.com/D4307
2017-03-31 12:08:48 -07:00
Juan Tejada
58cf02a824 [client-app] Don't hit contact rankings endpoint until account is ready
Summary:
This prevents us from hitting the endpoint before the database for that
account is initialized and it's actually syncing.

Before this commit, we would throw errors upon adding an account

Test Plan: manually add, remove accounts, run benchmark script

Reviewers: mark, halla, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4305
2017-03-31 11:21:56 -07:00
Halla Moore
af6e6e669a [client-sync] Wrap more syncWorker args
Turns out the SyncWorker isn't the only one who initiates SyncTasks
2017-03-31 11:12:50 -07:00
Halla Moore
ae2e2122e9 [client-sync] Properly wrap the SyncWorker arg in an object
The argument structure for runTask() changed and needs it this way.
2017-03-31 10:58:16 -07:00
Halla Moore
c6a339df1a [client-sync] Fix the new syncback-task structure
Summary:
- Change files to .es6 so they recognize the syntax
- Pass arguments down to _run()
- Make sure the responses get returned

Test Plan: manual, even though that somehow failed before. :(

Reviewers: evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4309
2017-03-31 10:52:42 -07:00
Nylas Coffee Machine
13ac2fa3b4 bump(version): 1.0.53 2017-03-31 09:07:20 -07:00
Evan Morikawa
ae278fa5f2 [client-app] add DatabaseStore.write which doesn't use Transactions
Summary:
Creates a new `DatabaseStore.write` method alongside
`DatabaseStore.inTransaction` that doesn't always wrap trivial queries in
a transaction. SQLite does this automatically.

Test Plan: manual

Reviewers: spang, mark, halla, juan

Reviewed By: halla, juan

Differential Revision: https://phab.nylas.com/D4303
2017-03-30 17:38:32 -07:00
Evan Morikawa
559ecb1cda [client-app] rename DatabaseTransaction -> DatabaseWriter
Summary: Renamed in prep for next few diffs

Test Plan: manual

Reviewers: juan, mark, spang, halla

Reviewed By: halla

Differential Revision: https://phab.nylas.com/D4302
2017-03-30 17:38:07 -07:00
Evan Morikawa
f9c0a93d1c [client-app] use debug library for DB
Summary:
Use the wonderful `debug` library instead of all our random flags.
You can now do things like: `DEBUG="app:*,sync:*" npm start` to print out
everything.

Test Plan: manual

Reviewers: mark, halla, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4297
2017-03-30 17:26:33 -07:00
Halla Moore
f5f5b60140 [client-sync] Add provider to syncbackTask perf metric 2017-03-30 16:24:32 -07:00
Halla Moore
a776e6dfb7 [client-sync] Report metrics about SyncbackTask runs
Summary:
Report how long they take and if they're stopped (due to timeout).

Part of T7978

Test Plan: manual

Reviewers: evan, juan

Reviewed By: evan, juan

Differential Revision: https://phab.nylas.com/D4272
2017-03-30 16:15:09 -07:00
Mark Hahnenberg
9270ff6c4c [client-app] Add script that benchmarks new commits
Summary:
We want to setup the initial sync benchmark to run automatically with each
commit on a separate machine. This script does just that and dumps the
results to a directory. We can then have a separate script process them
and upload them to whichever service we want.

Test Plan: Run locally

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4296
2017-03-30 16:11:49 -07:00
Mark Hahnenberg
5c08018382 [client-app] Add search support for "has:attachment"
Summary: See title

Test Plan: Run locally, verify that attachments are returned

Reviewers: spang, evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4293
2017-03-30 16:11:19 -07:00
Halla Moore
4c1af8f184 [client-sync] Convert syncback tasks to interruptible generators
Summary:
Make `run()` functions generators and change most awaits to yields

Part of T7978

Test Plan: specs from D4269, but really needs some heavy QA

Reviewers: evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4270
2017-03-30 15:51:25 -07:00