Commit graph

6035 commits

Author SHA1 Message Date
Juan Tejada fc7856b4d0 [client-sync] Fix runtime error when getting sync activity for account
Summary:
Make sure SyncActivity.getLastSyncActivityForAccount returns a sensible
default to prevent runtime errors when the SyncProcessManager uses it

Test Plan: manual

Reviewers: evan, halla

Reviewed By: halla

Differential Revision: https://phab.nylas.com/D4366
2017-04-05 14:10:01 -07:00
Nylas Coffee Machine 3653a66be2 bump(version): 2.0.1 2017-04-05 11:21:56 -07:00
Juan Tejada f708e3af0b [iso-core] IMAPConnectionPool now correctly disposes connections onDone
Summary:
Previously, calling the IMAPConnectionPool `onDone` callback when checking out a connection would just return the connection to the pool without re-establishing it the next time we checked it out of the pool.

When the `onDone` callback was called in an error scenario, this had the unintended consequence of returning a bad connection to the pool, and then re-using this bad connection. The only scenario when the connection was re-established was if `onConnected` threw an error, but if the connection was kept open outside the scope of onConnected, this logic would never run.

To make things simpler and more consistent, `onDone` will always destroy all connections and connections will always be re-established from scratch every time they are checked out of the pool. The pool acts more as a limit to the number of connections per account, than an actual shared pool of connections.

Test Plan: manual

Reviewers: evan, spang, halla, mark

Reviewed By: halla, mark

Differential Revision: https://phab.nylas.com/D4360
2017-04-05 11:10:41 -07:00
Juan Tejada 5269c11ee0 [client-sync] Prevent IMAP connection leaking in sync worker
Summary:
When the sync worker got stuck inside `performSync`, we would never
release the connection back to the connection pool. We would then try to
start a new sync worker, and it would claim a new connection. If this
happened enough times, we would eventually exhaust the whole pool and
keep the sync worker blocked forever.

Test Plan: manual

Reviewers: evan, spang, halla, mark

Reviewed By: halla, mark

Differential Revision: https://phab.nylas.com/D4359
2017-04-05 11:08:20 -07:00
Evan Morikawa 78a15700c8 [cloud-*] remove old SignalFX reporter & add docs
Summary:
This removes the old SignalFX reporter that we weren't using
Adding documentation around logging

Test Plan: manual

Reviewers: juan, halla, spang

Reviewed By: juan, halla, spang

Differential Revision: https://phab.nylas.com/D4341
2017-04-05 10:51:38 -07:00
Evan Morikawa 08a697e4c9 [cloud-*] log errors according to bunyan specs
Summary:
According to bunyan, https://github.com/trentm/node-bunyan, errors are
treated special. They either need to be the first argument of a log, or
they need to be attached to the special `err` (NOT `error`) key of the
logger object.

Test Plan: manual

Reviewers: halla, juan, spang

Reviewed By: halla, juan, spang

Differential Revision: https://phab.nylas.com/D4337
2017-04-05 10:47:56 -07:00
Juan Tejada a3072f6b04 [client-sync] 🎨 logging 2017-04-05 09:22:00 -07:00
Juan Tejada d75a5ee243 [client-app] Add better logging when restarting stuck sync worker
Summary: see title

Test Plan: manual

Reviewers: halla, mark, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4358
2017-04-04 21:37:55 -07:00
Juan Tejada f8d84a76e2 [iso-core] Remove unused vars from IMAPConnectionPool
Summary: see title

Test Plan: manual

Reviewers: mark, halla, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4357
2017-04-04 21:34:20 -07:00
Juan Tejada 2949ecffd5 [client-sync] 🎨 sync worker
Summary: Just renaming some stuff

Test Plan: manual

Reviewers: mark, spang, evan, halla

Reviewed By: spang, evan, halla

Differential Revision: https://phab.nylas.com/D4342
2017-04-04 17:53:13 -07:00
Halla Moore 5c8915f144 [client-app] Fix the unapply transformation for overlaid components
Summary:
The unapply transformation was incorrectly passing the props parameter
to buildAnchorTag() as a string, even though it expects an object.
buildAnchorTag() goes on to stringify this parameter, which causes extra
surrounding quotations and escape slashes in front of all the other
quotations. The unapply transform is applied on several re-renders,
which causes the number of escape characters to unboundedly increase.
Firstly, this was causing inline images to not appear properly in the
draft, and secondly, the large number of escape characters was making
the draft body large enough to make the app unresponsive.

This generally wasn't an issue because the unapply transformation is
only applied when there has been an apply transformation, and this is
triggered by the SyncbackDraftTask. This task was previously only queued
upon send, in which case the user never re-opens the draft. Now,
enabling send-later on a draft will queue the task, and these issues
appear if the user re-opens the draft for editing.

This diff makes it so that the unapply transform passes in the props as
an object, like the buildAnchorTag function expects.

Test Plan: manual

Reviewers: evan, juan

Reviewed By: evan, juan

Differential Revision: https://phab.nylas.com/D4348
2017-04-04 16:43:05 -07:00
Halla Moore 2b39af5aba [client-app] Allow the packageMigrationManager to re-enable packages
Summary:
Remove packages from the disabled package list when they have been
migrated to be enabled by default.

Also, the last migration would not have worked properly anyways because
the daily channel was already on 1.0.56, and we check for greater-than
rather than equal-to. Bump that version to match the next update.

Test Plan: manual

Reviewers: juan, evan

Reviewed By: juan, evan

Differential Revision: https://phab.nylas.com/D4335
2017-04-04 16:36:39 -07:00
Juan Tejada 7d99567855 [client-app] Make sure to fetch latest update url every time we check for updates
Summary:
Previously, is you signed out of your NylasID, you would not pick up new
updates because we never updated the autoupdater url to use the new id.

Test Plan: manual

Reviewers: mark, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4346
2017-04-04 16:30:52 -07:00
Juan Tejada d1b81b6afe [client-app] Prevent from making any requests when NylasID isn't present
Summary:
Sometimes, when logging out of your NylasID and restarting the app, we
would continue making some requests from the worker window that required a
NylasID. This would make the app enter a restart loop and become
completely unresponsive because when we made a request without a
NylasID, we would force the user to log out and restart the app, and
then we would again make the requests without the id, ad infinitum.

To fix this, we make sure we have a NylasID before making any requests
that require it

Test Plan: manual

Reviewers: halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4344
2017-04-04 16:26:32 -07:00
Christine Spang 2095f0907e [scripts] Batch upload each row
Makes the script like 4x faster.
2017-04-04 15:24:01 -07:00
Mark Hahnenberg 00e1df8491 [client-app] Add script to run benchmarks once per day at specified time
Summary: See title

Test Plan: Run locally

Reviewers: juan, evan, spang

Reviewed By: spang

Differential Revision: https://phab.nylas.com/D4345
2017-04-04 15:07:08 -07:00
Christine Spang bd933f1bc0 [scripts] Add script to upload benchmark data to Google Sheets
Test Plan: ran locally (need client secret for this)

Reviewers: mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4343
2017-04-04 15:04:03 -07:00
Mark Hahnenberg 4eef086e3e [client-app] Fix benchmark script
Summary:
We weren't changing into the directory of the git repository, so running
the script from anywhere but inside the git repo didn't work. We also
weren't pulling before checking for the latest commit. Now we do both of
these things.

Test Plan: Run locally

Reviewers: spang, juan, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4339
2017-04-04 13:28:46 -07:00
Juan Tejada a461490e41 [client-sync] Ensure we use refreshed access token for mail listener conn
Summary:
Before this commit, we would just establish a single mail listener imap
connection and never check if we needed to re-connect it due to an
expired access token. Even though we correctly refreshed the access
token at the beggining of each sync loop (hidden under
`ensureSMTPConnection`), we would never re-establish the mail listener
connection with the new access token.

This would cause the app to enter an `Invalid Credentials` error loop in
the sync loop, preventing from syncing any mail at all (T8064)

Test Plan: manual

Reviewers: spang, halla, mark, evan

Reviewed By: mark, evan

Differential Revision: https://phab.nylas.com/D4338
2017-04-04 13:27:43 -07:00
Juan Tejada 7dec807b28 [client-sync] Fix mail listener imap connection
Summary:
Given that we keep the imap mail listener conenction open outside of the
imap connection pool, it might close itself after a while or after
putting your computer to sleep.

Previously, if we had the connection object we just assumed it was
connected and proceeded with sync, and if it wasn't, we would fall into
an error loop in the sync worker preventing it from syncing at all (T8065)

To fix this, we just call `connect` every time to ensure the connection is open.
(This was the case before we introduced the connection pool)

Test Plan: manual

Reviewers: mark, spang, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4334
2017-04-04 13:27:04 -07:00
Mark Hahnenberg f629818075 [client-app] Remove sudo from benchmark-initial-sync.sh
Summary:
We created the process so we can also kill it with no problems. This
makes it easier to automatically run benchmarks without human
intervention.

Test Plan: Run locally

Reviewers: spang, evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4332
2017-04-04 12:49:00 -07:00
Mark Hahnenberg e06f729275 [client-app] Limit search to focused perspective
Summary:
Previously we would always search all mail. Now, if the user has focused
a particular folder we will limit our search to that folder. The inbox
is an exception--it will always search all mail unless the user
explicitly uses an "in:" clause.

Test Plan: Run locally, verify that searching folders returns the correct results.

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4328
2017-04-04 12:48:04 -07:00
Evan Morikawa f6bc2c4e5f bump(version): 2.0.0 2017-04-04 09:05:53 -07:00
Nylas Coffee Machine dfde1b9672 bump(version): 1.0.56 2017-04-04 09:03:34 -07:00
Mark Hahnenberg 7818f42d42 [client-app] Fix sqlite build for older versions of clang
Summary: See title

Test Plan: Run locally on older version of clang

Reviewers: juan, spang, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4329
2017-04-03 17:12:02 -07:00
Halla Moore 7079b21b05 [client-sync] Don't assign duplicate roles in folder fetch
Summary:
Check that there aren't any other categories with a role before we try
to assign it to the current category.

Addresses T7835

Test Plan: manual

Reviewers: juan, evan, mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4298
2017-04-03 17:04:25 -07:00
Evan Morikawa 2f082f8beb [client-app] re-setup IdentityStore in new window
Summary:
Before this patch, the IdentityStore would initialize in our empty hot
window. However, hot windows don't receive any `action-bridge-message`s,
which include DB updates. Since the hot window loads first, it was with a
stale verison of the Identity. The main window fetches a fresh identity,
but that fresh update failed to get to new composers because the hot
window wasn't listening to changes to the DB.

This makes it such that the IdentityStore properly boots up when the
window props change.

Test Plan: manual

Reviewers: halla, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4327
2017-04-03 16:44:01 -07:00
Evan Morikawa 368201f334 [client-app] add feature limits to reminders and send later
Summary:
This adds feature limit modals (graphics pending) for reminders and send
later

Test Plan: manual

Reviewers: juan, halla

Reviewed By: halla

Differential Revision: https://phab.nylas.com/D4322
2017-04-03 15:38:47 -07:00
Evan Morikawa 11684c6f9e [client-app] enable snooze, send later, reminders on 1.0.56
Summary: This enables packages on 1.0.56

Test Plan: manual

Reviewers: juan, halla

Reviewed By: juan, halla

Differential Revision: https://phab.nylas.com/D4320
2017-04-03 15:38:31 -07:00
Juan Tejada b01976ff10 [client-app] Remove rogue scripts-tmp folder 2017-04-03 14:07:29 -07:00
Juan Tejada 020fd79db1 [client-app] Update changelog 2017-03-31 17:41:22 -07:00
Juan Tejada be237d076e [client-app] Remove unecessary db setup for mail rules
Summary: This shaves off ~150ms trying to issue this query

Test Plan: manual

Reviewers: evan, spang, mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4317
2017-03-31 17:38:22 -07:00
Nylas Coffee Machine 7e0194e6c3 bump(version): 1.0.55 2017-03-31 16:14:55 -07:00
Evan Morikawa 96685bed01 [client-app] don't override existing bodies on K2
Summary:
Bodies that already exist on K2 shouldn't be overridden.
See the comment for how this caused open/link tracking to fail in certain
cases when a lot of messages were being sent

Test Plan: manual

Reviewers: halla, mark, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4319
2017-03-31 16:13:52 -07:00
Halla Moore 1864a2d48a [client-sync] Add comment about reason for try/catch block 2017-03-31 15:34:52 -07:00
Mark Hahnenberg 70bb602246 [client-sync] Split Contact ranking query
Summary:
We were using a join for the contact ranking query and for whatever
reason on large databases this was extremely slow in SQLite. This diff
splits the query into first finding the sent Folder/Label and then
searching for non-draft Messages in that category.

Test Plan: Run locally, verify the query is faster

Reviewers: juan, evan, spang

Reviewed By: spang

Differential Revision: https://phab.nylas.com/D4315
2017-03-31 15:29:20 -07:00
Juan Tejada c4763d4bd5 [client-app] Update changelog 2017-03-31 15:01:10 -07:00
Mark Hahnenberg f6a0ead259 [client-app] Remove getUnindexedAccounts
Summary: It was really slow, and we no longer need it.

Test Plan: Run locally, verify that we no longer do that super slow query

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4312
2017-03-31 14:49:31 -07:00
Evan Morikawa e6b399ea27 [client-app] remove the ANALYZE in favor of pragma.optimize
Summary:
Removes some VERY long running `ANLAYZE` queries. Was taking up to 50
seconds on my 9GB database on every boot
https://sqlite.org/pragma.html#pragma_optimize

Test Plan:
I tested to make sure the app still quits quickly. It does. The SQLite
docs also say this should be fast.

Reviewers: halla, spang, mark, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4314
2017-03-31 14:45:14 -07:00
Nylas Coffee Machine 328ea428be bump(version): 1.0.54 2017-03-31 12:22:21 -07:00
Mark Hahnenberg 0205ac3f61 [client-sync] Simplify Contact ranking query
Summary:
We were using a couple of left outer joins to avoid checking whether we
should be joining on Labels or Folders. We can greatly simplify the query
by just checking which we should use and issuing the correct inner join.

Test Plan: Run locally, verify contact ranking still works.

Reviewers: spang, juan, evan

Reviewed By: juan, evan

Differential Revision: https://phab.nylas.com/D4310
2017-03-31 12:19:58 -07:00
Evan Morikawa 4deef8b90c [client-app] delay building new hot window to improve win perf
Summary:
This improves window open times by about 70ms
We would spend a very long time in the backend browser process building a
new hot window when we didn't need to do that until much later

Test Plan: manual

Reviewers: halla, mark, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4311
2017-03-31 12:17:34 -07:00
Juan Tejada 590b17ae6f [client-app] Don't imeout removing sync worker when removing an account
Summary:
Most of the times when removing a sync worker (i.e. when removing an
account), we would see database errors in the console. This happened
because more often than not we would interrupt in the middle of message
processing, but `processMessage` is not interruptible, which means that
the worker would be interrupted right after processing its current
message. However, if `processMessage` would take more than 500ms (the
current timeout for stopping the worker), we would destroy the database
before processing was done and it would throw a bunch of errors.

To fix this, we just don't set a timeout when removing the worker as a consequence
of removing an account. However, when we are removing the worker when we
detect that it is stuck, we set a time out of 5 seconds.

Test Plan: manually test removing accounts, verify that it doesn't error

Reviewers: mark, halla, evan

Reviewed By: halla, evan

Differential Revision: https://phab.nylas.com/D4308
2017-03-31 12:09:20 -07:00
Juan Tejada 9181914453 [client-app] Ensure sync worker can never continue running after being destroyed
Summary:
Previously, it was possible for the sync worker to continue running after being
interrupted, e.g. it would break out of `performSync` and then try to run
`onSyncCompleted`. This is fine if we were just interrupting to restart the loop,
but when we stop it we don't want it to continue running anything at all.

This also refactors the syncworker to have a single `destroy` method, which sets
a `destroyed` flag and uses that one exclusively instead of the `stopped` flag.

Test Plan: manually check it works

Reviewers: spang, halla, mark, evan

Reviewed By: mark, evan

Differential Revision: https://phab.nylas.com/D4307
2017-03-31 12:08:48 -07:00
Juan Tejada 58cf02a824 [client-app] Don't hit contact rankings endpoint until account is ready
Summary:
This prevents us from hitting the endpoint before the database for that
account is initialized and it's actually syncing.

Before this commit, we would throw errors upon adding an account

Test Plan: manually add, remove accounts, run benchmark script

Reviewers: mark, halla, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D4305
2017-03-31 11:21:56 -07:00
Halla Moore af6e6e669a [client-sync] Wrap more syncWorker args
Turns out the SyncWorker isn't the only one who initiates SyncTasks
2017-03-31 11:12:50 -07:00
Halla Moore ae2e2122e9 [client-sync] Properly wrap the SyncWorker arg in an object
The argument structure for runTask() changed and needs it this way.
2017-03-31 10:58:16 -07:00
Halla Moore c6a339df1a [client-sync] Fix the new syncback-task structure
Summary:
- Change files to .es6 so they recognize the syntax
- Pass arguments down to _run()
- Make sure the responses get returned

Test Plan: manual, even though that somehow failed before. :(

Reviewers: evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4309
2017-03-31 10:52:42 -07:00
Nylas Coffee Machine 13ac2fa3b4 bump(version): 1.0.53 2017-03-31 09:07:20 -07:00
Evan Morikawa ae278fa5f2 [client-app] add DatabaseStore.write which doesn't use Transactions
Summary:
Creates a new `DatabaseStore.write` method alongside
`DatabaseStore.inTransaction` that doesn't always wrap trivial queries in
a transaction. SQLite does this automatically.

Test Plan: manual

Reviewers: spang, mark, halla, juan

Reviewed By: halla, juan

Differential Revision: https://phab.nylas.com/D4303
2017-03-30 17:38:32 -07:00