Commit graph

119 commits

Author SHA1 Message Date
Evan Morikawa
f23a96c481 [client-sync] limit attributes returned for messages
Summary:
I did a grep for `Message.find.*` to see if there were any other places
we could limit the attributes returned

There were.

This used to be INSANELY wasteful to return all bodies when we only needed
the ids.

Test Plan: Manually bootup app and ensure search still works

Reviewers: mark, juan, spang

Reviewed By: mark, juan, spang

Differential Revision: https://phab.nylas.com/D4042
2017-02-24 11:04:18 -08:00
Evan Morikawa
c7bdb5804a [client-sync] fix unbouded query
Summary:
Fixes T7783

This unbounded query, plus a yet-to-be-diagnosed bug causing large numbers
of messages with no folderIds, led to frequent crashes of the worker
window at the end of every sync loop.

This took a fair amount to diagnose. Notes on that are here:
https://paper.dropbox.com/doc/Diagnosing-Crash-X329VTxevrqtIMESBtHNV

Huge thanks to @mark.

Test Plan: Reboot app with patch, no more crashes!

Reviewers: spang, halla, mark, juan

Reviewed By: mark, juan

Subscribers: mark

Maniphest Tasks: T7783

Differential Revision: https://phab.nylas.com/D4039
2017-02-24 10:58:21 -08:00
Christine Spang
15eac9c9db [client-sync] Parse comma-separated References properly
This goes against the spec, but we've seen emails which use comma-separated
References instead of space-separated references. This could cause threading
to break if the message with comma-separated references was a reply and
we hadn't yet synced the message directly in the In-Reply-To header.
2017-02-23 18:58:13 -08:00
Christine Spang
49103367c1 [client-sync] Update comment about Gmail inbox prioritization 2017-02-23 15:36:35 -08:00
Christine Spang
4a1dde8830 [client-sync] Deduplicate UID downloads across sync loop interruptions
Summary:
Since we only persist updates to fetchedmin/fetchedmax at the end of a
batch, and a batch can contain many messages, if the sync loop is
getting interrupted often we will download the same messages over and
over again and not make much progress in downloading the message
backlog. This patch keeps a set of already downloaded messages in memory
for each batch and skips downloading UIDs we've processed in interrupted
sync loops.

Messages may still be redownloaded across app restarts.

Fixes T7798

Test Plan: manual

Reviewers: juan, mark

Reviewed By: juan, mark

Maniphest Tasks: T7798

Differential Revision: https://phab.nylas.com/D4040
2017-02-23 15:23:28 -08:00
Mark Hahnenberg
da6bc473f8 [client-app] Limit search to focused perspective
Summary:
Previously we would always search all mail. Now, if the user has focused
a particular folder we will limit our search to that folder. The inbox
is an exception--it will always search all mail unless the user
explicitly uses an "in:" clause.

Test Plan: Run locally, verify that searching folders returns the correct results.

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4032
2017-02-23 13:36:45 -08:00
Juan Tejada
dced0aa99e [client-sync] 🎨 Remove unecessary use of PromiseUtils
Given that this packages lives inside the NylasMail app, it has access
to Bluebird Promises, which already define a .each method.

Using PromiseUtils here is unecessary
2017-02-22 18:30:34 -08:00
Mark Hahnenberg
c77c8becb9 [client-sync] Use IMAPConnectionPool for IMAP search client
Summary: See title

Test Plan: Run locally, verify IMAP search still works

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4006
2017-02-22 10:57:42 -08:00
Mark Hahnenberg
a912dc971c [client-sync] Use IMAPConnectionPool for downloading raw messages
Summary: See title

Test Plan: Run locally, verify we can download raw messages

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4005
2017-02-22 10:55:08 -08:00
Mark Hahnenberg
c4ccd8aac4 [client-sync] Use IMAPConnectionPool for downloading files
Summary: See title

Test Plan: Run locally, verify downloading files works

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4004
2017-02-22 10:48:26 -08:00
Christine Spang
84a3b20839 [client-sync] Refresh SMTP client when auth credentials change
Summary:
Previously, we would create a nodemailer SMTP transport object when the
sync worker booted up. The transport object would be passed the account
SMTP credentials at the time of object creation. If the Google auth
token later expired, we would continue to try to send mail using the
expired token, resulting in "Invalid login" failures.

This patch makes it so we refresh the transport object if the auth token
changes, and also turns on SMTP connection pooling to limit simultaneous
SMTP connections (& maybe make sending multiple messages faster).

Fixes T7891

Test Plan: manual

Reviewers: juan, halla

Reviewed By: juan, halla

Subscribers: mark

Maniphest Tasks: T7891

Differential Revision: https://phab.nylas.com/D3997
2017-02-21 16:24:24 -08:00
Juan Tejada
a29dfcf56c [client-sync] Fix sending per recipient
Summary: This was broken, oh so broken

Test Plan: manual

Reviewers: halla, spang, evan, mark

Reviewed By: mark

Differential Revision: https://phab.nylas.com/D4001
2017-02-21 15:24:21 -08:00
Juan Tejada
9f78574c3d [*] metrics(Part 6) MetricsReporter.reportEvent now requires a nylasId
Summary:
This will help us aggregate metrics by user. This also makes it so we
don't report events in dev mode

Test Plan: manual

Reviewers: spang, evan

Reviewed By: spang, evan

Differential Revision: https://phab.nylas.com/D3981
2017-02-21 11:48:45 -08:00
Juan Tejada
301766722a [local-sync] metrics(Part 2) Move MetricsReporter to isomorphic-core
Summary:
Renamed it from SyncMetricsReporter to MetricsReporter and moved it to
iso-core.

The new metrics reporter can now be called from any environment and will
correctly report the metrics.

Test Plan: manual

Reviewers: mark, spang, evan

Reviewed By: spang, evan

Differential Revision: https://phab.nylas.com/D3967
2017-02-21 11:44:21 -08:00
Evan Morikawa
27c08d86e7 [*] update and add READMEs to each package
Summary:
Adding READMEs for easy and helpful browsing on GitHub.
Also add missing script and `--interpreter` flag

Test Plan: Run new launch commands

Reviewers: mark, spang, juan, halla

Reviewed By: spang, juan, halla

Differential Revision: https://phab.nylas.com/D3971
2017-02-17 17:28:09 -08:00
Halla Moore
57ff111925 [client-app, client-sync] Save imap folder names in the client-app
Summary:
Client-sync has the full imap folder names, but used to only pass the display
name to the application. The application needs the full imap names so that it
can pass them via metadata to cloud-workers that need to open imap boxes.

Test Plan: manual

Reviewers: evan, juan

Reviewed By: evan, juan

Subscribers: juan

Differential Revision: https://phab.nylas.com/D3951
2017-02-17 14:47:10 -08:00
Christine Spang
faf44193a7 [local-sync] Differentiate sync loop & other errors by additional fingerprint info
Summary:
If an exception has the same stack trace, by default Sentry will always group
it together in the same event. We don't want to do that for sync loop
errors---e.g. 'Invalid credentials' errors should not be grouped together with
stuff like 'Too many simultaneous connections'. Creating more unique groups
will allow us to better evaluate the effect of sync & other bugfixes.

Test Plan: writing unit test right now

Reviewers: juan, mark

Subscribers:

Differential Revision: https://phab.nylas.com/D3915
2017-02-17 14:03:24 -08:00
Evan Morikawa
918fa0b6dd [*] move to monorepo
[*] update babel

[client-app] remove flow-typed

[client-app] Move build/package.json to main package.json

[client-app] remove spec_integration

[client-app] fix babel support

Add client-private-plugins package.json

[client-app] add node_modules to global path for private-plugins

Move client-sync dependencies to client-app root

fix electron rebuild

[*] moved to monorepo

Summary: App now runs in monorepo

Test Plan: npm test

Reviewers: juan, mark, khamidou, halla, spang

Differential Revision: https://phab.nylas.com/D3947
2017-02-16 18:46:26 -08:00
Evan Morikawa
9407533f51 [client-*] Rename packages folders and update readme 2017-02-16 13:31:37 -08:00