Commit graph

767 commits

Author SHA1 Message Date
Halla Moore 1e0ae4fb03 [cloud-core, cloud-workers, isomorphic-core] Fix sending from cloud-workers
Summary:
The SendmailClient is in isomorphic-core, but wasn't working properly when
called from cloud-workers. This diff includes a variety of fixes to get it
working:

  - Add smtp config to connection settings in cloud-core
  - Don't use `PromiseUtils.promisify` for a function that needs `this` to
    be bound properly
  - Don't reference `NylasEnv` (That error gets caught and reported
    elsewhere anyway, we don't need to report it here)
  - Default `message.uploads` to `[]` to keep iterator happy
  - Add Gmail environment variables to cloud-worker app

Test Plan: manual

Reviewers: juan, khamidou, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D3954
2017-02-17 19:05:36 -08:00
Halla Moore 302f3b6215 [client-app, client-private-plugins] Update the reminders metadata structure
Summary:
The new send-reminders cloud-worker needs different information to check for
replies now that the sync engine is local. Make sure we save all of the
required information to the metadata. The current values are:

  - expiration: the time the reminder should be sent, if no new replies
  - folderImapNames: the folders to check for replies
  - messageIdHeaders: the Message-Id headers of messages in the thread
    (we search for In-Reply-To headers that match these values, and also
     use them to decide whether a message is a new reply or an old reply)
  - replyTo: the message to reply to when sending the reminder email
  - subject: the subject to use when sending the reminder email

Test Plan: manual

Reviewers: juan, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D3952
2017-02-17 19:03:07 -08:00
Evan Morikawa 12236753d5 [client-app] only show cloud deltas in the developer bar
Summary:
This removes client-sync deltas from the developerbar delta list.

We ONLY show cloud deltas now.

The connection between client-sync is no longer a network delta stream,
it's a direct function call. It makes no sense to show its status.

This now shows a single dot representing the state of the cloud delta
stream.

Test Plan: Manually connect and disconnect local cloud API and see icon change

Reviewers: halla, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3977
2017-02-17 17:33:18 -08:00
Evan Morikawa d1b5cb9952 [client-app] allow returning of message from DOM
Summary:
SFDC's task to upload an email to salesforce needs the stripped DOM of a
Message object to call `innerText`. The API was changed to return a string
instead of the DOM. This adds a flag to request the DOM instead of a
string.

Test Plan:
Manually assert `EnsureMessageOnSalesforceTask` properly can add the plain
text to the Salesforce Task object

Reviewers: halla, mark, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3976
2017-02-17 17:32:45 -08:00
Evan Morikawa 5577d1b4d3 [iso-core] fix error toJSON in node environments
Summary:
Node's native `Error` object does NOT implement toJSON. We attempt to call
toJSON when reporting errors. This wasn't noticed until now because
bunyan's pretty logger (which only run in dev mode) started JSONifying
errors

Test Plan:
Try and API auth with a bad username with local setup. See that it throws
toJSON error. After patch, error properly serializes

Reviewers: spang, halla, jerm, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3975
2017-02-17 17:31:36 -08:00
Evan Morikawa c149e9171d [client-app] fix errors with no message
Summary:
Some API Errors, like ECONNREFUSED, have no \.message.
Catch this in the error reporter

Test Plan: Manually create a non-message error and see better error message

Reviewers: spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3973
2017-02-17 17:29:33 -08:00
Evan Morikawa 27c08d86e7 [*] update and add READMEs to each package
Summary:
Adding READMEs for easy and helpful browsing on GitHub.
Also add missing script and `--interpreter` flag

Test Plan: Run new launch commands

Reviewers: mark, spang, juan, halla

Reviewed By: spang, juan, halla

Differential Revision: https://phab.nylas.com/D3971
2017-02-17 17:28:09 -08:00
Evan Morikawa 9994b2b3d7 [client-app] remove rouge-committed packages folder 2017-02-17 15:08:26 -08:00
Halla Moore 57ff111925 [client-app, client-sync] Save imap folder names in the client-app
Summary:
Client-sync has the full imap folder names, but used to only pass the display
name to the application. The application needs the full imap names so that it
can pass them via metadata to cloud-workers that need to open imap boxes.

Test Plan: manual

Reviewers: evan, juan

Reviewed By: evan, juan

Subscribers: juan

Differential Revision: https://phab.nylas.com/D3951
2017-02-17 14:47:10 -08:00
Halla Moore 393474a4d8 [isomorphic-core] Allow use of atob() in cloud environments
Summary: atob() is a global in browser environments, but needs to be imported otherwise.

Test Plan: manual

Reviewers: juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3950
2017-02-17 14:46:07 -08:00
Evan Morikawa ad65b7d77a [client-app] convert FocusedContactStore to es6 2017-02-17 14:17:30 -08:00
Mark Hahnenberg ebb19b4cd8 [client-app] Fix stale import
Summary: Moved a file and forgot to update this import :-/

Test Plan: Run locally

Reviewers: juan, evan, spang

Differential Revision: https://phab.nylas.com/D3970
2017-02-17 14:14:17 -08:00
Christine Spang faf44193a7 [local-sync] Differentiate sync loop & other errors by additional fingerprint info
Summary:
If an exception has the same stack trace, by default Sentry will always group
it together in the same event. We don't want to do that for sync loop
errors---e.g. 'Invalid credentials' errors should not be grouped together with
stuff like 'Too many simultaneous connections'. Creating more unique groups
will allow us to better evaluate the effect of sync & other bugfixes.

Test Plan: writing unit test right now

Reviewers: juan, mark

Subscribers:

Differential Revision: https://phab.nylas.com/D3915
2017-02-17 14:03:24 -08:00
Mark Hahnenberg 00299e446d [message-list] Add support for double-clicking inline images to open
Summary: See title

Test Plan: Run locally, verify that double clicking inline images opens them

Reviewers: evan, juan, spang

Reviewed By: juan, spang

Differential Revision: https://phab.nylas.com/D3963
2017-02-17 14:01:07 -08:00
Christine Spang 9cf703c6cd [*] Clean up some cruft
Summary: This stuff doesn't seem to be used for anything anymore and it's cluttering up the client-app dir.

Test Plan: ran the app, also did `npm start` with no ~/.nylas-dev

Reviewers: evan, juan

Reviewed By: evan, juan

Differential Revision: https://phab.nylas.com/D3961
2017-02-17 13:51:27 -08:00
Juan Tejada 4d5e69da7d [*] metrics(Part 1) Add env-helpers to iso core
Summary:
This is a set of functions which will allow isomorphic-core to detect
which environment it is running on.

This will be useful for moving the metrics reporter to iso-core

Test Plan: manual

Reviewers: mark, spang

Reviewed By: spang

Differential Revision: https://phab.nylas.com/D3966
2017-02-17 13:34:07 -08:00
Mark Hahnenberg a95ca8b07a [*] Fix electron launch args
Summary:
We were doing some incorrect processing of args passed to the main
function which was causing us to think we were launching NM by passing a
file (which creates a new draft and tries to attach that file). Since
were trying to attach 'packages/client-app', this was causing an error
dialogue to appear indicating that it wasn't possible to attach a
directory.

Test Plan: Run locally, verify no dialogue

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3962
2017-02-17 13:03:20 -08:00
Evan Morikawa c4a26e1dcd [*] convert everyting to rx-lite 2017-02-17 12:15:23 -08:00
Mark Hahnenberg 8267a14966 [iso-core] Add BackoffScheduler to exports
Summary: See title

Test Plan: Make sure NM loads

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3959
2017-02-17 11:03:26 -08:00
Juan Tejada 5a3a105e9d [*] Move BackoffSchedulers into isomorphic-core 2017-02-17 10:11:35 -08:00
Juan Tejada d385350808 [client-app] Update changelog 2017-02-17 10:11:35 -08:00
Evan Morikawa 3113fba25b [client-app] fix module spec
This changed because we now globally require node_modules
2017-02-17 10:00:43 -08:00
Evan Morikawa 918fa0b6dd [*] move to monorepo
[*] update babel

[client-app] remove flow-typed

[client-app] Move build/package.json to main package.json

[client-app] remove spec_integration

[client-app] fix babel support

Add client-private-plugins package.json

[client-app] add node_modules to global path for private-plugins

Move client-sync dependencies to client-app root

fix electron rebuild

[*] moved to monorepo

Summary: App now runs in monorepo

Test Plan: npm test

Reviewers: juan, mark, khamidou, halla, spang

Differential Revision: https://phab.nylas.com/D3947
2017-02-16 18:46:26 -08:00
Christine Spang 25d88589ba fix(linux-build): Update rpm spec to use new package name 2017-02-16 16:58:27 -08:00
Christine Spang 2bc6b7ac0f fix(linux-build): Change package name to Nylas Mail
Need this in order to allow parallel installation of Nylas Pro and Nylas
Mail.
2017-02-16 16:07:36 -08:00
Christine Spang 15697a9064 fix(cli): s/n1/nylas-mail/ 2017-02-16 16:02:08 -08:00
Evan Morikawa 9407533f51 [client-*] Rename packages folders and update readme 2017-02-16 13:31:37 -08:00
Evan Morikawa 84e2c75ce9 Merge remote-tracking branch 'k2/master' 2017-02-16 13:20:20 -08:00
Evan Morikawa 9f6889b03f [nylas-mail] move nylas-mail into /packages 2017-02-16 13:15:10 -08:00
Evan Morikawa 41bc1e5f5c [local-sync] fix related threads
Summary:
When saving a thread, we weren't properly setting the `participants`
object. Since Sequelize has object properties under getters and setters,
doing `this.participants.push()` did nothing.

Since threads had no `participants`, the related threads widget, which
queries through there, did nothing.

Test Plan: Manually inspect DB and use the related threads widget

Reviewers: halla, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3938
2017-02-16 12:33:52 -08:00
Evan Morikawa 9932fa4882 [*] update package.json to lerna specs
Summary:
This is the result of auto package.json fixing by lerna. Would be nice to
commit this so you can run script/bootstrap without it making local
changes.

I didn't manually bump any versions.

Test Plan: manual

Reviewers: mark, halla, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3934
2017-02-16 12:32:07 -08:00
Mark Hahnenberg d654e80a96 [local-sync] Use new BatteryStatusManager to compute sync loop delay
Summary:
We don't need to check folders that often while on battery. Check every
5 minutes rather than every 10 seconds.

Test Plan: Run locally, verify the timeout is longer while on battery

Reviewers: evan, spang, juan

Reviewed By: spang, juan

Differential Revision: https://phab.nylas.com/D3940
2017-02-16 10:35:24 -08:00
Juan Tejada e15079874b [local-sync] 🎨 sync loop error handler
Summary: See title

Test Plan: manual

Reviewers: evan, mark, spang

Reviewed By: spang

Subscribers: mark

Differential Revision: https://phab.nylas.com/D3932
2017-02-15 18:46:41 -08:00
Mark Hahnenberg 5f4e5cbbdf [sentry] Don't use breadcrumbs in dev mode
Summary: They obscure the location of our logs in the dev tools.

Test Plan: Run locally, verify that logs link to proper place

Reviewers: evan, spang, juan

Reviewed By: spang, juan

Differential Revision: https://phab.nylas.com/D3939
2017-02-15 18:38:55 -08:00
Christine Spang dce9743283 [local-sync] Refresh Google OAuth2 tokens when Invalid Credentials occurs in sync loop
Summary:
Previously, we would only refresh Google OAuth2 access tokens at the
beginning of the sync loop, and _only_ if the access token had already
expired. This meant that if an access token expired in the middle of a
sync loop iteration, the user would get prompted with the reauth red box
for their account and would have to either go through the oauth flow
again or restart the app for sync to continue.

This diff makes two changes:

1. Adds 5min of padding to the refresh window, so if a token will expire
in <5min, we'll go ahead and refresh the token. This will reduce the
possibility that an access token can expire during a sync loop
iteration.

2. Catches Invalid Credentials IMAPAuthenticationErrors for Gmail
accounts and forces a token refresh on the next sync loop.

These should prevent a user from _ever_ having to reauth their Gmail
account unless the refresh token is revoked, or we encounter some other
permanent error trying to refresh the token.

Fixes T7775 (at least some cases)

Test Plan: manual

Reviewers: khamidou, evan, juan

Reviewed By: juan

Maniphest Tasks: T7775, T7755

Differential Revision: https://phab.nylas.com/D3908
2017-02-15 14:37:31 -08:00
Christine Spang 56422eee10 Add TODOs about retries in sending 2017-02-15 14:23:38 -08:00
Juan Tejada d3f0847f79 [local-sync] Add exponential backoff when retrying syncback tasks
Summary:
Instead of re-implementing exponential backoff, throw the retryable
error so the sync loop handles it and backs-off

Test Plan: manual

Reviewers: spang, halla

Reviewed By: halla

Differential Revision: https://phab.nylas.com/D3914
2017-02-15 12:31:43 -08:00
Evan Morikawa f409bf8be1 [SFDC] Update SalesforceSearchIndexer for new search indexing
Summary:
Update Salesforce to use the new search indexer

Depends on D3911

Test Plan: Manually bootup SFDC and ensure it launches and indexes models properly

Reviewers: mark, halla, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3913
2017-02-15 11:33:28 -08:00
Christine Spang 04478f81c8 [cloud-api,cloud-workers,local-sync] Bump hapi version
We were two major versions behind. I upgraded and everything is working fine
in dev.
2017-02-15 07:55:48 -08:00
Juan Tejada 78f67d4a76 [cloud-api] KEEP Timeout streaming API connections every 15 minutes
This reverts commit a1b997f350.
This is actually working correctly to reduce REDIS connections
2017-02-14 18:28:07 -08:00
Evan Morikawa a1b997f350 Revert "[cloud-api] Timeout streaming API connections every 15 minutes"
This reverts commit 6e43c86a95.
2017-02-14 17:55:08 -08:00
Juan Tejada d1bd77d11b [cloud-*] Properly listen to stream disconnect events to close redis connections
See the following for why we need to set up the listeners on the raw
stream.
http://stackoverflow.com/questions/26221000/detecting-when-a-long-request-has-ended-in-nodejs-express
https://github.com/hapijs/discuss/issues/322#issuecomment-235999544

Hapi's disconnect event only fires on error or unexpected aborts: https://hapijs.com/api#response-events
2017-02-14 17:02:31 -08:00
Karim Hamidou 6e43c86a95 [cloud-api] Timeout streaming API connections every 15 minutes
Summary: Should help a fair bit with our redis connection pileup.

Test Plan: Tested manually.

Reviewers: mark, spang, juan, evan

Reviewed By: mark, spang, evan

Differential Revision: https://phab.nylas.com/D3916
2017-02-14 14:43:23 -08:00
Evan Morikawa 4245ec7aa5 [isomorphic-core] add accountId index definition to Transaction table 2017-02-14 10:48:12 -08:00
Juan Tejada fe57a5884b [local-sync] 🎨 comment 2017-02-13 21:52:21 -08:00
Juan Tejada e441553b50 [local-sync] syncback(Part 5): Always keep retrying tasks if error is retryable
Summary:
This commit makes it so we always continue retrying syncback tasks as long as they
error with a retryable error. There's really no reason to not continue retrying syncback
tasks after an arbitrary number of retries (especially such a low one) if we
encounter a retryable error. Before this commit, if for example we got 2 random
network errors in a row, we would just mark the task as failed even
though it would eventually succeed in subsequent attempts.

Previously, when N1 synced against the cloud api, we would indefinitely
retry a Task if we continued getting retryable errors. This ensures that
the app can work correctly offline and prevents displaying unecessary errors to
the user, and having actions bouncing back or sending messages without putting them
in the sent folder.

Additionally, this commit ensures that when cleaning up messages without a
folderImapUID we don't delete messages that are currently being added to the sent
folder. This is relevant to this commit because given that we could retry the
EnsureMessageInSentFolder task indefinitely, we might end up deleting that
message because it wont have a uid until the task succeeds.

Depends on D3898

Test Plan: manual

Reviewers: mark, spang, evan, halla

Reviewed By: spang, evan, halla

Differential Revision: https://phab.nylas.com/D3900
2017-02-13 21:50:02 -08:00
Christine Spang 3854051350 [local-sync] 🔥 Message syncback tasks
Summary:
Since all Nylas Mail actions are thread-based except for sending,
we never use any of these. Make `git grep` less confusing by getting rid
of them.

Test Plan: use the app

Reviewers: evan, halla, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D3910
2017-02-13 18:27:18 -08:00
Juan Tejada 964bc3208f [local-sync] syncback(Part 4): Don't always mark INPROGRESS tasks as failed at beginning of sync
Summary:
Previously, if you were to close the app while you had any tasks queued,
these would be marked as failed the next time you open the app, showing
an annoying error message and reverting any optimisitic actions.

However, we don't need to be so defensive about retrying tasks because
the only tasks we can't retry are the Sending tasks. All of the other
tasks like moving or changing labels are fine to retry (trying  move the same
set of uids twice wont cause an error)

This commit adds an extra status to syncback requests, "NOTRETRYABLE".
Only NOTRETRYABLE requests will be marked as failed at the beginning of
the sync loop, and any INPROGRESS tasks will be marked as NEW so they
can be retried

Depends on D3896

Test Plan: manual

Reviewers: mark, evan, spang, halla

Reviewed By: evan, halla

Differential Revision: https://phab.nylas.com/D3898
2017-02-13 13:48:29 -08:00
Juan Tejada 4e85993957 [local-sync] syncback(Part 3): Fixup runSyncbackTasks
Summary:
Now that we don't run Send tasks outside the sync loop, we don't need
that awful hack wich required passing a `runTask` callback to
`runSyncbackTask` in order to customize how to run the task.

Instead, runSyncbackTask now knows 2 ways to run a task, either via imap, or
via smtp, depending on the resource declared by task to run. So now
SyncbackTasks declare a resource type they need to run, and that will be
passed as their second argument when running.

Depends D3894

Test Plan: manual

Reviewers: mark, halla, spang, evan

Reviewed By: halla, spang, evan

Differential Revision: https://phab.nylas.com/D3896
2017-02-13 13:07:20 -08:00
Juan Tejada c1ecd045d7 [local-sync] syncback(Part 2): Reinstate send tasks back into the sync loop
Summary:
We had previously ripped send tasks outside the sync loop to make them run faster,
but they run fast enough inside the loop.

This commit will also fix the scenario where if you closed the app in the
middle of a send task, the task would just hang forever and never succeed or
fail (T7818); given that it was excluded from the loop, we also had to exclude it
from the cleanup step to mark any INPROGRESS tasks as failed at the beginning
of each loop, which caused send tasks in progress to never get cleaned.

Putting them back inside the loop allows us to  fix this without adding more messy
logic, and it cleans up ugly duplicated code. Additionally, it will prevent more
duplicated code for upcoming diffs that will improve syncback task reliability
when the app is closed or window is restarted in the middle of a task.

Depends on D3893

Test Plan:
manually test sending, it still works, it's still fast. Restarted
window in the middle of send task, task fails.

Reviewers: mark, spang, halla, evan

Reviewed By: spang, halla, evan

Differential Revision: https://phab.nylas.com/D3894
2017-02-13 13:05:11 -08:00