Summary:
This diff modifies the SearchIndexer class to handle limiting the search
index size. It does this by periodically re-evaluating the window of
the n most recent items in a particular index where n is the max size of
the index. It then unindexes the items which are marked as indexed but
are no longer in the window and indexes the things that are in the window
but aren't marked as indexed.
Test Plan:
Run locally with a reduced thread index size, verify that the index
includes the most recent items and that it is the correct size. Also verify that
the queries used properly use fast sqlite indices.
Reviewers: evan, juan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3741
Summary:
When the search index got very big the queries we were running during
after keypress would cause jank in the UI which was very noticeable on
subsequent keypresses. This diff backgrounds these queries and adds a
LIMIT to the fts MATCH clauses to avoid having to send too much stuff
across the IPC bridge.
Test Plan: Run locally, make sure that typing is smooth while searching
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3757
Summary: See title
Test Plan: tested locally
Reviewers: juan, mark, evan
Reviewed By: juan, mark
Subscribers: juan
Differential Revision: https://phab.nylas.com/D3744
Summary:
I thought it was gonna be OK that we kept all HTML parts in a multipart/alternative
MIME structure because the world is a sane place and nobody would ever put more
than one HTML part in a multipart/alternative structure.
I was wrong.
We have found extraterrestrial life^W^WI mean emails which contain duplicate,
exactly the same MIME parts within a multipart/alternative MIME structure: two
text/plain parts and two text/html parts. This is likely due to a broken MIME
implementation, or perhaps a bug in someone's email script. So, we should
only keep one text/html MIME part if there are multiple.
Test Plan:
manual for now---added this to my mail parsing regression test list
for implementation once we unify the DBs and have a roughly stable code
structure
Reviewers: halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3750
Summary:
Rather than having a strict model where we don't decode the message if we
don't specifically recognize the CTE, treat any CTEs we don't recognize as
having no encoding. There are several CTE strings that could mean this (e.g.
7bit, 7BITS, 8-bit, binary, NONE, utf8), and we don't want to check for them
all. Additionally, if there is a CTE we don't support, the user will likely
see rendering issues and contact support. This will allow us to obtain more
concrete information about these messages.
Test Plan: manual
Reviewers: spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3748
When handling unhandled promise rejections, our error reporter would
completely freeze the worker window when trying to send a Promise
through the ipc bridge, rendering the app unusuable
This fixes T7621
Summary:
We need people to manually drag into their /Applications folders due to
OSX gatekeeper protections.
Test Plan: manual
Reviewers: juan, khamidou
Reviewed By: khamidou
Differential Revision: https://phab.nylas.com/D3756
Summary:
Given that we were marking the account as errored if we've encountered
enough RetryableErrors, we would show the red box to the user when in
fact the problem was the user was offline, causing confusion
If the user is offline, we will constantly get RetryableErrors in the
sync loop, and we can't mark the account as errored in that case.
Test Plan: manual
Reviewers: evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3752
Summary:
This commit ensures that we handle transient errors correctly when refreshing
tokens
Test Plan: manual
Reviewers: khamidou, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3740
Summary:
- Add a new button to the sync error notification to "Debug" sync. This will open the activity window and dev tools in that window. Depends on D3736
- Update the "Contact Support" link in error notifications to prepopulate the support ticket with the sync error in the account
- Make the "Check Again" button react when clicking it
Test Plan: manual
Reviewers: evan, khamidou
Reviewed By: evan, khamidou
Differential Revision: https://phab.nylas.com/D3737
Summary:
In the sync worker:
- Move the backoff logic inside `scheduleNextSync`, where all logic to schedule the next sync loop now lives
- If we've retried a RetryableError a bunch of times, show the error to the user, otherwise the user might think the app is not working for no reason
- Clean up logging
In the message processor:
- Report message processing errors to sentry!
Sync Process Manager:
- Listen to new `Actions.debugSync` to show the Activity Window and open dev tools
Test Plan: manual
Reviewers: khamidou, evan
Reviewed By: khamidou, evan
Differential Revision: https://phab.nylas.com/D3736
Summary:
The isSearchIndexed attribute gets bulk reset when we drop the search
index, however, the JSON values were not updated to reflect the new
column values. We don't care about notifications to this field, so in
order to make bulk clearing ok this diff adds a new loadFromColumn option to
Attributes which causes the value to be loaded from the SQL table column
rather than the JSON blob.
Test Plan:
Run locally, drop the search index, make sure we load from the
column rather than the JSON when re-indexing
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D3739
Summary:
This commit makes it so our syncback tasks send as few imap commands as possible by passing a set of UIDs whenever possible. Previously, we would send 1 command per message, with a single UID, which was very wasteful given that we can pass a set of UIDs. This is especially helpful for operating on threads with a large number of messages.
Syncback actions will now group all messages in a thread by the folder they belong to, and issue a single operation on the folder box. When removing all labels from a thread (setting labels to []), we need to issue a command of the form `box.delLabels(uids, labels)`, so we also group messages by set their set of labels to issue as few commands as possible.
This commit only batches imap commands, but we can still batch syncback actions themselves, which can be implemented in a separate patch.
Test Plan: manual
Reviewers: evan, mark, spang
Reviewed By: spang
Subscribers: halla, mg
Differential Revision: https://phab.nylas.com/D3719
Summary:
- Make the try again button react when clicked
- Don't mark as offline until after we've tried again at least 3 times
Test Plan: manual
Reviewers: evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3738
Summary:
- Refactor Gmail auth functions
- Add Analytics in
- Show error states
Depends on D3735
Test Plan: manual
Reviewers: khamidou, juan
Reviewed By: khamidou, juan
Differential Revision: https://phab.nylas.com/D3732
Summary:
Very occasionally the db can become malformed for some reason. If this
happens all DB requests will fail and flood sentry with errors. This will
attempt to reset the db if we encounter this error
Depends on D3730
Test Plan: manual
Reviewers: juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3731
Summary: Double-firing protection since the DatabaseStore can now fire this
Test Plan: manual
Reviewers: juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3730
Summary:
We are getting to many imap timeout connection errors because the
authTimeout was just 5 secs
Test Plan: manual
Reviewers: khamidou, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3733
Summary:
We changed the index format (splitting participants into to and from) so
we need to update the index version so that it gets rebuilt.
Test Plan: Run locally, make sure index gets rebuilt
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3728
Summary:
Previously we indexed all participants together for a single thread.
This lead to confusing search results when people would search for
'from' and get back lots of results that were only 'to'. This diff
splits the participants index into separate to and from indices and
updates the query generator appropriately.
Fixes T7580
Test Plan: Run locally, verify from doesn't return to and vice versa.
Reviewers: evan, juan
Reviewed By: juan
Maniphest Tasks: T7580
Differential Revision: https://phab.nylas.com/D3724
Summary:
Specifically, these imap connections have been known to hang when you close your laptop or go offline/online, even though we are passing a `socketTimeout` to node-imap. When they hang, everything freezes because the promise waiting for the result never resolves.
`_createConnectionPromise` wraps the operations with a timeout we implemented ourselves, and correctly rejects on timeout.
This commit wraps other imap operations that were missing. (I notices because I encountered the hanging on one of these operations)
Test Plan: manual
Reviewers: evan, mark, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D3720