Summary:
This commit fixes the following bugs in message parsing:
- we were unilaterally decoding MIME bodies as UTF-8; instead, decode according
to the charset data in the mimepart header
- '7bit' content-transfer-encoding means us-ascii, NOT utf-7
- only interpret valid content-transfer-encodings (previously we were trying
to treat various charsets as transfer-encodings)
- clearer naming: s/values/parsedMessage/
- unify snippet cleanup between plaintext & stripped HTML (merging
whitespace etc.)
Test Plan: units tests coming
Reviewers: juan
Differential Revision: https://phab.nylas.com/D3491
We were getting sql unique constraint violation errors for ids because
we were attempting to create objects with the same id within the same
transaction.
This commit ensures that only attempt to write a contact with the same
id once, and that we check for all exsiting contacts before hand.
For file, we ensure that we don't attempt to write 2 files with the same
id more than once
We were using the stripped version of label/folder names for the id
hash, e.g. [Gmail]/Drafts would be Drafts.
However, we can't do this because it might collide with other names. e.g. if the
user created a Drafts label, it would end up colliding with [Gmail]/Drafts
Minor lint fix
Summary:
This commit also fixes snippets for HTML-only messages to strip out HTML
tags, and makes us preserve whitespace for plaintext emails by
displaying them in <pre class="nylas-plaintext"> tags, and makes us log
messages that fail to parse at all to a tempdir.
The only issue I found with using <pre> tags for plaintext email was
that some lines may trigger scrolling, so there is an associated commit
(D3484) that changes the CSS for <pre class="nylas-plaintext"> to wrap
lines.
In the future, we can add regression tests to this test suite whenever
we fix parsing bugs.
Test Plan: unit tests included
Reviewers: bengotow
Reviewed By: bengotow
Differential Revision: https://phab.nylas.com/D3483
When running syncback requests, if a task is meant to change the UID of
a message (e.g. move it to a new folder), that task should be run after
other tasks that don't affect the UID. Otherwise, when trying to run the
other tasks, they would reference a UID that is no longer valid.
This commit will make sure that we run any tasks that will change message uids last,
/and/ make sure that we don't run more than 1 task that will affect the uids of
the same messages in a row (i.e. without running a sync loop in between)
This commit will correctly keep track of folder and label ids when
creating them from N1.
Previously, when we sent the request to create a folder or label to our api,
we would immediately get back a serverId because it was created optimistically
in the back end— given that K2 is strictly non-optimistic, we won’t have a serverId
until some undetermined time in the future, and we need to somehow reference
the object that /was/ optimistically created in N1 to update the ui when
we do get the server id.
Since we can deterministically generate ids for folders and labels,
we "guess" what its going to be, and include it in the props of the syncback request
returned to N1. This is the simplest solution to get thing working
correctly right now, but we’ll need to revisit this in the future for
other types of objects (drafts, contacts, events), and revisit how we
will manage optimistic updates in N1 when we merge the 2 codebases
with K2 (given that K2 was designed to be non-optimisitc).
Summary:
Fixes T7283. We weren't creating deltas for contacts because we were inserting contacts using `UPSERT`, which requires us to add another sequelize hook. Unfortunately, support for this hook is only available on sequelize 4.x.
I didn't want to upgrade our sequelize version right now and cause of bunch of mysterious failures, so I've simply changed the `UPSERT` to be either an `INSERT`or `UPDATE`. We didn't really need it anyway.
Test Plan: Checked that N1 was receiving contact deltas.
Reviewers: bengotow
Reviewed By: bengotow
Maniphest Tasks: T7283
Differential Revision: https://phab.nylas.com/D3481
Also renames JSONType() -> buildJSONColumnOptions() and
JSONARRAYType() -> buildJSONARRAYColumnOptions() to prevent passing
those return values in as just the type value instead of the entire
options object.
Since message IDs are now static but there's no good way to generate
static thread IDs while syncing an account from newest message first,
we give threads the ID of any message on that thread and, when setting
metadata, look up the local thread ID by first going through the
message table.
[cloud-api] move Dockerfile and pm2 files into cloud-api
[*] additions to .gitignore
[cloud-workers]: move pm2 file into cloud-workers
updates to readme
Revert "[cloud-workers]: move pm2 file into cloud-workers"
This reverts commit 952292124b6e8dfa304c5fbd945c0b00f427e154.
Revert "[cloud-api] move Dockerfile and pm2 files into cloud-api"
This reverts commit aa92f691bf991848a51c00a2d01a5848f20a35a2.
Update deploy readme
- Don’t need functions in delta.js which must be called to return promsies. Fun of promsies is that you don’t need to care when they’re built to attach a .then.
- Make boundary between route handler and delta stream builder more explicit, don’t do query parsing in helpers, always reply from handler.
- Remove pushJSON extension to outputStream which never actually received JSON.
- Remove `takeUntil` - disposing of the downstream observable should dispose of all the merged/upstream observables
- Rename inflate => stringify since the returned value is a string not an object.
- Remove support for delta streams with no cursors. Don’t think this was supposed to be a feature.
- Add accountId to Transaction models
- Make database hooks shared in isomorphic core
Summary:
T7253 has two related parts:
1. Stripping the '[Gmail]/' prefix from any canonical Gmail folders
2. Handling nested IMAP folders.
This diff fixes 1. Changes for 2. are forthcoming.
Test Plan: Tested manually. Checked that the part was stripped from the N1 folder list.
Reviewers: juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3475
Moving to between gmail folders (all, spam, trash) or moving to inbox,
involves changing labels /and/ folders, simultaneously. For this I added
a task to perform both operations, and apply labels first before
attempting to move the folder
