Commit graph

533 commits

Author SHA1 Message Date
Evan Morikawa
5a5aeb6bb3 [local-private] move old edgehill src/pro into packages/local-private
Summary: This is a test

Test Plan: Testing

Reviewers: juan

Differential Revision: https://phab.nylas.com/D3493

[local-private] add old edgehill src/pro into packages/local-private
2016-12-12 10:09:58 -05:00
Mark Hahnenberg
f32d0df7e0 [local-sync] Increment streamAll offset by chunkSize (#5)
Otherwise we'll infinite loop if there are more than 2000 results.
2016-12-09 11:15:04 -08:00
Mark Hahnenberg
60bca4acb2 [local-sync] Add index for Thread.id (#4) 2016-12-09 11:12:47 -08:00
Christine Spang
587f7787a6 fix(local-sync): Fix charset interpretation in message parsing
Summary:
This commit fixes the following bugs in message parsing:
- we were unilaterally decoding MIME bodies as UTF-8; instead, decode according
  to the charset data in the mimepart header
- '7bit' content-transfer-encoding means us-ascii, NOT utf-7
- only interpret valid content-transfer-encodings (previously we were trying
  to treat various charsets as transfer-encodings)
- clearer naming: s/values/parsedMessage/
- unify snippet cleanup between plaintext & stripped HTML (merging
  whitespace etc.)

Test Plan: units tests coming

Reviewers: juan

Differential Revision: https://phab.nylas.com/D3491
2016-12-09 11:01:04 -08:00
Jackie Luo
162dbbd141 fix(error): Treat error as object, not string 2016-12-08 18:32:23 -08:00
Jackie Luo
4a11bfe977 fix(message-factory): Unlink circular dependency 2016-12-08 18:10:17 -08:00
Jackie Luo
6e111c073a fix(message-ids): Use correct hashing for headers 2016-12-08 17:55:39 -08:00
Jackie Luo
fae855f0fe feat(message-ids): Hash message IDs and replace in draft before sending 2016-12-08 17:48:34 -08:00
Evan Morikawa
947eb99b8d [local-sync] fix builds. Routes with trailing slash and main extension 2016-12-08 14:17:42 -08:00
Jackie Luo
7a763b604e feat(tracking): Add routes for open and link tracking 2016-12-08 13:38:29 -08:00
Juan Tejada
1ba56c5c05 [local-sync] Fix lint error 2016-12-08 10:28:29 -08:00
Evan Morikawa
2f786a8e91 [local-sync] fix delta sync not getting threads with updated labels 2016-12-07 17:47:49 -08:00
Evan Morikawa
d161a30a34 [local-sync] Add global dbs and cleanup orphan messages 2016-12-07 16:39:01 -08:00
Juan Tejada
878735f52e [local-sync] Add todos 2016-12-07 16:16:45 -08:00
Juan Tejada
a185a8a5fe [local-sync] Fix file/contact creation
We were getting sql unique constraint violation errors for ids because
we were attempting to create objects with the same id within the same
transaction.

This commit ensures that only attempt to write a contact with the same
id once, and that we check for all exsiting contacts before hand.

For file, we ensure that we don't attempt to write 2 files with the same
id more than once
2016-12-07 14:48:57 -08:00
Halla Moore
56f8d41b8c [local-sync] feat(send): Add support for attachments
Also move some helper function logic onto the Message model
2016-12-07 14:16:36 -08:00
Juan Tejada
8307976df8 [local-sync]: Fix label/folder id creation
We were using the stripped version of label/folder names for the id
hash, e.g. [Gmail]/Drafts would be Drafts.
However, we can't do this because it might collide with other names. e.g. if the
user created a Drafts label, it would end up colliding with [Gmail]/Drafts

Minor lint fix
2016-12-07 13:39:41 -08:00
Evan Morikawa
edbf869ff7 [isomorphic-core] add more auth error states 2016-12-07 10:10:49 -08:00
Christine Spang
718cdbe9ae Enable nodejs completion with tern 2016-12-07 07:25:48 -08:00
Christine Spang
a23c68092e [local-sync] Add specs for message parsing
Summary:
This commit also fixes snippets for HTML-only messages to strip out HTML
tags, and makes us preserve whitespace for plaintext emails by
displaying them in <pre class="nylas-plaintext"> tags, and makes us log
messages that fail to parse at all to a tempdir.

The only issue I found with using <pre> tags for plaintext email was
that some lines may trigger scrolling, so there is an associated commit
(D3484) that changes the CSS for <pre class="nylas-plaintext"> to wrap
lines.

In the future, we can add regression tests to this test suite whenever
we fix parsing bugs.

Test Plan: unit tests included

Reviewers: bengotow

Reviewed By: bengotow

Differential Revision: https://phab.nylas.com/D3483
2016-12-07 07:25:28 -08:00
Ben Gotow
c3bd3dc297 [local-sync] Don’t update contacts if name is the same 2016-12-06 17:51:12 -08:00
Evan Morikawa
6ac46d1079 remove console log 2016-12-06 17:20:43 -08:00
Evan Morikawa
7e485228af [cloud-api] fix param names 2016-12-06 16:03:31 -08:00
Evan Morikawa
83ef47d049 [*] package.json updates from lerna 2016-12-06 16:03:31 -08:00
Evan Morikawa
8cbda7505a [*] fix auth 2016-12-06 16:03:31 -08:00
Evan Morikawa
4c53247df1 [*] update lerna to 2.0.0-beta.30 2016-12-06 16:03:31 -08:00
Evan Morikawa
2cbb90bb3b [*] DRY Auth 2016-12-06 16:03:31 -08:00
Evan Morikawa
896f981408 [isomorphic-core] extract AuthHelpers to DRY 2016-12-06 16:03:31 -08:00
Juan Tejada
7468e8123e [local-sync] Update DeleteSentMessage task for ordering 2016-12-06 15:54:45 -08:00
Juan Tejada
fda7fe18cc [local-sync] Ensure order correct order when running syncback requests
When running syncback requests, if a task is meant to change the UID of
a message (e.g. move it to a new folder), that task should be run after
other tasks that don't affect the UID. Otherwise, when trying to run the
other tasks, they would reference a UID that is no longer valid.

This commit will make sure that we run any tasks that will change message uids last,
/and/ make sure that we don't run more than 1 task that will affect the uids of
the same messages in a row (i.e. without running a sync loop in between)
2016-12-06 15:45:34 -08:00
Halla Moore
e785d73bdc [local-sync, iso-core] feat(send): add finishing touches to gmail multi-send
Delete the sent messages that gmail automatically creates and save our
generic form of the draft as a sent message.
2016-12-06 11:37:47 -08:00
Juan Tejada
1a24840062 [local-sync] Correctly sync folders and labels
This commit will correctly keep track of folder and label ids when
creating them from N1.

Previously, when we sent the request to create a folder or label to our api,
we would immediately get back a serverId because it was created optimistically
in the back end— given that K2 is strictly non-optimistic, we won’t have a serverId
until some undetermined time in the future, and we need to somehow reference
the object that /was/ optimistically created in N1 to update the ui when
we do get the server id.

Since we can deterministically generate ids for folders and labels,
we "guess" what its going to be, and include it in the props of the syncback request
returned to N1. This is the simplest solution to get thing working
correctly right now, but we’ll need to revisit this in the future for
other types of objects (drafts, contacts, events), and revisit how we
will manage optimistic updates in N1 when we merge the 2 codebases
with K2 (given that K2 was designed to be non-optimisitc).
2016-12-05 18:56:36 -08:00
Ben Gotow
aed1d59916 [local-sync] fix(folder-list): Support children when parent has no attribs 2016-12-05 16:20:17 -08:00
Ben Gotow
1e3b346c94 [local-sync] feat(specs): Add basic tests for folder sync 2016-12-05 16:07:49 -08:00
Karim Hamidou
04b1763965 [fix] Fix transaction creation for contacts
Summary:
Fixes T7283. We weren't creating deltas for contacts because we were inserting contacts using `UPSERT`, which requires us to add another sequelize hook.  Unfortunately, support for this hook is only available on sequelize 4.x.

I didn't want to upgrade our sequelize version right now and cause of bunch of mysterious failures, so I've simply changed the `UPSERT` to be either an `INSERT`or `UPDATE`. We didn't really need it anyway.

Test Plan: Checked that N1 was receiving contact deltas.

Reviewers: bengotow

Reviewed By: bengotow

Maniphest Tasks: T7283

Differential Revision: https://phab.nylas.com/D3481
2016-12-05 15:37:17 -08:00
Ben Gotow
269d61171a [local-sync] fix(specs): New specs for threading 2016-12-05 15:00:37 -08:00
Halla Moore
63032a2449 [local-sync] Fix(send): change some validation fields for multi-send 2016-12-05 12:55:56 -08:00
Ben Gotow
30c8bedd7a [local-sync] fix(specs): run npm test in local-sync dir 2016-12-05 12:16:53 -08:00
Ben Gotow
017e22c88d [*] Allow zero elements in the replyTo field 2016-12-05 09:57:47 -08:00
Halla Moore
6a51036e48 [local-sync, iso-core, cloud-core] feat(send): add multi-send support
Also renames JSONType() -> buildJSONColumnOptions() and
JSONARRAYType() -> buildJSONARRAYColumnOptions() to prevent passing
those return values in as just the type value instead of the entire
options object.
2016-12-02 15:20:31 -08:00
Evan Morikawa
2ba326dfc6 [local-sync] only enable logging in dev mode of N1 2016-12-02 17:34:05 -05:00
Christine Spang
9752eea9f7 [local-sync] Set thread IDs to the ID of a message in the thread
Since message IDs are now static but there's no good way to generate
static thread IDs while syncing an account from newest message first,
we give threads the ID of any message on that thread and, when setting
metadata, look up the local thread ID by first going through the
message table.
2016-12-02 14:13:47 -08:00
Evan Morikawa
d9f3edad0f [isomorphic-core] ignore folder deltas due to just syncState version issues 2016-12-02 16:25:42 -05:00
Evan Morikawa
b06566d8bd [local-sync] make syncbackRequest objects N1-ready 2016-12-02 16:14:42 -05:00
Juan Tejada
b36f61812d [local-sync]: Don't create or sync \NonExistent or \NoSelect folders 2016-12-02 12:14:09 -08:00
Juan Tejada
f1491fa4cc fix observable disposable 2016-12-02 12:09:35 -08:00
Juan Tejada
5ac3fe9ea8 [local-sync] Fix starred role on folders 2016-12-02 12:07:21 -08:00
Evan Morikawa
40791028d5 [cloud-api] Update cloud-api readme & remove unused files
[cloud-api] move Dockerfile and pm2 files into cloud-api

[*] additions to .gitignore

[cloud-workers]: move pm2 file into cloud-workers

updates to readme

Revert "[cloud-workers]: move pm2 file into cloud-workers"

This reverts commit 952292124b6e8dfa304c5fbd945c0b00f427e154.

Revert "[cloud-api] move Dockerfile and pm2 files into cloud-api"

This reverts commit aa92f691bf991848a51c00a2d01a5848f20a35a2.

Update deploy readme
2016-12-02 13:28:05 -05:00
Ben Gotow
5ff7a92197 [*] fix(deltas): 0 as first cursor, deltas from redis are already JSON 2016-12-01 18:54:26 -08:00
Ben Gotow
7712269402 [*] fix(deltas): Cloud-API not filtering deltas at all, refactor a few things
- Don’t need functions in delta.js which must be called to return promsies. Fun of promsies is that you don’t need to care when they’re built to attach a .then.

- Make boundary between route handler and delta stream builder more explicit, don’t do query parsing in helpers, always reply from handler.

- Remove pushJSON extension to outputStream which never actually received JSON.

- Remove `takeUntil` - disposing of the downstream observable should dispose of all the merged/upstream observables

- Rename inflate => stringify since the returned value is a string not an object.

- Remove support for delta streams with no cursors. Don’t think this was supposed to be a feature.

- Add accountId to Transaction models

- Make database hooks shared in isomorphic core
2016-12-01 18:41:46 -08:00