Karim Hamidou
47e0683cac
First step to getting persistent id in K2
...
Summary:
This diff adds persistent unique ids for messages and contacts. For messages, we just take a hash of the headers. For contacts, we hash the contact's email address.
This diff bundles a couple of tiny fixes too, like always trying to restart an account's sync, even after an exception.
Note that since there's no reliable way to have persistent ids for threads, we'll have to change our code to use message ids instead. Alas, that's a story for another (massive) diff.
Test Plan: Tested manually.
Reviewers: bengotow
Reviewed By: bengotow
Differential Revision: https://phab.nylas.com/D3468
2016-12-01 11:15:13 -08:00
Ben Gotow
3d79f9b8be
[local-sync] Run message processor for one message at a time
...
This avoids issues that arise when we process two messages on the same thread concurrently!
2016-11-30 17:26:23 -08:00
Ben Gotow
94722f27ab
[local-sync] Fix “Add INBOX label” issue
2016-11-30 16:26:53 -08:00
Ben Gotow
bf9db2738c
[local-sync] Fix ingestion of flags and labels
2016-11-30 16:26:53 -08:00
Ben Gotow
935d8cf141
[local-sync] Fix transaction log to only ignore syncState
2016-11-30 16:25:28 -08:00
Ben Gotow
2476cae297
[local-sync] Sync a bit faster for testing
2016-11-30 16:25:28 -08:00
Ben Gotow
1e01878e5a
fix(sync): Fix attribute updates applying to threads
2016-11-30 13:55:49 -08:00
Evan Morikawa
a36b1e1f28
[*] add delta endpoints and DRY deltaStreamBuilder
...
[cloud-core] add objectType to Metadata
[*] binding fixes to delta stream
2016-11-30 13:41:09 -05:00
Juan Tejada
a37b90a209
[local-sync] Remove unecessary db queries in detect-thread
2016-11-30 10:17:59 -08:00
Ben Gotow
ece519553f
fix(sync): Issue with CONDSTORE args not passed to fetch
2016-11-29 18:02:25 -08:00
Ben Gotow
82f505b659
refactor(sync-worker): Convert IMAP operations to async/await
2016-11-29 17:04:01 -08:00
Ben Gotow
fb9d770cc6
refactor(sync-worker) async/await for Karim
2016-11-29 16:20:18 -08:00
Ben Gotow
0c79ebb86a
[local-sync] Synchronize K2 accounts with N1 accounts
2016-11-29 16:07:33 -08:00
Karim Hamidou
0879b6390e
Make db reset synchronous.
2016-11-29 15:13:42 -08:00
Juan Tejada
20be7cc513
[local-sync] properly save folders when detecting thread
2016-11-29 15:04:45 -08:00
Karim Hamidou
ac818f30fe
Add a shortcut to restart the sync for an account from scratch.
2016-11-29 14:59:37 -08:00
Ben Gotow
48fff4f4e1
fix(dashboard): Remove sync policy
2016-11-29 14:58:52 -08:00
Juan Tejada
b459076415
[local-sync] minor fix
2016-11-29 12:52:36 -08:00
Juan Tejada
b7b0808689
[local-sync]: Load all db models
2016-11-29 12:38:49 -08:00
Juan Tejada
a9be3bb3ed
[local-sync]: Refactor message ingestion to write new messages only once
...
This will only generate 1 delta per new message instead of 2
Adds a MessageFactory to create messages from imapmessage objects, and
unifies and cleans up that logic
Also modified the message-processor to be more suited for local sync,
made emphasis that it's only for processing new messages, and got rid of
the while "pipeline" concept
- TODO message-processor specs are broken
2016-11-29 11:24:27 -08:00
Evan Morikawa
fc24ffea63
[isomorphic-core]: move Transaction model to isomorphic-core
2016-11-29 13:57:19 -05:00
Ben Gotow
f804c53522
[dashboard] Put dashboard into work window
2016-11-28 18:02:42 -08:00
Halla Moore
10f62f6b5a
[local-sync, iso-core] feat(send): Support basic sending for gmail accounts
2016-11-28 14:46:31 -08:00
Ben Gotow
217ba1e020
[sync]: Sync again immediately if more emails remain
2016-11-28 12:00:57 -08:00
Juan Tejada
f9b6e63958
[local-sync] Support parsing 8bit encoded messages
2016-11-28 11:41:16 -08:00
Evan Morikawa
048e077cc5
[local-sync] retry sync after error
2016-11-28 10:39:23 -08:00
Evan Morikawa
2c0c6164ec
[local-sync] print sync error when in error state
2016-11-28 10:37:04 -08:00
Juan Tejada
64cb03f4cd
Move non shared-models to local sync / remove duplicate DatabaseTypes
2016-11-26 12:48:48 -08:00
Ben Gotow
4bbfd6b906
fix(storage): Move to ~/.nylas-k2
2016-11-23 17:08:38 -08:00
Ben Gotow
5c37abc578
[n1] fix(sync): Add support for message encodings
2016-11-23 17:07:09 -08:00
Ben Gotow
dd28d6b146
[n1] fix(api): Add support for messages?thread_id=1
2016-11-23 16:25:45 -08:00
Ben Gotow
42e20f106c
fix(account):
2016-11-23 15:47:49 -08:00
Ben Gotow
e7386ff4ae
Gmail auth: resolved settings tests
2016-11-23 15:23:46 -08:00
Juan Tejada
3f050ea150
fix dangling paren
2016-11-23 14:26:24 -08:00
Juan Tejada
f0caf042bd
[local-sync]: Restore message-processor
...
Don't use pubsub for scheduling a message for processing, just use
nextTick
2016-11-23 14:25:15 -08:00
Evan Morikawa
78b96d24ca
[local-sync] fix promise instance of check
2016-11-23 14:22:38 -08:00
Evan Morikawa
931943bea6
[local-sync] fix unnecessary transaction JSON parse and binding issue
2016-11-23 13:43:00 -08:00
Juan Tejada
f45425a7b6
[local-sync] Remove unused message-types
2016-11-23 13:30:51 -08:00
Karim Hamidou
43030789a8
Make the delta stream work
...
Also replaces our LocalPubsubConnector by a TransactionConnector which
just uses EventEmitter.
2016-11-23 13:08:59 -08:00
Evan Morikawa
60801262c7
[local-sync] return as auth_token instead of token for N1
2016-11-23 12:35:48 -08:00
Juan Tejada
f98c38fef4
[iso-core]: Make loadModels reusable by cloud-core
2016-11-23 12:16:58 -08:00
Evan Morikawa
eacbb99f15
rearrange logging so message comes first in console
2016-11-23 12:01:47 -08:00
Karim Hamidou
c05bc8ee63
Remove redis dependency
...
Conflicts:
packages/local-sync/src/local-sync-worker/sync-process-manager.js
2016-11-23 11:50:13 -08:00
Juan Tejada
f2a032c87c
[iso-core]: Update api for loadModels
2016-11-23 11:49:32 -08:00
Evan Morikawa
edd49c0f4a
Use 2578 as port on example
2016-11-23 11:48:48 -08:00
Evan Morikawa
0243edeee3
[local-sync] Remove Gmail auth from local-sync
2016-11-23 11:07:04 -08:00
Juan Tejada
53a1f6e824
[local-sync]: Properly remove all refs to global.Metrics
2016-11-23 11:01:48 -08:00
Evan Morikawa
9707092cc9
Add logger fallback
2016-11-23 10:53:02 -08:00
Juan Tejada
5b7214e464
[local-sync]: Let local-sync start, add loadModels to iso-core
2016-11-23 10:18:35 -08:00
Evan Morikawa
3fac21458c
Update local port to 2578
2016-11-23 10:12:02 -08:00
