Commit graph

249 commits

Author SHA1 Message Date
Halla Moore
ef11508b38 Added MarkThreadAsUnread task 2016-06-29 17:23:18 -07:00
Evan Morikawa
4d7f728a75 categoryUID -> categoryImapUID and messageId -> messageHeaderId 2016-06-29 17:10:34 -07:00
Halla Moore
d3fd671ff6 Update MarkThreadAsRead task 2016-06-29 17:01:30 -07:00
Evan Morikawa
6cb67e417c Fix to use Imap BOX and simplify move helper 2016-06-29 16:25:51 -07:00
Halla Moore
c7fa9a782f Merge branch 'master' of github.com:nylas/k2 2016-06-29 16:12:44 -07:00
Halla Moore
825ce2eba3 Use categoryUID instead of categoryId, change default value to match node-imap 2016-06-29 16:12:29 -07:00
Ben Gotow
b00b1e6d21 Add replyTo to message model 2016-06-29 16:10:45 -07:00
Halla Moore
da27959009 Changed some variables so that first letter is lowercase 2016-06-29 15:07:29 -07:00
Evan Morikawa
fc3796ff66 Fix transaction case issue 2016-06-29 14:58:38 -07:00
Evan Morikawa
5a56c9a4e4 AccountId -> accountId 2016-06-29 14:43:20 -07:00
Evan Morikawa
9ba3ac1bfd Merge branch 'syncback' 2016-06-29 14:25:52 -07:00
Evan Morikawa
905d797fd2 Remove comments 2016-06-29 14:25:42 -07:00
Evan Morikawa
124f0a9579 Add sigterm 2016-06-29 14:19:04 -07:00
Halla Moore
a45306729f Merge branch 'syncback' of github.com:nylas/k2 into syncback 2016-06-29 14:01:01 -07:00
Halla Moore
bf39b51e08 Correct some function names/params 2016-06-29 14:00:47 -07:00
Evan Morikawa
6a6e1f4f60 Remove loggin in api 2016-06-29 12:29:21 -07:00
Halla Moore
2721617665 Use CategoryId instead of messageId 2016-06-29 12:28:22 -07:00
Evan Morikawa
0e80bce433 readonly false 2016-06-29 12:26:45 -07:00
Evan Morikawa
3753d9743a Fix return 2016-06-29 12:25:38 -07:00
Evan Morikawa
d26824771d Clean up loggin 2016-06-29 12:08:31 -07:00
Halla Moore
bdfee404e5 Add move imap wrapper, make sure wrapper methods are called 2016-06-29 12:07:28 -07:00
Evan Morikawa
7cf59c3c11 Fix retrun value in MoveToFolderIMAP task 2016-06-29 12:04:46 -07:00
Ben Gotow
a25c399652 Downcase remaining model names 2016-06-29 12:04:22 -07:00
Ben Gotow
53d6c6284e Support for /threads?view=count/expanded, thread & message JSON includes folders 2016-06-29 12:02:31 -07:00
Evan Morikawa
941180c1df Syncback WIP 2016-06-29 11:44:30 -07:00
Ben Gotow
9ff7d3aea5 Add folders and labels to /threads response, version numbers to all models 2016-06-29 11:22:38 -07:00
Ben Gotow
6ecacdd591 Add account_id to all responses, downcase model ref columns 2016-06-29 10:36:35 -07:00
Evan Morikawa
ab19133b20 Add SyncbackTask super class 2016-06-29 10:13:56 -07:00
Evan Morikawa
cf87325c5d Fix syncback request object 2016-06-29 10:13:56 -07:00
Halla Moore
8a53dca755 Merge branch 'master' of github.com:nylas/k2 2016-06-29 10:10:49 -07:00
Evan Morikawa
ba2d865a5d Cleanup syncback request 2016-06-28 19:07:49 -07:00
Evan Morikawa
11a3a8f58e Cleanup syncback task 2016-06-28 19:02:24 -07:00
Ben Gotow
c7422ffaea Fix bug where always closing connection after sync 2016-06-28 18:43:53 -07:00
Ben Gotow
0189e3e77d Break Gmail mailboxes into folders and labels 2016-06-28 18:31:40 -07:00
Evan Morikawa
7180ebbb70 PUT route works on threads to create SyncbackRequests 2016-06-28 18:17:11 -07:00
Ben Gotow
ad484e8ae1 Add object key to all JSON responses 2016-06-28 18:14:54 -07:00
Evan Morikawa
306086091d syncbackMessageActions refactor 2016-06-28 18:13:49 -07:00
Halla Moore
f5af9c0cb5 Added a syncback task to mark a thread as read, untested 2016-06-28 18:13:49 -07:00
Ben Gotow
222c68a598 Deprecate thread.cleanedSubject, since the only subject is the clean one 2016-06-28 18:11:55 -07:00
Ben Gotow
1d1186cb34 Add more attributes to /threads, move processor to it's own process 2016-06-28 18:10:52 -07:00
Evan Morikawa
cea7783a5f syncbackMessageActions refactor 2016-06-28 17:12:45 -07:00
Halla Moore
b867bcb31e Merge branch 'master' of github.com:nylas/k2 2016-06-28 15:56:35 -07:00
Evan Morikawa
2d90c5bb14 Fixes to sync worker 2016-06-28 15:44:38 -07:00
Ben Gotow
ca0120a824 Allow sync after errors with env var 2016-06-28 15:37:22 -07:00
Evan Morikawa
14cffcf8a5 Change to notify / observe for syncback requests 2016-06-28 15:35:59 -07:00
Ben Gotow
a10543c1c8 Chunk sync of large mailboxes, keep track of synced UID range, not just uidnext 2016-06-28 15:30:56 -07:00
Halla Moore
0f64b6e58e Add category filtering to threads api endpoint 2016-06-28 15:01:41 -07:00
Evan Morikawa
0f4ed7f4a1 Fix package json 2016-06-28 14:09:56 -07:00
Ben Gotow
8108a875ee Allow trailing slashes on API endpoints 2016-06-28 13:57:07 -07:00
Jackie Luo
eaf19cc522 Add files endpoint for metadata 2016-06-28 13:55:00 -07:00
Jackie Luo
a00f86660c Create file objects from MIME parts 2016-06-28 11:32:18 -07:00
Juan Tejada
c5d753a434 Add /very/ basic initial error status to dashboard
- TODO: add option to solve the error
2016-06-27 23:59:22 -07:00
Ben Gotow
f68948c23f Fix population of db fields in processor 2016-06-27 16:38:40 -07:00
Ben Gotow
91887a6e11 Speed up sync by fetching preferred body block only 2016-06-27 16:11:36 -07:00
Juan Tejada
ed749e0f51 Add sync worker error handling
- Handles sync errors in a single place. For now, if error is not a
socket error, will treat as a permanent error, save the error to the
account object, and prevent any other syncing until the error is cleared
from the account object
- Adds a NylasError class that can be extended and serialized. Adds it
to global namespace on all packages and replaces all uses of regular
Error
2016-06-27 16:03:38 -07:00
Jackie Luo
cf421cbb2d Check In-Reply-To instead of References 2016-06-27 14:58:09 -07:00
Jackie Luo
179f24449b Refactor code and add test for threading 2016-06-27 14:52:05 -07:00
Halla Moore
5077b3f027 Add subject query to threads route 2016-06-27 13:45:11 -07:00
Halla Moore
2e56d3ced4 Add timestamp queries to threads route 2016-06-27 12:30:28 -07:00
Ben Gotow
76d873c2b2 Reintroduce concept of provider, start work on Gmail 2016-06-27 10:43:53 -07:00
Halla Moore
ab4c8cf32c Add id query to threads route 2016-06-27 10:35:34 -07:00
Juan Tejada
1fe7fdf4b1 More sync error handling WIP 2016-06-27 10:27:38 -07:00
Halla Moore
621f8b5c2b Add starred query to threads route 2016-06-27 10:15:05 -07:00
Juan Tejada
63b929e59b Fix missing promise based error handling in sync-worker 2016-06-26 09:52:03 -07:00
Juan Tejada
dd350a5081 Update IMAPConnection api + error handling fixes + misc
- `IMAPConnection::openBox` now returns a Promise that resolves to an
IMAPBox, and IMAPBox contains all of the `fetch` operations. This makes
the dependency between fetch operations and the currently open mailbox
explicit rather than implicit and by forcing the operations to be called on
a box instance and hopefully prevent errors. It will also throw an error
if the constraint is no longer satisfied.

- `fetch` operations now return an observable stream of messages (or Promise for single value),
while preserving the same logic of the original implementation. You can use `.toPromise()` on
the observable to get a Promise that resolves when the observable stream has
completely drained.

- Fixes error handling in a few places and renames some variables
2016-06-26 01:57:33 -07:00
Juan Tejada
63f48ae817 Adds hapi-boom for easy error responses 2016-06-26 01:44:39 -07:00
Juan Tejada
1336dea0d4 Fix auth response + log errors 2016-06-25 22:19:05 -07:00
Juan Tejada
caaef8d5b5 Add /messages/<id> enpoint + support for raw message/rfc822
- When message/rfc822 is requested, fetch ray message from IMAP
connection
2016-06-25 11:25:34 -07:00
Ben Gotow
69e87cbf49 Gmail auth at http://localhost:5100/auth/gmail 2016-06-24 16:46:41 -07:00
Halla Moore
bc5a4ecf3c Add unread query to threads route 2016-06-24 16:14:04 -07:00
Halla Moore
18ad15937f Change uppercase instance of Joi to lowercase 2016-06-24 14:07:10 -07:00
Juan Tejada
8f945888e4 Add missing test fixtures and fix path for reading them 2016-06-24 10:39:30 -07:00
Juan Tejada
ee2dc6c3f6 Expect true to be true 2016-06-24 10:36:57 -07:00
Juan Tejada
0ac3c8c313 Add jasmine for message-processor 2016-06-24 10:35:34 -07:00
Ben Gotow
b67a3ae3e4 Launch dashboard when in dev mode, publish change on acct creation 2016-06-23 18:17:04 -07:00
Ben Gotow
f678045c4f sync db <=> redis with sequelize hooks 2016-06-23 16:28:51 -07:00
Ben Gotow
eac019f536 Rather than changing policies, just set expiring redis key to track activity 2016-06-23 16:28:48 -07:00
Ben Gotow
09bb7874f8 Dashboard with a sweet background. Also realtime assignment / policy view. 2016-06-23 15:52:53 -07:00
Evan Morikawa
3f5cac4342 Adding in syncback message actions 2016-06-23 16:47:07 -06:00
Jackie Luo
b6f57f3ce8 Add threading algorithm 2016-06-23 15:44:03 -07:00
Jackie Luo
f9fe836899 Fix syntax error 2016-06-23 15:21:54 -07:00
Jackie Luo
5cc4841ac6 Add check for Gmail thread ID 2016-06-23 15:19:20 -07:00
Evan Morikawa
6ad9cdd322 Fixing broken processors 2016-06-23 14:15:30 -06:00
Evan Morikawa
8160acc81e Add imap dependency to package.json 2016-06-23 13:18:33 -06:00
Evan Morikawa
1460a0ae9f Rename Refresh to Sync 2016-06-23 13:16:15 -06:00
Ben Gotow
8e692982bb Remove concept of self-limiting workers, will use cloudwatch metrics collection to scale fleet instead of queue length 2016-06-23 12:02:57 -07:00
Ben Gotow
f7c647f7ba Return syncs to unclaimed queue after CLAIM_DURATION, just because it's healthy 2016-06-23 11:45:33 -07:00
Ben Gotow
c2e9093b42 Escalate sync policy based on number of stream connections 2016-06-23 11:45:33 -07:00
Jackie Luo
ae54192ed6 Update threading and fix typo 2016-06-23 10:35:47 -07:00
Juan Tejada
a917022505 Update order of threading processor 2016-06-23 10:26:41 -07:00
Ben Gotow
ad1683c9a5 Fix reference to __base 2016-06-23 10:11:07 -07:00
Ben Gotow
12d9db8dd9 Redis coordination of sync processes / assignment 2016-06-23 00:49:26 -07:00
Ben Gotow
2e9bfa68b5 Refactor DeltaStreamQueue (connection per subscription) 2016-06-23 00:49:26 -07:00
Jackie Luo
cb95574378 Fix associations 2016-06-22 17:34:29 -07:00
Jackie Luo
e64c2ae4c0 Add basic threading 2016-06-22 17:34:29 -07:00
Ben Gotow
f5f435236f /auth + remove hardcoded stub in favor of curl request 2016-06-22 17:19:48 -07:00
Evan Morikawa
06393dd07c Move categories & messages to new monorepo structure 2016-06-22 14:44:09 -07:00
Ben Gotow
95fbb64d17 Add /messages, /folders, /account 2016-06-22 14:42:08 -07:00
Evan Morikawa
18f2925b43 Convert to monorepo 2016-06-22 14:41:32 -07:00