Commit graph

697 commits

Author SHA1 Message Date
Evan Morikawa
1460a0ae9f Rename Refresh to Sync 2016-06-23 13:16:15 -06:00
Ben Gotow
8e692982bb Remove concept of self-limiting workers, will use cloudwatch metrics collection to scale fleet instead of queue length 2016-06-23 12:02:57 -07:00
Ben Gotow
f7c647f7ba Return syncs to unclaimed queue after CLAIM_DURATION, just because it's healthy 2016-06-23 11:45:33 -07:00
Ben Gotow
c2e9093b42 Escalate sync policy based on number of stream connections 2016-06-23 11:45:33 -07:00
Jackie Luo
ae54192ed6 Update threading and fix typo 2016-06-23 10:35:47 -07:00
Juan Tejada
a917022505 Update order of threading processor 2016-06-23 10:26:41 -07:00
Ben Gotow
ad1683c9a5 Fix reference to __base 2016-06-23 10:11:07 -07:00
Ben Gotow
12d9db8dd9 Redis coordination of sync processes / assignment 2016-06-23 00:49:26 -07:00
Ben Gotow
2e9bfa68b5 Refactor DeltaStreamQueue (connection per subscription) 2016-06-23 00:49:26 -07:00
Jackie Luo
cb95574378 Fix associations 2016-06-22 17:34:29 -07:00
Jackie Luo
e64c2ae4c0 Add basic threading 2016-06-22 17:34:29 -07:00
Ben Gotow
f5f435236f /auth + remove hardcoded stub in favor of curl request 2016-06-22 17:19:48 -07:00
Evan Morikawa
d5d019f9d9 Plain npm install 2016-06-22 14:53:23 -07:00
Evan Morikawa
71167328dd Update readme 2016-06-22 14:49:36 -07:00
Evan Morikawa
06393dd07c Move categories & messages to new monorepo structure 2016-06-22 14:44:09 -07:00
Ben Gotow
95fbb64d17 Add /messages, /folders, /account 2016-06-22 14:42:08 -07:00
Evan Morikawa
18f2925b43 Convert to monorepo 2016-06-22 14:41:32 -07:00
Evan Morikawa
2c5da59d74 Properly pluck out transactionModel 2016-06-22 13:57:51 -07:00
Evan Morikawa
02f0788232 Improving observable based delta stream 2016-06-22 13:35:55 -07:00
Evan Morikawa
54599a21c0 Ignore storage folder 2016-06-22 12:19:17 -07:00
Evan Morikawa
135f16f8d6 Add initial observable based delta stream endpoint 2016-06-22 12:16:41 -07:00
Juan Tejada
5bf63f8ea6 Fix message processor pipeline initialization
- Was not actually returning an array of functions from
`processors/index.js`
2016-06-22 11:57:24 -07:00
Juan Tejada
8172cad622 Fix syntax error 2016-06-22 11:01:51 -07:00
Juan Tejada
c3ed7cbdef Adds message parsing module + other updates
- Adds an order for the message processing pipeline to ensure that
parsing occurs first
- Adds JSONARRAYType
- Other misc updates
2016-06-22 11:00:35 -07:00
Ben Gotow
19de776e00 Encrypt connction credentials, move conn settings to acct 2016-06-21 18:37:00 -07:00
Ben Gotow
b7cd644a83 More quietly update category sync state 2016-06-21 18:30:07 -07:00
Evan Morikawa
647dc1f78c Default to port 5100 2016-06-21 17:51:47 -07:00
Evan Morikawa
e75d0ea16b Add procfile.dev with external redis launcher 2016-06-21 17:45:25 -07:00
Ben Gotow
644af22d40 Fix merge conflicts 2016-06-21 17:25:25 -07:00
Juan Tejada
2170ee4143 WIP message-processor: building the message 2016-06-21 17:18:32 -07:00
Evan Morikawa
b4e05fcb30 Bind to proper port 2016-06-21 17:17:59 -07:00
Ben Gotow
05795692f1 More sync improvements, use CONDSTORE, gmail coming soon 2016-06-21 17:15:36 -07:00
Evan Morikawa
02fbd6e625 Add core as package dependency 2016-06-21 17:13:48 -07:00
Evan Morikawa
7602620742 Add Procfile and heroku readiness 2016-06-21 17:11:31 -07:00
Evan Morikawa
9dad99801c Add params 2016-06-21 16:32:11 -07:00
Evan Morikawa
fa4ed23621 Put single token in shared db 2016-06-21 16:11:47 -07:00
Evan Morikawa
2d4b17ee52 Add a delta streaming endpoint 2016-06-21 15:57:50 -07:00
Ben Gotow
29a6448922 Initial concept of sync policy 2016-06-20 17:33:27 -07:00
Evan Morikawa
f995f6fda1 Add delta stream queue on sync side 2016-06-20 17:28:26 -07:00
Evan Morikawa
cebce1081f Fix error & and turn on transaction logging 2016-06-20 16:23:09 -07:00
Evan Morikawa
8ed94442c4 Add transaction logging 2016-06-20 16:03:25 -07:00
Juan Tejada
83dfb664e1 Add initial version of message-processor package 2016-06-20 15:03:02 -07:00
Ben Gotow
de8e09d6b5 Unify operations, fix a few bugs, logging issues 2016-06-20 14:44:07 -07:00
Evan Morikawa
6577bd9358 Update readme 2016-06-20 13:21:10 -07:00
Ben Gotow
3a677a8c2d Create README.md 2016-06-20 12:19:01 -07:00
Ben Gotow
3dc03688e8 Cleaner sync state machine 2016-06-20 00:19:16 -07:00
Ben Gotow
25270c0b75 Initial commit 2016-06-19 03:02:32 -07:00