💌 A beautiful, fast and fully open source mail client for Mac, Windows and Linux.
Find a file
Juan Tejada 83ef8c12b3 [local-sync] Restore global queue for message processing to improve perf
Summary:
Sync operations are mostly bound by I/O and the imap connection.
What we believe that is mostly affecting cpu and battery life is that node’s event
loop is being hosed with cpu intensive message processing operations.

To alleviate this, we do a few things:

- Restore a global message processing queue to process messages serially and meter cpu usage (message processing continues to be a fire and forget call from within sync operations)
- Move actual cpu intensive work to the message processing queue, i.e. `MessageFactory.parseFromImap`
- Keep track of message processing queue length, and skip sync operations if queue is too big to prevent massive memory consumption

This commit also renames the package from new-message-processor to
message-processor, given that now it processes both new and existing
messages, and we like to minimize confusion.

Test Plan: manual

Reviewers: spang, khamidou, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D3602
2017-01-06 14:28:33 -08:00
.ebextensions Fixes to Docker and better docs/debugging 2017-01-03 09:32:08 -08:00
arclib [local-private] move old edgehill src/pro into packages/local-private 2016-12-12 10:09:58 -05:00
packages [local-sync] Restore global queue for message processing to improve perf 2017-01-06 14:28:33 -08:00
scripts [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
.arcconfig [local-private] move old edgehill src/pro into packages/local-private 2016-12-12 10:09:58 -05:00
.arclint [local-private] move old edgehill src/pro into packages/local-private 2016-12-12 10:09:58 -05:00
.babelrc [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
.dockerignore Fixes to Docker and better docs/debugging 2017-01-03 09:32:08 -08:00
.eslintrc Fix linter issues 2016-11-21 14:00:35 -08:00
.gitignore [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
.tern-project Enable nodejs completion with tern 2016-12-07 07:25:48 -08:00
Dockerfile [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
lerna.json [*] update lerna to 2.0.0-beta.30 2016-12-06 16:03:31 -08:00
package.json [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
pm2-dev.yml [cloud-api] refactor cloud API routes to use es6 & fix Gmail Auth 2016-12-19 09:25:07 -08:00
pm2-prod-api.yml [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
pm2-prod-workers.yml [*] Add basic babel toolchain 2016-12-16 13:08:21 -05:00
README.md Readme updates 2016-11-28 14:13:23 -08:00
test_accounts.txt First step to getting persistent id in K2 2016-12-01 11:15:13 -08:00

K2 - Local Sync Engine & Cloud Services for Nylas N1

This is a collection of all sync and cloud components required to run N1.

  1. Cloud API: The cloud-based auth and metadata APIs for N1
  2. Cloud Core: Shared code used in all remote cloud services
  3. Cloud Workers: Cloud workers for services like send later
  4. Isomorphic Core: Shared code across local client and cloud servers
  5. Local Sync: The local mailsync engine integreated in Nylas N1

See /packages for the separate pieces. Each folder in /packages is designed to be its own stand-alone repositoriy. They are all bundled here for the ease of source control management.

Initial Setup for All Local & Cloud Services:

New Computer (Mac):

  1. Install Homebrew
  2. Install NVM brew install nvm
  3. Install Node 6 via NVM: nvm install 6
  4. Install Redis locally brew install redis

New Computer (Linux - Debian/Ubuntu):

  1. Install Node 6+ via NodeSource (trusted):
  2. curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
  3. sudo apt-get install -y nodejs
  4. Install Redis locally sudo apt-get install -y redis-server redis-tools benefit of letting us use subdomains.