Summary:
There are some settings that apply to dev and prod modes that we don't want
to use while benchmarking. E.g. the folder where we store messages,
whether we generate long stack traces in our bluebird promises, etc.
This diff adds a benchmark mode so that we can change these settings to
something that works better for benchmarking.
Test Plan: Run locally, verify it works
Reviewers: evan, juan, spang
Reviewed By: juan, spang
Differential Revision: https://phab.nylas.com/D4374
Summary: See title
Test Plan: Run on the coffee machine
Reviewers: juan, evan, spang
Reviewed By: spang
Differential Revision: https://phab.nylas.com/D4368
Summary:
We weren't changing into the directory of the git repository, so running
the script from anywhere but inside the git repo didn't work. We also
weren't pulling before checking for the latest commit. Now we do both of
these things.
Test Plan: Run locally
Reviewers: spang, juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D4339
Summary:
We created the process so we can also kill it with no problems. This
makes it easier to automatically run benchmarks without human
intervention.
Test Plan: Run locally
Reviewers: spang, evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D4332
Summary: See title
Test Plan: Run locally on older version of clang
Reviewers: juan, spang, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D4329
Summary:
We want to setup the initial sync benchmark to run automatically with each
commit on a separate machine. This script does just that and dumps the
results to a directory. We can then have a separate script process them
and upload them to whichever service we want.
Test Plan: Run locally
Reviewers: evan, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D4296
Summary:
We want to be able to benchmark initial sync, so this diff adds two
scripts. The first, drop-stuff.sh, drops all data from the app and sync
databases that isn't related to account credentials. The second,
benchmark-initial-sync.sh, runs a fixed number of iterations (current 5) that
invokes drop-stuff.sh then opens the app, waits a fixed amount of time
(currently 120 seconds), and then kills the app and measures how many messages
it synced which it prints to the console. This is sufficient for us to start
measuring how quickly we can sync messages. This diff also includes the
sqlite3 amalgamation which drop-stuff.sh requires to function correctly
due to depending on the FTS5 extension which doesn't come built-in on
some platforms.
Test Plan: Run benchmark locally
Reviewers: evan, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D4275
Summary:
This commit makes it so postinstall.es6 only bootstraps the packages we
are interested in based on a new env variable, `INSTALL_TARGET`.
`INSTALL_TARGET` can be set to `cloud`, `client` or `all`, or not set at all,
and our postinstall will bootstrap the packages accordingly. This will make
build and deploy times faster.
Test Plan: manual
Reviewers: spang, halla, evan, khamidou
Reviewed By: evan, khamidou
Differential Revision: https://phab.nylas.com/D4271
Summary:
We don't need to spend the time running script/bootstrap in our cloud-*
folders!
Test Plan: manual
Reviewers: spang, juan, halla
Reviewed By: juan, halla
Differential Revision: https://phab.nylas.com/D4214
Summary:
This is going to be a diff way shorter than the previous one! Basically, it adds a new endpoint, `/blobs` to our API to store send later attachments. When a user schedules a draft to be sent, we send all attachments to this endpoint. Separately, we store the rest of the message as metadata.
When it's time to send the message, we fetch the attachments from S3, fetch the metadata and merge them together to get a message we can send.
Test Plan: Tested manually. Will make a final QA pass before landing.
Reviewers: juan, halla, evan
Reviewed By: halla, evan
Differential Revision: https://phab.nylas.com/D4196
Summary:
build-client, aka packaging, aka bundling is now separate from uploading.
This is both to compartmentalize our tasks a bit more and so we can add a
non-grunt windows task inbetween packaging and uploading.
No more heavily-overloaded PUBLISH_BUILD flag.
Added SIGN_BUILD flag instead.
No more TRAVIS and TRAVIS_PULL_REQUEST flag.
Test Plan: Manual
Reviewers: halla, juan, spang
Reviewed By: juan, spang
Differential Revision: https://phab.nylas.com/D4208
Summary:
Generating the daily build always takes a bit of effort, so I wrote the
script to automate the process and allow anyone to do it as well.
This script will:
- Check your current working directory is in a good state
- Bump the version number in package.json
- Edit CHANGELOG.md with the git log diff between current master and the last version tag (you can also manually edit it). This is so writing the CHANGELOG before a release takes less effort; you will just need to open the CHANGELOG file and edit the commit messages.
- Commit the changes with a commit message of: bump(version) <next_version>
- Add new version tags to nylas-mail and k2
- Push the changes to Github
- If provided with the flag, it will build the app. This is intended to be used in our mac mini machine
There are a few TODO's left in here, like updating the daily channel,
and emailing a notification when the channel is updated with the buid urls.
When this lands, we will configure our mac mini to run this script daily/nightly.
Also, any engineer should be able to trigger a daily build easily by just
running this script.
For sample output, see this commit which was generated by this script:
000fa88ebb
Test Plan: manually running it
Reviewers: halla, spang, mark, khamidou, evan
Reviewed By: evan
Subscribers: tomasz
Differential Revision: https://phab.nylas.com/D4123
Summary:
Given that `lerna bootstrap` does not install optional dependencies, we
need to manually run `npm install` inside `client-app` so
`node-mac-notifier` get's correctly installed and included in the build
See https://github.com/lerna/lerna/issues/121
Fixes T7887
Test Plan: manual
Reviewers: halla, evan
Reviewed By: evan
Maniphest Tasks: T7887
Differential Revision: https://phab.nylas.com/D4076
Summary:
Put them in the client-app/spec folder instead of
client-app/internal_packages
Test Plan: Ran the tests
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D4080
Summary:
Move the base Jasmine spec runner into isomorphic-core to prevent
code duplication. Jasmine will look for the config file relative to
the directory it's being run in though, so we need to symlink the
config file into each package that will need it.
Test Plan: Run tests once the suites are integrated
Reviewers: evan, spang, juan
Reviewed By: spang, juan
Differential Revision: https://phab.nylas.com/D4056
Summary:
Convert the isomorphic-core specs to Jasmine 1 and symlink them into
client-app/internal_packages so they are run as part of the client
test suite.
Test Plan: Ran the suite with fdescribes in iso-core to ensure they were being called properly
Reviewers: evan, spang, juan
Reviewed By: spang, juan
Differential Revision: https://phab.nylas.com/D4055
Summary:
Now that .babelrc is in client-app, we need to make it an exception in the
.dockerignore file. Additionally, this means that the client-app directory will
exist during the postinstall script in the cloud environment, so we need a
stricter check to figure out if the client-app needs to be built or not.
Test Plan: deployed to staging
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D4008
Partially revert 3071856a8e
Symlinking instead of copying the error-logger-extensions prevents the
app from finding the appropriate dependencies required in the error
logger extensions in the main process and in the renderer processes
because globalPaths are only set in renderer processes and /after/
error-logger-extensions are loaded.
We should fix this properly in a separate diff, and move the error
logger to isomorphic-core instead of copying it or symlinking it
Summary:
The prod build was failing because it couldn't find the .babelrc.
I decided to put the babelrc in the main nylas-mail repo with the
expectation that it'll need to be in the open source version too. To DRY
up this for us building, we sylink the root one to the client-app babelrc
Also since we now dereference symlinks we don't need to do the full copy
of nylas-private-resources so we don't have two error reporters floating
around
Test Plan: npm run build-client
Reviewers: juan, spang
Reviewed By: juan, spang
Differential Revision: https://phab.nylas.com/D3993
Summary:
We weren't actually running npm install inside of apm. This meant that
/apm/node_modules was empty causing the build to fail. This will install
with the proper env required for old apm
Wow we need to get rid of this thing soon…
Test Plan: Run `npm install`
Reviewers: jerm, spang, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3986
[*] update babel
[client-app] remove flow-typed
[client-app] Move build/package.json to main package.json
[client-app] remove spec_integration
[client-app] fix babel support
Add client-private-plugins package.json
[client-app] add node_modules to global path for private-plugins
Move client-sync dependencies to client-app root
fix electron rebuild
[*] moved to monorepo
Summary: App now runs in monorepo
Test Plan: npm test
Reviewers: juan, mark, khamidou, halla, spang
Differential Revision: https://phab.nylas.com/D3947
Summary:
This diff adds support for database migration to our cloud API. It's partially inspired by Halla's local-sync migration diff (D3809). You can run a migration by calling "node-babel scripts/migrate-db up|down" or by calling "npm script upgrade-db|downgrade-db".
Note that for simplicity reasons we assume that we're only writing migrations for our MySQL database – people developing locally may have to blow up there dbs whenever there's a schema change, though in practice `ALTER TABLE ADD COLUMN`statements work the same on both dbs.
Test Plan: Tested locally. Will run the metadata migration on staging.
Reviewers: evan, spang, halla
Reviewed By: halla
Differential Revision: https://phab.nylas.com/D3840
Summary:
I've decided to break my snooze patches in multiple parts to make it easier to review. This diff contains all the code related to running workers in the cloud. Cloud workers all inherit from the `Worker` class which defines a bunch of useful things like error handling.
What's left to do:
- spawn workers based on the plugin type
- add monitoring (I'm going to add a simple HTTP endpoint for that)
- writing a migration for the local sync db and the prod metadata db.
Test Plan: Tested manually.
Reviewers: halla, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3814
We added new presets and plugins to the k2 .babelrc but didn't add the
respective dependencies in our package.json, so the build was failing.
Add the required dependencies, and make sure that `build-n1-cloud.js`
uses the correct babel presets and plugins
Summary:
Adds babel to K2
Creates a simple build script so it'll run on prod.
Test Plan: manual
Reviewers: jackie, halla, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3527
