Summary:
This diff removes significant cruft from N1's compilation and build tooling:
- Electron-Packager replaces most of the code in build/tasks/* used to copy things,
bundle things, download electron, etc.
- script/bootstrap has been replaced with a much simpler script that does not use
APM, does not download Electron (we just use electron as an NPM dep) and does
not manully compile sqlite. It requires NPMv3, but I think that's safe.
- babel and eslint are now devDependencies of the main project. The main project
also supports optionalDependencies now.
- npm test and npm start replace ./N1.sh
- APM is still around, and is only put into N1 so it can install plugins at runtime.
It should be removed as soon as we notify package maintainers and have them provide zips.
- N1 no longer has it's own compile-cache or babel/typescript/coffeescript compilers.
It delegates to electron-compile and electron-compilers. Both of these packages had
to be forked and modified slightly, but I'm hopeful the modifications will make it back
in to the projects and you can still consult their documentation for more info.
+ In the near future, I think we should stop shipping electron-compilers with N1. This
would mean that all plugins would need to be compiled on pre-publish, just like NPM
packages, and would complicate the local development story a bit, but would make the
app smaller. electron-compile is not supposed to compile at runtime in the prod app,
just pull from the compile cache.
- I've re-organized Grunt according to Grunt best practices, where each tasks/* file
specifies it's own config and imports grunt tasks.
- Unfortunately, I was not able to use any open source projects for the deb and rpm builds,
because we have things like postinst hooks and start menu items which are not supported
by the electron installer-generators.
WIP
Turn off all LESS compilation, because themes. Doh.
Use Grunt for new build process too, just remove tasks
More changes
Add babel-eslint
Remove unused react-devtools
WIP
Add name
Ignore nonexistent
Switch to more modern approach to config for grunt
Move zipping to mac installer task
Restructure publish task so it aggregates first, can log useful info if publishing is disabled
Fix build dirs
Fix win installer
Fix linux installer
Fix linux installer
Try making linux
A few more
Updates
Upadtes
fixes
fixes
Get rid of non-meaningful variables
Resolve assets path
Insert nylas.sh
Clean up args more
Actually use description
Fix display name ugh
More tweaks
Expliclty write /usr/bin/nylas
Improve vars
Use old nylas.sh
Reinstate APM to better scope this diff
Test Plan: Test on Mac, Windows, Linux
Reviewers: evan, jackie, juan
Reviewed By: jackie, juan
Differential Revision: https://phab.nylas.com/D3411
node-mac-notifier build process is currently breaking. This will be
fixed when we revamp the build process, so for now we are punting the
node-mac-notifier feature
Summary:
Better-SQLite3 is a fork of node-sqlite3 which includes a re-written JavaScript interface. It’s more synchronous, but better reflects what is actually sync vs. async in sqlite’s C++ API. (Not much is really async under the hood.) This diff uses a branch of better-sqlite3 I’ve edited to support Node 6.
In my tests, this branch spends 3.24x less time executing queries than `master`. (Measured time spent in calls to `this._db[run|all|get]` over the first 5000 queries of initial sync. It also increased the performance of starring a thread in the thread list by 28%.
This library also allows us to use a prepared statement cache, which is great because we often make the same queries repeatedly as query subscriptions refresh the UI and deltas are dumped into the app. The old interface didn’t expose statements (cached query plans) to JS.
better-sqlite3 advertises that it uses the JS garbage collector instead of lower level C++ memory management. I tested syncing my entire mailbox to verify that memory usage is not significantly different on this branch after a lot of queries have been made.
Finally, it looks like we can finally stop building sqlite3 from scratch in `script/bootstrap`. This library builds properly with `apm install`. 🎉
We might want to change the DatabaseStore and DatabaseTransaction classes more, now that it’s possible to execute queries synchronously. It could make things cleaner and get us out of promise-hell in a few places. In this diff I tried to change as little as possible.
Test Plan: Run tests, everything still works
Reviewers: juan, jackie
Reviewed By: juan, jackie
Differential Revision: https://phab.nylas.com/D3315
Summary:
This diff includes several updates:
- Update sqlite version to use FTS5
- Adds new methods to DatabaseStore to create and update Search Indexes
- Currently indexing subject, participants, and thread message bodies or snippets if the body is not available
- Update initial sync to fetch 5000 message bodies per account
- Adds a new SearchMatcher
- Add new thread-search-index package to run in the worker window to
init and keep thread search index up to date
- Converts Thread to ES6
TODO:
- Remove/Update suggestions dropdown
- Add tests
Test Plan: - TODO
Reviewers: evan, bengotow
Reviewed By: evan, bengotow
Differential Revision: https://phab.nylas.com/D2826
Summary:
Until now, we've been hiding transactions beneath the surface. When you call persistModel, you're implicitly creating a transaction.
You could explicitly create them with `atomically`..., but there were several critical problems that are fixed in this diff:
- Calling persistModel / unpersistModel within a transaction could cause the DatabaseStore to trigger. This could result in other parts of the app making queries /during/
the transaction, potentially before the COMMIT occurred and saved the changes. The new, explicit inTransaction syntax holds all changes until after COMMIT and then triggers.
- Calling atomically and then calling persistModel inside that resulted in us having to check whether a transaction was present and was gross.
- Many parts of the code ran extensive logic inside a promise chained within `atomically`:
BAD:
```
DatabaseStore.atomically =>
DatabaseStore.persistModel(draft) =>
GoMakeANetworkRequestThatReturnsAPromise
```
OVERWHELMINGLY BETTER:
```
DatabaseStore.inTransaction (t) =>
t.persistModel(draft)
.then =>
GoMakeANetworkRequestThatReturnsAPromise
```
Having explicit transactions also puts us on equal footing with Sequelize and other ORMs. Note that you /have/ to call DatabaseStore.inTransaction (t) =>. There is no other way to access the methods that let you alter the database. :-)
Other changes:
- This diff removes Message.labels and the Message-Labels table. We weren't using Message-level labels anywhere, and the table could grow very large.
- This diff changes the page size during initial sync from 250 => 200 in an effort to make transactions a bit faster.
Test Plan: Run tests!
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D2353
comment
Adding test harness
Using key strokes in main window test
Tests work now
Clean up argument variables
Rename list manager and get rid of old spec-helper methods
Extract out time overrides from spec-helper
Spectron test for contenteditable
fix spec exit codes and boot mode
fix(spec): cleanup N1.sh and make specs fail with exit code 1
Revert tests and get it working in window
Move to spec_integration and add window load tester
Specs pass. Console logs still in
Remove console logs
Extract N1 Launcher ready method
Make integrated unit test runner
feat(tests): adding integration tests
Summary:
The /spectron folder got moved to /spec_integration
There are now unit tests (the old ones) run via the renamed
`script/grunt run-unit-tests`
There are now integration tests run via the command `script/grunt
run-integration-tests`.
There are two types of integration tests:
1. Tests that operate on the whole app via Selenium/Chromedriver. These
tests have access to Spectron APIs but do NOT have access to any JS object
running inside the application. See the `app-boot-spec.es6` for an example
of these tests. This is tricky because we want to test the Main window,
but Spectron may latch onto any other of our loading windows. Code in
`integration-helper` give us an API that finds and loads the main window
so we can test it
2. Tests that run in the unit test suite that need Spectron to perform
integration-like behavior. These are the contentedtiable specs. The
Spectron server is accessed from the app and can be used to trigger
actions on the running app, from the app. These tests use the
windowed-test runner so Spectron can identify whether the tests have
completed, passed, or failed. Unfortunately Spectron can't access the logs
, nor the exit code of the test script thereby forcing us to parse the
HTML DOM. (Note this is still a WIP)
I also revamped the `N1.sh` file when getting the launch arguments to work
properly. It's much cleaner. We didn't need most of the data.
Test Plan: new tests
Reviewers: juan, bengotow
Differential Revision: https://phab.nylas.com/D2289
Fix composer specs
Tests can properly detect when Spectron is in the environment
Report plain text output in specs
fixing contenteditable specs
Testing slow keymaps on contenteditable specs
Move to DOm mutation
Spell as `subtree` not `subTree`
- Upgrade node-sqlite3 from patched 3.0.2 to mainline 3.1.0
- Upgrade Electron to 0.30.8: NOTE: This is the latest in the 0.30 line.
The 0.31+ line did a Chrome upgrade that we want to save for a later
day. We've had a history of serious instabilities with these upgrades
and are doing them conservatively
- Upgrade coffee-script from 1.9.0 to 1.10.0
- Package updates in /build folder
node-gyp header layout change in node v4, causes package builds to fail.
This change forces node-gyp to download the correct version of node so
native extensions can be built.
Summary:
- We now build sqlite3 manually from source in script/bootstrap
- We now allow queries to run in parallel outside of transaction blocks
- When signining in and out, the main window creates the database file and then advances the database "phase", which allows all the windows to connect to the initialized database.
This diff also fixes T2411 where popout drafts opened twice, and several issues around Windows icons and install.
Test Plan: Run existing tests
Reviewers: evan
Reviewed By: evan
Maniphest Tasks: T2411
Differential Revision: https://phab.nylas.com/D1815
