Summary: This will address the longstanding concern in #417
Test Plan: No new tests
Reviewers: juan, evan
Reviewed By: juan, evan
Maniphest Tasks: T7065
Differential Revision: https://phab.nylas.com/D3322
Summary:
Better-SQLite3 is a fork of node-sqlite3 which includes a re-written JavaScript interface. It’s more synchronous, but better reflects what is actually sync vs. async in sqlite’s C++ API. (Not much is really async under the hood.) This diff uses a branch of better-sqlite3 I’ve edited to support Node 6.
In my tests, this branch spends 3.24x less time executing queries than `master`. (Measured time spent in calls to `this._db[run|all|get]` over the first 5000 queries of initial sync. It also increased the performance of starring a thread in the thread list by 28%.
This library also allows us to use a prepared statement cache, which is great because we often make the same queries repeatedly as query subscriptions refresh the UI and deltas are dumped into the app. The old interface didn’t expose statements (cached query plans) to JS.
better-sqlite3 advertises that it uses the JS garbage collector instead of lower level C++ memory management. I tested syncing my entire mailbox to verify that memory usage is not significantly different on this branch after a lot of queries have been made.
Finally, it looks like we can finally stop building sqlite3 from scratch in `script/bootstrap`. This library builds properly with `apm install`. 🎉
We might want to change the DatabaseStore and DatabaseTransaction classes more, now that it’s possible to execute queries synchronously. It could make things cleaner and get us out of promise-hell in a few places. In this diff I tried to change as little as possible.
Test Plan: Run tests, everything still works
Reviewers: juan, jackie
Reviewed By: juan, jackie
Differential Revision: https://phab.nylas.com/D3315
The clean script fails since productName is "Nylas N1" which contains a
space and it appears to pkill 2 patterns. This is causing the script to
stop in my linux machine. Now the productName is wrapped with quotes to
prevent this from happening.
Summary:
This diff includes several updates:
- Update sqlite version to use FTS5
- Adds new methods to DatabaseStore to create and update Search Indexes
- Currently indexing subject, participants, and thread message bodies or snippets if the body is not available
- Update initial sync to fetch 5000 message bodies per account
- Adds a new SearchMatcher
- Add new thread-search-index package to run in the worker window to
init and keep thread search index up to date
- Converts Thread to ES6
TODO:
- Remove/Update suggestions dropdown
- Add tests
Test Plan: - TODO
Reviewers: evan, bengotow
Reviewed By: evan, bengotow
Differential Revision: https://phab.nylas.com/D2826
Summary:
Until now, we've been hiding transactions beneath the surface. When you call persistModel, you're implicitly creating a transaction.
You could explicitly create them with `atomically`..., but there were several critical problems that are fixed in this diff:
- Calling persistModel / unpersistModel within a transaction could cause the DatabaseStore to trigger. This could result in other parts of the app making queries /during/
the transaction, potentially before the COMMIT occurred and saved the changes. The new, explicit inTransaction syntax holds all changes until after COMMIT and then triggers.
- Calling atomically and then calling persistModel inside that resulted in us having to check whether a transaction was present and was gross.
- Many parts of the code ran extensive logic inside a promise chained within `atomically`:
BAD:
```
DatabaseStore.atomically =>
DatabaseStore.persistModel(draft) =>
GoMakeANetworkRequestThatReturnsAPromise
```
OVERWHELMINGLY BETTER:
```
DatabaseStore.inTransaction (t) =>
t.persistModel(draft)
.then =>
GoMakeANetworkRequestThatReturnsAPromise
```
Having explicit transactions also puts us on equal footing with Sequelize and other ORMs. Note that you /have/ to call DatabaseStore.inTransaction (t) =>. There is no other way to access the methods that let you alter the database. :-)
Other changes:
- This diff removes Message.labels and the Message-Labels table. We weren't using Message-level labels anywhere, and the table could grow very large.
- This diff changes the page size during initial sync from 250 => 200 in an effort to make transactions a bit faster.
Test Plan: Run tests!
Reviewers: juan, evan
Reviewed By: juan, evan
Differential Revision: https://phab.nylas.com/D2353
comment
Adding test harness
Using key strokes in main window test
Tests work now
Clean up argument variables
Rename list manager and get rid of old spec-helper methods
Extract out time overrides from spec-helper
Spectron test for contenteditable
fix spec exit codes and boot mode
fix(spec): cleanup N1.sh and make specs fail with exit code 1
Revert tests and get it working in window
Move to spec_integration and add window load tester
Specs pass. Console logs still in
Remove console logs
Extract N1 Launcher ready method
Make integrated unit test runner
feat(tests): adding integration tests
Summary:
The /spectron folder got moved to /spec_integration
There are now unit tests (the old ones) run via the renamed
`script/grunt run-unit-tests`
There are now integration tests run via the command `script/grunt
run-integration-tests`.
There are two types of integration tests:
1. Tests that operate on the whole app via Selenium/Chromedriver. These
tests have access to Spectron APIs but do NOT have access to any JS object
running inside the application. See the `app-boot-spec.es6` for an example
of these tests. This is tricky because we want to test the Main window,
but Spectron may latch onto any other of our loading windows. Code in
`integration-helper` give us an API that finds and loads the main window
so we can test it
2. Tests that run in the unit test suite that need Spectron to perform
integration-like behavior. These are the contentedtiable specs. The
Spectron server is accessed from the app and can be used to trigger
actions on the running app, from the app. These tests use the
windowed-test runner so Spectron can identify whether the tests have
completed, passed, or failed. Unfortunately Spectron can't access the logs
, nor the exit code of the test script thereby forcing us to parse the
HTML DOM. (Note this is still a WIP)
I also revamped the `N1.sh` file when getting the launch arguments to work
properly. It's much cleaner. We didn't need most of the data.
Test Plan: new tests
Reviewers: juan, bengotow
Differential Revision: https://phab.nylas.com/D2289
Fix composer specs
Tests can properly detect when Spectron is in the environment
Report plain text output in specs
fixing contenteditable specs
Testing slow keymaps on contenteditable specs
Move to DOm mutation
Spell as `subtree` not `subTree`
