Mailspring/internal_packages/search-index
Ben Gotow d53edbffea feat(db): background flag to exec queries in a single forked node process
Summary:
This is an initial attempt to fix an issue we’ve had with long-running queries interrupting the N1 user experience. Node-sqlite3 used an async approach that ran sqlite’s synchronous query methods on a worker thread, but doing that involves copying memory more and node-sqlite3 was just generally slow.

However, moving to better-sqlite3 made /everything/ synchronous. Even with the right indexes some of our queries just suck.

This diff adds `DatabaseStore.findAll(…).background().then()` which allows you to mark a query as “unimportant”. These queries are run in a separate process which is forked from the window and can take an extra 10-50ms to complete. That said, they’re totally async and don’t jam up the app.

I’m personally a fan of the flag and less a fan of the implementation. The “agent” process can handle many queries in it’s lifetime if they keep coming and quits after 10 seconds of inactivity. (Both to save memory and to avoid scenarios where it might end up oprhaned and running forever). While running it uses about 40MB of RAM, which is a bit on the crazy side.

Test Plan: No new tests yet

Reviewers: evan, juan

Reviewed By: evan, juan

Differential Revision: https://phab.nylas.com/D3420
2016-11-16 17:47:24 -08:00
..
lib feat(db): background flag to exec queries in a single forked node process 2016-11-16 17:47:24 -08:00
package.json deps(*): Lift deps out of packages for simpler install / better deduping 2016-11-09 14:24:10 -08:00