Commit graph

4 commits

Author SHA1 Message Date
Ben Gotow 07861f64be perf(db): Use subselect to improve thread list query perf (53%!)
Summary:
- Use a sub-select query with much better performance to display the thread list
- Perform analyze on tables after launch

The new query is:

```
SELECT `Thread`.`data` FROM `Thread` WHERE `Thread`.`id` IN (SELECT `id` FROM `ThreadCategory` AS `M26` WHERE `M26`.`value` IN ('9m9ks71k06n5rmx82kgues09p','9s9k25q6j1krjgpkovbcjm7d','13b7ufruoymvih07ki0uahlto','dtmhlzz6phr47zp512knhjgf8','16dvjb84bszfh15kgfrjj37i3','aclwmgncdqjfibp51bvgbeik','17qad7jhbp6tozog3klm5zagt','4x4bkbawiq825u4eu3aus8tll','7axr9f5f1lzpwm2rw2ghkirhq','dsnn660af0pmou2gg3nstga8a','361qr5rva1ieby2r0ec3sn0bm','10fyvba7pjyjgeyr5i65i1zri') AND `M26`.`in_all_mail` = 1  ORDER BY `M26`.`last_message_received_timestamp` DESC LIMIT 200 OFFSET 0) ORDER BY `Thread`.`last_message_received_timestamp` DESC;
`
0|0|0|SEARCH TABLE Thread USING INDEX Thread_id (id=?)
0|0|0|EXECUTE LIST SUBQUERY 1
1|0|0|SCAN TABLE Thread-Category AS M26 USING COVERING INDEX ThreadFancyIndex
1|0|0|EXECUTE LIST SUBQUERY 2
0|0|0|USE TEMP B-TREE FOR (only on 200 result items)
```

Which is twice as performant as:
```
SELECT `Thread`.`data` FROM `Thread` INNER JOIN `ThreadCategory` AS `M26` ON `M26`.`id` = `Thread`.`id` WHERE `M26`.`value` IN ('9m9ks71k06n5rmx82kgues09p','9s9k25q6j1krjgpkovbcjm7d','13b7ufruoymvih07ki0uahlto','dtmhlzz6phr47zp512knhjgf8','16dvjb84bszfh15kgfrjj37i3','aclwmgncdqjfibp51bvgbeik','17qad7jhbp6tozog3klm5zagt','4x4bkbawiq825u4eu3aus8tll','7axr9f5f1lzpwm2rw2ghkirhq','361qr5rva1ieby2r0ec3sn0bm','10fyvba7pjyjgeyr5i65i1zri') AND `M26`.`in_all_mail` = 1  ORDER BY `M26`.`last_message_received_timestamp` DESC LIMIT 200 OFFSET 0;

0|0|1|SCAN TABLE Thread-Category AS M26 USING COVERING INDEX ThreadFancyIndex
0|0|0|EXECUTE LIST SUBQUERY 1
0|1|0|SEARCH TABLE Thread USING INDEX Thread_id (id=?)
```

Test Plan: Broken!

Reviewers: evan, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D2869
2016-04-11 13:29:05 -07:00
Ben Gotow a2c901559e DatabaseTransaction / QueryBuilder spec fixes 2016-01-25 11:11:57 -08:00
Ben Gotow e1882ab61a feat(counts): Unread counts for all folders and labels across all accounts
Summary:
This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries.

When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO".

I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in.

The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters.

I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily.

The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts.

Test Plan: Tests are a work in progress, want to get feedback first!

Reviewers: juan, evan

Reviewed By: evan

Differential Revision: https://phab.nylas.com/D2232
2015-11-23 17:12:22 -08:00
Evan Morikawa 57fef805cd refactor(spec) move spec-nylas to spec 2015-10-01 21:39:44 -07:00
Renamed from spec-nylas/fixtures/db-test-model.coffee (Browse further)