Summary:
Sometimes you have to perform a query that needs to do a table scan. For those
times you don't want to silence entire classes of queries (e.g. by table name),
you can now silence individual queries.
Test Plan: Run locally, verify that noisy stuff disappears from logs
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3763
Summary:
This diff modifies the SearchIndexer class to handle limiting the search
index size. It does this by periodically re-evaluating the window of
the n most recent items in a particular index where n is the max size of
the index. It then unindexes the items which are marked as indexed but
are no longer in the window and indexes the things that are in the window
but aren't marked as indexed.
Test Plan:
Run locally with a reduced thread index size, verify that the index
includes the most recent items and that it is the correct size. Also verify that
the queries used properly use fast sqlite indices.
Reviewers: evan, juan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3741
Summary:
We had to add the id to the index as well (along with is_search_indexed)
so that the query worked correctly since it sorts by id. Also added a
default value of 0 for boolean attributes so that we don't have to query
for two different values (0 and null), which was causing sqlite to have to
do a final b-tree sort which is slow.
Test Plan: Run locally, verify query plan with EXPLAIN QUERY PLAN
Reviewers: evan, juan
Reviewed By: juan
Differential Revision: https://phab.nylas.com/D3712
Summary:
This diff adds a new SearchIndexer class that each of the concrete
indexer implementations register with. This new class uses the
`isSearchIndexed` field in searchable classes to split indexing work
into chunks. It then times how long it takes to index each chunk and
schedules more work based on a target CPU percent. For example, if it
takes 150 ms to index the last chunk of work and the target CPU fraction
is 0.05 (i.e. 5% CPU use) then it will schedule the next increment of
indexing 3 seconds in the future.
Test Plan: Run locally, verify that indexing occurs in increments
Reviewers: juan, evan
Reviewed By: evan
Differential Revision: https://phab.nylas.com/D3707
