1442374 - Create XCUITest that loads 500+ sites for SQL performance measurement on nimbledroid

Status: RESOLVED FIXED

(Firefox for iOS :: Build & Test, enhancement, P2)

Description

[No-Jun Park [:njpark]]

We need to create a test that loads browser.db with:
- 500+ (probably from Alexa sites list) site / favicon URL in history
- We can visit first link of each page if we want to increase the site count to 1000+
and measure the time for SQL transaction, (more specifically, https://bugzilla.mozilla.org/attachment.cgi?id=8954215&action=edit)

In order to do this, we need to:

1. Create a browser.db with populated data, and put it in test-fixtures folder. (The test launch arguments takes an argument for the name of the test-fixtures db to load for your test.)
2. Add nimbledroid bookend surrounding above method
3. Load the browser db when a specific XCUITest starts
4. The test will invoke the method in 2.

Comment 1

[No-Jun Park [:njpark]]

So the approach to collect database is to create an XCUITest that visits 500 sites, and save the db.  The script would most likely to read off the list of websites from a .csv or .json file, and put on a loop to visit each site.  When finished, we'll retrieve the browser.db from the sim folder and save it.

Isabel, can you look into creating this test?  I can point you to the .csv file containing the sites.

Comment 2

[Isabel Rios[:isabel_rios]]

Sure, I can try to get this working. Yes, please, point me to that .csv file

Comment 3

[Isabel Rios[:isabel_rios]]

Attachment: Pull Request

Comment 4

[Isabel Rios[:isabel_rios]]

In the PR I added a DB that has 378 sites loaded. I'm trying to get a bigger one, at least to have all the sites in the csv file but the test fails when loading some heavy sites and so far this is the bigger DB I got.

Also, although the DB has more than 100 entries it looks like there is a limitation in number of entries in the history list to be shown. There are always 100 no matter how big the DB is, is that right? Not sure who is the best person to ask, maybe Garvan? (since the next question would be for you...)

In oder to have significant and useful data measured by Nimbledroid, is that ok with this limitation although the DB is bigger or should we allow to load all the entries?

Comment 5

[Isabel Rios[:isabel_rios]]

ni Justin too as DB expert :) to know your opinion about having a bigger DB although only a limited number is shown in the list, would the DB size affect the performance we could measure? Would it make a difference having 500 or 1000.. Thinking on creating a bigger one as commented on description

Comment 6

[:garvan]

Yes, always 100 limit on the history query. If DB query perf is being tested, many more entries are needed as filtering the history to 100 entries has a time cost.

Comment 7

[Justin D'Arcangelo [:justindarc]]

I concur with Garvan's comments above. Probably 500 entries would be good enough to get a decent load test. Even though the History query is limited to 100 items, having more in the DB still puts a load on that query.

Comment 8

[Isabel Rios[:isabel_rios]]

Thanks for your replies! Will update the PR with a bigger DB, got one with around 700 entries. If you think it is a good idea we can keep some DBs, like high load, medium load or simlar...I have different DBs stored locally (100, 300, 500 entries) in case they are needed.

Comment 9

[No-Jun Park [:njpark]]

Comment on attachment 8958764 [details] [review]
Pull Request

Let's mark this script disabled by default - otherwise the script will run as part of the daily XCUITest run, taking a lot of time.

Comment 10

[Isabel Rios[:isabel_rios]]

Just to clarify, this test does not generate the DB, it only checks that the DB can be preloaded and is accessible. It only loads it and checks that 100 entries are shown in the History.

Comment 11

[Isabel Rios[:isabel_rios]]

Merged on master: https://github.com/mozilla-mobile/firefox-ios/commit/66947b6fe8f6a57a90724696f05de14795b5a8db