Bugzilla

Sorry about the lack of updates here.

Sorry about the lack of updates here.

# Summary of test changes for Puppeteer 13.0.0

## Breakdown per test file

### elementhandle.spec.ts

New tests: 4

PASS: ElementHandle specs Element.waitForSelector should wait correctly with waitForSelector on an element
PASS: ElementHandle specs ElementHandle.isIntersectingViewport should work with threshold
PASS: ElementHandle specs ElementHandle.isIntersectingViewport should work with threshold of 1
PASS: ElementHandle specs Custom queries should wait correctly with waitForSelector on an element


### emulation.spec.ts

New tests: 1

FAIL: Emulation Page.emulateCPUThrottling should change the CPU throttling rate successfully


### frame.spec.ts

New tests: 1

PASS: Frame specs Frame.evaluate allows readonly array to be an argument

Existing tests: 2

PASS -> TIMEOUT: Frame specs Frame.evaluate should throw for detached frames
PASS -> FAIL: Frame specs Frame Management should report different frame instance when frame re-attaches


### jshandle.spec.ts

New tests: 5

PASS: JSHandle JSHandle.getProperty should work
PASS: JSHandle JSHandle.getProperty should return a JSHandle even if the property does not exist
PASS: JSHandle JSHandle.clickablePoint should work
PASS: JSHandle JSHandle.clickablePoint should work for iframes
FAIL: JSHandle JSHandle.click should work

Existing tests: 1

REMOVED: JSHandle Page.evaluateHandle should work with primitives


### launcher.spec.ts

New tests: 4

PASS: Launcher specs Puppeteer Puppeteer.launch should pass the timeout parameter to browser.waitForTarget
PASS: Launcher specs Puppeteer Puppeteer.launch should set the debugging port
PASS: Launcher specs Puppeteer Puppeteer.launch should not allow setting debuggingPort and pipe
PASS: Launcher specs Puppeteer Puppeteer.executablePath returns executablePath for channel

Existing tests: 2

PASS -> FAIL: Launcher specs Puppeteer Puppeteer.connect should support targetFilter option
PASS -> FAIL: Launcher specs Puppeteer Puppeteer.connect should be able to reconnect to a disconnected browser

Note: Both failures started with https://github.com/puppeteer/puppeteer/pull/7722


### navigation.spec.ts

Existing tests: 2

TIMEOUT -> FAIL: navigation Frame.goto should return matching responses
PASS -> TIMEOUT: navigation Frame.waitForNavigation should fail when frame detaches


### network.spec.ts

New tests: 6

FAIL: network Request.initiator shoud return the initiator
TIMEOUT: network Response.buffer should throw if the response does not have a body
PASS: network Response.statusText handles missing status text
PASS: network raw network headers Same-origin set-cookie navigation
PASS: network raw network headers Same-origin set-cookie subresource
FAIL: network raw network headers Cross-origin set-cookie


### page.spec.ts

New tests: 11

FAIL: Page removing and adding event handlers should correctly added and removed request events
FAIL: Page BrowserContext.overridePermissions should grant persistent-storage
PASS: Page Page.waitForNetworkIdle should work
PASS: Page Page.waitForNetworkIdle should respect timeout
PASS: Page Page.waitForNetworkIdle should respect idleTime
PASS: Page Page.waitForNetworkIdle should work with no timeout
FAIL: Page Page.exposeFunction should fallback to default export when passed a module object
FAIL: Page Page.setUserAgent should work with additional userAgentMetdata
PASS: Page Page.addScriptTag should add id when provided
PASS: Page printing to PDF can print to PDF and stream the result
PASS: Page printing to PDF should respect timeout

Existing tests: 1

TIMEOUT -> FAIL: Page Page.close should run beforeunload if asked for

Some tests failed to run because of an earlier test. Missing tests:

MISSING: Page Page.title should return the page title
MISSING: Page Page.select should select single option
MISSING: Page Page.select should select only first option
MISSING: Page Page.select should not throw when select causes navigation
MISSING: Page Page.select should select multiple options
MISSING: Page Page.select should respect event bubbling
MISSING: Page Page.select should throw when element is not a <select>
MISSING: Page Page.select should return [] on no matched values
MISSING: Page Page.select should return an array of matched values
MISSING: Page Page.select should return an array of one element when multiple is not set
MISSING: Page Page.select should return [] on no values
MISSING: Page Page.select should deselect all options when passed no values for a multiple select
MISSING: Page Page.select should deselect all options when passed no values for a select
MISSING: Page Page.select should throw if passed in non-strings
MISSING: Page Page.select should work when re-defining top-level Event class
MISSING: Page Page.Events.Close should work with window.close
MISSING: Page Page.Events.Close should work with page.close
MISSING: Page Page.browser should return the correct browser instance
MISSING: Page Page.browserContext should return the correct browser context instance 


### requestinterception-experimental.spec.ts

New tests: 49

FAIL: All 49 tests of the file

I suggest to keep the describeFailsFirefox for this one, we are not going to accidentally start supporting request interception.


### screenshot.spec.ts

New tests: 1

PASS: Screenshots Page.screenshot should work with webp


### waittask.spec.ts

New tests: 1

PASS: waittask specs Page.waitFor should allow you to select an element with parenthesis-starting xpath

Existing tests: 

PASS -> TIMEOUT: waittask specs Frame.waitForSelector should throw when frame is detached
PASS -> TIMEOUT: waittask specs Frame.waitForXPath should throw when frame is detached


## Analysis

There is a trend of issues in existing tests related to detached frames. Support for OOP iframes was added recently in Puppeteer, this might be related to the breakage. https://github.com/puppeteer/puppeteer/pull/7556

For the new failures in launcher.spec, a new assert was added in https://github.com/puppeteer/puppeteer/pull/7722. It relates to setting the context several times, which for some reason would only happen in Firefox? The following issue has been linked from the code: https://github.com/puppeteer/puppeteer/issues/4197

The situation with page.spec.ts is concerning since many tests seem to fail to even run. Will try a few other pushes to see if that's a consistent behavior or if it's intermittent. But we might have to skip another test in order to make sure all those existing tests can at least run.