860965 - PJS: Move 1D ParallelArray operations to Array

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Shu-yu Guo [:shu]]

This is part of a bigger plan to split the current ParallelArray API into two: a new Matrix (ParallelMatrix)? constructor for higher-dimensioned computations, and migrating the existing 1D operations onto Array.prototype.

Pros:
  - Allows interleaving of parallel and sequential computation without changing data representation as ParallelArray would require.
  - Smaller API surface; more familiar ergonomics.
  - Lets us leverage existing engine optimizations for arrays without duplication. With ParallelArray being a different constructor, there'd be another []-path to optimize in all the JITs.

We do the following migration:

ParallelArray.prototype.map -> Array.prototype.pmap
ParallelArray.prototype.reduce -> Array.prototype.preduce
ParallelArray.prototype.scan -> Array.prototype.pscan
ParallelArray.prototype.scatter -> Array.prototype.pscatter
ParallelArray.prototype.filter -> Array.prototype.pfilter

Comment 1

[Shu-yu Guo [:shu]]

Attachment: WIP

The current WIP patch does not remove ParallelArray. The plan is such that until Matrix is ready, ParallelArray will remain. It refactors the following methods: map, reduce, scan, scatter, filter and shares the implementation of those between Array and ParallelArray.

The comprehension form is not refactored.

Comment 2

[Shu-yu Guo [:shu]]

Comment on attachment 736535 [details] [diff] [review]
WIP

Carrying f+ from (presumably) accidental r+ from nmatsakis.

r? module owner for adding to Array.prototype.

Comment 3

[Luke Wagner [:luke]]

Is this all behind an (about:config) pref?

Comment 4

[Shu-yu Guo [:shu]]

Forgot to mention an important point: currently all PJS is behind ENABLE_PARALLEL_JS, which is only on for Nightly and Aurora, though we are thinking of turning it off even for Aurora.

Comment 5

[Luke Wagner [:luke]]

Comment on attachment 736535 [details] [diff] [review]
WIP

Ah, I see.  Seems low-risk to inject these names then.

Comment 6

[Shu-yu Guo [:shu]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/47e92fbb6153

try: https://tbpl.mozilla.org/?tree=Try&rev=c43e81733706

Comment 7

[Ed Morley [:emorley]]

Backed out for Win8 xpcshell assertions (virtually permaorange):
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=winnt.*debug.*xpcshell&fromchange=d68224f5325b&tochange=4be4617827de

https://hg.mozilla.org/integration/mozilla-inbound/rev/4be4617827de

Comment 8

[Ed Morley [:emorley]]

(These were being starred as bug 845190)

Comment 9

[Jesse Ruderman]

In the new world, is there a way to construct an array in parallel using an elemental function?  I believe (new ParallelArray(10000, x => x)) runs in parallel.

Will there be a "pupdate" to modify an array in parallel in-place, or will arrays still be treated as immutable during all parallel operations?

What happens if you pass a non-conforming array to a parallel operation?  (e.g. an array with holes, or a sparse array, or an array with element getters.)

Will typed arrays (such as Int8Array) support these parallel operations?  Will there functions like pmap_to_Int8Array for *creating* typed arrays in parallel?

Comment 10

[Shu-yu Guo [:shu]]

Attachment: Part 1: Copy parallel operations to Array

Re-requesting review, since the strategy has changed since last review.

In the current strawman spec, the single-dimensioned ops on Array are going to be fairly different than the higher-dimensioned ops on Binary Data, such that I don't think refactoring them out as a separate file is worth the effort.

This patch copies the appropriate methods into Array.js.

The naming scheme has also changed to use a 'Par' suffix, per the strawman.

The ParallelArray implementation will remain in place until Binary Data is ready and we can move the higher-dimensioned operations there. We might rename it something like ParallelMatrix/Matrix in the interim once we implement the rest of the higher-dimensioned ops.

Comment 11

[Niko Matsakis [:nmatsakis]]

Comment on attachment 777433 [details] [diff] [review]
Part 1: Copy parallel operations to Array

It is a lot of code to effectively duplicate. I wonder if we can find a way to share something later.

Comment 12

[Shu-yu Guo [:shu]]

Attachment: Part 2: Add Array.build

Adds the sequential version of Array.build and some preliminary tests.

Comment 13

[Shu-yu Guo [:shu]]

Attachment: Part 3: Migrate tests to use Arrays

Tests that could be moved to use Arrays were moved to use Arrays. Other tests are *untouched* until we land Matrix, at which point we would transition them to Matrix (and later to BD). Fuzz bug tests (bugNNNNNN.js) are also untouched, but once ParallelArray is removed they should also be removed.

Also did some cleanup while I was at it:

 - Tests for Array methods are renamed to be prefixed with 'Array-methodName-'
 - Tests for features that should work in parallel are left as is.
 - Renamed the entire directory to 'parallel'

Comment 14

[Shu-yu Guo [:shu]]

Forgot to mention that for part 3, I did not touch the scatter tests since it's uncertain how useful it is for JS arrays.

Comment 15

[Shu-yu Guo [:shu]]

https://tbpl.mozilla.org/?tree=Try&rev=e75fde1e4f06

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/23dda916c3d0
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/3bdb29aa201e
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/f0c3a58f4d22

Comment 16

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/23dda916c3d0
https://hg.mozilla.org/mozilla-central/rev/3bdb29aa201e
https://hg.mozilla.org/mozilla-central/rev/f0c3a58f4d22

Comment 17

[Florian Scholz (Open Web Docs)]

With bug 1117724 (Remove PJS), I guess that it makes no sense to document the PJS API on MDN at all for now.

Array.prototype.pmap
Array.prototype.preduce
Array.prototype.pscan
Array.prototype.pscatter
Array.prototype.pfilter

These are Nightly-only and are going to be removed. Please correct me if I'm wrong.

Comment 18

[Lars T Hansen [:lth]]

(In reply to Florian Scholz [:fscholz] (MDN) from comment #17)
> With bug 1117724 (Remove PJS), I guess that it makes no sense to document
> the PJS API on MDN at all for now.

Right.

> Array.prototype.pmap
> Array.prototype.preduce
> Array.prototype.pscan
> Array.prototype.pscatter
> Array.prototype.pfilter
> 
> These are Nightly-only and are going to be removed. Please correct me if I'm
> wrong.

It is true that they are Nightly-only and will be removed.

However, they are not called that any longer (and haven't been called that for a while, at least since I started almost a year ago).  Instead they are mapPar, reducePar, scanPar, scatterPar, and filterPar.