1404719 - Fix the 'Provider "jrt" not found' error when running t-b-v jobs via Airflow

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, defect, P4)

Description

[Mark Reid [:mreid]]

This is an annoying red herring. It generally doesn't have anything to do with whether the job ran successfully or not, but leads to confusion when checking job logs and such.

My current theory is that it's actually an error raised when running `sbt assembly` as part of the job initialization, but I haven't confirmed.

Comment 1

[Mark Reid [:mreid]]

The full error is:

Exception in thread "main" java.nio.file.ProviderNotFoundException: Provider "jrt" not found
	at java.nio.file.FileSystems.getFileSystem(FileSystems.java:224)
	at io.github.retronym.java9rtexport.Export.main(Export.java:36)

Comment 2

[Mark Reid [:mreid]]

The error is coming from sbt, and happens on both EMR 5.2.1 and 5.8:

$ sbt version
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/mnt1/ -Xmx15000M -Xms1000M
Copying runtime jar.
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/mnt1/ -Xmx15000M -Xms1000M
Exception in thread "main" java.nio.file.ProviderNotFoundException: Provider "jrt" not found
	at java.nio.file.FileSystems.getFileSystem(FileSystems.java:224)
	at io.github.retronym.java9rtexport.Export.main(Export.java:36)
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/mnt1/ -Xmx15000M -Xms1000M
Waiting for lock on /home/hadoop/.sbt/boot/sbt.boot.lock to be available...
Updated file /home/hadoop/project/build.properties setting sbt.version to: 0.13.15
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /home/hadoop/project
[info] Set current project to hadoop (in build file:/home/hadoop/)
[info] 0.1-SNAPSHOT
-bash-4.2$ echo $?
0

Comment 3

[Mark Reid [:mreid]]

This was added in the first item under "Improvements" at [1], in the PR at [2].

[1] https://github.com/sbt/sbt/blob/1.x/notes/0.13.14.markdown
[2] https://github.com/sbt/sbt-launcher-package/pull/143

Comment 4

[Mark Reid [:mreid]]

So... looks like there is (was?) an issue with Java version detection that hits the OpenJDK java, when it prints out that "Picked up _JAVA_OPTIONS" line.

It appears to have been fixed in [1]:
https://github.com/sbt/sbt-launcher-package/commit/c487e3166e5decfa2463a1155f216de7de217ee9

Comment 5

[Mark Reid [:mreid]]

I filed an issue with sbt:

https://github.com/sbt/sbt/issues/3593

Comment 6

[Jeff Klukas [:klukas] (UTC-4)]

TBV is no longer actively used.