Closed Bug 1628068 Opened 4 years ago Closed 4 years ago

Implement Viaduct backend

Categories

(Firefox :: Sync, task, P1)

Product:
Firefox
Component:
Sync
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Firefox 78
Tracking Flags:
Tracking Status
firefox78 --- fixed

People

(Reporter: eoger, Assigned: eoger)

References

Details

(Whiteboard: SACI)

Attachments

(2 files)

Bug 1628068 p1 - Vendor viaduct crate.
47 bytes, text/x-phabricator-request
Details | Review
Bug 1628068 p2 - Implement viaduct gecko backend.
47 bytes, text/x-phabricator-request
Details | Review

Viaduct is a way for Application Services Rust components (https://github.com/mozilla/application-services) to make network requests using gecko.
I suspect some other in-tree Rust users might want such a capability too.

Whiteboard: SACI

I suspect some other in-tree Rust users might want such a capability too.

For reference: https://github.com/mozilla/application-services/issues/2930. (That said we've been using it for a lot of stuff without issues, but we also know the kinds of things we're looking for from servers).

(In reply to Edouard Oger [:eoger] from comment #0)

I suspect some other in-tree Rust users might want such a capability too.

I feel seen. Lemme just add a quick See Also here...

See Also: → 1623304
Blocks: 1631630
Blocks: 1623304
Blocks: 1632897
No longer blocks: 1631630
Pushed by eoger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3aa2f99843e0
p1 - Vendor viaduct crate. r=lina
https://hg.mozilla.org/integration/autoland/rev/1ea8f742ca69
p2 - Implement viaduct gecko backend. r=tcsc,dragana,froydnj

Backed out 2 changesets (bug 1628068) for ViaductRequest.cpp related bustages

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=1ea8f742ca69f20559fb199693d8596ff8985e05&tochange=059578fc663a4b18ada37c93f583df80dbbe6ce8&searchStr=build&selectedTaskRun=FCoc5MafT9GVj0S1Y3xkBg-0

Backout link: https://hg.mozilla.org/integration/autoland/rev/059578fc663a4b18ada37c93f583df80dbbe6ce8

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301776792&repo=autoland&lineNumber=19207

...
[task 2020-05-11T19:54:07.597Z] 19:54:07     INFO -  [encoding_c 0.9.7] cargo:include-dir=/builds/worker/checkouts/gecko/third_party/rust/encoding_c/include
[task 2020-05-11T19:54:07.597Z] 19:54:07     INFO -       Running `/builds/worker/workspace/obj-build/release/build/lucet-wasi-2b191ae17f8ed620/build-script-build`
[task 2020-05-11T19:54:07.597Z] 19:54:07     INFO -       Running `/builds/worker/workspace/obj-build/release/build/cose-0fed5d53752e3b6d/build-script-build`
[task 2020-05-11T19:54:07.597Z] 19:54:07     INFO -     Compiling http v0.2.0
[task 2020-05-11T19:54:07.598Z] 19:54:07    ERROR -  [anyhow 1.0.28] error[E0554]: `#![feature]` may not be used on the stable release channel
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  [anyhow 1.0.28]  --> /builds/worker/workspace/obj-build/release/build/anyhow-2fb262225263c2f8/out/probe.rs:2:5
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  [anyhow 1.0.28]   |
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  [anyhow 1.0.28] 2 |     #![feature(backtrace)]
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  [anyhow 1.0.28]   |     ^^^^^^^^^^^^^^^^^^^^^^
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  [anyhow 1.0.28]
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -       Running `CARGO=/builds/worker/fetches/rustc/bin/cargo CARGO_MANIFEST_DIR=/builds/worker/checkouts/gecko/third_party/rust/http CARGO_PKG_AUTHORS='Alex Crichton <alex@alexcrichton.com>:Carl Lerche <me@carllerche.com>:Sean McArthur <sean@seanmonstar.com>' CARGO_PKG_DESCRIPTION='A set of types for representing HTTP requests and responses.
[task 2020-05-11T19:54:07.598Z] 19:54:07     INFO -  ' CARGO_PKG_HOMEPAGE= CARGO_PKG_NAME=http CARGO_PKG_REPOSITORY='https://github.com/hyperium/http' CARGO_PKG_VERSION=0.2.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= LD_LIBRARY_PATH='/builds/worker/workspace/obj-build/release/deps:/builds/worker/fetches/rustc/lib:/builds/worker/fetches/clang/lib:/builds/worker/fetches/clang/lib32' /builds/worker/fetches/sccache/sccache /builds/worker/fetches/rustc/bin/rustc --crate-name http --edition=2018 /builds/worker/checkouts/gecko/third_party/rust/http/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C panic=abort -C metadata=2f37b6fc2c00bd20 -C extra-filename=-2f37b6fc2c00bd20 --out-dir /builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -C linker=/builds/worker/checkouts/gecko/build/cargo-linker -L dependency=/builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/deps -L dependency=/builds/worker/workspace/obj-build/release/deps --extern bytes=/builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/deps/libbytes-89a11ceb29576901.rmeta --extern fnv=/builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/deps/libfnv-c1384af85553b793.rmeta --extern itoa=/builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/deps/libitoa-58d8159d1eff9ccd.rmeta --cap-lints warn -C opt-level=2 -C debuginfo=2 -C force-frame-pointers=yes -Dwarnings -C codegen-units=1`
[task 2020-05-11T19:54:07.598Z] 19:54:07    ERROR -  [anyhow 1.0.28] error: aborting due to previous error
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -  [anyhow 1.0.28]
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -  [anyhow 1.0.28] For more information about this error, try `rustc --explain E0554`.
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -       Running `/builds/worker/workspace/obj-build/release/build/encoding_c_mem-a71970bebc9cc07f/build-script-build`
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -  [encoding_c_mem 0.2.5] cargo:rerun-if-changed=
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -  [encoding_c_mem 0.2.5] cargo:include-dir=/builds/worker/checkouts/gecko/third_party/rust/encoding_c_mem/include
[task 2020-05-11T19:54:07.599Z] 19:54:07     INFO -       Running `/builds/worker/workspace/obj-build/release/build/authenticator-bf89e37cd1076531/build-script-build`
...
...
[task 2020-05-11T20:08:23.282Z] 20:08:23     INFO -  make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/components/viaduct'
[task 2020-05-11T20:08:23.286Z] 20:08:23     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ -std=gnu++17 -o Unified_cpp_components_viaduct0.o -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -I/builds/worker/workspace/obj-build/dist/system_wrappers -include /builds/worker/checkouts/gecko/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DOS_POSIX=1 -DOS_LINUX=1 -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/toolkit/components/viaduct -I/builds/worker/workspace/obj-build/toolkit/components/viaduct -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/checkouts/gecko/ipc/glue -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/obj-build/mozilla-config.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wunused-function -Wunused-variable -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -funwind-tables -Werror -fexperimental-new-pass-manager  -MD -MP -MF .deps/Unified_cpp_components_viaduct0.o.pp   Unified_cpp_components_viaduct0.cpp
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -  In file included from Unified_cpp_components_viaduct0.cpp:11:
[task 2020-05-11T20:08:23.287Z] 20:08:23    ERROR -  /builds/worker/checkouts/gecko/toolkit/components/viaduct/ViaductRequest.cpp:39:3: error: bad implicit conversion constructor for 'HeaderVisitor'
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -    HeaderVisitor(google::protobuf::Map<std::string, std::string>* aHeaders)
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -    ^
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -  /builds/worker/checkouts/gecko/toolkit/components/viaduct/ViaductRequest.cpp:39:3: note: consider adding the explicit keyword to the constructor
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -    HeaderVisitor(google::protobuf::Map<std::string, std::string>* aHeaders)
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -    ^
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -    explicit
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -  1 error generated.
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -  /builds/worker/checkouts/gecko/config/rules.mk:750: recipe for target 'Unified_cpp_components_viaduct0.o' failed
[task 2020-05-11T20:08:23.287Z] 20:08:23    ERROR -  make[4]: *** [Unified_cpp_components_viaduct0.o] Error 1
[task 2020-05-11T20:08:23.287Z] 20:08:23     INFO -  make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/components/viaduct'
[task 2020-05-11T20:08:23.288Z] 20:08:23     INFO -  /builds/worker/checkouts/gecko/config/recurse.mk:74: recipe for target 'toolkit/components/viaduct/target-objects' failed
[task 2020-05-11T20:08:23.288Z] 20:08:23    ERROR -  make[3]: *** [toolkit/components/viaduct/target-objects] Error 2
[task 2020-05-11T20:08:23.288Z] 20:08:23     INFO -  make[3]: *** Waiting for unfinished jobs....
Flags: needinfo?(eoger)

Seems like the 1st issue is just a probe and would not prevent us from compiling anyhow (https://github.com/dtolnay/anyhow/blob/b51c48ee918f23b6ca6d98c40f01c54fb2cbcfb8/build.rs#L9-L35).

I've pushed a patch for the 2nd issue.

Flags: needinfo?(eoger)
Pushed by eoger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b6fe5d357bed
p1 - Vendor viaduct crate. r=lina
https://hg.mozilla.org/integration/autoland/rev/928a5891c55d
p2 - Implement viaduct gecko backend. r=tcsc,dragana,froydnj

Backed out 2 changesets (bug 1628068) for ViaductRequest.cpp related bustages

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=928a5891c55dd27ecf504d3eec0e7fa30d5be471&searchStr=build&tochange=d548cbc23a677054edc3043bed998a7e2c18a5cb&selectedTaskRun=Zcza2BI9SuCeeD53OWJ6Qw-0

Backout link: https://hg.mozilla.org/integration/autoland/rev/d548cbc23a677054edc3043bed998a7e2c18a5cb

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301786674&repo=autoland&lineNumber=32918

[task 2020-05-11T20:44:10.800Z] 20:44:10     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/gcc/bin/g++ -std=gnu++17 -o Unified_cpp_components_viaduct0.o -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -I/builds/worker/workspace/obj-build/dist/system_wrappers -include /builds/worker/checkouts/gecko/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DOS_POSIX=1 -DOS_LINUX=1 -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/toolkit/components/viaduct -I/builds/worker/workspace/obj-build/toolkit/components/viaduct -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/checkouts/gecko/ipc/glue -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/obj-build/mozilla-config.h -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wduplicated-cond -Wimplicit-fallthrough -Wunused-function -Wunused-variable -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object -Wformat -Wformat-overflow=2 -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -freorder-blocks -O2 -fno-omit-frame-pointer -funwind-tables -Werror  -MD -MP -MF .deps/Unified_cpp_components_viaduct0.o.pp   Unified_cpp_components_viaduct0.cpp
[task 2020-05-11T20:44:10.800Z] 20:44:10     INFO -  In file included from Unified_cpp_components_viaduct0.cpp:11:0:
[task 2020-05-11T20:44:10.800Z] 20:44:10     INFO -  /builds/worker/checkouts/gecko/toolkit/components/viaduct/ViaductRequest.cpp: In function 'nsCString mozilla::ConvertMethod(mozilla::appservices::httpconfig::protobuf::Request_Method)':
[task 2020-05-11T20:44:10.800Z] 20:44:10    ERROR -  /builds/worker/checkouts/gecko/toolkit/components/viaduct/ViaductRequest.cpp:181:1: error: control reaches end of non-void function [-Werror=return-type]
[task 2020-05-11T20:44:10.800Z] 20:44:10     INFO -   }
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -   ^
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -  cc1plus: all warnings being treated as errors
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -  /builds/worker/checkouts/gecko/config/rules.mk:750: recipe for target 'Unified_cpp_components_viaduct0.o' failed
[task 2020-05-11T20:44:10.801Z] 20:44:10    ERROR -  make[4]: *** [Unified_cpp_components_viaduct0.o] Error 1
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -  make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/components/viaduct'
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -  /builds/worker/checkouts/gecko/config/recurse.mk:74: recipe for target 'toolkit/components/viaduct/target-objects' failed
[task 2020-05-11T20:44:10.801Z] 20:44:10    ERROR -  make[3]: *** [toolkit/components/viaduct/target-objects] Error 2
[task 2020-05-11T20:44:10.801Z] 20:44:10     INFO -  make[3]: *** Waiting for unfinished jobs....
Flags: needinfo?(eoger)
Flags: needinfo?(eoger)
Pushed by eoger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/dcda2ce7b7f9
p1 - Vendor viaduct crate. r=lina
https://hg.mozilla.org/integration/autoland/rev/013a469557c1
p2 - Implement viaduct gecko backend. r=tcsc,dragana,froydnj

Please consider pushing to try first.

Backed out 2 changesets (bug 1628068) for ViaductRequest.cpp related bustages.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=928a5891c55dd27ecf504d3eec0e7fa30d5be471&tochange=acd33e2a4fb390468d211d1778291ae6786183df&searchStr=build&selectedTaskRun=HyzRGrhcQxOwG4GF_xbcTg-0

Backout link: https://hg.mozilla.org/integration/autoland/rev/acd33e2a4fb390468d211d1778291ae6786183df

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301790934&repo=autoland&lineNumber=33931

[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  z:/build/build/src/toolkit/components/viaduct/fetch_msg_types.pb.cc(686,33): note: consider adding the explicit keyword to the constructor
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  Response_HeadersEntry_DoNotUse::Response_HeadersEntry_DoNotUse(::PROTOBUF_NAMESPACE_ID::Arena* arena)
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -                                  ^
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -                                  explicit
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  4 errors generated.
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  z:/build/build/src/config/rules.mk:750: recipe for target 'Unified_cpp_components_viaduct0.obj' failed
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  mozmake.EXE[4]: *** [Unified_cpp_components_viaduct0.obj] Error 1
[task 2020-05-11T21:30:32.418Z] 21:30:32     INFO -  mozmake.EXE[4]: Leaving directory 'z:/build/workspace/obj-build/toolkit/components/viaduct'
[task 2020-05-11T21:30:32.419Z] 21:30:32     INFO -  z:/build/build/src/config/recurse.mk:74: recipe for target 'toolkit/components/viaduct/target-objects' failed
[task 2020-05-11T21:30:32.419Z] 21:30:32     INFO -  mozmake.EXE[3]: *** [toolkit/components/viaduct/target-objects] Error 2
[task 2020-05-11T21:30:32.419Z] 21:30:32     INFO -  mozmake.EXE[3]: *** Waiting for unfinished jobs....
Flags: needinfo?(eoger)
See Also: → 1637169

Apologies about the previous failures. I've pushed to try against autoland and it looks all green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8f1a6f7158674498639b2fed65b6326ad8c3eb5a

Flags: needinfo?(eoger)
Pushed by eoger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e1d5a153eaf7
p1 - Vendor viaduct crate. r=lina
https://hg.mozilla.org/integration/autoland/rev/c9ca86d98171
p2 - Implement viaduct gecko backend. r=tcsc,dragana,froydnj

https://hg.mozilla.org/mozilla-central/rev/e1d5a153eaf7
https://hg.mozilla.org/mozilla-central/rev/c9ca86d98171

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
status-firefox78: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 78
Regressions: 1800295
Blocks: 1845815
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: