1496172 - Attachments with unicode filenames cause timeout errors when retreiving attachments

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, enhancement)

Description

[Dylan Hardison [:dylan] (he/him)]

For instance:

https://bug1495965.bmoattachments.org/attachment.cgi?id=9013900

will timeout.

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Mojo super powers time.

I connected to one of the web-workers, and do this:

perl bugzilla.pl get 'https://bug1495965.bmoattachments.org/attachment.cgi?id=9013900'


the output below:

(logging omitted)
Inactivity timeout at local/lib/perl5/Mojolicious/Command/get.pm line 77.

isn't very helpful yet, but I can now use the debugger or other tools to figure out why this is taking so long.

Comment 2

[Kohei Yoshino [:kohei]]

I have spot-checked files attached in 24 hours and these are fine. Also checked several past PDF files and these are also fine.

Comment 3

[Kohei Yoshino [:kohei]]

Attachment: Sample PDF

Comment 4

[Kohei Yoshino [:kohei]]

And this PDF also looks good.

Comment 5

[Dylan Hardison [:dylan] (he/him)]

The underlying error is this:

Mojo::Reactor::EV: I/O watcher failed: Wide character in subroutine entry at /app/local/lib/perl5/Mojo/IOLoop/Stream.pm line 83.

which suggests the attachment data is encoded as characters and not bytes; however it seems like attachment->data is returning bytes.

Comment 6

[Dylan Hardison [:dylan] (he/him)]

Mojo::Reactor::EV: I/O watcher failed: whee at /app/local/lib/perl5/Mojo/IOLoop/Stream.pm line 86.
	Mojo::IOLoop::Stream::write(Mojo::IOLoop::Stream=HASH(0x9efc668), "HTTP/1.1 200 OK\x{d}\x{a}Link: </static/v20181002.3/skins/standard/fo"..., CODE(0xa0a0828)) called at /app/local/lib/perl5/Mojo/Server/Daemon.pm line 247
	Mojo::Server::Daemon::_write(Mojo::Server::Daemon=HASH(0x9ec1be8), "6ad5338f90159e7441fba0ad2da0abce") called at /app/local/lib/perl5/Mojo/Server/Daemon.pm line 114
	Mojo::Server::Daemon::__ANON__(Mojo::Transaction::HTTP=HASH(0xa090148)) called at /app/local/lib/perl5/Mojo/EventEmitter.pm line 15
	Mojo::EventEmitter::emit(Mojo::Transaction::HTTP=HASH(0xa090148), "request") called at /app/local/lib/perl5/Mojo/Transaction/HTTP.pm line 60
	Mojo::Transaction::HTTP::server_read(Mojo::Transaction::HTTP=HASH(0xa090148), "GET /attachment.cgi?id=9013900 HTTP/1.1\x{d}\x{a}User-Agent: Mojolici"...) called at /app/local/lib/perl5/Mojo/Server/Daemon.pm line 221
	Mojo::Server::Daemon::_read(Mojo::Server::Daemon=HASH(0x9ec1be8), "6ad5338f90159e7441fba0ad2da0abce", "GET /attachment.cgi?id=9013900 HTTP/1.1\x{d}\x{a}User-Agent: Mojolici"...) called at /app/local/lib/perl5/Mojo/Server/Daemon.pm line 202
	Mojo::Server::Daemon::__ANON__(Mojo::IOLoop::Stream=HASH(0x9efc668)) called at /app/local/lib/perl5/Mojo/EventEmitter.pm line 15
	Mojo::EventEmitter::emit(Mojo::IOLoop::Stream=HASH(0x9efc668), "read", "GET /attachment.cgi?id=9013900 HTTP/1.1\x{d}\x{a}User-Agent: Mojolici"...) called at /app/local/lib/perl5/Mojo/IOLoop/Stream.pm line 102
	Mojo::IOLoop::Stream::_read(Mojo::IOLoop::Stream=HASH(0x9efc668)) called at /app/local/lib/perl5/Mojo/IOLoop/Stream.pm line 48
	Mojo::IOLoop::Stream::__ANON__(Mojo::Reactor::EV=HASH(0x24b2298)) called at /app/local/lib/perl5/Mojo/Reactor/Poll.pm line 143
	eval {...} called at /app/local/lib/perl5/Mojo/Reactor/Poll.pm line 143
	Mojo::Reactor::Poll::_try(Mojo::Reactor::EV=HASH(0x24b2298), "I/O watcher", CODE(0xa09f340), 0) called at /app/local/lib/perl5/Mojo/Reactor/EV.pm line 46
	Mojo::Reactor::EV::__ANON__(EV::IO=SCALAR(0xa08ff08), 1) called at /app/local/lib/perl5/Mojo/Reactor/EV.pm line 25
	eval {...} called at /app/local/lib/perl5/Mojo/Reactor/EV.pm line 25
	Mojo::Reactor::EV::start(Mojo::Reactor::EV=HASH(0x24b2298)) called at /app/local/lib/perl5/Mojo/IOLoop.pm line 136
	Mojo::IOLoop::start(Mojo::IOLoop=HASH(0x24b1f50)) called at /app/local/lib/perl5/Mojo/UserAgent.pm line 65
	Mojo::UserAgent::start(Mojo::UserAgent=HASH(0x9eba478), Mojo::Transaction::HTTP=HASH(0x9edb580)) called at local/lib/perl5/Mojolicious/Command/get.pm line 76
	Mojolicious::Command::get::run(Mojolicious::Command::get=HASH(0x9eba028), "-H", "Host: bug1495965.bmoattachments.org", "-i", 0, "/attachment.cgi?id=9013900", "-v") called at /app/local/lib/perl5/Mojolicious/Commands.pm line 54
	Mojolicious::Commands::run(Mojolicious::Commands=HASH(0x9eb9f50), "get", "-H", "Host: bug1495965.bmoattachments.org", "-i", 0, "/attachment.cgi?id=9013900", "-v") called at /app/local/lib/perl5/Mojolicious.pm line 203
	Mojolicious::start(Bugzilla::Quantum=HASH(0x7072c78)) called at /app/local/lib/perl5/Mojolicious/Commands.pm line 71
	Mojolicious::Commands::start_app("Mojolicious::Commands", "Bugzilla::Quantum") called at bugzilla.pl line 20

Comment 7

[Kohei Yoshino [:kohei]]

Attachment: Sample PDF with unicode file name

Comment 8

[Rishi Chopra]

Not sure if it's of any help (likely not)... but the attachments for [bug 1495965] were created using the 'Print to PDF' feature of an El Capitan Mac (i.e. without any modification of default operating system settings and/or third-party software)...

Comment 9

[Rishi Chopra]

(In reply to Dylan Hardison [:dylan] (he/him) from comment #0)
> For instance:
> 
> https://bug1495965.bmoattachments.org/attachment.cgi?id=9013900
> 
> will timeout.

O.K. to ask what change was made to the linked attachment (per the other bug report)?  In contrast to the other three attachments on [bug 1495965] the file does, now, load - but attempting to edit the filename (changing the second dash) did not result in the same behavior; worth noting is that the second dash appears to be the only change made... though that particular character was used by Safari in the name chosen for the file (likely pulled from the title of the page), i.e. the only change made when using 'Print to PDF' was prepending the name with an ordinal & an ordinary dash...

Comment 10

[Kohei Yoshino [:kohei]]

Attachment: GitHub Pull Request by dylan

Comment 11

[Kohei Yoshino [:kohei]]

The fix already merged and deployed today.

Comment 12

[Rishi Chopra]

Manually changing the filenames didn't help (please note the previous question for Dylan) - but the fix did (appears to be working without issue on the original/default file names)...