1844952 - Support adding alt text to image annotations in PDFs

Status: VERIFIED FIXED

(Firefox :: PDF Viewer, enhancement)

Description

[Adrian Roselli]

Attachment: PDF-UA_FF-nightly-add-img.pdf

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

1. Using Firefox Nightly (117) open a tagged PDF: https://adrianroselli.com/files/xfr/PDF-UA.pdf
2. Activate the "Add an image" button
3. Click or tap in the document (I see no way to do this with keyboard alone)
4. Choose an image from your computer
5. Embed the image

Actual results:

An image was embedded at the point where I clicked. I could drag it around and resize it.

1. I was not prompted to provide alternative text, nor could I find a setting to add alternative text; and
2. When I saved the PDF, the image was not added to the tag structure of the document, making it unavailable to my screen reader.

Expected results:

1. I should be immediately prompted to enter an image description to be used as its alternative text; and
2. When the PDF is saved the image should be folded into the document tags.

My original PDF: https://adrianroselli.com/files/xfr/PDF-UA.pdf

I attached the PDF I created with Firefox Nightly 117 to this issue so you can compare them.

Comment 1

[Adrian Roselli]

This is related to the feature added through this request: Bug 1790255 Support importing images.

No idea if this prior fixed bug is useful Bug 1823296 PDF editor removes tags from tagged PDFs.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::PDF Viewer' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Marco Castelluccio [:marco]]

This is something we are already planning to do, not sure yet about the UX (should we prompt the user as soon as they add an image? Should we add a context menu option to add alt text?).

Comment 4

[Adrian Roselli]

You changed the name of issue, which now obfuscates that two things are buggy:

1. Users cannot add alt text to images, and
2. The image in the resultant PDF is not added into the tagging structure of the PDF (meaning a PDF/UA-conformant file would no longer conform).

Please restore the issue name as filed.

This is something we are already planning to do, …

That is ace. I encourage Firefox not to release the feature to add images to PDFs until the tagging and alt text issues are in place (and they support keyboard-only users).

…not sure yet about the UX (should we prompt the user as soon as they add an image? Should we add a context menu option to add alt text?).

Yes and yes. Prompt users when added and add a (keyboard accessible) option to add/edit the alt text after the image has been added.

Comment 5

[Marco Castelluccio [:marco]]

(In reply to Adrian Roselli from comment #4)

You changed the name of issue, which now obfuscates that two things are buggy:

1. Users cannot add alt text to images, and
2. The image in the resultant PDF is not added into the tagging structure of the PDF (meaning a PDF/UA-conformant file would no longer conform).

Please restore the issue name as filed.

Since they are two separate issues, and this one is now about the first, could you file a separate bug for the second?
We prefer having one bug for each problem, it makes things easier to track.

…not sure yet about the UX (should we prompt the user as soon as they add an image? Should we add a context menu option to add alt text?).

Yes and yes. Prompt users when added and add a (keyboard accessible) option to add/edit the alt text after the image has been added.

This is not for us to say, we need an UX expert to tell us what is the best way to achieve the effect we want (having people add alt text).
For example, prompting when added could have the adverse effect if people are not happy with having to do it every time and so use another editor that maybe doesn't even support alt text.

Comment 6

[Adrian Roselli]

Since they are two separate issues, and this one is now about the first, could you file a separate bug for the second?
We prefer having one bug for each problem, it makes things easier to track.

Understood and done (I realized the keyboard thing should also be an issue):

• 1845087: Images imported to PDFs are not tagged
• 1845088: Make importing images into PDF keyboard operable

This is not for us to say, we need an UX expert to tell us what is the best way to achieve the effect we want (having people add alt text).

While that is my job, fair enough not to consider me one. What, then, is the plan/timeline to get UX expertise engaged?

Comment 7

[Marco Castelluccio [:marco]]

(In reply to Adrian Roselli from comment #6)

Since they are two separate issues, and this one is now about the first, could you file a separate bug for the second?
We prefer having one bug for each problem, it makes things easier to track.

Understood and done (I realized the keyboard thing should also be an issue):

• 1845087: Images imported to PDFs are not tagged
• 1845088: Make importing images into PDF keyboard operable

Thanks! That's helpful.

This is not for us to say, we need an UX expert to tell us what is the best way to achieve the effect we want (having people add alt text).

While that is my job, fair enough not to consider me one. What, then, is the plan/timeline to get UX expertise engaged?

Oh I wasn't aware :)
The plan is to have somebody review the overall UX soon.
Given you are an expert, do you have suggestions on how we could approach this? Is prompting when adding too noisy? If yes, how could we prompt when adding without looking too noisy for the user?

Comment 8

[Adrian Roselli]

Oh I wasn't aware

No reason you would be. Frankly, I am not aware half the time.

The Microsoft Office workflow is a reasonable model (with the fewest complaints I have heard from users) where it opens a panel that prompts the user for alt text. The user can accept what is there (perhaps the image has metadata that can be parsed?) or replace it with their own. The panel can be dismissed or ignored.

This is less intrusive than a modal that the user must dismiss to continue.

Comment 9

[Marco Castelluccio [:marco]]

Thanks! It looks like Office is sending images to the cloud to use AI to generate alt text (without user consent?). We won't be able to do that for privacy reasons.

Comment 10

[Adrian Roselli]

I did not mean to suggest that Firefox should generate the alt text, but that if the image metadata contained some then it could be used in an analogous way.

Comment 11

[Marco Castelluccio [:marco]]

Oh yes, I understand, I was saying that we can't replicate what Microsoft is doing there. The image metadata is mostly technical stuff, it doesn't describe the contents of the image.
So we have to find a solution that doesn't include the autogeneration (with AI or from metadata). Opening a panel automatically or something like that might work, as long as it is not modal. Maybe there could be a small panel shown right below the image after you have added it or when you select it, with a field to add alt text.

Comment 12

[Calixte Denizet (:calixte)]

Here are the specifications from UX/UI:
https://www.figma.com/file/elSBKpUxHCZe6yTqoohUu9/PDF.js-Designs?node-id=3511%3A50300&mode=dev
for the flow we want for adding an alt text to an image.

Comment 13

[BMO Github Automation]

Attachment: [mozilla/pdf.js] [Editor] Add a dialog box in order to get alt-text data (bug 1844952) (#16952)

Comment 14

[Adrian Roselli]

Thank you for sharing that! I am tight on time, but will ask some other UX folks to take a look too!

Comment 15

[BMO Github Automation]

Attachment: [mozilla/pdf.js] [Editor] Add a button to trigger a dialog for adding an alt text (bug 1844952) (#16955)

Comment 16

[BMO Github Automation]

Attachment: [mozilla/pdf.js] [Editor] Allow the user to add and save an alt-text for images (bug 1844952) (#16977)

Comment 17

[BMO Github Automation]

Attachment: [mozilla/pdf.js] [Editor] Add a tooltip showing the alt text when hovering the alt-text button (bug 1844952) (#16983)

Comment 18

[Dana Lucaci, Desktop QA]

This is verified fixed with Firefox 120.0a1 (20231004094640) and Firefox 119.0b4 (20231002091755) on macOS 12 ARM, Win 10 and Ubuntu 22.04. The alt text is successfully added to the tag structure of the document.