Closed Bug 1232476 Opened 6 years ago Closed 3 months ago

Request for live captions or very detailed notes for deaf and hard of hearing contributors to have more effective communication during meetings

Categories

(Webtools :: Air Mozilla, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: linuxwebdeveloper, Unassigned)

Details

(Whiteboard: Access)

Request for live captions or very detailed notes  for deaf and hard of hearing contributors to have more effective communication during meetings.
This would be a great feature for air mozilla
Whiteboard: Access
(In reply to David Weir (satdav) from comment #1)
> This would be a great feature for air mozilla

I agree. And this is for what I have requested by this bug.  :)
We're working on this as an experiment. Automated speech-to-text is infamously rough and I suspect we won't succeed, with this experiment, to pipe the speech-to-text to become real closed captions. What our experiment is all about is to pipe the live stream into a speech-to-text server which pipes the text into an Etherpad that humans can watch and edit. 

Thank you for raising the concern and filing a bug.
Component: Events → Air Mozilla
Product: Air Mozilla → Webtools
Target Milestone: --- → 0.1
Version: unspecified → other
(In reply to Peter Bengtsson [:peterbe] from comment #3)
> We're working on this as an experiment. Automated speech-to-text is
> infamously rough and I suspect we won't succeed, with this experiment, to
> pipe the speech-to-text to become real closed captions. What our experiment
> is all about is to pipe the live stream into a speech-to-text server which
> pipes the text into an Etherpad that humans can watch and edit. 
> 
> Thank you for raising the concern and filing a bug.

I understand. You're welcome.
We're going to seriously struggle with live TTS, especially since most of our events have a wide variety of accents and dialects

Is there a bug open for making amara work on archived video?
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

I will check to see if there is.
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

So far, I don't see any, but I will do further research on this.
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

I contacted Amara using the section of the Amara website titled 'professional inquiry. I await their response back about this (I contacted them this way because I could not find a phone number, nor an e-mail for them to contact them).
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

But, I just found this video that has at the need of it about that they don't think they are yet ready for live captioning on YouTube or elsewhere, yet. But, for regular way of captioning, they are.
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

https://www.youtube.com/watch?v=Tn87VUM8f4k

(The link of that video)
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

I just left a comment to that video to which I had just given the link. I gave them my and your e-mail addresses so that they can contact about this if they respond to comments.  Here is what I actually stated to them in my comment, just now:

"I am a contributor for Mozilla. I want to know if Mozilla can use it. We (Mozilla) are trying to use it for deaf and hard of hearing contributors, as well as for contributors who speak different languages and want to use closed captioning and live captioning, as well, to benefit from better understanding. The deaf and hard of hearing Mozilla contributors and reps of Mozilla really need the captions, though, too (myself, included). Please e-mail me to linuxwebdeveloper@gmail.com or to AprilMorone@mozillausa.org, or contact Tom Farrow at tom@mozilla.org.uk, in regards to this. Thank you."
(In reply to Tom Farrow [:Tad] from comment #5)
> We're going to seriously struggle with live TTS, especially since most of
> our events have a wide variety of accents and dialects
> 
> Is there a bug open for making amara work on archived video?

I also contacted Amara on the Vimeo site that connects closed captioning Amara questions to Amara.
I also contacted Vidyo, who said we should confer with Guillermo of Mozilla would need to get a hold of Karen Chang of Vidyo about requesting live and/or closed captioning for video conferences done through Vidyo.
OMG, I almost forget to mention that a court reporter/captionor named Michele Durig stated to me of that she wouldn't ind helping live caption Mozilla meetings for contributors.  I met her in one of my Facebook groups.  May she do this?
Hi April

Thanks for your enthusiasm, but I'm worried that the pace you've moving at here might make disappoint you because Mozilla will be largely unable to kepe up. I've already consulted with the Amara team to discuss how to implement a community around this, and the Air Mo guys are working on the technical side as time permits.
(In reply to Tom Farrow [:Tad] from comment #15)
> Hi April
> 
> Thanks for your enthusiasm, but I'm worried that the pace you've moving at
> here might make disappoint you because Mozilla will be largely unable to
> kepe up. I've already consulted with the Amara team to discuss how to
> implement a community around this, and the Air Mo guys are working on the
> technical side as time permits.

Lol.  Ok.  :)  I usually am a go-getter and therefore, tend to go a bit fast, along. Sorry about that. :) I will try to be more patient.
(I'm ONLY talking about transcribing/translating ARCHIVED videos)

There is an API. http://amara.readthedocs.org/en/latest/api.html
It looks pretty straight forward. I can create a video (i.e. a page like this http://www.amara.org/en/videos/O6ZONmBLXuQq/en/1203639/) using the API. When you do that you get an Amara Video ID which we can store to presumably make a link on our page to say "This video is being transcribed and translated on Amara. Click here"

Using the API we should be able to also download the .srt files which I think we can incorporate when we play the video. (We play videos using Vid.ly's JWPlayer integration). Also, having the .srt on file means we'd be able to show the subtitles in some nice fashion underneath the video. And we'd be able to inject that text from the .srt file so you can find videos (using the airmo search function) by typing in words that are uttered. 

What I'm not clear on is how to start things. And how to continue. Also, I don't know how to commence a professional transcription. I.e. get the Amara team to do the first version, in English. And when/if we do, how does billing work?

Suppose that anybody can start a transcription project on any video. Can a second person help out, or is locked to who started?
Is there version control and/or workflow, which might be useful if we get a rogue transcriber who does a terrible or offensive job transcribing. 

I guess we can simply just get started with some basics and see how it goes.
One thing I noticed when I attempted to transcribe a randomish video I chose is that it takes time. I'm a pretty fast typer but it's a lot of work to get it "right". For example, if you make a typo, you have to pause and correct and possibly go back a bunch. And it takes a lot of patience and practice to do the reviewing where you perfect the timings.

My fear is that whoever does this will need to dedicate time. It's not enough to be able to type fast and it might be inefficient to just do a part of video and hope that someone else finishes it. And how do we know when it's good enough?

With that in mind, I'm really intrigued to get started with the professional transcription and let volunteers do translations. Or corrections.
Lastly, does anybody know they have the legal framework to professionally transcribe a private video?

(Passing by.) This old bug is probably obsolete now, since Panopto has captions?

Flags: needinfo?(akochendorfer)
Status: NEW → RESOLVED
Closed: 3 months ago
Flags: needinfo?(akochendorfer)
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.