894622 - display URL/address when hovering over a link

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: General, enhancement)

Description

[Helge Hielscher]

Please display the target URL of a link in the bottom left corner like the link preview of desktop firefox when one hovers (onMouseOver) a link.

This should work with a pen (S-Pen) as well.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

I think this would be a pretty useful feature but it's going to be low on our priority list because there aren't that many mouse/s-pen users out there for android. This might make a good mentored bug for a more experienced contributor, as it will require exploring different options to see what works best and also getting feedback from UX. I can mentor it but the mentorship will be mostly pointing the contributor to different ways that this might be implemented and having them see which works best.

Comment 2

[Jayesh]

M ready to solve this bug.. Please guide me how can I move ahead with this..

Comment 3

[Teodora Vermesan (:TeoVermesan)]

What's the status of this? I've changed the status to NEW based on comment1. Feel free to change it.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Yeah, this is waiting for somebody to pick it up. I totally didn't see Jayesh's comment 2. Sorry about that! If you're still interested in working on this please let me know.

Comment 5

[Jayesh]

Hi Kartikaya,

I am already working on some bug and in a midway to that. I can start working on this as that is done. And also I do not have any experience on Android. However, can work with guidance.

Thanks,
Jayesh

Comment 6

[Kartikaya Gupta (email:[e-mail])]

Great, thanks! Let me know when you're done with the other bug and I can help you get set up for this one.

Comment 7

[Jayesh]

Hi Kartikaya, 

Can you please give me some pointers on how can i move ahead with this....

Thanks,
Jayesh

Comment 8

[Kartikaya Gupta (email:[e-mail])]

Sure! The first thing to do is set up a build environment for Fennec. You can follow the instructions at https://wiki.mozilla.org/Mobile/Fennec/Android to get set up.

Actually I also just realized that to do this bug properly you'll probably need an actual Android device with a mouse or s-pen. I don't think the emulator mouse support work enough for the purposes of this bug. If you do have such a device, carry on!

The simplest way to prototype this, I think, would be to add a mousemove listener in the Fennec browser.js (mobile/android/chrome/content/browser.js) and when the mouse moves, check to see what is under the mouse cursor. If it is a link, then you want to extract the link URL and display something that shows the URL. For now you can probably just log the URL to console to make sure you're finding the URL properly, and then we can loop in UX to discuss how it should actually be displayed on-screen.

Comment 9

[Shekhar Prasad Rajak]

I want to work on this .Can I work on this bug  ???

Comment 10

[Kartikaya Gupta (email:[e-mail])]

Hi Shekhar, you can definitely work on this bug if you're interested. Just be warned that if this is your first bug then it might be a little more complex than you want to take on right away. However, if you're interested please take a look at comment 8 on how to get started. Let me know once you have a build environment going and have successfully got a local build of Firefox running on your phone; I can assign the bug to you then and we can discuss how to work on fixing the actual bug.

Comment 11

[Prabhjyot Sodhi [:psd]]

Hey kats!
I haven't coded in javascript before but have played around with listeners in java. 
I think I could help fix this
But I didn't understand where in the file is the listener supposed to be added, 
I don't quite understand the flow of code here

Comment 12

[Kartikaya Gupta (email:[e-mail])]

The way Fennec is set up is that there is a root window [1] which contains a stack of xul:browser elements [2]. Adding a mouse listener on either of these should work for the purpose of this bug. There is some code already at [3] that adds an event listener to the deck, so that might be the best way to go.

[1] http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.xul?rev=cddde1da19ae#8
[2] http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.xul?rev=cddde1da19ae#15
[3] http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.js?rev=a5db16049be1#365

Comment 13

[Prabhjyot Sodhi [:psd]]

Attachment: 894622.patch

I have added the event listener.
Though I still haven't understood how js code is used in fennec, and would love it if you could elaborate.
would also love some pointers for the road ahead.
btw off topic: I needed some help on understanding what kind of skills would I require to implement bug 518263, It'd be awesome if you could check it out once 
Thanks a lot :)

Comment 14

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8552590 [details] [diff] [review]
894622.patch

Review of attachment 8552590 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Prabhjyot Sodhi [:psd] from comment #13)
> Created attachment 8552590 [details] [diff] [review]
> 894622.patch
> 
> I have added the event listener.

This looks like a good start. Do you have a device that can simulate mouse input though? I forgot to mention again that without such a device it's going to be hard to work on this bug. If you do have such a device then you should be able to simulate mouse input to exercise the code you added. In particular you can do a console.log(url) at the end of your function and you want to make sure that when you mouseover a link it gets logged to the logcat output.

> Though I still haven't understood how js code is used in fennec, and would
> love it if you could elaborate.

So all the content in Fennec outside of the URL bar and menu stuff is drawn by Gecko. The browser.xul document that I linked to before is the root document that Gecko loads, and that's in XUL which is Gecko's native UI language. The XUL also loads the browser.js file which allows JS to control the whole browser engine. Individual <browser> elements are used in XUL to create tabs which allow web content to be loaded. That's kind of a high level view but you can see the code that creates the browser elements at http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.js?rev=a2651cba8673#3332.

> btw off topic: I needed some help on understanding what kind of skills would
> I require to implement bug 518263, It'd be awesome if you could check it out
> once 

Honestly I don't know. That bug is "before my time" and I'm not sure what would be involved there, if it's something that the Fennec team is still interested in pursuing.

Comment 15

[Prabhjyot Sodhi [:psd]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #14)
> This looks like a good start. Do you have a device that can simulate mouse
> input though? I forgot to mention again that without such a device it's
> going to be hard to work on this bug. If you do have such a device then you
> should be able to simulate mouse input to exercise the code you added. In
> particular you can do a console.log(url) at the end of your function and you
> want to make sure that when you mouseover a link it gets logged to the
> logcat output.

I tried to use a bluetooth mouse that I could connect to my device and try it out,
I then checked the adb shell logcat for the log, but did not find anything that validates this added feature( I did add the console.log(url) at the end)
Am I doing something wrong?

Comment 16

[Kartikaya Gupta (email:[e-mail])]

I'm not sure. Probably the best way to debug this is to attach the remote debugger in desktop Firefox to fennec and stick a breakpoint in the listener to see if it is getting run. If it is not then maybe we are not dispatching the mouse event in response to the input we get in Java.

https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging

Comment 17

[Prabhjyot Sodhi [:psd]]

Hey Kats!
I have managed to get the remote connection up, but I am not sure how to go about it further.
Sorry for the delay, it just slipped out of my mind for a couple of days. :)

Comment 18

[Kartikaya Gupta (email:[e-mail])]

So when you connect with the remote debugger you'll want to debug the "Main Process" rather than the specific browser tab. And then the first thing would be to look at the console while you move the mouse around to see if there are any exceptions getting reported, or if the logging you added is showing up. You can also click in the left margin of the source view to set a breakpoint to see if your code is getting run at all. If not then I would suggest starting to trace the code from the LayerView onGenericMotionEvent/onHoverEvent functions to see what happens to the mouse input events once we get them in Java, and make sure they get propagated to Gecko and the JS code.

Comment 19

[Prabhjyot Sodhi [:psd]]

Just cannot get the remote connection up :(

Comment 20

[Kartikaya Gupta (email:[e-mail])]

Which steps did you complete successfully? Which ones fail? How do they fail?

Comment 21

[Prabhjyot Sodhi [:psd]]

Hey!
I have followed instructions given here [1] and [2],
but the connection has failed.

the command adb shell cat /proc/net/unix | grep firefox-debugger-socket
returns nothing, according to what I have read, it should return some numbers + ... firefox-debugger-socket

[1] : https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Firefox_for_Android
[2] : https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Debugging_Firefox_for_Android_with_WebIDE

Comment 22

[Kartikaya Gupta (email:[e-mail])]

You need to provide more detail before I can actually help. The connection can fail for any number of reasons. Put in your own words what you did, which commands you ran, what you saw as output, and so on.

Comment 23

[Prabhjyot Sodhi [:psd]]

Alrighty,
I connect a device through usb.
run adb devices to check device and get : 
hexa number       device
then I run the adb forward tcp:6000 localfilesystem:/data/data/org.mozilla.fennec_psd/firefox-debugger-socket
which returns no output
then I run |mach install| and install the application 
Now, I try to enter settings and turn on remote debugging
and finally I try to connect!


Now I have realised I am able to connect if the 894622.patch is not applied.
When I apply the patch, the connection fails with the message : connection timeout 
Also, when I apply the patch, and install on device, I am not able to access any sites.
ping me on irc, when you find time :)

Comment 24

[Prabhjyot Sodhi [:psd]]

I am not even able to access settings (the option is shown to be invalid), 
so the problem just maybe due to disabled remote debugging, but I doubt that since Normally when I reinstall fennec through ./mach install, I find my settings to be intact

Comment 25

[Kartikaya Gupta (email:[e-mail])]

The problem then sounds like there's a syntax error in browser.js and so it's not getting loaded, making the browser useless. Look in logcat when you start the browser for JavaScript errors in browser.js

Comment 26

[Prabhjyot Sodhi [:psd]]

Awesome!!
I got it working finally, As in I can see my log message in adb logcat,
but the problem now is that I am not able to get the correct url
In the log all I get is: 
Mouse Over: undefined
over and over again.

I believe the javascript that I have written to find the url, is faltered somehow. I am trying to find how

off topic though, I can be vouched for, now, right?

Comment 27

[Kartikaya Gupta (email:[e-mail])]

Great! It might help to log (or use the debugger to look at) the element that you're getting to make sure that's correct. And yeah, I can vouch for you - do you have a mozillians profile?

Comment 28

[Prabhjyot Sodhi [:psd]]

Hey!
I tried logging the obj[see code] that I obtain by calling document.elementFromPoint(x, y)
in logcat I now get:
Mouse Over:[object XULElement]  undefined

I am stuck here and am not sure how to proceed. How should I extract url from obj?

Comment 29

[Kartikaya Gupta (email:[e-mail])]

so the XUL element you're getting is likely just the browser element that contains the webpage. What you want is to get the element in the subdocument that's *inside* the browser element. Instead of just using "document.elementFromPoint", which will use the top-level XUL document, try using BrowserApp.selectedBrowser.contentDocument.elementFromPoint, which will look inside the webpage that's currently displayed.

Alternatively try looking at evt.target (the target of the mousemove event) which might already have the correct element.

Comment 30

[Kartikaya Gupta (email:[e-mail])]

Following up from the IRC discussion, I think getting a prototype of this up and working would be good. One thing you might be able to do is add some sort of box into the XUL tree in browser.xul and give it a CSS style of position:fixed; top:0; display:none. Then, inside your mousemove handler if there is an href you copy the URL into the box and make it display:block so that it shows up on the screen. You can hide it again by making it display:none when the mouse moves off a link.

That would probably be a good next step here, to get the URL somehow displayed to the user. You could also try to find the existing URL display tooltip in the desktop browser and see how that is implemented, perhaps copying or modeling your solution based on that. Don't spend too much time on the look and feel of it right now though, since UX will probably want to decide that.

Comment 31

[Prabhjyot Sodhi [:psd]]

Attachment: 894622.patch

Hey Hey!
I did add a label here
But I just cannot see it on the screen 
notice something fishy in the code?

Comment 32

[Kartikaya Gupta (email:[e-mail])]

Yeah the value shouldn't be specified as a CSS property. I added this in my browser.xul and it shows up as a green bar the top of the content:

diff --git a/mobile/android/chrome/content/browser.xul b/mobile/android/chrome/content/browser.xul
index 8072a7a..552f7bd 100644
--- a/mobile/android/chrome/content/browser.xul
+++ b/mobile/android/chrome/content/browser.xul
@@ -7,11 +7,12 @@
 
 <window id="main-window"
         onload="BrowserApp.startup();"
         windowtype="navigator:browser"
         xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
 
   <script type="application/javascript" src="chrome://browser/content/browser.js"/>
 
+  <description style="position:fixed; top:0; background-color: green" value="here is some text"/>
   <deck id="browsers" flex="1"/>
 
 </window>

Comment 33

[Prabhjyot Sodhi [:psd]]

Attachment: 894622.patch

Updates url only when it comes to new link, 
I can just remove the box when it the element it is on has no url

Comment 34

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8561643 [details] [diff] [review]
894622.patch

Review of attachment 8561643 [details] [diff] [review]:
-----------------------------------------------------------------

Yeah, please remove the box when there is no URL. You can use vie.style.display = 'none' to hide and = 'block' to make it appear again, if it works the same as other HTML elements. Please also make sure it is hidden to start with so that it only appears when needed.

Comment 35

[Prabhjyot Sodhi [:psd]]

Attachment: 894622.patch

Comment 36

[Prabhjyot Sodhi [:psd]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #34)
> it works the same as other HTML elements. Please also make sure it is hidden
> to start with so that it only appears when needed.

I haven't made any change for the starting Since as soon as we are on some element(that has undefined href), it will immediately hide itself.
Is that okay?

Comment 37

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8562292 [details] [diff] [review]
894622.patch

Review of attachment 8562292 [details] [diff] [review]:
-----------------------------------------------------------------

This is fine, but play with it more. This bug is much more open-ended and you have room to try different things and see what feels the best both in terms of look and user interaction. Once you have something you're happy with grab a screenshot/video and we can ask UX for feedback based on that.

Comment 38

[Prabhjyot Sodhi [:psd]]

Yups, Okay will do that!!

Comment 39

[Prabhjyot Sodhi [:psd]]

Hey Hey!
Really Sorry, 
Caught up in a lot of stuff right now
Is this urgent?
I'll try to work on this asap :)

Comment 40

[Kartikaya Gupta (email:[e-mail])]

It's been a year and a half since the bug was filed, I'm sure a bit more time won't hurt :)

Comment 41

[Prabhjyot Sodhi [:psd]]

Thanks a lot :D

Comment 42

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.