Set screen coordinates during HTML5 drag event

NEW
Unassigned

Status

()

P2
normal
9 years ago
3 months ago

People

(Reporter: sebastian, Unassigned)

Tracking

(5 keywords)

unspecified
x86
All
DevAdvocacy, html5, parity-chrome, parity-ie, parity-safari
Points:
---
Bug Flags:
firefox-backlog +

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: p=0 [DevRel:P2])

Attachments

(2 attachments, 3 obsolete attachments)

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.33 Safari/530.5
Build Identifier: 

During the dragstart, drag and dragend events, the mouse position (pageX, clientX, screenX) is set to 0 (zero).



Reproducible: Always

Steps to Reproduce:
Create an element with draggable attribute set to true. Listen to the dragstart, drag and dragend events.
Actual Results:  
The pageX, clientX, screenX values are zero.

Expected Results:  
They should report the current mouse position, even if the mouse is currently outside of the document (just like mousedown + mousemove does).

Even if a DragEvent is initialized manually using initDragEvent, they values are set to zero.

As far as I know there are no known security issues with regards to mouse position and drag operations. Other browsers implement it fully. It's already possible to get the same mouse position during mousemove events even outside of the document if the mouse button is down.

Comment 1

9 years ago
We don't support coordinates on the drag and dragend events as they are not triggered by mouse events and don't occur at a specific point. As an special exception, the screen coordinates where the drop occurred are available during the dragend event.

I don't see a problem retrieving the coordinates during the dragstart event however, where they should be set. Can you please provide a testcase demonstrating this problem?
(Reporter)

Comment 2

9 years ago
dragstart does retain the screen coordinates. I was a wrong there. drag and dragend doesn't.

The HTML5 drag & drop API is suppose to be UI/input device agnostic. However, DragEvent inherits MouseEvent because the typical case would be a mouse gesture.

For example, the drag event fills no significant purpose than to trigger an event when the mouse moves or something else moves in front of or behind it.

I know the spec defines that the events should trigger during certain time intervals but this doesn't make sense. All browsers triggers the sequence faster if the mouse is moved. In which case mouse position should be available.

I've brought it to the WHATWG list that this should be specified more clearly.

WebKit and IE currently treats the drag event as a mouse event similar to dragover or mousemove.
(Reporter)

Comment 3

9 years ago
The current HTML5 spec describes that all DragEvent properties should be available during all the events - according to editor Ian Hickson.

http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/021917.html

All other implementations does this so I don't think the Mozilla implementation should be more limited in this regard.
(Reporter)

Updated

9 years ago
Flags: wanted1.9.2?
Keywords: html5
Priority: -- → P2

Comment 4

9 years ago
(In reply to comment #3)
> The current HTML5 spec describes that all DragEvent properties should be
> available during all the events - according to editor Ian Hickson.

Note though that it doesn't specify what the properties should be set to, just that they should be set and we currently set them to 0.
(Reporter)

Comment 5

9 years ago
Well I'm pretty sure it's implied that it's set to something useful. I guess it needs to be abundantly clear. But IMO that's just an excuse not to fix it. There's still an issue here.

Recently a comment has been added that they should be set to 0 if there is no relevant pointing device.

http://html5.org/tools/web-apps-tracker?from=3587&to=3588

But that just applies to keyboard actions, alternative input devices etc. In this case there IS a relevant pointing device - the mouse input. So implicitly it should be set to a relevant and useful property.

Comment 6

9 years ago
I'm not arguing it shouldn't be changed. But the spec doesn't specify what it should be set to. The 'drag' event isn't fired due to a mouse action. Since it is unspecified, any number of possible values could be used:
 - the coordinates should be set to 0
 - the coordinates are the location of the element the drag event is fired on
 - the coordinates are the x,y relative to the window the mouse is over, if any
 - the coordinates are the x,y relative to the window containing the element the drag event is over

What do other browsers do here?
(Reporter)

Comment 7

9 years ago
Oh, I see what you mean.

Other browsers uses coordinates relative to the source node. I.e. the event target in this case. It works just like the mousemove event. If a mouse button is pressed "mousemove" is continually triggered even outside of the current document. (Values outside the bounds of the document is valid.) "drag" should work the same as "mousemove" in this regard.

Comment 8

9 years ago
What's up with this annoying issue?
 
I'd like to display a custom drag proxy when the user is dragging an element on the page (the "drag" event seems pretty appropriated to me), unfortunately as Sebastian said above the clientX and clientY are always set to zero.

I've tried to use setDragImage but it doesn't keep any reference to the DOM element being dragged, nor giving you the possibility to update this image anywhere else than the "dragstart" event. 

I've found a workaround to retrieve the mouse position by adding a "dragover" event listener at the document level, but that's a bit overkill and I don't understand why these properties aren't exposed on all events of the drag source.

My two cents.

Updated

8 years ago
Blocks: 455694

Updated

8 years ago
No longer blocks: 455694

Updated

8 years ago
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: HTML5 Drag and Drop: DragEvent hides mouse position during source node events → Set screen coordinates during drag event

Updated

7 years ago
Flags: wanted1.9.2?

Comment 9

7 years ago
(In reply to Neil Deakin from comment #6)
> I'm not arguing it shouldn't be changed. But the spec doesn't specify what
> it should be set to. The 'drag' event isn't fired due to a mouse action.
> Since it is unspecified, any number of possible values could be used:
>  - the coordinates should be set to 0
>  - the coordinates are the location of the element the drag event is fired on
>  - the coordinates are the x,y relative to the window the mouse is over, if
> any
>  - the coordinates are the x,y relative to the window containing the element
> the drag event is over

My answer is that the coordinates should be given relative to the screen containing the element that is the target of the drag event. More precisely, screen is the screen whose top-left corner is used for calculating element.boxObject.screenX/Y, where element is the target of the drag event. In most or all cases, this will return the same values that checking screenX/Y would for a mouse event would.


> What do other browsers do here?

(In reply to Sebastian Markbåge from comment #7)
> Other browsers uses coordinates relative to the source node.

This is absolutely incorrect. In Chrome, screen coordinates are given relative to the screen's top-left corner as expected.

Comment 10

7 years ago
I was going to report this bug too, glad I found the existing report myself for once. :)
Here's my testcase, in case it helps anyone: http://jsfiddle.net/leaverou/89AtH/

The D&D API would be incredibly useful not only for dragging data outside or from outside the browser or for dragging an element onto another (as most demos show), but also for the typical dragging an element from one place in the document to another. This bug renders this completely impossible.

Updated

7 years ago
Blocks: 674925
(In reply to Frank Yan (:fryn) from comment #9)
> My answer is that the coordinates should be given relative to the screen
> containing the element that is the target of the drag event. More precisely,
> screen is the screen whose top-left corner is used for calculating
> element.boxObject.screenX/Y, where element is the target of the drag event.
> In most or all cases, this will return the same values that checking
> screenX/Y would for a mouse event would.

I absolutely concur. This is essential if you want to replace the default drag image with a custom one or do drop target detection on your own. At the moment I have to use the workaround from comment #8 which is really not that ideal.
Created attachment 571632 [details] [diff] [review]
patch v1

I decided to take a stab and implemented it for GTK. This works quite well on my machine but there might be some edge cases I don't know of. Also I'm really not sure that's the right approach but I'm not too familiar with all the DnD and platform code :)
Attachment #571632 - Flags: feedback?(enndeakin)

Comment 13

7 years ago
(In reply to Tim Taubert [:ttaubert] from comment #11)
> I absolutely concur. This is essential if you want to replace the default
> drag image with a custom one or do drop target detection on your own.

Can you explain why you think you need this?

Comment 14

7 years ago
(In reply to Neil Deakin from comment #13)
> (In reply to Tim Taubert [:ttaubert] from comment #11)
> > I absolutely concur. This is essential if you want to replace the default
> > drag image with a custom one or do drop target detection on your own.
> 
> Can you explain why you think you need this?

In order to drag tabs with animated images designating what will happen if they drop, for example...

Comment 15

7 years ago
I think this should get a higher priority, since it is blocking more than a few bugs, most of which are encountered regularly. (see the dependency graph)

Comment 16

7 years ago
(In reply to Notlost from comment #14)
> > Can you explain why you think you need this?
> 
> In order to drag tabs with animated images designating what will happen if
> they drop, for example...

More details please?

You can't create animated images with the drag/drop api, (unless you use <panel> but you can already obtain its coordinates)

Comment 17

7 years ago
See bug 455694

Comment 18

7 years ago
That bug may be a reason to implement this, but it has nothing to do with animated or custom drag images.

Various people above are asserting that the screen coordinates are necessary for using custom drag images (animated or otherwise). I assert that it is not the case.

I want to know why specifically people disagree such that this can be implemented to address those usecases, or, explain why what they are doing doesn't need the coordinates.
Created attachment 574562 [details] [diff] [review]
patch v2

FWIW, added Mac support.
Attachment #571632 - Attachment is obsolete: true
Attachment #571632 - Flags: feedback?(enndeakin)
(In reply to Neil Deakin from comment #13)
> Can you explain why you think you need this?

Sorry to step in so late, I obviously forgot to add me to the CC list :/

So, I'm implementing a feature that needs to implement a custom drag image. The default one is not what the UX wants to have (it's at 0.8 opacity or something and you can't change that). To achieve this I need to re-position the custom drag image on every 'drag' event. Currently, there's no way to do that but adding a 'dragover' listener to the documentElement. This does not feel like the right way at all when there's an event like 'drag' that could provide these coordinates to move the drag image accordingly.

Comment 21

7 years ago
I thought you were using a <panel> here? The transparency of the panel should be used. If not, why not, as it handles all of this for you?
(In reply to Neil Deakin from comment #21)
> I thought you were using a <panel> here? The transparency of the panel
> should be used. If not, why not, as it handles all of this for you?

I didn't know about the <panel> but (unfortunately) I'm implementing this feature as a HTML page so that's not available to me. I was talking about the HTML5 DnD API as were the other reporters (I think :).
Created attachment 575971 [details] [diff] [review]
patch v3

Added Windows support.
Attachment #574562 - Attachment is obsolete: true
(In reply to Tim Taubert [:ttaubert] from comment #22)
> (In reply to Neil Deakin from comment #21)
> > I thought you were using a <panel> here? The transparency of the panel
> > should be used. If not, why not, as it handles all of this for you?
> 
> I didn't know about the <panel> but (unfortunately) I'm implementing this
> feature as a HTML page so that's not available to me.

You can mix XUL and HTML.
(In reply to Neil Deakin from comment #21)
> I thought you were using a <panel> here? The transparency of the panel
> should be used. If not, why not, as it handles all of this for you?

Ok, after Dão's hint I tried to use the XUL panel but had no luck in specifying a custom opacity. While that actually may be fixable I don't see any difference between passing a <panel> and passing an arbitrary HTML element to dataTransfer.setDragImage() - there's a copy of that element (with less opacity) used as the drag image.

The second thing about that is, that we implement a custom drop target detection (in the New Tab Page) based on the current drag image's position. So even when using a <panel> I can't get the drag image's position and use it to determine the current drop target. I'd still need to use pointer coordinates in the 'drag' event. Please tell me if I'm missing something here.

Comment 26

7 years ago
(In reply to Tim Taubert [:ttaubert] from comment #25)
> I don't see
> any difference between passing a <panel> and passing an arbitrary HTML
> element to dataTransfer.setDragImage() - there's a copy of that element
> (with less opacity) used as the drag image.

The difference is that you can change the styling of the contents of the panel on-the-fly, but that point isn't relevant to this bug.

The main problem here is that it is convoluted to have to attach a dragover handler to the drop target within a dragstart handler for an element when the handling is specific to the element. Just because it does not immediately add power to the API doesn't mean it shouldn't be done. We also implemented mouseenter and mouseleave to make things easier for developers.
Can someone clearly list what each of the other browsers use for coordinate values for drag events? Neil asked for that earlier in the comments, but there doesn't seem to be a definitive answer, only disputed assertions.

Comment 28

7 years ago
Yes, this is a bit confusing. My issue is about the use cases here, and there is confusing discussions from multiple people about drag images (comments 8, 10, 11, 14)  that makes it very unclear what people are doing 

Non-privileged code is prevented from using non-static drag images. If you are using privileged xul code, you should use <panel> to create non-static drag images. You cannot use non-static drag images at all in html as standalone <panels> do not render in html documents. (although you may be able to use menu trickery).

The patch itself is a good start, although I haven't tested it to see what coordinate system it is in. I think you may be able to combine mCurrentDragPoint and mEndDragPoint as well.

Updated

7 years ago
Duplicate of this bug: 590355
Created attachment 579313 [details] [diff] [review]
patch v4

Combined mCurrentDragPoint and mEndDragPoint.

(In reply to Dietrich Ayala (:dietrich) from comment #27)
> Can someone clearly list what each of the other browsers use for coordinate
> values for drag events? Neil asked for that earlier in the comments, but
> there doesn't seem to be a definitive answer, only disputed assertions.

With the help of Lea's testcase http://jsfiddle.net/leaverou/89AtH/ I checked the current browser support:

* Chrome, IE and Safari support drag event coordinates.
* Opera does not support drag-and-drop at all.

Chrome, IE and Safari (and Firefox) fill the pageX/Y properties to have values relative to the document containing the drag event target.
Assignee: nobody → ttaubert
Attachment #575971 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #579313 - Flags: review?(enndeakin)

Comment 31

7 years ago
Comment on attachment 579313 [details] [diff] [review]
patch v4

>   if (dragSession) {
>     if (aMessage == NS_DRAGDROP_OVER) {
>+      // set current drag position
>+      nsDragService* dragService = static_cast<nsDragService *>(mDragService);
>+      dragService->SetCurrentDragPoint(nsIntPoint(NSToIntRound(localPoint.x), NSToIntRound(localPoint.y)));

You're passing in values in a different coordinate system from dragend here. localPoint is relative to the 'self' whereas for dragend, screen coordinates are passed in.
Attachment #579313 - Flags: review?(enndeakin) → review-
Not going to block the New Tab Feature on this (anymore). We'll be using the workaround from comment #8 until this bug gets resolved. Alas, I don't have time at the moment to continue digging further into this.
Assignee: tim.taubert → nobody
No longer blocks: 455553
Status: ASSIGNED → NEW

Updated

6 years ago
Summary: Set screen coordinates during drag event → Set screen coordinates during HTML5 drag event

Comment 33

5 years ago
I would also greatly appreaciate this bug fixed. Here's a usecase for you, Neil, without all the convoluted arguments with ghost image adjustments:

I use the DnD API to resize elements (in my case some time intervals in a calendar). It's pretty straightforward using dragstart, drag, and dragend events. In other major browsers I can easily resize the element in question during the drag, because I get the current mouse position from the 'drag' event. In Firefox I'm screwed (or left to use the document dragover hotfix mentioned above). I understand that this is not exactly a fit for "drag & DROP" mechanism, but the "drag" part fits pretty well.

Comment 34

5 years ago
Since the drag/drop api shouldn't be used for resizing elements, that isn't a good usecase. The right way to do that is with ordinary mouse events, using setCapture if you need to go outside the window boundary.

Comment 35

5 years ago
Created attachment 781842 [details]
Test case

I was about to report this issue, but saw this in the "Possible Duplicates" section after entering a title.

Attached is a test case.  It's not a use case, just a test case :)

Comment 36

5 years ago
My use case is selecting multiple items, then moving them all by dragging one. I need the coordinates in the `drag` event to move my custom ghost images of the other items.

Comment 37

5 years ago
(In reply to Forrest O. from comment #36)
> My use case is selecting multiple items, then moving them all by dragging
> one. I need the coordinates in the `drag` event to move my custom ghost
> images of the other items.

You should instead just be drawing them to an image/canvas and calling setDragImage. You won't be able to properly emulate the drag image behaviour otherwise.

Comment 38

5 years ago
We don’t need to talk about use cases. The question is: why does the drag event fire at all? It presents no useful data. Without position data the drag event is completely useless.
This thread started 2009 when Firefox 3.5 was the actual version. Now , 4 years later we have Firefox 23 and this sloppiness still exists!
Just drop the drag event in Firefox 24. It is useless this way.

Comment 39

5 years ago
We wanted use cases so we can identify the best way to implement the coordinates. Perhaps removing the event is one such solution.

Comment 40

5 years ago
Use case:

Problem
Imagine a puzzle you want to solve with a mouse. You want to drag the puzzle pieces to the right gaps in the puzzle. You could use drop targets, but drop targets react on the cursor position and not position of the moved element. So if you grab the puzzle piece at the lower right corner and drag the pointer to the upper left corner of the gap, the drop would work, but the puzzle piece would be far from being on the place where it belongs.

Solution
Don’t use the standard drop events, write your own drop method that measures the click position on dragstart, calculates the real position of the dragged element and only allows a drop if the element is over the gap. But that is only possible if the drag position is known while dragging.

Since an element is dragged with mouse or touch, why not simply output the same position of the mouse or finger as mousemove or touchmove does? It is the only purpose for a drag event.

Comment 41

5 years ago
Here’s a simple Fiddle for the use case:
http://jsfiddle.net/gimmel/tV8Hk/

Comment 42

5 years ago
In that last case, I'm not clear why you need to drag event at all. You can retrieve the current mouse coordinates within the dragover event.

Comment 43

5 years ago
That would allow you to detect and deal with the case where the mouse pointer was inside the gap but most of the dragged puzzle piece was outside the gap, but it would not let you detect the case where the mouse pointer was outside the gap but most of the dragged puzzle piece was inside the gap.

Comment 44

5 years ago
I was mistaken, retrieving the mouse coordinates manually should allow you to detect and deal with either of those cases.

Michael's right though, there is currently no useful information presented in the drag event, besides the fact that the mouse is moving.

Updated

5 years ago
Blocks: 950073
Whiteboard: p=0

Comment 45

5 years ago
Another use case that I am up against is trying to get the screen to scroll during the dragging action. Given that it is the primary event controlling the behavior of the window at that point, it makes more sense to receive the data from the drag event than from an alternative event like mousedown. It just makes more semantic sense to me, at least. If the drag event is what's causing the screen to scroll, then it shouldn't be information from another pointer event that is defining the action.

Updated

5 years ago
No longer blocks: 950073
Flags: firefox-backlog+

Updated

4 years ago
No longer blocks: 674925

Comment 46

4 years ago
A use case is demonstrated by the packery draggable functionality:  http://packery.metafizzy.co/draggable.html

There is a draggable demo there.  While dragging, I want to reposition other elements on the screen to give a preview of the effects of a drop to the user.  I need the coords during the drag event in order to do this. 

A quick test in chrome shows that the clientX, clientY, pageX, and pageY props are all set, and seem to be set to the same values: (pageX === clientX, and pageY === clientY).  They are set to the absolute window positions of the mouse cursor.  They only show movement within the window - the min value is 0,0.  If needed, you can record the original positions in the dragStart event to figure out relative movement.
Comment hidden (off-topic)

Comment 48

4 years ago
The fact that IE and Chrome are both returning the mouse coordinates on drag event should be the main reason why this issue should be addressed ASAP.

Comment 49

4 years ago
I noticed that using the approach suggested at entry #8 will not work when I hover over the target area.Any idea how I can overcome this ?

Comment 50

4 years ago
This thread started 2009 when Firefox 3.5 was brand new. Now, FIVE years later (we have Firefox 32 now!) this topic is still discussed and Firefox still sets all position coordinates to 0 in the drag event.
The drag event in Firefox is still useless. I have lost any hope that this will change some day.
Whiteboard: p=0 → p=0 [parity-IE][parity-webkit]

Comment 51

4 years ago
It would be great if someone could fix this, please.
The lack of mouse coordinates makes it impossible to implement "resizable columns" in Firefox.

Comment 52

4 years ago
(In reply to Thibaut Courouble from comment #51)
> It would be great if someone could fix this, please.
> The lack of mouse coordinates makes it impossible to implement "resizable
> columns" in Firefox.

You shouldn't be using the drag and drop api for that. Instead you should use setCapture.

Comment 53

4 years ago
(In reply to Neil Deakin from comment #52)
> You shouldn't be using the drag and drop api for that. Instead you should
> use setCapture.

setCapture is non-standard and not supported in Blink/WebKit.

Comment 54

3 years ago
It's insane this is still a problem. How do you suggest we get the coordinates during a drag then? What's the point of drag and drop if I can't drag below the view and get scrolling. Is this ever going to be fixed?
Keywords: DevAdvocacy

Comment 56

3 years ago
Firefox 45 (or 44) broke the screenX property on the "dragend" event (its value is now always 0, same as pageX and clientX), making it impossible to implement any kind of UI drag feature (like resizing) in Firefox. Please fix.

Comment 57

3 years ago
Please file a new bug on that with a testcase and/or steps to reproduce and platform details.

Comment 58

3 years ago
(In reply to Neil Deakin from comment #57)
> Please file a new bug on that with a testcase and/or steps to reproduce and
> platform details.

Done: https://bugzilla.mozilla.org/show_bug.cgi?id=1244533
Whiteboard: p=0 [parity-IE][parity-webkit] → p=0 [parity-IE][parity-webkit][DevRel:P2]

Updated

2 years ago
Flags: platform-rel?

Updated

2 years ago
platform-rel: --- → ?

Comment 59

2 years ago
Another use case of the mouse coordinate during the drag event. I disabled the ghost image of the dragged element because I want to build a custom one with some data inside (how many item I have selected to drag, names etc..)

I respectably disabled the setDragImage, and make my other element to follow the cursor during the drag event.

Here is a quick example : http://jsfiddle.net/e1wqafyr/26/

As you can see the pageX and pageY aren't updated while we should have it.

Comment 60

2 years ago
I need this feature to :(

MDM claims, it is supported since Firefox 3.5
https://developer.mozilla.org/en-US/docs/Web/Events/drag
platform-rel: ? → ---

Comment 61

2 years ago
I'm trying to mimic the functionality of dragging files on drive.google.com.  
The only workaround I can think of is a document wide dragover handler which is pretty sloppy.

Updated

a year ago
Duplicate of this bug: 1396403

Comment 63

a year ago
This works fine in Safari, Chrome, Opera and Edge. What would we need to do to land a fix?
Duplicate of this bug: 1428225

Comment 65

6 months ago
9 years since this bug was reported and you still couldn't fix it? 
Please do something as this is overtly annoying.
Whiteboard: p=0 [parity-IE][parity-webkit][DevRel:P2] → p=0 [parity-IE][parity-webkit][parity-chrome][DevRel:P2]
Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)
Keywords: parity-chrome, parity-ie, parity-safari
Whiteboard: p=0 [parity-IE][parity-webkit][parity-chrome][DevRel:P2] → p=0 [DevRel:P2]
You need to log in before you can comment on or make changes to this bug.