Closed Bug 1715386 Opened 3 years ago Closed 2 years ago

Unexpected function offsets returned by Eliot symbol server

Categories

(Eliot :: General, defect, P2)

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: mstange, Unassigned)

References

()

Details

The following request gives a function offset of 0x100e3 with the current symbolication API, which is correct, and a function offset of 0x0 with the Eliot implementation, which is incorrect.

curl -d '{"stacks":[[[0,3849667]]],"memoryMap":[["XUL","395F7EC110E43068A7EE11BEA458C5210"]]}' https://symbolication.stage.mozaws.net/symbolicate/v5 | jq

The address is correctly attributed to the function void draw_quad_spans<unsigned int>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&).

In the sym file, there's only one entry for this function:

FUNC m 39bce0 2e3ec 0 void draw_quad_spans<unsigned int>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&)

So the sym file says it starts at 0x39bce0. But the staging server somehow thinks that there's another instance of this function which starts at 0x3abdc3.

I tried a number of addresses.

Current symbolication API:

0x3b7b74 - 0x1be94 = 0x39bce0
0x3a5860 - 0x9b80 = 0x39bce0
0x39cb00 - 0xe20 = 0x39bce0
0x39c22c - 0x54c = 0x39bce0
0x3a9cac - 0xdfcc = 0x39bce0
0x3b7b84 - 0x1bea4 = 0x39bce0
0x3aec08 - 0x12f28 = 0x39bce0
0x3ab26c - 0xf58c = 0x39bce0
0x3a5574 - 0x9894 = 0x39bce0
0x3a9d1c - 0xe03c = 0x39bce0
0x3aec78 - 0x12f98 = 0x39bce0
0x3b7c1c - 0x1bf3c = 0x39bce0
0x39c298 - 0x5b8 = 0x39bce0
0x39cae4 - 0xe04 = 0x39bce0

Eliot:

0x3b7b74 - 0xbdb1 = 0x3abdc3
0x3a5860 - 0x9b80 = 0x39bce0
0x39cb00 - 0xe20 = 0x39bce0
0x39c22c - 0x54c = 0x39bce0
0x3a9cac - 0xdfcc = 0x39bce0
0x3b7b84 - 0xbdc1 = 0x3abdc3
0x3aec08 - 0x2e45 = 0x3abdc3
0x3ab26c - 0xf58c = 0x39bce0
0x3a5574 - 0x9894 = 0x39bce0
0x3a9d1c - 0xe03c = 0x39bce0
0x3aec78 - 0x2eb5 = 0x3abdc3
0x3b7c1c - 0xbe59 = 0x3abdc3
0x39c298 - 0x5b8 = 0x39bce0
0x39cae4 - 0xe04 = 0x39bce0

The bad symbol address is introduced during symcache generation.

% cargo run --release -p symcache_debug -- -c long-drawquad-xul.symcache 
[...]
          39b4f4 ps_text_run_DEBUG_OVERDRAW_TEXTURE_2D_frag::swgl_drawSpanRGBA8()
          39bb64 ClipRect::ClipRect(Texture const&)
          39bce0 void draw_quad_spans<unsigned int>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3abdc3 void draw_quad_spans<unsigned int>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3bbe06 void draw_quad_spans<unsigned int>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3ca0cc void draw_quad_spans<unsigned char>(int, glsl::vec2_scalar*, unsigned int, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3ce7d8 void draw_perspective_spans<unsigned int>(int, glsl::vec4_scalar*, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3de8b9 void draw_perspective_spans<unsigned int>(int, glsl::vec4_scalar*, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3ee8ff void draw_perspective_spans<unsigned int>(int, glsl::vec4_scalar*, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3f421c void draw_perspective_spans<unsigned char>(int, glsl::vec4_scalar*, glsl::vec3*, Texture&, Texture&, ClipRect const&)
          3f7a94 void prepare_row<unsigned char>(Texture&, int, int, int, bool, DepthRun*, unsigned int, DepthCursor*)
          3f819c Context::~Context()
[...]

This function is longer than 65k. I'll try to see if there's a better way to handle this in the symcache format, which doesn't expect such long functions.

You're using the words "production" and "staging", but you're comparing the current symbolication API in production with the new Eliot symbolication API in stage. You could also compare the current symbolication API in stage with the new Eliot symbolication API in stage. We have two different server environments (stage and production) and two different services (current symbolication API and the new Eliot symbolication API). Eliot only exists in stage because it's not production-ready, yet.

I appreciate you looking into this and filing a bug against symbolic. I'll keep tabs on that issue and update Eliot as soon as there's a new symbolic release. Thank you!

Priority: -- → P2

Ah, thanks for the clarification w.r.t. production/staging.

This is a regression but should not block rollout of Eliot. We can work around it on the profiler side.
It only affects a very small number of functions.

Oh! I didn't realize--I thought this was a blocker. My bad.

I'll look into scheduling getting Eliot to prod.

The underlying problem here was a bug in symbolic (comment #3) and that appears to be fixed, so I think this is fixed now.

Markus: When you have some time, can you verify whether this is fixed now or not?

Flags: needinfo?(mstange.moz)

It is very likely to be fixed. I can reopen this bug if I see it happening again.

Status: NEW → RESOLVED
Closed: 2 years ago
Flags: needinfo?(mstange.moz)
Resolution: --- → WORKSFORME

Moving to Eliot product.

Component: Symbolication → General
Product: Tecken → Eliot
You need to log in before you can comment on or make changes to this bug.