[dev] IARC submissionID/secCode form fails + pingback never comes on -dev

VERIFIED FIXED in 2014-01-21

Status

Marketplace
Developer Pages
P1
major
VERIFIED FIXED
4 years ago
4 years ago

People

(Reporter: AndreiH, Unassigned)

Tracking

2014-01-21
Points:
---

Details

(Whiteboard: [fromAutomation] [qa+] [incorrect_implementation], URL)

(Reporter)

Description

4 years ago
Our automated tests fail when submitting an app at Content Ratings step.

STR:

1. Go to https://marketplace-dev.allizom.org/developers and login
2. Upload an app, or if you already have one try to set the Content Ratings
3. When you get to the Content Ratings step, enter submission_id and security_code if you already have them set.
4. Click Submit button.

Expected results:
"Congratulations, your app submission is now complete and will be reviewed shortly!" message should appear and app submission is complete.

Actual results:
"Invalid submission_id or security_code message appears when testing manually, but nothing appears/happens when automated test is running, not even a warning message appears.

Here is a screencast:
http://screencast.com/t/z0PfyUyJGRz
(Reporter)

Updated

4 years ago
OS: Linux → All
Hardware: x86_64 → All
Whiteboard: [fromAutomation]

Comment 1

4 years ago
Also please note that the content rating should be automatically set on MP-dev when IARC form is completed , but nothing happens and the content rating cannot be set.

Please let me know if a new bug is needed, but I am pretty sure that both issues are related.


NOT reproducible on Stage or Production.
Severity: normal → major
Target Milestone: --- → 2014-01-21
Rob, Kevin, would one of you want to take a look at this? Not sure how to triage.
There's a couple things going on here...

1. Clicking the button that opens the pop-up window, answering the questions, etc should result in a content rating. If you go that route you do not need to add the submission ID and secret code in the 2nd island.

2. The pingback we get from IARC seems to be taking longer than expected. Once you're done with the questionnaire and have a content rating, IARC ping us with the results. When that happens the spinner on the 1st island should stop and redirect to the content ratings page. This doesn't seem to be happening from what I see in the screencast. Perhaps it's just taking longer than expected?

If that pingback is never coming that's an important issue to address. The confusion between the two methods of obtaining a content rating would be a less urgent thing to consider as well.
Priority: -- → P1
Blocks: 929812
After testing on -dev and watching the logs it appears the pingback never comes. I received the content rating email but not the pingback and the submission ID indeed fails to be found.
The IARC has been observing several timeouts per day - I wonder if this is one of them.  From their side they see that they send the ping request, wait 30s, and then disconnect with a 500 error.  Recently they also saw a 502 (bad gateway) from production.

I investigated a single instance of this yesterday and in that case they saw a timeout but on our end we got the info and returned a 200, which made me think it was outside of our app.  May or may not be related to this.
https://github.com/mozilla/zamboni/commit/9a858ae 

Flipped IARC_ENV to 'test'. Let's see if that helps.
(In reply to Rob Hudson [:robhudson] from comment #7)
> https://github.com/mozilla/zamboni/commit/9a858ae 
> 
> Flipped IARC_ENV to 'test'. Let's see if that helps.

This fixed the problem of manually entering the content rating credentials finding nothing. But the pingbacks still do not seem to be flowing through.

Comment 10

4 years ago
Updating the status*
+ESRB folks - this may be the cause of the timeouts.

Comment 12

4 years ago
Please let us know if there's anything we can provide or do to help triage this.
(Reporter)

Comment 13

4 years ago
Also if I set the Content Ratings on app and then use the submission_id and security_code on another app I get "Invalid submission ID or security code." message. This happens on dev and stage also. Can't we use the same ID and security code on more apps?

Comment 14

4 years ago
Based on some of the comments here, I took a look at our logs for web service calls. It looks like there have been a number of calls for older submissions, particularly submissions 813 and 825. Before you went live on Tuesday, we went through and cleaned our data, retiring all of the "test" submissions put in place by all the various testers in our system, including Mozilla testers. Subsequent to that clean up, when your system went live, the valid Submission IDs started at 860. Any submission prior to 860 has been retired and won't return data.

Additionally, I noticed a number of calls for Submission IDs in the 200s and 300s. Those Submission IDs would have originated from the Demo system and would not correspond to data in the Prod system.

I hope this helps clear up some of the confusion. If you need us to unretire a submission before 860 for testing purposes, let us know and we'll try to help you out.
(Reporter)

Comment 15

4 years ago
When testing we used submission ID 813 on all our apps that were submitted. I guess that's why this won't work anymore. So we should change our submission ID in our tests?
Is this the correct way to submit our apps? (using the same submission ID on all of our test apps) Or you will need to get the data cleaned up again?

Comment 16

4 years ago
We could potentially bring 813 out of retirement for use in your testing. However, making it live will make it look like a real submission made by a developer in the system. This means it will be visible to the various rating authorities in their interface. The rating authorities are much more concerned with submissions that have been connected to an actual application and subsequently released (as indicated by the SET_STOREFRONT_DATA call) so having a single test submission look live may be acceptable (as long as the process doesn't complete the release call, indicating to the rating authorities that the application is available to the public and opening it to potential testing and review). However, I cannot speak to your test process. Let us know if you need us to make this change.

Updated

4 years ago
Summary: [dev] App submission fails on Content Ratings step → [dev] IARC submissionID/secCode form fails + pingback never comes on -dev

Comment 17

4 years ago
Christine, have you gotten any web service calls to the DEMO box from 1/16/2014 2pm to 3pm PST? And if so, where were the pingback requests being sent to?

Comment 18

4 years ago
Kevin, I am not home yet for the evening and won't be for another couple hours, but as soon as I am, I will log onto the server and check for those calls. I can tell you then whether any ping backs have gone out, but not the URLs pinged, we don't store that. I could, however provide the tokens you passed, which may help you recreate the URLs?

Comment 19

4 years ago
That would be helpful. Thanks, Christine.

Comment 20

4 years ago
Kevin, I don't see any calls since 1/7 in the DEMO system. Over the weekend, we made a significant change to the way the pingback works in the DEMO system, I am assuming you are testing that change? From the sounds of it, we're having trouble with it. I will take a look and see if I can find any issue with the new set up.

Comment 21

4 years ago
The change had caused an issue with the pingback's database connection. I got that repaired. Please try again. Sorry for the inconvenience.

Comment 22

4 years ago
Christine, 

Is there a way we can use a Content rating in our automated test which won't trigger false alarm to the rating bodies? How would you recommend we proceed ahead with our test?

Thanks!

Comment 23

4 years ago
I will discuss this with our team in the morning. We will let you know any suggestions we have.

Comment 24

4 years ago
Thanks Christine, the pingback worked for me on -dev.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED

Comment 25

4 years ago
Kevin, the the issue described in comment #1 is still reproducible(both -dev and stage). Should we log another bug for this issue?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 26

4 years ago
This may or not fix it. 

https://github.com/mozilla/zamboni/commit/51845d2f2a796911b650521e181ea542c81d56f9
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Whiteboard: [fromAutomation] → [fromAutomation] [qa+]
(Reporter)

Comment 27

4 years ago
Thanks for the fix! We currently want to use Subm-381 as our next Submission ID to test on apps that are submitted on dev. But what I saw it's that we can't use the same submission ID on dev and also on stage. Do we need another generated ID also for stage?
Flags: needinfo?(cmccarthy)

Comment 28

4 years ago
Andrei, I'm not sure I have a direct answer for you. Here's the information I can provide:

We have given Mozilla access to 2 versions of the IARC system, DEMO (which is a sandbox) and PROD (which is live). Submission IDs in the two systems are not cross-compatible. The DEMO system is generally producing lower ID numbers than the PROD system. A 300s ID like 381 is probably from the DEMO system (PROD is nearing or has recently passed 1000).

I do not know which version of our system your dev and stage tiers are utilizing. If they are both using DEMO, I can take a look when I get to the office and see if there is an issue  with ID 381.

I'm sorry I don't have a more direct answer. Let me know if there is anything else I can provide.
Flags: needinfo?(cmccarthy)

Comment 29

4 years ago
I can still reproduce this issue in https://marketplace-dev.allizom.org/developers/app/test-app-rememgrber/content_ratings/edit on FF29 (Win 7). The Content Rating page is not updated automatically.
Please see screencast http://screencast.com/t/AZFPRvNnuxD9
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 30

4 years ago
I took a quick look in our webservice log and don't see an entry for ID 386 (depicted in the screencast). This suggests that the pingback was never sent at all, perhaps bad data passed in? Or the new pingback executable could have failed before sending it. Hard to pin that down without knowing all of the data.

Comment 31

4 years ago
Christine, is there any data we could give you that would help?

Comment 32

4 years ago
Kevin, I'd be interested whether another test produces the same result. The last pingback I see in the log is 3:30 this morning (EST). I don't know whether there have been many attempted since then or just 1. If I know you're trying, I can watch the server for the executable to launch, which will determine whether the issue is with the web app not calling the executable (since it escapes around the call if there's no token and pingback URL) or if the executable is failing.

Comment 33

4 years ago
I had just tried again 5 minutes with Submission ID 387.

At 10:40am PST, I will do it several times in a row. Let me know if you see anything.

Thanks!

Comment 34

4 years ago
Kevin, I'm not seeing the executable starting up. I took a quick look at the launching page in the application and made a change. There haven't been changes made to either the executable or the application today, this is very odd. If you've finished your tests, one more could help see if that change helped at all.

Comment 35

4 years ago
Sorry Christine, I didn't do it. I'll do it again now.

Comment 36

4 years ago
There's currently a compilation error on the IARC DEMO tool.

Comment 37

4 years ago
That's what I get for hot fixing. Forgot to escape a character. Should be good now.

Comment 38

4 years ago
Trying now!

Comment 39

4 years ago
I see a pingback for submission 388. The executable popped up and I have the result in the database.

Comment 40

4 years ago
The pingback came and everything came, looks like that change did something. Thanks, Christine! We'll have some other people verify.
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED

Comment 41

4 years ago
Verified as fixed
Status: RESOLVED → VERIFIED

Updated

4 years ago
Whiteboard: [fromAutomation] [qa+] → [fromAutomation] [qa+] [incorrect_implementation]
You need to log in before you can comment on or make changes to this bug.