Closed Bug 960044 Opened 10 years ago Closed 10 years ago

[dev] IARC submissionID/secCode form fails + pingback never comes on -dev

Categories

(Marketplace Graveyard :: Developer Pages, defect, P1)

Product:
Marketplace Graveyard
Component:
Developer Pages
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
2014-01-21

People

(Reporter: AndreiH, Unassigned)

References

(
URL
)

Details

(Whiteboard: [fromAutomation] [qa+] [incorrect_implementation]) 

Our automated tests fail when submitting an app at Content Ratings step.

STR:

1. Go to https://marketplace-dev.allizom.org/developers and login
2. Upload an app, or if you already have one try to set the Content Ratings
3. When you get to the Content Ratings step, enter submission_id and security_code if you already have them set.
4. Click Submit button.

Expected results:
"Congratulations, your app submission is now complete and will be reviewed shortly!" message should appear and app submission is complete.

Actual results:
"Invalid submission_id or security_code message appears when testing manually, but nothing appears/happens when automated test is running, not even a warning message appears.

Here is a screencast:
http://screencast.com/t/z0PfyUyJGRz
URL: https://marketplace-dev.allizom.org/d...
OS: Linux → All
Hardware: x86_64 → All
Whiteboard: [fromAutomation]
Also please note that the content rating should be automatically set on MP-dev when IARC form is completed , but nothing happens and the content rating cannot be set.

Please let me know if a new bug is needed, but I am pretty sure that both issues are related.


NOT reproducible on Stage or Production.
Severity: normal → major
Target Milestone: --- → 2014-01-21
Rob, Kevin, would one of you want to take a look at this? Not sure how to triage.
There's a couple things going on here...

1. Clicking the button that opens the pop-up window, answering the questions, etc should result in a content rating. If you go that route you do not need to add the submission ID and secret code in the 2nd island.

2. The pingback we get from IARC seems to be taking longer than expected. Once you're done with the questionnaire and have a content rating, IARC ping us with the results. When that happens the spinner on the 1st island should stop and redirect to the content ratings page. This doesn't seem to be happening from what I see in the screencast. Perhaps it's just taking longer than expected?

If that pingback is never coming that's an important issue to address. The confusion between the two methods of obtaining a content rating would be a less urgent thing to consider as well.
Priority: -- → P1
Blocks: 929812
After testing on -dev and watching the logs it appears the pingback never comes. I received the content rating email but not the pingback and the submission ID indeed fails to be found.
The IARC has been observing several timeouts per day - I wonder if this is one of them.  From their side they see that they send the ping request, wait 30s, and then disconnect with a 500 error.  Recently they also saw a 502 (bad gateway) from production.

I investigated a single instance of this yesterday and in that case they saw a timeout but on our end we got the info and returned a 200, which made me think it was outside of our app.  May or may not be related to this.
https://github.com/mozilla/zamboni/commit/9a858ae 

Flipped IARC_ENV to 'test'. Let's see if that helps.
(In reply to Rob Hudson [:robhudson] from comment #7)
> https://github.com/mozilla/zamboni/commit/9a858ae 
> 
> Flipped IARC_ENV to 'test'. Let's see if that helps.

This fixed the problem of manually entering the content rating credentials finding nothing. But the pingbacks still do not seem to be flowing through.
Updating the status*
+ESRB folks - this may be the cause of the timeouts.
Please let us know if there's anything we can provide or do to help triage this.
Also if I set the Content Ratings on app and then use the submission_id and security_code on another app I get "Invalid submission ID or security code." message. This happens on dev and stage also. Can't we use the same ID and security code on more apps?
Based on some of the comments here, I took a look at our logs for web service calls. It looks like there have been a number of calls for older submissions, particularly submissions 813 and 825. Before you went live on Tuesday, we went through and cleaned our data, retiring all of the "test" submissions put in place by all the various testers in our system, including Mozilla testers. Subsequent to that clean up, when your system went live, the valid Submission IDs started at 860. Any submission prior to 860 has been retired and won't return data.

Additionally, I noticed a number of calls for Submission IDs in the 200s and 300s. Those Submission IDs would have originated from the Demo system and would not correspond to data in the Prod system.

I hope this helps clear up some of the confusion. If you need us to unretire a submission before 860 for testing purposes, let us know and we'll try to help you out.
When testing we used submission ID 813 on all our apps that were submitted. I guess that's why this won't work anymore. So we should change our submission ID in our tests?
Is this the correct way to submit our apps? (using the same submission ID on all of our test apps) Or you will need to get the data cleaned up again?
We could potentially bring 813 out of retirement for use in your testing. However, making it live will make it look like a real submission made by a developer in the system. This means it will be visible to the various rating authorities in their interface. The rating authorities are much more concerned with submissions that have been connected to an actual application and subsequently released (as indicated by the SET_STOREFRONT_DATA call) so having a single test submission look live may be acceptable (as long as the process doesn't complete the release call, indicating to the rating authorities that the application is available to the public and opening it to potential testing and review). However, I cannot speak to your test process. Let us know if you need us to make this change.
Summary: [dev] App submission fails on Content Ratings step → [dev] IARC submissionID/secCode form fails + pingback never comes on -dev
Christine, have you gotten any web service calls to the DEMO box from 1/16/2014 2pm to 3pm PST? And if so, where were the pingback requests being sent to?
Kevin, I am not home yet for the evening and won't be for another couple hours, but as soon as I am, I will log onto the server and check for those calls. I can tell you then whether any ping backs have gone out, but not the URLs pinged, we don't store that. I could, however provide the tokens you passed, which may help you recreate the URLs?
That would be helpful. Thanks, Christine.
Kevin, I don't see any calls since 1/7 in the DEMO system. Over the weekend, we made a significant change to the way the pingback works in the DEMO system, I am assuming you are testing that change? From the sounds of it, we're having trouble with it. I will take a look and see if I can find any issue with the new set up.
The change had caused an issue with the pingback's database connection. I got that repaired. Please try again. Sorry for the inconvenience.
Christine, 

Is there a way we can use a Content rating in our automated test which won't trigger false alarm to the rating bodies? How would you recommend we proceed ahead with our test?

Thanks!
I will discuss this with our team in the morning. We will let you know any suggestions we have.
Thanks Christine, the pingback worked for me on -dev.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Kevin, the the issue described in comment #1 is still reproducible(both -dev and stage). Should we log another bug for this issue?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
This may or not fix it. 

https://github.com/mozilla/zamboni/commit/51845d2f2a796911b650521e181ea542c81d56f9
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Whiteboard: [fromAutomation] → [fromAutomation] [qa+]
Thanks for the fix! We currently want to use Subm-381 as our next Submission ID to test on apps that are submitted on dev. But what I saw it's that we can't use the same submission ID on dev and also on stage. Do we need another generated ID also for stage?
Flags: needinfo?(cmccarthy)
Andrei, I'm not sure I have a direct answer for you. Here's the information I can provide:

We have given Mozilla access to 2 versions of the IARC system, DEMO (which is a sandbox) and PROD (which is live). Submission IDs in the two systems are not cross-compatible. The DEMO system is generally producing lower ID numbers than the PROD system. A 300s ID like 381 is probably from the DEMO system (PROD is nearing or has recently passed 1000).

I do not know which version of our system your dev and stage tiers are utilizing. If they are both using DEMO, I can take a look when I get to the office and see if there is an issue  with ID 381.

I'm sorry I don't have a more direct answer. Let me know if there is anything else I can provide.
Flags: needinfo?(cmccarthy)
I can still reproduce this issue in https://marketplace-dev.allizom.org/developers/app/test-app-rememgrber/content_ratings/edit on FF29 (Win 7). The Content Rating page is not updated automatically.
Please see screencast http://screencast.com/t/AZFPRvNnuxD9
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I took a quick look in our webservice log and don't see an entry for ID 386 (depicted in the screencast). This suggests that the pingback was never sent at all, perhaps bad data passed in? Or the new pingback executable could have failed before sending it. Hard to pin that down without knowing all of the data.
Christine, is there any data we could give you that would help?
Kevin, I'd be interested whether another test produces the same result. The last pingback I see in the log is 3:30 this morning (EST). I don't know whether there have been many attempted since then or just 1. If I know you're trying, I can watch the server for the executable to launch, which will determine whether the issue is with the web app not calling the executable (since it escapes around the call if there's no token and pingback URL) or if the executable is failing.
I had just tried again 5 minutes with Submission ID 387.

At 10:40am PST, I will do it several times in a row. Let me know if you see anything.

Thanks!
Kevin, I'm not seeing the executable starting up. I took a quick look at the launching page in the application and made a change. There haven't been changes made to either the executable or the application today, this is very odd. If you've finished your tests, one more could help see if that change helped at all.
Sorry Christine, I didn't do it. I'll do it again now.
There's currently a compilation error on the IARC DEMO tool.
That's what I get for hot fixing. Forgot to escape a character. Should be good now.
Trying now!
I see a pingback for submission 388. The executable popped up and I have the result in the database.
The pingback came and everything came, looks like that change did something. Thanks, Christine! We'll have some other people verify.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Verified as fixed
Status: RESOLVED → VERIFIED
Whiteboard: [fromAutomation] [qa+] → [fromAutomation] [qa+] [incorrect_implementation]
You need to log in before you can comment on or make changes to this bug.