Closed Bug 910789 Opened 11 years ago Closed 9 years ago

update fjord sampledata generation to include api-like data

Categories

(Input Graveyard :: Code Quality, defect)

Product:
Input Graveyard
Component:
Code Quality
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

Details

(Whiteboard: u=dev c=codequality p=1 s=input.2015q1) 

To help contributors and developers get started on the Fjord codebase which drives Input, we have a sampledata generator to generate sample data. This makes it easier to "bootstrap" the environment so you can work on interesting things. Otherwise, you'd have to go fill out a bunch of feedback responses on your own. Ew.

We're adding an Input API (see bug #895675). We need to enhance the sampledata generator to add sample data that looks like it came from the API.
Putting this in the 2013q3 backlog, but it's not crucial it gets done during q3.

Adding the mentor flag--this is a good bug for someone new to work on.

The code for generation sample feedback response data is in fjord/feedback/sampledata.py. 

This bug covers two changes:

1. add another loop at the bottom like the "create 5 happy responses" and "create 5 sad responses" that covers "create 5 responses from the api"

2. modify the section that runs when the samplesize was specified to also create api-like feedback


This is a mentored bug. If you're interested in working on it, comment here or ping willkg in #input on irc.freenode.net.
Whiteboard: u=dev c=codequality p= s=input.2013q3 [mentor=willkg]
Nixing the sprint information. 2013q3 is over and we don't have to do this for 2013q4.
Whiteboard: u=dev c=codequality p= s=input.2013q3 [mentor=willkg] → u=dev c=codequality p= s= [mentor=willkg]
Hi,

I've wanted to get started contributing to Mozilla, and this looks like a good first assignment. I like doing work to support TDD, and have done a lot of Python programming.
Awesome! Assigning this to robb!

I'm on PTO until Monday, December 2nd, but I'll be reading email once a day. If you run into problems say something in the bug comments.
Assignee: nobody → robb
Hi Rob! How are you faring with this bug?
Hi Will, thanks for checking in. I was gone during Thanksgiving as well. I'm getting my dev environment set up using vagrant and Ubuntu 12.04. I'm caught for the moment on less via npm, but I'm sure I'll figure it out.
Hmm - I'm stuck at trying to run the web server. It returns a 503. I don't see a log directory anywhere - is there a place I can find error output? I ran the tests, and I get 23 errors.
Hi Will, I've got my dev environment all set up and running the app now. I'm looking forward to getting started. 

I get this, though, when trying to run generatedata: ConnectionError: HTTPConnectionPool(host='localhost', port=9200) Is there some additional process I need to run that listens on port 9200?  Or is it trying to connect to the http server there (I started it up at 8000, following the recipe.)
Mentor: willkg
Whiteboard: u=dev c=codequality p= s= [mentor=willkg] → u=dev c=codequality p= s=
Robb: I really apologize! I don't think I ever saw your question and it's been 7 months and that sucks. 

What's going on is that Fjord tries to connect to Elasticsearch at localhost port 9200. You need to be running Elasticsearch for that to work.

In the last 7 months, I've made changes to the sampledata stuff so it does a bulk create which skips the Elasticsearch indexing. Further, we've created a new Vagrant-based development environment which has a functional Elasticsearch in it. So the issues you bumped into shouldn't be a problem anymore.

Let me know if you want to keep working on this.
I haven't heard back from Robb in two months, so I'm unassigning this and putting it back in the pool.

Robb: If your're still working on this, let me know.
Assignee: robb → nobody
Hi, 

I would like to work on this bug. Looking forward to your help.

Thanks
Adnan
Hi, 

I've set my dev environment all and Fjord running perfectly. I'm looking forward to getting started.

Thank you.
Adnan
Awesome! I'm assigning it to you. Let me know if you have any problems.
Assignee: nobody → adnan.ayon
(In reply to Will Kahn-Greene [:willkg] from comment #13)
> Awesome! I'm assigning it to you. Let me know if you have any problems.

Thanks for your response. Yes, I've a question regrading the first change. 

> add another loop at the bottom like the "create 5 happy responses" and "create 5 sad responses" that covers "create 5 responses from the api"

Should I create 5 responses from fjord API which I found here.

http://fjord.readthedocs.org/en/latest/api.html
Yes.

It's worth looking at the data that is generated using the API and basing it on that. For example, the following are all feedback responses that were posted via the API:

* https://input.mozilla.org/en-US/dashboard/response/4643080
* https://input.mozilla.org/en-US/dashboard/response/4642985
* https://input.mozilla.org/en-US/dashboard/response/4640271
Hi, Can I use Requests(http://docs.python-requests.org/en/latest/) module?
I'm not sure I understand why requests helps here. How were you thinking of implementing this?
Tagging Adnan with a needinfo to answer the question in comment #17.
Flags: needinfo?(adnan.ayon)
(In reply to Will Kahn-Greene [:willkg] from comment #17)
> I'm not sure I understand why requests helps here. How were you thinking of
> implementing this?

Sorry for late reply. Because, I think I can achieve this objective "create 5 responses from the api" with writing less lines of codes with maintaining proper efficiency. Actually, I'm quite new in python. Please correct me, if I'm wrong. 

Thank you.
Flags: needinfo?(adnan.ayon)
(In reply to Adnan from comment #19)
> (In reply to Will Kahn-Greene [:willkg] from comment #17)
> > I'm not sure I understand why requests helps here. How were you thinking of
> > implementing this?
> 
> Sorry for late reply. Because, I think I can achieve this objective "create
> 5 responses from the api" with writing less lines of codes with maintaining
> proper efficiency. Actually, I'm quite new in python. Please correct me, if
> I'm wrong. 

I'm not sure what you mean by that.

Can you tell me specifically how you were thinking about implementing this? An algorithm or outline of the code would be a good way to help me understand.
(In reply to Will Kahn-Greene [:willkg] from comment #20)
> (In reply to Adnan from comment #19)
> > (In reply to Will Kahn-Greene [:willkg] from comment #17)
> > > I'm not sure I understand why requests helps here. How were you thinking of
> > > implementing this?
> > 
> > Sorry for late reply. Because, I think I can achieve this objective "create
> > 5 responses from the api" with writing less lines of codes with maintaining
> > proper efficiency. Actually, I'm quite new in python. Please correct me, if
> > I'm wrong. 
> 
> I'm not sure what you mean by that.
> 
> Can you tell me specifically how you were thinking about implementing this?
> An algorithm or outline of the code would be a good way to help me
> understand.

Using Requests:

https://gist.github.com/anonymous/a7805991ed762067568c

Using urllib2

https://gist.github.com/anonymous/ae65db6e623038074729

Well, I was able to post feedback responses via API.
Ahh... ok. That's not what this bug is about. This bug is about adding some code to sampledata.py to create entries in the database that have the same sort of data an entry created by using the API would have.

Here's the code that's creating happy/sad feedback that looks like it was submitted using one of the feedback forms on Input:

https://github.com/mozilla/fjord/blob/68508f8b75a86f25fb07798451afdb1849bea57a/fjord/feedback/sampledata.py#L109

We need to add some code that creates happy/sad feedback that looks like it was submitted using the API.

This is running via the command line, so there's no server listening to API requests. Ergo you won't be able to use urllib2 or the requests library.

Does that clarify what this bug is about enough that you can work on it?
How's it going? Are you still working on this?
Flags: needinfo?(adnan.ayon)
I haven't heard from the assignee in a while, so I'm putting this back in the pool.
Assignee: adnan.ayon → nobody
Flags: needinfo?(adnan.ayon)
Tossing this in Adam's queue.
Mentor: willkg
Whiteboard: u=dev c=codequality p= s= → u=dev c=codequality p= s=input.adam
Moving things out of the input.adam sprint.
Whiteboard: u=dev c=codequality p= s=input.adam → u=dev c=codequality p= s=input.2015q2
Grabbing this to do this week.
Assignee: nobody → willkg
Whiteboard: u=dev c=codequality p= s=input.2015q2 → u=dev c=codequality p= s=input.2015q1
I also did a minor refactoring of the code so that we were doing combinations of things that were sane.

In a PR: https://github.com/mozilla/fjord/pull/544

Landed in master: https://github.com/mozilla/fjord/commit/7e6defdae42afb8f3b892ba2bc8ae2e09a058a61

Marking this as FIXED given this is just dev environment stuff.
Whiteboard: u=dev c=codequality p= s=input.2015q1 → u=dev c=codequality p=1 s=input.2015q1
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: Input → Input Graveyard
You need to log in before you can comment on or make changes to this bug.