Closed Bug 577613 Opened 11 years ago Closed 9 years ago

[tracker] create a webapp to analyze .dlls and where users/community members can enter more information pertaining to a DLL

Categories

(Webtools Graveyard :: Dragnet, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aphadke, Assigned: brandon)

References

Details

(Whiteboard: [Q42011wanted] [Q12012wanted])

DLL names play a crucial role in Windows and lot of spyware/malware/adware infect users PCs with bad DLLs. Affected users usually search the DLL name but end up going in circles as there isn't an easy way to find information about the DLL.
This project entails building a wiki site where users can update information for given DLLs and thereby (hopefully) help users in identifying + cleaning up the right DLL.
Depends on: 577617
Whiteboard: ETA 7/28
Whiteboard: ETA 7/28 → ETA 7/28 to start work
Whiteboard: ETA 7/28 to start work → ETA 8/22 to start work
Target Milestone: --- → 2010-08.2
Whiteboard: ETA 8/22 to start work → ETA 9/10 to start work - fixing SOLR bug (high priority)
Target Milestone: 2010-08.2 → 2010-09.2
Whiteboard: ETA 9/10 to start work - fixing SOLR bug (high priority) → moving at later date
Target Milestone: 2010-09.2 → 2010-11.1
Assignee: aphadke → nobody
Group: metrics-private
Component: Hadoop Cluster → Socorro
Product: Mozilla Metrics → Webtools
QA Contact: hadoop-cluster → socorro
Target Milestone: 2010-11.1 → ---
Version: unspecified → Trunk
I'm finding that bing and a new search engine at http://blekko.com/ are better choices for searching for .dll info.   They don't seem to have been attacked with SEO that hogs all the top links in search results for .dll names and directs you to ad's rather than useful information about .dll's and problems associated with them.
chofmann - what should be our next logical steps?
my suggestion:
I can generate a list of top 1000 most common DLLs in firefox 4.0, install media-wiki on one of our servers, open a mechanical turk project and crowd-source the information?
We will need some USD budget, approx. $500 for crowdsourcing the first 1000 DLLs.

thoughts?
yeah, lets get the media-wiki site or some quick app populated with a 1000 or more .dll names.

we should also try and catch:
- some measure of relative frequency of the appearance of the .dll in the module list
- list or link to signatures where the .dll is named in the signatures of crashes
- OS version(s) where the .dll appears
- version info associated with the .dll name if provided
- "first appearance" of the .dll (this may be too hard).
- "# of total crashes at all addresses"
"# of unique crashing addresses"

and any other usefull info we think we could begin to populate the pages with.

then we also want standard places on the .dll info pages for a description and links to other references about the .dll.

the mechanical turk part can come later if we think we need that for additional crowd sourcing.
I'll try to finish some of the things mentioned in comment #3 by 4/1

-anurag
Assignee: nobody → aphadke
Whiteboard: moving at later date
also want a place to link in particular bugs where the .dll's have come into play and maybe bugzilla contact info where people have helped from companies behind the .dll development (this later might have to be behind auth)

some of this info really could be populated and updated from several sources like bugzilla, crash-stats, dbaron's reports so maybe thinking about this more as a web app than just a static page might be the way to approach the problem.  combining the hand written narative part, with the other updating sections could make this a very effective tool.

here is rough example for one that particular troublesome .dll that's been showing up lately.

RadioWMPCore.dll

Highly Correlated crash signatures (from dbaron's reports?)

   RadioWMPCore.dll shows up in 14%  of 54658 of 388604 reports for Firefox 4.0

Some signatures where the .dll has stronger correlation

 [@ mozalloc_abort(char const* const) | mozalloc_handle_oom() | nsHtml5Tokenizer::stateLoop(int, unsigned short, int, unsigned short*, int, int, int) ]

   21% have RadioWMPCore.dll 
     21% (188/900) vs.  14% (54658/388604) RadioWMPCore.dll

Narative:

   RadioWMPCore.dll appears to be distributed as part of Conduit ToolBars 

   Adware.Ezula is also known to package Conduit ToolBars 

Some Bugs Where RadioWMPCore.dll is mentioned (bugzilla search?):

Bug 637279 - Crash [@ mozalloc_abort(char const* const) | mozalloc_handle_oom() | nsHtml5Tokenizer::stateLoop(int, unsigned short, int, unsigned short*, int, int, int) ] mainly at hasznaltauto.hu and menetrendek.hu

Bug 633445 - Crash [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ]


References:

Adware.Ezula - http://www.symantec.com/security_response/writeup.jsp?docid=2003-080116-4112-99


Common Install locations:

Directory:    
%APPDATA%\Mozilla\Firefox\Profiles\5s6uj0hw.default\extensions\{66f2e20d-0da8-4c11-a9c8-dd8477b88acd}\components\RadioWMPCore.dll

also found in the firefox app directory... 

Recent Crashes for the .dll ( from daily bug stats )

   1 20110326-crashdata.csv:RadioWMPCore.dll@0x71 Firefox 4.0 Windows NT 6.1.7600
   1 20110326-crashdata.csv:RadioWMPCore.dll@0x69 Firefox 4.0 Windows NT 6.1.7601 Service Pack 1
   1 20110326-crashdata.csv:RadioWMPCore.dll@0x18300 Firefox 4.0 Windows NT 5.1.2600 Service Pack 3
   1 20110326-crashdata.csv:RadioWMPCore.dll@0x1 Firefox 4.0 Windows NT 6.1.7600
   1 20110326-crashdata.csv:RadioWMPCore.dll@0x0 Firefox 4.0 Windows NT 6.1.7601 Service Pack 1
   1 20110325-crashdata.csv:RadioWMPCore.dll@0x0 Firefox 4.0 Windows NT 6.1.7600
   1 20110324-crashdata.csv:RadioWMPCore.dll@0x3a Firefox 4.0b12 Windows NT 6.0.6002 Service Pack 2
   1 20110324-crashdata.csv:RadioWMPCore.dll@0x3870 Firefox 4.0 Windows NT 6.1.7601 Service Pack 1
   1 20110324-crashdata.csv:RadioWMPCore.dll@0x0 Firefox 4.0 Windows NT 6.1.7601 Service Pack 1
   1 20110323-crashdata.csv:RadioWMPCore.dll@0x16751 Firefox 4.0 Windows NT 6.1.7601 Service Pack 1
   1 20110323-crashdata.csv:RadioWMPCore.dll@0x14941 Firefox 4.0 Windows NT 6.1.7600
   1 20110321-crashdata.csv:RadioWMPCore.dll@0x70 Firefox 4.0b10 Windows NT 6.1.7600
   1 20110321-crashdata.csv:RadioWMPCore.dll@0x10075 Firefox 4.0b10 Windows NT 6.1.7600
Please hold off on the wiki install for the moment. That isnt really in our domain and we should check with Laura and/or other webdevs about that part of it.  Especially since this site would eventually be publically accessible.
bugzilla stuff could come from the simple query

https://bugzilla.mozilla.org/buglist.cgi?quicksearch=RadioWMPCore.dll

that turns up a pretty interesting list.
here is another idea for the index of all the .dll's that we want to cover.

http://people.mozilla.org/~chofmann/crash-stats/windows-dlls-around-at-time-of-topcrash.txt

this is a list of about 6000 .dll's that were in reports from 4.0 users that dbaron used in the module correation report.

left number is number of instances where the dll was observed in crash reports
next number is number of reports for firefox 4.0 sample on the particular day
followed by the .dll name.

its interesting that about 5000 of the .dll's show up more than 3 times so maybe thats a good long tail cut off point.

to this listing we want to start to hook up the other meta data in additional colums or maybe links to more detailed pages.
it would be cool if this could be searchable and might return results like this for all the .dll names that contained the search term "hook", for processes that might be hooking into the firefox process.

http://people.mozilla.org/~chofmann/crash-stats/hook-dlls.txt
Based on above comments, we seem to be having enough information to have a basic site up and running. 

Given the complexity of this project, we need more structure around this, either to be integrated as part of Socorro UI or as a different site. Adding Laura Thomson as CC list so she can help prioritize the tasks.
Assignee: aphadke → nobody
Quick question for chofmann: I know when we last talked about this idea you had
concerns about malware vendors spamming the site if it was a wiki.  Is this no
longer a concern?
(In reply to comment #11)
> Quick question for chofmann: I know when we last talked about this idea you had
> concerns about malware vendors spamming the site if it was a wiki.  Is this no
> longer a concern?

yeah, any community generated and maintained content is open to that kind of attack.  we probably want login and levels of trusted participation and review to edit the sensitive parts of the content and ways to avoid spam.

finding ways to prohib the inclusion of links to untrusted malware removal tools or "so called solutions to .dll problems" would also be a concern, but we have the same kind of risks on sumo.
here is some interesting data with more scrapping from dbaron's collrealtions and a bit of knowledge about what particular .dll's do.

combined with a bit of research on what sahook is we know that its part of 
Adware.Zango.AN and possibly other malware.

In this case sahook.dll is around more than 90% of the time when a particular set of crash signatures occur.

http://people.mozilla.org/~chofmann/crash-stats/sahook-around-90+pct.txt

stuff like this provides more context for the crashes, and might help us toward blocking or AV outreach to protect firefox users.
Summary: create a wiki page/site where users/community members can enter more information pertaining to a DLL → create a webapp to analyze .dlls and where users/community members can enter more information pertaining to a DLL
also wanted to say that in this case we might save 10,000+ crashes a day if we got sahook and its companion malware off of the crashing systems, but the first step is removal, then remeasure.
(In reply to comment #10)
> Given the complexity of this project, we need more structure around this,
> either to be integrated as part of Socorro UI or as a different site. Adding
> Laura Thomson as CC list so she can help prioritize the tasks.

If part of Socorro UI is something you probably need to figure out on the Socorro team side, but it would be really nice if it could be (probably as a further step) integrated into Socorro UI so one could get some info (we'll need to figure out what exactly) directly in modules and correlation lists.
assigning to laura as she is leading this effort.
Assignee: nobody → laura
Will write up a spec in the next week or two.  Plan on delivering version 1 in 1.9.
Target Milestone: --- → 1.7.8
Assignee: laura → bsavage
Target Milestone: 1.7.8 → 2.0
Whiteboard: Q3
Target Milestone: 2.0 → 2.1
Blocks: 439679
Blocks: 525098
Target Milestone: 2.1 → 2.2
Target Milestone: 2.2 → 2.3
Component: Socorro → Dragnet
QA Contact: socorro → dragnet
Summary: create a webapp to analyze .dlls and where users/community members can enter more information pertaining to a DLL → [tracker] create a webapp to analyze .dlls and where users/community members can enter more information pertaining to a DLL
Target Milestone: 2.3 → 1.0
The application has been built, and is awaiting hardware for staging and testing. Resolving fixed.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Whiteboard: Q3 → Q42011wanted
Whiteboard: Q42011wanted → [Q42011wanted] [Q12012wanted]
any update on when the hardware will be availible?
It's staged and awaiting secreview.  I'll cc you on the appropriate bugs.
Blocks: 441462
Blocks: 497876
Duplicate of this bug: 525098
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.