993758 - Access to public mailman list archives for Michael Trinkala

Status: RESOLVED FIXED

(Infrastructure & Operations :: Infrastructure: Mail, task)

Description

[Clarissa Sorenson [:CSO]]

From SN RITM0025373 -

One of my team is interested in using our corpus of public email lists to generate a searchable database. Is there an API he could access to slurp down this data, or some other way? Thanks!

Comment 1

[:Atoll]

Note that a usenet or google groups slurper would be able to pull them down easily, since they're all mirrored to usenet/etc (for the most part).

Comment 2

[Toby Elliott [:telliott]]

Adding trink to the bug, since he's the one who would actually be accessing the data.

Comment 3

[Ed Lim [:limed]]

Anything actionable here? Only way this is possible is to use something like the archives in usenet per comment 1. Other than that there is no API on the mailman part of things

Comment 4

[Mike Trinkala [:trink]]

Reading directly from the file system would be preferred.

Comment 5

[Ed Lim [:limed]]

(In reply to Mike Trinkala [:trink] from comment #4)
> Reading directly from the file system would be preferred.

For security reasons we cannot provide you with access to the server to actually read these files if they are indeed archived. Best course of action here for you is to read from NNTP.

The list of our lists that are synced via NNTP is listed here https://lists.mozilla.org/newsgroups.txt

Comment 6

[:Atoll]

If it's of any help, https://docs.python.org/3/library/nntplib.html has a brief, although useful, example of how to scrape a newsgroup archive of a mailing list using Python.

Comment 7

[Dan Parsons [:lerxst]]

Seems like we did the best we could here. R/Fing for now, feel free to reopen if necessary.