Use Embed.ly for temporary metadata collection

RESOLVED FIXED

Status

P1
normal
Rank:
11
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: ianbicking, Assigned: mancas)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [akita-alpha])

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
Until we get some of the other metadata stuff fully stood up, it will be nice to see a table of contents with some good data in it.

The easiest way is to use the Embed.ly API to get some data.  This should be done on PAGE ADD TIME (not at ToC render time).  You'll make a request to:

  GET https://api.embedly.com/1/extract?url={url}&key={api_key}

API key has been provided to some people on the team via email.

I propose we hardcode the API key in the client for now.  It has a fairly low limit (5000 requests per month), but this is a very temporary approach.

Note then that when you add a page you'll do something like:

function getPageMetadata(url) {
  return new Promise((resolve, reject) => {
    let result = {url: url};
    let req = new XMLHttpRequest();
    req.open("GET", `https://api.embedly.com/1/extract?url=${encodeURIComponent(url)}&key=${encodeURIComponent(api_key)}`);
    req.onload = function () {
      if (req.status != 200) {
        reject(req);
        return;
      }
      let extracted = JSON.parse(req.responseText);
      result.title = extracted.title;
      result.images = extracted.images;
      result.favicon_url = extracted.favicon_url;
      //maybe result.embedly = extracted ?
      resolve(result);
    };
  });
}

Docs are here: http://docs.embed.ly/docs/extract

The .images property is fairly rich, and some stuff (like entropy) we're unlikely to do ourselves, but we can keep it for now.
(Reporter)

Updated

3 years ago
Rank: 11
Priority: -- → P1
Whiteboard: [akita-alpha]
(Reporter)

Updated

3 years ago
Blocks: 1278584
(Assignee)

Updated

3 years ago
Assignee: nobody → b.mcb
Status: NEW → ASSIGNED
Created attachment 8761121 [details] [review]
[loop] mancas:bug1278677 > mozilla:akita
(Assignee)

Updated

3 years ago
Blocks: 1273506
(Assignee)

Updated

3 years ago
Attachment #8761121 - Flags: review?(standard8)
Attachment #8761121 - Flags: review?(fernando.campo)
Attachment #8761121 - Flags: review?(edilee)
Attachment #8761121 - Flags: review?(dmose)
Comment on attachment 8761121 [details] [review]
[loop] mancas:bug1278677 > mozilla:akita

left couple of questions on GH. code looks good, needs error treatment and tests. 

Or maybe we can avoid it, being a temporary solution and close deadline (although I'm afraid we will forget about it when exchanging for final solution if we don't make them now)
Attachment #8761121 - Flags: review?(fernando.campo) → feedback+

Comment 3

3 years ago
Ian -- if I read this correctly, we are planning to send every URL visited by Hello users to embedly as they visit them. Is this right?
Flags: needinfo?(ianb)
(In reply to Adam Roach [:abr] from comment #3)
> Ian -- if I read this correctly, we are planning to send every URL visited
> by Hello users to embedly as they visit them. Is this right?

That's correct, however this is temporary on our Akita branch and only going to be used for demos/initial development - it won't be released to users (for which we're planning alternate options).
Flags: needinfo?(ianb)
Blocks: 1273495
No longer blocks: 1278584
I'd say just add an XXXakita comment with a bug number for writing the tests and error handling.
Comment on attachment 8761121 [details] [review]
[loop] mancas:bug1278677 > mozilla:akita

At this point, this wants Fernando's questions answered and probably just a spin-off bug filed before a final review/landing.
Attachment #8761121 - Flags: review?(standard8)
Attachment #8761121 - Flags: review?(edilee)
Attachment #8761121 - Flags: review?(dmose)
(Assignee)

Updated

3 years ago
Attachment #8761121 - Flags: review?(fernando.campo)
(Assignee)

Comment 7

3 years ago
Created attachment 8761515 [details]
Snackbar demo
Attachment #8761515 - Flags: ui-review?(b.pmm)
Comment on attachment 8761121 [details] [review]
[loop] mancas:bug1278677 > mozilla:akita

Thanks for adding the error handling.
And cool feature with the snackbar :) Let's wait for UX approval before merging it
Attachment #8761121 - Flags: review?(fernando.campo) → review+

Comment 9

3 years ago
Comment on attachment 8761515 [details]
Snackbar demo

It's a good start. Thanks, Manu!
Attachment #8761515 - Flags: ui-review?(b.pmm) → ui-review+
(Assignee)

Comment 10

3 years ago
Landed in akita: https://github.com/mozilla/loop/commit/aeb3cf69d5ff1b74ad21056257d902141c0ca38f
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.