1246013 - Reduce the size of nsEffectiveTLDService from 256 KiB to 128 KiB on 64-bit platforms

Status: RESOLVED FIXED

(Core :: Networking: DNS, defect)

Description

[Nicholas Nethercote [inactive]]

In nsEffectiveTLDService we have a {char name[], bool ex, bool wild} triple for
each TLD. This is stored in a three layer data structure.

1. nsDomainEntry::etld_strings is a static array (in a union) of tightly packed
  TLD names, i.e.:

> { str1:"com.ac", str2:"edu.ac", str3:"gov.ac", ... }

2. nsDomainEntry::entries is a static array, each entry of which holds a
  representation of a TLD triple:

> struct ETLDEntry {
>   uint32_t strtab_index : ETLD_ENTRY_N_INDEX_BITS;
>   uint32_t exception : 1;
>   uint32_t wild : 1; 
> };

  strtab_index is an index into etld_strings.

3. nsEffectiveTLDHash::mHash is a PLDHashTable<DomainEntry>. Each DomainEntry
  has a pointer to an ETLDEntry.

  - On 32-bit platforms, sizeof(DomainEntry) is 8: 4 bytes for the HashNumber,
    and 4 bytes for the pointer.

  - On 64-bit platforms, sizeof(DomainEntry) is 16: 4 bytes for the HashNumber,
    4 bytes of padding, and 8 bytes for the pointer.

What this means is that on 64-bit platforms, the pointer stored in each hash
table entry is bigger than the thing it is pointing to! So instead of holding a
pointer to an ETLDEntry, each hash table entry can instead hold a copy of that
ETLDEntry. This reduces sizeof(DomainEntry) on 64-bit from 16 bytes to 8 bytes,
making it the same as on 32-bit platforms.

There are 6,271 TLDs. This results in a hash table with 16,384 entries.
Reducing the size of each hash table entry from 16 bytes to 8 bytes on 64-bit
platforms reduces the size of this table from 16384*16 to 16384*8, i.e. from
256 KiB to 128 KiB. I've confirmed this is the case in about:memory.

This change also results in less pointer-chasing, on all plaforms, because we
can read values directly from the hash table entry.

Comment 1

[Nicholas Nethercote [inactive]]

glandium raised the idea of using gperf, but looks like that was considered in bug 704848 and rejected due to disk space concerns.

I also wonder if switching to binary search would be worthwhile. It'd reduce the storage required for the top-level data structure from 128 KiB to 6271*4 = 25,084 bytes. But that's a bigger chance, and I think it's worth doing this one for now.

Comment 2

[Nicholas Nethercote [inactive]]

Attachment: Store a copy of the ETLDEntry directly in DomainEntry instead of a pointer to it

Comment 3

[Nicholas Nethercote [inactive]]

Attachment: (part 2) - Change "explicit/xpcom/effective-TLD-service" path to the more sensible "explicit/network/effective-TLD-service"

I'm not sure why this was under "explicit/xpcom/". Probably a thinko on my part
when I first added the reporter.

Comment 4

[Jason Duell]

Comment on attachment 8716089 [details] [diff] [review]
Store a copy of the ETLDEntry directly in DomainEntry instead of a pointer to it

Review of attachment 8716089 [details] [diff] [review]:
-----------------------------------------------------------------

Nicholas,

Note that there is a high chance bug 1196364 will conflict with your patch (but only IIUC in terms of nearby lines of code changed, not semantic conflicts).  My read is that this bug could land before bug 119364, but if it doesn't, be prepared for that.

Comment 5

[Nicholas Nethercote [inactive]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/181e4142e652a62ec6d1dbf96d0efcd382b31173
Bug 1246013 (part 1) - Store a copy of the ETLDEntry directly in DomainEntry instead of a pointer to it. r=jduell.

https://hg.mozilla.org/integration/mozilla-inbound/rev/7f6fbaf77a9b327b8fc2156c61002682bdf4ef26
Bug 1246013 (part 2) - Change "explicit/xpcom/effective-TLD-service" path to the more sensible "explicit/network/effective-TLD-service". r=jduell.

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/181e4142e652
https://hg.mozilla.org/mozilla-central/rev/7f6fbaf77a9b