There is this popular AtomEncoder snippet that is suggested on various websites. I used it many years ago with success, but I never understood how it works.

class AtomEncoder(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(AtomEncoder, self).__init__()

        self.embeddings = torch.nn.ModuleList()

        for i in range(9):
            self.embeddings.append(Embedding(100, hidden_channels))

    def reset_parameters(self):
        for embedding in self.embeddings:
            embedding.reset_parameters()

    def forward(self, x):
        if x.dim() == 1:
            x = x.unsqueeze(1)

        out = 0
        for i in range(x.size(1)):
            out += self.embeddings[i](x[:, i])
        return out

With hidden_channels equal to, say, 12, this is what it creates:

  (embeddings): AtomEncoder(
    (embeddings): ModuleList(
      (0-8): 9 x Embedding(100, 12)
    )
  )

Does that mean that I get 9 embedding layers?

Where do they go? All 9 between each linear layer, or one between 9 layers?

Given the lack of explanation, the number 9 itself sounds arbitrary to me. Why not just one layer?

I have searched the documentations and the internet thoroughly for answers, queried Stack Overflow about "AtomEncoder" embedding, as well as watched videos explaining embedding in details.

If I were to make a guess, it would probably be that all 9 layers come all together between every pair of linear layers, but I don't actually know it for sure. It's never explicitly stated anywhere. Could someone please help me understand this?

Sincerely thanks,