Research on graph representation learning has received great attention in recent years. However, most of the studies so far have focused on the embedding of single-layer graphs. The few studies dealing with the problem of representation learning of multilayer structures rely on the strong hypothesis that the inter-layer links are known, and this limits the range of possible applications. Here we propose MultiplexSAGE, a generalization of the GraphSAGE algorithm that allows embedding multiplex networks. We show that MultiplexSAGE is capable to reconstruct both the intra-layer and the inter-layer connectivity, outperforming competing methods. Next, through a comprehensive experimental analysis, we shed light also on the performance of the embedding, both in simple and multiplex networks, showing that both the density of the graph and the randomness of the links strongly influences the quality of the embedding.