Multilingual Molecular Representation Learning via Contrastive Pre-training

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3441 - 3453 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Multilingual Molecular Representation Learning via Contrastive Pre-training Zhihui Guo, Pramod Sharma, Andy Martinez, Liang Du, Robin Abraham Microsoft Corporation {zhihui.guo,pramod.sharma,andy.martinez,liang.du,robin.abraham}@microsoft.com Abstract Molecular representation learning plays an es- sential role in cheminformatics. Recently, lan- guage model-based approaches have gained popularity as an alternative to traditional expert- designed features to encode molecules. How- ever, these approaches only utilize a single molecular language for representation learning. Motivated by the fact that a given molecule can be described using different languages such as Simplified Molecular Line Entry System (SMILES), the International Union of Pure and Applied Chemistry (IUPAC), and the IU- PAC International Chemical Identifier (InChI), we propose a multilingual molecular embed- ding generation approach called MM-Deacon (multilingual molecular domain embedding analysis via contrastive learning). MM-Deacon is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules. We evaluated the robustness of our method on seven molecular property prediction tasks from MoleculeNet benchmark, zero-shot cross- lingual retrieval, and a drug-drug interaction prediction task. 1 Introduction Drug discovery process involves screening of mil- lions of compounds in the early stages of drug design, which is time consuming and expensive. Computer-aided drug discovery can reduce the time and cost involved in this process via automating various cheminformatics tasks (Kontogeorgis and Gani, 2004; Xu et al., 2017; Winter et al., 2019). Traditional methods to encode molecules such as fingerprint generation rely heavily on molecu- lar fragment-level operations on top of molecule graph constructed by molecular atoms and bonds (Burden, 1989; Bender and Glen, 2004; Vogt and Bajorath, 2008; Muegge and Mukherjee, 2016). An example of such methods is Morgan fingerprint, also known as Extended-Connectivity Fingerprint (ECFP) (Morgan, 1965; Rogers and Hahn, 2010), where a fixed binary hash function is applied on each atom and its neighborhood. These kinds of approaches focus on local features, hence they may not capture global information. In addition to molecule graph, a given molecule can also be described with different languages such as Simplified Molecular Line Entry System (SMILES), the International Union of Pure and Applied Chemistry (IUPAC), and the IUPAC Inter- national Chemical Identifier (InChI). Particularly, SMILES is widely used to represent molecule struc- tures as ASCII strings (Weininger, 1988; Favre and Powell, 2013) at an atom and bond level. IUPAC nomenclature, on the other hand, serves the pur- pose of systematically naming organic compounds by basic words that indicate the structure of the compound and prioritize on functional groups to facilitate communication (Panico et al., 1993). Fig. 1 shows a comparison of SMILES and IUPAC char- acteristics for the same molecule. The SMILES string is created by traversing the molecule graph, where each letter in the SMILES string (such as C, F, N, O in Fig. 1) corresponds to an atom on the graph, and other characters represent positions and connectivity. However, IUPAC names are akin to a natural language, …

Content truncated to 500 words