Improving Latent Space Diversity in SMILES Generation Using Transformer VAE
DOI:
https://doi.org/10.11113/ijic.v16n1-2.691Keywords:
Artificial Intelligence, Latent Space Diversity, Transformer, Variational Autoencoder, SMILES, Drug DiscoveryAbstract
This work presents TransMolVAE, a Transformer-based variational autoencoder for molecular generation using SMILES. RNN-based models such as SmilesVAE often face posterior collapse. This happens when the latent space is not well used. To solve this, TransMolVAE applies self-attention. This allows the model to capture long-range patterns in SMILES strings. It improves reconstruction accuracy, keeps KL divergence stable, and makes better use of the latent space. The experiments showed strong results. Token accuracy increased steadily, reaching 0.72 by the fifth epoch. In contrast, SmilesVAE stayed low at 0.25. Validity, uniqueness, and novelty of molecules generated by TransMolVAE also reached above 70%. For SmilesVAE, these values were only between 50% and 55%. These findings show that TransMolVAE can generate more valid, novel, and diverse molecules compared to the baseline. The number of active units also stayed high, proving that the latent space was fully used. There was a trade-off in computation. TransMolVAE needed longer CPU time per epoch compared to SmilesVAE. This is because Transformer models are more complex and require higher computation. Even so, the improved quality of molecule generation makes TransMolVAE more suitable for drug discovery, where novelty and diversity are more important than speed.
References
Berdigaliyev, N., & Aljofan, M. (2020). An overview of drug discovery and development. Future Medicinal Chemistry, 12(10), 939–947.
Ocana, A., Pandiella, A., Privat, C., Bravo, I., Luengo-Oroz, M., Amir, E., & Gyorffy, B. (2025). Integrating artificial intelligence in drug discovery and early drug development: A transformative approach. Biomarker Research, 13(1).
Bikku, T., Malligunta, K. K., Thota, S., & Surapaneni, P. P. (2025). Improved quantum algorithm: A crucial stepping stone in quantum-powered drug discovery. Journal of Electronic Materials.
Jusoh, A. S., Remli, M. A., Mohamad, M. S., Cazenave, T., & Fong, C. S. (2025). How generative artificial intelligence can transform drug discovery? European Journal of Medicinal Chemistry, 295, 117825.
Pracht, P., Bohle, F., & Grimme, S. (2020). Automated exploration of the low-energy chemical space with fast quantum chemical methods. Physical Chemistry Chemical Physics, 22(14), 7169–7192.
Degenhardt, Y., Poss, M., & Gao, X. (2025). Computer-aided design for cancer-targeted peptide drugs. In Methods in Pharmacology and Toxicology (pp. 45–62).
Niazi, S. K., & Mariam, Z. (2025). Artificial intelligence in drug development: Reshaping the therapeutic landscape. Therapeutic Advances in Drug Safety, 16.
Hagen, J., Hornung, L., Barham, W., Mukhopadhyay, S., Bess, A., Contrera, K., Basu, D., Sandulache, V., Spielmann, G., & Kansara, S. (2025). Artificial intelligence in head and neck cancer: Towards precision medicine. Cancers, 17(18), 3023.
Suveena, S., Rekha, A. A., Rani, J. R., Oommen, O. V., & Ramakrishnan, R. (2025). The translational impact of bioinformatics on traditional wet lab techniques. Advances in Pharmacology.
Vogt, M. (2023). Exploring chemical space—Generative models and their evaluation. Artificial Intelligence in the Life Sciences, 3, 100064.
Bilodeau, C., Jin, W., Jaakkola, T., Barzilay, R., & Jensen, K. F. (2022). Generative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5).
Iqbal, T., & Qureshi, S. (2022). The survey: Text generation models in deep learning. Journal of King Saud University – Computer and Information Sciences, 34(6), 2515–2528.
Ochiai, T., Inukai, T., Akiyama, M., Furui, K., Ohue, M., Matsumori, N., Inuki, S., Uesugi, M., Sunazuka, T., Kikuchi, K., Kakeya, H., & Sakakibara, Y. (2023). Variational autoencoder-based chemical latent space for large molecular structures with 3D complexity. Communications Chemistry, 6(1).
Alperstein, Z., Cherkasov, A., & Rolfe, J. T. (2023). All SMILES variational autoencoder for molecular property prediction and optimization. In Challenges and Advances in Computational Chemistry and Physics (pp. 85–115).
Li, C., & Yamanishi, Y. (2025). AI-driven transcriptome profile-guided hit molecule generation. Artificial Intelligence, 104239.
Li, C., & Yamanishi, Y. (2026). Gx2Mol: De novo generation of hit-like molecules from gene expression profiles (pp. 333–349).
Yamanaka, C., Uki, S., Kaitoh, K., Iwata, M., & Yamanishi, Y. (2023). De novo drug design based on patient gene expression profiles via deep learning. Molecular Informatics, 42(8–9).
Lipton, Z. C., Berkowitz, J., & Elkan, C. (2015). A critical review of recurrent neural networks for sequence learning. arXiv.
Yazdani, S., Singh, A., Saxena, N., Wang, Z., Palikhe, A., Pan, D., Pal, U., Yang, J., & Zhang, W. (2025). Generative AI in depth: A survey of recent advances, model variants, and real-world applications. Journal of Big Data, 12(1).
Gangwal, A., & Lavecchia, A. (2025). Artificial intelligence in natural product drug discovery: Current applications and future perspectives. Journal of Medicinal Chemistry.
Tiezzi, M., Casoni, M., Betti, A., Guidi, T., Gori, M., & Melacci, S. (2025). Back to recurrent processing at the crossroad of transformers and state-space models. Nature Machine Intelligence, 7(5), 678–688.
Jiang, J., Ke, L., Chen, L., Dou, B., Zhu, Y., Liu, J., Zhang, B., Zhou, T., & Wei, G. (2024). Transformer technology in molecular science. Wiley Interdisciplinary Reviews: Computational Molecular Science, 14(4).
Sultan, A., Sieg, J., Mathea, M., & Volkamer, A. (2024). Transformers for molecular property prediction: Lessons learned from the past five years. Journal of Chemical Information and Modeling, 64(16), 6259–6280.
Qiu, X., Shao, S., Wang, H., & Tan, X. (2025). Bio-K-Transformer: A pre-trained transformer-based sequence-to-sequence model for adverse drug reactions prediction. Computer Methods and Programs in Biomedicine, 260, 108524.
Jørgensen, P. B., Schmidt, M. N., & Winther, O. (2018). Deep generative models for molecular science. Molecular Informatics, 37(1–2).
Khater, T., Alkhatib, S. A., AlShehhi, A., Pitsalidis, C., Pappa, A. M., Ngo, S. T., Chan, V., & Truong, V. K. (2025). Generative artificial intelligence-based model optimization towards molecule design enhancement. Journal of Cheminformatics, 17(1), 116.
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019). Deep learning for molecular design: A review of the state of the art. Molecular Systems Design & Engineering, 4(4), 828–849.
Özçelik, R., Brinkmann, H., Criscuolo, E., & Grisoni, F. (2025). Generative deep learning for de novo drug design—A chemical space odyssey. Journal of Chemical Information and Modeling.
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023). Applications of transformer-based language models in bioinformatics: A survey. Bioinformatics Advances, 3(1).
Khambhawala, A., Lee, C. H., Pahari, S., & Kwon, J. S.-I. (2025). Minimizing late-stage failure in drug development with transformer models: Enhancing drug screening and pharmacokinetic predictions. Chemical Engineering Journal, 508, 160423.
Ünlü, A., Çevrim, E., Yiğit, M. G., Sarıgün, A., Çelikbilek, H., Bayram, O., Kahraman, D. C., Olğaç, A., Rifaioglu, A. S., Banoğlu, E., & Doğan, T. (2025). Target-specific de novo design of drug candidate molecules with graph-transformer-based generative adversarial networks. Nature Machine Intelligence, 7(9), 1524–1540.
Sagawa, T., & Kojima, R. (2025). ReactionT5: A pre-trained transformer model for accurate chemical reaction prediction with limited data. Journal of Cheminformatics, 17(1), 126.
Sadaghiyanfam, S., Kamberaj, H., & Isler, Y. (2025). Leveraging ChemBERTa and machine learning for accurate toxicity prediction of ionic liquids. Journal of the Taiwan Institute of Chemical Engineers, 171, 106030.
Masood, M. A., Kaski, S., Ceulemans, H., Herman, D., & Heinonen, M. (2024). Balancing imbalanced toxicity models: Using MolBERT with focal loss. In Lecture Notes in Computer Science (pp. 82–97).
Bagal, V., Aggarwal, R., Vinod, P. K., & Priyakumar, U. D. (2022). MOLGPT: Molecular generation using a transformer-decoder model. Journal of Chemical Information and Modeling, 62(9), 2064–2076.
Liyaqat, T., Ahmad, T., & Saxena, C. (2025). Advancements in molecular property prediction: A survey of single and multimodal approaches. Archives of Computational Methods in Engineering.
Li, D., Chen, X., Becchi, M., & Zong, Z. (2016). Evaluating the energy efficiency of deep convolutional neural networks on CPUs and GPUs. In IEEE International Conference on Big Data and Cloud Computing (BdCloud).
Song, T., Huang, Z., Liu, X., & Sun, J. (2025). Preventing posterior collapse with DVAE for text modeling. Entropy, 27(4), 423.
Bhadwal, A. S., Kumari, M., & Kumar, A. (2025). PCF-VAE: Posterior collapse free variational autoencoder for de novo drug design. Scientific Reports, 15(1).
Bakhtiary, N., Ghalandari, B., Ghorbani, F., & Liu, C. (2025). Dry-lab computational approaches for simulating bioinspired materials/structures for regenerative medicine. In Principles of Bioinspired and Biomimetic Regenerative Medicine (pp. 861–899).
Chakraborty, R., & Hasija, Y. (2023). Utilizing deep learning to explore chemical space for drug lead optimization. Expert Systems with Applications, 229, 120592.
Vogt, M. (2022). Using deep neural networks to explore chemical space. Expert Opinion on Drug Discovery, 17(3), 297–304.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Innovative Computing
This work is licensed under a Creative Commons Attribution 4.0 International License.
