Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Planning chemical syntheses with deep neural networks and symbolic AI

Nature volume 555, pages 604–610 (2018)Cite this article

Subjects

Abstract

To plan the syntheses of small organic molecules, chemists use retrosynthesis, a problem-solving technique in which target molecules are recursively transformed into increasingly simpler precursors. Computer-aided retrosynthesis would be a valuable tool but at present it is slow and provides results of unsatisfactory quality. Here we use Monte Carlo tree search and symbolic artificial intelligence (AI) to discover retrosynthetic routes. We combined Monte Carlo tree search with an expansion policy network that guides the search, and a filter network to pre-select the most promising retrosynthetic steps. These deep neural networks were trained on essentially all reactions ever published in organic chemistry. Our system solves for almost twice as many molecules, thirty times faster than the traditional computer-aided search method, which is based on extracted rules and hand-designed heuristics. In a double-blind AB test, chemists on average considered our computer-generated routes to be equivalent to reported literature routes.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Translation of the traditional chemists’ retrosynthetic route representation to the search tree representation.
Figure 2: Schematic of MCTS methodology.
Figure 3: An exemplary six-step synthesis route for an intermediate in a drug candidate synthesis.
Figure 4: Influence of the time per query on performance.
Figure 5: Double-blind AB testing of MCTS-derived routes against literature and BFS routes.

Similar content being viewed by others

References

  1. Clayden, J., Greeves, N., Warren, S. & Wothers, P. Organic Chemistry 2nd edn (Oxford Univ. Press, 2008)

  2. Brückner, R. Reaktionsmechanismen: Osganische Reaktionen, Stereochemie, Moderne Synthesemethoden (Springer, 2014)

  3. Robinson, R. LXIII. A synthesis of tropinone. J. Chem. Soc. Trans. 111, 762–768 (1917)

    Article  CAS  Google Scholar 

  4. Corey, E. & Cheng, X. The Logic of Chemical Synthesis (Wiley, 1989)

  5. Kurti, L. & Czakó, B. Strategic Applications of Named Reactions in Organic Synthesis (Elsevier, 2005)

  6. Evans, J. in The Oxford Handbook of Thinking and Reasoning (eds Holyoak, K. J. & Morrison, R. G.) 115–133 (Oxford Univ. Press, 2012)

  7. Collins, K. D. & Glorius, F. A robustness screen for the rapid assessment of chemical reactions. Nat. Chem. 5, 597–601 (2013)

    Article  CAS  PubMed  Google Scholar 

  8. Ley, S. V., Fitzpatrick, D. E., Ingham, R. & Myers, R. M. Organic synthesis: march of the machines. Angew. Chem. Int. Ed. 54, 3449–3464 (2015)

    Article  CAS  Google Scholar 

  9. Schneider, P. & Schneider, G. De novo design at the edge of chaos: miniperspective. J. Med. Chem. 59, 4077–4086 (2016)

    Article  CAS  PubMed  Google Scholar 

  10. Segler, M. H., Kogej, T., Tyrchan, C. & Waller, M. P. Generating focussed molecule libraries for drug discovery with recurrent neural networks. ACS Cent. Sci. 4, 120–131 (2018)

    Article  CAS  PubMed  Google Scholar 

  11. Vléduts, G. Concerning one system of classification and codification of organic reactions. Inform. Storage Retrieval 1, 117–146 (1963)

    Article  Google Scholar 

  12. Todd, M. H. Computer-aided organic synthesis. Chem. Soc. Rev. 34, 247–266 (2005)

    Article  CAS  PubMed  Google Scholar 

  13. Szymkuć, S. et al. Computer-assisted synthetic planning: the end of the beginning. Angew. Chem. Int. Ed. 55, 5904–5937 (2016)

    Article  CAS  Google Scholar 

  14. Cook, A. et al. Computer-aided synthesis design: 40 years on. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2, 79–107 (2012)

    Article  CAS  Google Scholar 

  15. Ihlenfeldt, W.-D. & Gasteiger, J. Computer-assisted planning of organic syntheses: the second generation of programs. Angew. Chem. Int. Edn Engl. 34, 2613–2633 (1996)

    Article  Google Scholar 

  16. Fick, R. Konzepte zur Syntheseplanung: Strukturelle Ähnlichkeit und Strategische Bindungen. PhD thesis, Friedrich-Alexander-Universität (1996)

  17. Ugi, I. et al. Models, concepts, theories, and formal languages in chemistry and their use as a basis for computer assistance in chemistry. J. Chem. Inf. Comput. Sci. 34, 3–16 (1994)

    Article  CAS  Google Scholar 

  18. Kayala, M. A., Azencott, C.-A., Chen, J. H. & Baldi, P. Learning to predict chemical reactions. J. Chem. Inf. Model. 51, 2209–2222 (2011)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Minsky, M. A Framework for Representing Knowledge. Technical Report (Massachusetts Institute of Technology, 1974)

  20. Bøgevig, A. et al. Route design in the 21st century: the ICSYNTH software tool as an idea generator for synthesis prediction. Org. Process Res. Dev. 19, 357–368 (2015)

    Article  CAS  Google Scholar 

  21. Law, J. et al. Route designer: a retrosynthetic analysis tool utilizing automated retrosynthetic rule generation. J. Chem. Inf. Model. 49, 593–602 (2009)

    Article  CAS  PubMed  Google Scholar 

  22. Christ, C. D., Zentgraf, M. & Kriegl, J. M. Mining electronic laboratory notebooks: analysis, retrosynthesis, and reaction based enumeration. J. Chem. Inf. Model. 52, 1745–1756 (2012)

    Article  CAS  PubMed  Google Scholar 

  23. Segler, M. H. & Waller, M. P. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry 23, 5966–5971 (2017)

    Article  CAS  PubMed  Google Scholar 

  24. Boda, K., Seidel, T. & Gasteiger, J. Structure and reaction based evaluation of synthetic accessibility. J. Comput. Aided Mol. Des. 21, 311–325 (2007)

    Article  ADS  CAS  PubMed  Google Scholar 

  25. Ertl, P. & Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J. Cheminform. 1, 8 (2009)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In Int. Conf. on Computers and Games 72–83 (Springer, 2006)

  27. Kocsis, L. & Szepesvári, C. Bandit based Monte-Carlo planning. In 17th Eur. Conf. on Machine Learning 282–293 (Springer, 2006)

  28. Browne, C. B. et al. A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4, 1–43 (2012)

    Article  Google Scholar 

  29. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction 2nd edn (MIT Press, in the press)

  30. Coulom, R. Computing “elo ratings” of move patterns in the game of go. ICGA J. 30, 198–208 (2007)

    Article  Google Scholar 

  31. Stern, D., Herbrich, R. & Graepel, T. Bayesian pattern ranking for move prediction in the game of Go. In Int. Conf. on Machine Learning 873–880 (Omni Press, 2006)

  32. Maddison, C. J., Huang, A., Sutskever, I. & Silver, D. Move evaluation in Go using deep convolutional neural networks. In 3rd Int. Conf. on Learning Representations (2015); preprint at https://arxiv.org/abs/1412.6564

  33. Clark, C. & Storkey, A. Training deep convolutional neural networks to play Go. In 32nd Int. Conf. on Machine Learning 1766–1774 (PMLR, 2015); http://proceedings.mlr.press/v37/clark15.html

  34. Winands, M. Neural networks for video game AI. In Artificial and Computational Intelligence in Games: Integration (Dagstuhl Seminar 15051) Vol. 5 (eds Lucas, S. M. et al.) 224 (2015)

  35. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)

    Article  ADS  CAS  Google Scholar 

  36. Reaxyshttp://www.reaxys.com (Elsevier Life Sciences, 2017)

  37. Srivastava, R. K ., Greff, K. & Schmidhuber, J. Training very deep networks. In Advances in Neural Information Processing Systems 2377–2385 (MIT Press, 2015); preprint at https://arxiv.org/abs/1507.06228

  38. Clevert, D.-A., Unterthiner, T. & Hochreiter, S. Fast and accurate deep network learning by exponential linear units (ELUs). In 4th Int. Conf. on Learning Representations (2016); preprint at https://arxiv.org/abs/1511.07289

  39. Sheridan, R. P. Time-split cross-validation as a method for estimating the goodness of prospective prediction. J. Chem. Inf. Model. 53, 783–790 (2013)

    Article  CAS  PubMed  Google Scholar 

  40. Marcou, G. et al. Expert system for predicting reaction conditions: the Michael reaction case. J. Chem. Inf. Model. 55, 239–250 (2015)

    Article  ADS  CAS  PubMed  Google Scholar 

  41. Segler, M. H. & Waller, M. P. Modelling chemical reasoning to predict and invent reactions. Chemistry 23, 6118–6128 (2017)

    Article  CAS  PubMed  Google Scholar 

  42. Coley, C. W., Barzilay, R., Jaakkola, T. S., Green, W. H. & Jensen, K. F. Prediction of organic reaction outcomes using machine learning. ACS Cent. Sci. 3, 434–443 (2017)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Murphy, K. P. Machine Learning: a Probabilistic Perspective (MIT Press, 2012)

  44. Nirogi, R. V., Badange, R., Reballi, V. & Khagga, M. Design, synthesis and biological evaluation of novel benzopyran sulfonamide derivatives as 5-HT6 receptor ligands. Asian J. Chem. 27, 2117–2124 (2015)

    Article  CAS  Google Scholar 

  45. Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and think like people. Behav. Brain Sci. 40, 1–101 (2016)

    Google Scholar 

  46. Sierra, M. A. & de la Torre, M. C. Dead ends and detours en route to total syntheses of the 1990s. Angew. Chem. Int. Ed. 39, 1538–1559 (2000)

    Article  CAS  Google Scholar 

  47. Rocktäschel, T. & Riedel, S. End-to-end differentiable proving. In Advances of Neural Information Processing Systems (eds Guyon, I. et al.) 3788–3800 (Curran Associates, 2017); https://papers.nips.cc/paper/6969-end-to-end-differentiable-proving

  48. Peng, Q., Duarte, F. & Paton, R. S. Computing organic stereoselectivity—from concepts to quantitative calculations and predictions. Chem. Soc. Rev. 45, 6093–6107 (2016)

    Article  CAS  PubMed  Google Scholar 

  49. Lin, A. I. et al. Automatized assessment of protective group reactivity: a step toward big reaction data analysis. J. Chem. Inf. Model. 56, 2140–2148 (2016)

    Article  CAS  PubMed  Google Scholar 

  50. Gini, A., Segler, M., Kellner, D. & Garcia Mancheno, O. Dehydrogenative tempo-mediated formation of unstable nitrones: easy access to n-carbamoyl isoxazolines. Chemistry 21, 12053–12060 (2015)

    Article  CAS  PubMed  Google Scholar 

  51. Andersen, J. L., Flamm, C., Merkle, D. & Stadler, P. F. Generic strategies for chemical space exploration. Int. J. Comput. Biol. Drug Des. 7, 225–258 (2014)

    Article  PubMed  Google Scholar 

  52. Steinbeck, C. et al. Recent developments of the chemistry development kit (CDK)-an open-source Java library for chemo- and bioinformatics. Curr. Pharm. Des. 12, 2111–2120 (2006)

    Article  CAS  PubMed  Google Scholar 

  53. Landrum, G. RDKit: Open-Source Cheminformaticshttp://www.rdkit.org

  54. Silver, D. Reinforcement Learning and Simulation-Based Search. PhD thesis, Univ. Alberta (2009)

  55. Reymond, J.-L., Ruddigkeit, L., Blum, L. & van Deursen, R. The enumeration of chemical space. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2, 717–733 (2012)

    Article  CAS  Google Scholar 

  56. Färber, M ., Kaliszyk, C. & Urban, J. Monte Carlo connection prover. Preprint at https://arxiv.org/abs/1611.05990 (2016)

  57. Rosin, C. D. Multi-armed bandits with episode context. Ann. Math. Artif. Intell. 61, 203–230 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  58. Winands, M. H., Björnsson, Y. & Saito, J.-T. Monte-Carlo tree search solver. In Int. Conf. on Computers and Games 25–36 (Springer, 2008)

  59. Schneider, N., Lowe, D. M., Sayle, R. A. & Landrum, G. A. Development of a novel fingerprint for chemical reactions and its application to large-scale reaction classification and similarity. J. Chem. Inf. Mod. 55, (2015)

  60. Coley, C. W., Rogers, L., Green, W. H. & Jensen, K. F. Computer-assisted retrosynthesis based on molecular similarity. ACS Cent. Sci. 3, 1237–1245 (2017)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Gelernter, H., Rose, J. R. & Chen, C. Building and refining a knowledge base for synthetic organic chemistry via the methodology of inductive and deductive machine learning. J. Chem. Inf. Comput. Sci. 30, 492–504 (1990)

    Article  CAS  Google Scholar 

  62. Rose, J. R. & Gasteiger, J. Horace: an automatic system for the hierarchical classification of chemical reactions. J. Chem. Inf. Comput. Sci. 34, 74–90 (1994)

    Article  CAS  Google Scholar 

  63. Liu, B. et al. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS Cent. Sci. 3, 1103–1113 (2017)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Kingma, D.P. & Ba, J. ADAM: a method for stochastic optimization. In 3rd Int. Conf. for Learning Representations; preprint at https://arxiv.org/abs/1412.6980 (2015)

  65. Chollet, F. et al. Kerashttps://github.com/fchollet/keras (2015)

  66. The Theano Development Team Theano: a Python framework for fast computation of mathematical expressions. Preprint at https://arxiv.org/abs/1605.02688 (2016)

  67. Rogers, D. & Hahn, M. Extended-connectivity fingerprints. J. Chem. Inf. Model. 50, 742–754 (2010)

    Article  CAS  PubMed  Google Scholar 

  68. Wei, J. N., Duvenaud, D. & Aspuru-Guzik, A. Neural networks for the prediction of organic chemistry reactions. ACS Cent. Sci. 2, 725–732 (2016)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  69. Socorro, I. M. & Goodman, J. M. The ROBIA program for predicting organic reactivity. J. Chem. Inf. Model. 46, 606–614 (2006)

    Article  CAS  PubMed  Google Scholar 

  70. Satoh, H. & Funatsu, K. Sophia, a knowledge base-guided reaction prediction system—utilization of a knowledge base derived from a reaction database. J. Chem. Inf. Comput. Sci. 35, 34–44 (1995)

    Article  CAS  Google Scholar 

  71. Patel, H., Bodkin, M. J., Chen, B. & Gillet, V. J. Knowledge-based approach to de novo design using reaction vectors. J. Chem. Inf. Model. 49, 1163–1184 (2009)

    Article  CAS  PubMed  Google Scholar 

  72. Zhang, Q.-Y. & Aires-de Sousa, J. Structure-based classification of chemical reactions without assignment of reaction centers. J. Chem. Inf. Model. 45, 1775–1783 (2005)

    Article  CAS  PubMed  Google Scholar 

  73. Polishchuk, P. et al. Structure–reactivity modeling using mixture-based representation of chemical reactions. J. Comput. Aided Mol. Des. 31, 829–839 (2017)

    Article  ADS  CAS  PubMed  Google Scholar 

  74. Carrera, G. V., Gupta, S. & Aires-de Sousa, J. Machine learning of chemical reactivity from databases of organic reactions. J. Comput. Aided Mol. Des. 23, 419–429 (2009)

    Article  ADS  CAS  PubMed  Google Scholar 

  75. Neese, F. The ORCA program system. WIREs Comput. Mol. Sci. 2, 73–78 (2012)

    Article  CAS  Google Scholar 

  76. Butina, D. Unsupervised data base clustering based on Daylight’s fingerprint and Tanimoto similarity: a fast and automated way to cluster small and large data sets. J. Chem. Inf. Comput. Sci. 39, 747–750 (1999)

    Article  CAS  Google Scholar 

  77. Parsy, C. C. et al. Discovery and structural diversity of the hepatitis C virus NS3/4a serine protease inhibitor series leading to clinical candidate IDX320. Bioorg. Med. Chem. Lett. 25, 5427–5436 (2015)

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

M.H.S.S. and M.P.W. thank the Deutsche Forschungsgemeinschaft (SFB858) for funding. M.H.S.S. and M.P.W. also thank D. Evans (RELX Intellectual Properties) and J. Swienty-Busch (Elsevier Information Systems) for the reaction dataset. We thank all AB-test participants in Shanghai and Münster, and J. Guo for assistance in AB testing. M.H.S.S. thanks M. Wiesenfeldt, the Studer group, D. Barton, S. McAnanama-Brereton, R. Vidyadharan and T. Kogej for discussions. M.P. thanks M. Winands and J. Togelius for insights.

Author information

Authors and Affiliations

  1. Institute of Organic Chemistry and Center for Multiscale Theory and Computation, Westfälische Wilhelms-Universität, Münster, Germany

    Marwin H. S. Segler

  2. BenevolentAI, London, UK

    Marwin H. S. Segler

  3. European Research Center for Information Systems, Westfälische Wilhelms-Universität Münster, Germany

    Mike Preuss

  4. Department of Physics and International Centre for Quantum and Molecular Structures, Shanghai University, Shanghai, China

    Mark P. Waller

Authors
  1. Marwin H. S. Segler
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mike Preuss
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mark P. Waller
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

M.H.S.S. conceived the project, M.P.W. and M.P. contributed ideas. M.H.S.S., M.P. and M.P.W. designed the experiments. M.H.S.S. implemented the program. M.H.S.S. and M.P.W. conducted the experiments. M.P.W. supervised the project. All authors co-wrote the manuscript.

Corresponding authors

Correspondence to Marwin H. S. Segler or Mark P. Waller.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Additional information

Reviewer Information thanks D. Duvenaud, W. H. Green and the other anonymous reviewer(s) for their contribution to the peer review of this work.

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Figure 1 Receiver operation characteristic curve for the in-scope filter.

The area under the curve is 0.99.

Extended Data Figure 2 An exemplary 10-step synthesis route for a complex intermediate in a drug synthesis.

It resembles the published route77 (with intermediates A and B) and was found by our algorithm autonomously within 30?s. The target was not contained in the training set.

Extended Data Figure 3 Example of reaction used in the AB testing, where the MCTS-derived route was less favoured.

In this task, the participants preferred the literature solution, as its key step was presumably perceived to be more convergent.

Extended Data Figure 4 Architectures of the employed neural networks.

(‘dim’, dimensions.)

Extended Data Figure 5 Rediscovering physicochemical properties with the in-scope filter.

The output logit score of the neural network correlates surprisingly well with calculated quantum-mechanical properties (LUMO energies, in Hartree) in Diels–Alder reactions (r2?=?0.74) (a) and with empirically measured Hammond parameters in electrophilic brominations (r2?=?0.78) (b), even though the input features (ECFP4 fingerprints) do not contain electronic information.

Extended Data Table 1 Metrics for the supervised neural network policies
Full size table

Supplementary information

Supplementary Information

This file contains the DOE, route diversity analysis, and failed molecules (including Supplementary Figures 1-7, Supplementary Table 1 and Supplementary References). Available on figshare (DOI 10.6084/m9.figshare.5832054) are 2 files, mcts_examples.pdf which contains routes found by the 3N-MCTS algorithm and heuristicBFS_examples.pdf which contains routes found by heuristic best first search without policy network and in-scope filter. (PDF 692 kb)

Supplementary Information

This file contains the AB test. (PDF 5648 kb)

Supplementary Data

This file contains the experiment for correlating the in-scope filter to physicochemical properties. (XLSX 45 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Segler, M., Preuss, M. & Waller, M. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 604–610 (2018). https://doi.org/10.1038/nature25978

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature25978

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics