Machine Learning in Electron Microscopy Towards the Attosecond Resolution

Ultrafast Electron Diffraction and Microscopy imaging have been demonstrated to be pivot tools for imaging the atomic motion in real-time and space ^1-3. The generation of a few hundred femtoseconds electron pulses enabled recording movies for molecular and atomic motion⁴. However, the technical challenges in electron pulse compression have limited the temporal resolution of electron imaging experiments to a hundred femtoseconds. Here, we demonstrate the attosecond temporal resolution in the transmission electron microscope by optical gating ⁵ to establish what we so-called “attomicroscope” ⁶. Moreover, we utilized the attomicroscope to image the electron motion dynamics in graphene. In a strong field, the electron is moving in the reciprocal space following the waveform of the driver field. The attosecond electron diffraction experiment allowed us to study the electron density distribution in the reciprocal space at different time instants and connect it with the electron motion in real space. The demonstrated attomicroscopy imaging tool opens the avenue to study electron motion in neutral matter and promises new electron imaging applications in physics, chemistry, and biochemistry ⁶.

2. [1] Hassan, M. T. J. Phys. B: At. Mol. Opt. Phys. 51, 032005, (2018).

3. [2]  Miller, R. J. D. Science 343, 1108-1116, (2014).

4. [3]  Zewail, A. H. Science 328, 187-193, (2010).

5. [4]  Yang, J. et al. Science 368, 885-889, (2020).

6. [5]  Hassan, M. T., Baskin, J. S., LiaoB & Zewail, A. H. Nat. Photon. 11, 425-430, (2017).

7. [6]  Dandan Hui et al. ,Attosecond electron microscopy and diffraction.Sci.

Adv.10,eadp5805(2024).

Revolutionising Electron Microscopy: The Impact of Machine Learning on Nanoscale Imaging and Analysis

 In recent years, electron microscopy has undergone a significant transformation, driven by the integration of machine learning (ML) and artificial intelligence (AI) techniques. The talk will revise this path from its origin till nowadays, by providing a comprehensive overview of the current state-of-the-art in applying ML to electron microscopy, highlighting the advancements and future directions in this rapidly evolving field. [1]

The talk will go through traditional challenges in electron microscopy, such as data acquisition, processing, and interpretation, which have historically limited the technique’s efficiency and accuracy. By leveraging ML algorithms, researchers have developed innovative solutions to automate and enhance various aspects of electron microscopy, from image reconstruction and denoising to feature extraction and classification.

 Key developments discussed include the application of convolutional neural networks (CNNs) for high-resolution imaging, the use of generative adversarial networks (GANs) for realistic image synthesis, and the implementation of reinforcement learning for optimizing experimental parameters. I will also focus on spectroscopy and briefly go through tomography and techniques like compressive sensing that enable a more precise and multidimensional analysis of materials at the nanoscale.

The talk will feature a set of practical guidelines for microscopists and materials scientists, offering insights into selecting and applying ML tools effectively, even for those without extensive ML expertise, as parallelly done in other scientific disciplines. Finally, I will go through recent examples of a successful application of ML, deep learning, computer vision or generally AI to analyse high resolution (scanning) transmission electron microscopy data and unveil new physics from materials, with special emphasis on quantum materials. [2]

References:

[1] Nanoscale Horiz., 2022, 7, 1427-1477

[2] Botifoll, M et al. Submitted/In arXiv soon!

Inhomogeneous and disordered atomic arrangement, and machinery/thermal drifts, often vary (micro)spectroscopy signals of sub-micron materials, albeit the (bulk) composition and experimental parameters are identical. Such diversity can be harnessed to measure material’s purity, unveiling several concealed features via statistical inspection of heterogeneous signals. However, this approach requires efficient categorization of a substantial number of data, which is currently encumbered by laborious calculations, computational hurdles, and manual intervention. We present a robust unsupervised machine learning module that automatically clusters real-time high-dimensional microscopy signals and perform class-wise power spectral density calculation. This workflow is explicitly tested for wide-field fluorescence image analysis of semiconductor nanocrystal thin-films, while also proves beneficial for scanning tunnelling spectroscopy of electron donor-acceptor (2D) self-assemblies. We additionally demonstrate the impact of data-preprocessing on the clustering efficiency. As a programmed interface, our methodology would enable fast big-data-analytics for contemporary (micro)spectroscopes to achieve instant material characterization, spanning diverse spectrum of interests.

Traditionally, materials discovery has been driven more by evidence and intuition than by systematic design. However, the advent of “big data” and an exponential increase in computational power have reshaped the landscape. Today, we use simulations, artificial intelligence (AI), and machine learning (ML) to predict materials characteristics, which dramatically accelerates the discovery of novel materials. For instance, combinatorial megalibraries, where millions of distinct nanoparticles are created on a single chip, have spurred the need for automated characterization tools. This talk will present an ML model specifically developed to perform real-time binary classification of grayscale high-angle annular dark-field images of nanoparticles sourced from these megalibraries. We elaborate on the computational challenges and our solutions, including managing memory constraints, optimizing training time, and utilizing Neural Architecture Search tools. The final model outperformed our expectations, achieving over 95% precision and a weighted F-score of more than 90% on our test data set.