由美國伊利諾伊大學香檳分校材料科學與工程系的Qian Chen教授領導的團隊提出了UsiNet，這是一個無監督投影圖修復方法，旨在解決電子斷層掃描中常見的缺失楔形效應問題。

UsiNet的無監督訓練機制免除了對基準真值、人工標注或傾斜圖像模擬的依賴，顯著提升了其在真實電子斷層掃描數據集處理中的實用性，特別是在無法獲取全角度傾斜序列的場合。

該方法在訓練時只需極少量的數據集（甚至低至20個納米顆粒）和較小的傾斜范圍（±45°），使其在處理那些對束流敏感的聚合物和生物材料時變得尤為寶貴，因為這些材料的傾斜范圍可能由于束流累積傷害而受到限制。

UsiNet對窄傾斜范圍的容忍性在原位電子斷層掃描的研究中至關重要，比如在研究電化學循環、催化作用或腐蝕過程中納米顆粒形態變化時，由于需要保持時間分辨率，僅能收集有限的傾斜序列。

Fig. 5 Orientation-dependent missing wedge artifact and comparison between different reconstruction algorithms.

此外，UsiNet無需進行樣品平均化處理，因此可廣泛適用于諸如用于可充電離子電池的電極納米顆粒、催化劑納米顆粒和納米塑料等多種異質納米顆粒系統。在這些系統中，缺失楔形效應尤其棘手，因為它會引起明顯的各向異性失真。盡管此次展示主要聚焦于膠體納米顆粒，但UsiNet無監督修復的基本原理也同樣適用于其他包含3D納米尺度形態細節的樣本，如合金的微觀結構域和聚酰胺分離膜的皺折。UsiNet的出現極大地擴展了電子斷層掃描技術在解析材料的形態、合成及性能之間關聯性方面的潛力。

Fig. 6 Comparison of 3D reconstructions of experimentally synthesized NPs with and without inpainting.

UsiNet的應用前景極為廣闊，它不僅能夠揭示電池或催化納米材料的退化機理，還能幫助理解自然形成的納米塑料的形態與聚集行為，并能優化不同組成的納米顆粒的合成流程。該文近期發表于npj Computational Materials?10:?28 (2024).

Fig. 7 Visualizing the heterogeneity of experimentally synthesized NPs.

Editorial Summary

Electron tomography is favored for its high-resolution in 3D characterization of nanomaterials but is limited by the “missing wedge effect”, leading to distortions in the reconstructed images. Modern algorithms, especially neural network technologies in machine learning, have made advances in correcting these distortions, yet they still face challenges in accuracy due to differences between training data and actual conditions.?

A team led by Prof. Qian Chen from Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, USA, purposed UsiNet, an unsupervised sinogram inpainting method to correct the missing wedge effect in electron tomography. The unsupervised training in UsiNet does not require ground truth, manual annotation, or tilt image simulation, and thus is practically applicable to real electron tomography datasets where full angle tilt series are not obtainable. The authors demonstrate that UsiNet works with a small number of training dataset (down to 20 NPs) and narrow tilt range (±45°), which can be immediately useful for beam sensitive polymeric and biological materials where the tilt range can be limited by accumulated beam damage. The tolerance with a narrow tilt range could be critical for studies involving in-situ electron tomography—for example, on the evolution of the 3D shapes of NPs during chemical reactions such as electrochemical cycling, catalysis, and corrosion—where only scarce tilt series can be collected to ensure temporal resolution. Moreover, UsiNet does not require sample averaging and can thus apply to a broad range of heterogeneous NP systems such as electrode NPs used in rechargeable ion batteries, catalytical NPs, and nanoplastics. The missing wedge effect is otherwise particularly problematic for heterogeneous systems by generating anisotropic distortion. Although the demonstration focuses on colloidal NPs, the principle of unsupervised inpainting is expected to work for other samples containing 3D nanoscale morphology details, such as microstructural domains in alloys and crumples in polyamide separation membranes. UsiNet brings the full potential of electron tomography in charting the relationships of morphology with synthesis and performance of materials. A wide scope of applications can be enabled by UsiNet, such as uncovering degradation mechanisms of battery or catalytical nanomaterials, understanding morphologies and aggregation behaviors of naturally formed nanoplastics, and optimizing synthetic protocols of NPs with varying compositions. This article was recently published in?npj Computational Materials?10:?28 (2024).

原文Abstract及其翻譯

No ground truth needed: unsupervised sinogram inpainting for nanoparticle electron tomography (UsiNet) to correct missing wedges?(無需基準真值：用于校正缺失楔形區的無監督納米顆粒電子斷層掃描圖像重建的投影圖修復)

Lehan Yao,?Zhiheng Lyu,?Jiahui Li?&?Qian Chen?

Abstract?Complex natural and synthetic materials, such as subcellular organelles, device architectures in integrated circuits, and alloys with microstructural domains, require characterization methods that can investigate the morphology and physical properties of these materials in three dimensions (3D). Electron tomography has unparalleled (sub-)nm resolution in imaging 3D morphology of a material, critical for charting a relationship among synthesis, morphology, and performance. However, electron tomography has long suffered from an experimentally unavoidable missing wedge effect, which leads to undesirable and sometimes extensive distortion in the final reconstruction. Here we develop and demonstrate Unsupervised Sinogram Inpainting for Nanoparticle Electron Tomography (UsiNet) to correct missing wedges. UsiNet is the first sinogram inpainting method that can be realistically used for experimental electron tomography by circumventing the need for ground truth. We quantify its high performance using simulated electron tomography of nanoparticles (NPs). We then apply UsiNet to experimental tomographs, where >100 decahedral NPs and vastly different byproduct NPs are simultaneously reconstructed without missing wedge distortion. The reconstructed NPs are sorted based on their 3D shapes to understand the growth mechanism. Our work presents UsiNet as a potent tool to advance electron tomography, especially for heterogeneous samples and tomography datasets with large missing wedges, e.g. collected for beam sensitive materials or during temporally-resolved in-situ imaging.

摘要?復雜的自然和合成材料，如細胞亞結構器官、集成電路中的器件架構，以及具有微觀結構領域的合金，需要能夠在三維（3D）中研究這些材料的形態和物理性質的表征方法。電子斷層掃描在成像材料的3D形態方面具有無與倫比的（亞）納米分辨率，這對于描繪合成、形態和性能之間的關系至關重要。然而，電子斷層掃描長期以來一直受到實驗上不可避免的缺失楔形效應的困擾，這導致最終重建中出現了不希望的、有時甚至是大量的失真。在這里，我們開發并展示了用于納米顆粒電子斷層掃描的無監督投影圖修復（UsiNet）來校正缺失的楔形區。UsiNet是第一個可以在現實實驗電子斷層掃描中使用的投影圖修復方法，它避開了對基準真值的需求。我們使用模擬的納米顆粒電子斷層掃描來量化其高性能。然后我們將UsiNet應用于實驗層析圖，其中同時重建了100多個十面體納米顆粒和極為不同的副產物納米顆粒，且沒有缺失楔形失真。根據它們的3D形態對重建的納米顆粒進行分類，以理解生長機制。我們的工作將UsiNet作為推進電子斷層掃描的強大工具，特別適用于異質樣品和具有大量缺失楔形區的斷層數據集，例如為了敏感材料而收集的數據或在時間解析的原位成像過程中收集的數據。