Sparse-scan SEM reconstruction

A scanning electron microscope builds an image one pixel at a time. Measuring only some of the pixels makes a scan faster and gentler on delicate samples, if software can fill in the rest. This project tests which methods do that best.

Code on GitHub Full write-up (PDF) Technical details

Try it

Pick an example or use your own SEM image. The page simulates a sparse scan of it, then fills in the gaps two ways. It all runs in your browser; your image is never uploaded.

Scan pattern

20%
Noise

Shot noise from counting electrons: about 100, 20 or 5 electrons per pixel at full brightness.

The score (PSNR, in decibels) measures how close each result is to the original: higher is better, and 3 dB more means half the error. The network was trained on materials-science SEM images with simulated noise, so it can do worse on very different images or on real detector noise.

What we found

  1. The neural network fills in the gaps best. In simulation it beat every traditional method in all 30 test conditions, with over a third less error, and it takes a fraction of a second.
  2. Skip whole lines, not scattered pixels. For the same scan time, measuring complete lines gives far better images. Random pixels make the beam jump before almost every measurement, which wastes time and suffers most when the beam lands slightly off target.
  3. A larger generative model (diffusion) can match the network, but not beat it, and only at about 600 times the cost.
  4. A network must be trained on the scan pattern it will be used with. One trained only on random pixels did worse than a traditional method on line scans.
  5. On a real microscope, the network wins only when the real noise resembles its training noise. At short exposures the real noise behaves differently from the simulation's, and the traditional method won. At longer exposures the network won again. Training it on realistic noise is the obvious next step; it hasn't been tried yet.

The full write-up explains each finding with figures. The technical details have every table, and the README shows how to reproduce them.

How it was tested

We took real SEM images from the public NFFA-Europe collection and simulated sparse scans of them, including the random electron-counting noise and the time the beam needs to move. Six methods filled in the gaps, and each was scored against the original on 100 test images under 30 scan settings. The results were then checked on real scans of brain tissue from a different microscope.