Longcreek Blog
Notes on physics, math and technology, particularly in microscopy and imaging with Python for calculations
https://longcreek.me/
Thu, 14 Jul 2022 08:47:28 +0000
Thu, 14 Jul 2022 08:47:28 +0000
Jekyll v3.4.0

Georeferencing Chernarus and visiting in real life đź—şď¸Ź
<p>In this post I will walk through how to</p>
<ol>
<li>Obtain a highresolution raster map by stitching map tiles from a <a href="https://en.wikipedia.org/wiki/Tiled_web_map">tileserver</a> with a <a href="https://gitlab.com//snippets/2167748">shell script</a>.</li>
<li>Georeference a fictional map to a real location by manually matching features and their corresponding real locations using <a href="https://www.qgis.org/">QGIS</a>.</li>
<li>Convert the georeferenced map to a Garmin compatible KMZ file with a <a href="https://gitlab.com/kogens/gdal2custommap">Python script</a> using <a href="https://pypi.org/project/GDAL/">GDAL</a> for use on a handheld GPS device.</li>
</ol>
<p>I will apply it to the fictional map of <a href="https://community.bistudio.com/wiki/Chernarus">Chernarus</a> which is based on a real area near ĂšstĂ nad Labem in Czechia. The end product is a georeferenced Garmin compatible <a href="/assets/20210901chernarusirl/chernarus.kmz">KMZ file</a> which you can put on your GPS handheld if you ever visit the area.</p>
<p>The map below shows Chernarus (left) overlayed with the real region (right)  you can move the map and drag the slider to compare:</p>
<iframe width="100%" height="500" src="https://longcreek.me/leaflets/chernarus?center=50.705/14.082&collapseLayers=1&zoom=14&disableZoom=1" frameborder="0" allowfullscreen=""></iframe>
<p>Click <a href="/chernarustilemap">here</a> for a full screen version of the map.</p>
<h2 id="background">Background</h2>
<p><a href="https://armedassault.fandom.com/wiki/Chernarus">Chernarus</a> is a fictional country introduced in the game <a href="https://en.wikipedia.org/wiki/ARMA_2">Arma II</a>, and the name of a large piece of terrain featured in the game spanning roughly 15x15 km. It was also the main location used in the <a href="https://store.steampowered.com/app/224580/Arma_II_DayZ_Mod/">DayZ mod</a> which modified the game from a â€śmilitary simulatorâ€ť to a zombie survival game, which was later expanded<sup id="fnref:itsinalpha"><a href="#fn:itsinalpha" class="footnote">1</a></sup> into a <a href="https://en.wikipedia.org/wiki/DayZ_(video_game)">standalone game</a> where new details and towns were added to the original terrain (known as Chernarus+).</p>
<!
On the left below is a screenshot of the real [location in OpenTopoMap](http://127.0.0.1:4000/blog/2021/chernarusirl), and on the right the whole region of Chernarus as seen in the games.
![Comparison of Chernarus with area around Ăšsti nad Labem](/assets/20210901chernarusirl/map before.jpg)
>
<p><a href="https://dayz.ginfo.gg/chernarus/">The map of Chernarus</a> is closely based on an area in nothern Bohemia in Czechia, <a href="https://opentopomap.org/#map=12/50.7215/14.1411">between ĂšstĂ nad Labem and DÄ›ÄŤĂn</a> and appears very similar to the real location, as many of the real buildings are also appearing in the game. Some people in the community would occasionally post pictures comparing the game with the real location either by <a href="https://imgur.com/a/Oeph3">Google Street View</a> or from <a href="https://imgur.com/a/qecHz">actual visits</a> and note how familiar the place felt even though it was their first time there.</p>
<p>The map can be even be georeferenced and overlayed on the real location quite accurately:</p>
<p><img src="/assets/20210901chernarusirl/chernarusoverlayed.jpg" alt="Chernarus after georeferencing" class="centerimage" /></p>
<p>A few years ago I was in the area and decided to put the map on my GPS handheld and try to use it as reference while hiking around the area for fun.</p>
<p><img src="/assets/20210901chernarusirl/irl_gps_elektro.jpg" alt="Novy Sobor IRL" class="halfimage centerimage" /></p>
<p>It took a bit of effort and learning for me to get it working, and in this post I will walk through how I did it. The same process can be used for any map sufficiently similar to a real location, be it other fictional, historical or rare maps that you might have a copy of.</p>
<h2 id="downloadandstitchhighresolutionmapfromatileserver">Download and stitch highresolution map from a tileserver</h2>
<p>To obtain a highresolution map of Chernarus, we are going to download tiles from a tileserver and stitch them together into one large map. A tileserver is a common way to serve maps for the web, where a large map is rendered as many small tiles, so we donâ€™t have to load the full map. For this project I used the <a href="https://dayz.ginfo.gg/chernarus/">iZurvive Chernarus map</a> as a source.</p>
<p>The address for the tileserver is</p>
<p><code class="highlighterrouge">https://maps.izurvive.com/maps/CHMod/1.7.9/tiles/${z}/${x}/${y}.png</code></p>
<p>In this format <code class="highlighterrouge">${z}</code> is the zoom level, and <code class="highlighterrouge">${x}</code> and <code class="highlighterrouge">${y}</code> is the x and y integer tile coordinates, or simply the tile number.
Thus at zoom level <code class="highlighterrouge">z=6</code> (the highest of the server) the tile with <code class="highlighterrouge">x=4</code> and <code class="highlighterrouge">y=3</code> is found at <code class="highlighterrouge">https://maps.izurvive.com/maps/CHMod/1.7.9/tiles/6/4/3.png</code> and looks like this</p>
<p><img src="/assets/20210901chernarusirl/643.jpeg" alt="Chernarus tile 6/4/3" class="centerimage" /></p>
<p>The whole map is split into pieces like this, and our goal is now to download all tiles for the Chernarus map and stitch them together.
To download all the tiles, I used <a href="https://gitlab.com//snippets/2167748">a shell script</a> (modified from <a href="https://gis.stackexchange.com/questions/73946/automaticallydownloadandmergewebmaptilesintoonebigimage">this question on GIS Stackexchange</a>).</p>
<p>First we download the tiles:</p>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="c"># Extent of map</span>
<span class="nv">X1</span><span class="o">=</span>0
<span class="nv">X2</span><span class="o">=</span>50
<span class="nv">Y1</span><span class="o">=</span>0
<span class="nv">Y2</span><span class="o">=</span>42
<span class="c"># Zoom level</span>
<span class="nv">Z</span><span class="o">=</span>6
<span class="c"># Iterate through tiles and download</span>
TILE_URL <span class="o">=</span> https://maps.izurvive.com/maps/CHMod/1.7.9/tiles/<span class="k">${</span><span class="nv">Z</span><span class="k">}</span>/<span class="k">${</span><span class="nv">x</span><span class="k">}</span>/<span class="k">${</span><span class="nv">y</span><span class="k">}</span>.png
<span class="k">for </span>x <span class="k">in</span> <span class="sb">`</span>seq <span class="nv">$X1</span> <span class="nv">$X2</span><span class="sb">`</span>; <span class="k">do
for </span>y <span class="k">in</span> <span class="sb">`</span>seq <span class="nv">$Y1</span> <span class="nv">$Y2</span><span class="sb">`</span>; <span class="k">do
</span><span class="nb">echo</span> <span class="s2">"Getting </span><span class="k">${</span><span class="nv">x</span><span class="k">}</span><span class="s2">,</span><span class="k">${</span><span class="nv">y</span><span class="k">}</span><span class="s2">"</span>
curl s <span class="nv">$TILE_URL</span> o <span class="k">${</span><span class="nv">Z</span><span class="k">}</span>_<span class="k">${</span><span class="nv">y</span><span class="k">}</span>_<span class="k">${</span><span class="nv">x</span><span class="k">}</span>.png &
<span class="k">done
</span><span class="nb">wait
</span><span class="k">done
</span><span class="nb">echo</span> <span class="s2">"Rename .png to .jpg as files are wrongly named"</span>
<span class="k">for </span>i <span class="k">in</span> ./<span class="k">${</span><span class="nv">Z</span><span class="k">}</span>_<span class="k">*</span>.png ; <span class="k">do </span>mv <span class="s2">"</span><span class="nv">$i</span><span class="s2">"</span> <span class="s2">"</span><span class="k">${</span><span class="nv">i</span><span class="p">/.png/.jpg</span><span class="k">}</span><span class="s2">"</span>; <span class="k">done</span></code></pre></figure>
<p>The images are actually JPEG files, despite the .png file extension, which is why they get renamed to .jpg file extension.</p>
<p>To stitch the images together I use <code class="highlighterrouge">montage</code> from <a href="https://imagemagick.org/script/commandlineoptions.php#mode">Imagemagick</a>.
First we iterate through <code class="highlighterrouge">x</code> coordinates and for each <code class="highlighterrouge">x</code> we vertically stack all the relevant tiles from <code class="highlighterrouge">Y1</code> to <code class="highlighterrouge">Y2</code> together giving us a bunch of tall images<sup id="fnref:montageerror"><a href="#fn:montageerror" class="footnote">2</a></sup>:</p>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="k">for</span> <span class="o">((</span><span class="nv">x</span><span class="o">=</span>0; x<<span class="o">=</span><span class="nv">$X2</span>; x++<span class="o">))</span>; <span class="k">do
</span><span class="nb">echo</span> <span class="s2">"Stitching x=</span><span class="k">${</span><span class="nv">x</span><span class="k">}</span><span class="s2">"</span>
montage mode concatenate tile <span class="s2">"1x"</span> <span class="k">$(</span>ls <span class="k">${</span><span class="nv">Z</span><span class="k">}</span>_<span class="k">*</span>_<span class="k">${</span><span class="nv">x</span><span class="k">}</span>.jpg  sort Vr<span class="k">)</span> out_z<span class="k">${</span><span class="nv">Z</span><span class="k">}</span>_x<span class="k">${</span><span class="nv">x</span><span class="k">}</span>.jpg
<span class="k">done</span></code></pre></figure>
<p>Finally we stack all of these images horizontally</p>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="nb">echo</span> <span class="s2">"Stitching Everything together"</span>
montage mode concatenate tile <span class="s2">"x1"</span> <span class="k">$(</span>ls out_z<span class="k">${</span><span class="nv">Z</span><span class="k">}</span>_<span class="k">*</span>.jpg  sort V<span class="k">)</span> full_map.jpg</code></pre></figure>
<p>If everything worked out, we now have a full map of Chernarus in high resolution on our hands!</p>
<p><img src="/assets/20210901chernarusirl/full_map_800px.jpg" alt="Full stitched map of Chernarus" class="centerimage" /></p>
<h2 id="georeferencingchernarustoreallife">Georeferencing Chernarus to real life</h2>
<p>The next step is to overlay the Chernarus map on the real life counterpart by referencing some points on the game map with the same points on the real map. These are known as Ground Control Ppoints (GCP). To do this, I am using <a href="https://www.qgis.org/en/site/">QGIS</a>, a free and open source application for working with GIS (Geographical Information System) and the built in Georeferencer tool. Have a look at the <a href="https://docs.qgis.org/3.16/en/docs/user_manual/working_with_raster/georeferencer.html">QGIS documentation on Georeferencing</a> for more details.</p>
<h3 id="addingothertilemapstoqgis">Adding other tilemaps to QGIS</h3>
<p>If you just installed QGIS you may want to add some more map sources, as only the standard Open Street Map tiles are provided by default. On the left panel, simply right click on â€śXYZ Tilesâ€ť and choose â€śNew Connectionâ€ť.</p>
<p><img src="/assets/20210901chernarusirl/add_xyz_connection.jpg" alt="Add XYZ connection" class="centerimage" /></p>
<p>In the following dialogue you can add sources.</p>
<p><img src="/assets/20210901chernarusirl/opentopomap_connection.jpg" alt="OpenTopoMap parameters" class="centerimage" /></p>
<p>I like to use the <a href="https://opentopomap.org/">OpenTopoMap</a> for topographical maps at<code class="highlighterrouge">https://tile.opentopomap.org/{z}/{x}/{y}.png</code>, and <a href="https://docs.microsoft.com/enus/bingmaps/restservices/directlyaccessingthebingmapstiles">Bing Aerial</a> for satelite views at <code class="highlighterrouge">http://ecn.t3.tiles.virtualearth.net/tiles/a{q}.jpeg?g=1</code>.</p>
<p>You can find more open tile servers <a href="https://wiki.openstreetmap.org/wiki/Tile_servers">here</a>.</p>
<p>Now start a new project in QGIS and simply double click on the tilemaps you want to add them as a layer in the project.</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/add_layer_to_project.jpg" alt="Add layer to project" />
<img src="/assets/20210901chernarusirl/first_layers.jpg" alt="Add layer to project" /></p>
<p>Since we are using a web map for reference we will set the Coordinate Reference System (CRS) of the project to the <a href="https://en.wikipedia.org/wiki/Web_Mercator_projection">Web Mercator projection</a> or <code class="highlighterrouge">EPSG: 3857</code> so it renders properly. QGIS automatically sets the project CRS to be the same as the first layer added, but yo be sure you can set it by right clicking on the OpenTopoMap layer, then â€śLayer CRSâ€ť > â€śSet project CRS from layerâ€ť. This should set the project CRS to <code class="highlighterrouge">EPS 3857  WGS 84 / PseudoMercator</code>.</p>
<p><strong>NOTE</strong>: When we later transform the Chernarus map after georeferencing, it should be transformed into <code class="highlighterrouge">EPSG: 4326</code> if you want to use it on your Garmin device to show up properly.</p>
<p><img src="/assets/20210901chernarusirl/crsfromlayer.jpg" alt="Set project CRS from OpenTopoMap CRS" class="centerimage" /></p>
<p>Zoom in on ĂšstĂ nad Labem (in north western Czechia, north of Prague) in preparation for the next step.</p>
<p><img src="/assets/20210901chernarusirl/ustitopo.jpg" alt="ĂšstĂ nad Labem in OpenTopoMap" class="centerimage" /></p>
<h3 id="georeferencingpoints">Georeferencing points</h3>
<p>In order to load the Chernarus map into QGIS we first convert it to a PNG<sup id="fnref:jpginqgis"><a href="#fn:jpginqgis" class="footnote">3</a></sup>, e.g. with <code class="highlighterrouge">convert</code> from Imagemagick. Or just load it into <a href="https://krita.org/">your favourite image editor</a> and save as PNG.</p>
<figure class="highlight"><pre><code class="languagebash" datalang="bash">convert full_map.jpg chernarus.png</code></pre></figure>
<p>Now open the Georeferencer with Raster > Georeferencer</p>
<p><img src="/assets/20210901chernarusirl/rastergeoreferencer.jpg" alt="Raster > Georeferencer" class="centerimage" /></p>
<p>Load in the Chernaus map using Open raster (Ctrl+O), and it should look like this
<img src="/assets/20210901chernarusirl/openraster.jpg" alt="Chernarus map opened for georeferencing" class="centerimage" /></p>
<p>Now comes the fun part! We need to find points on the Chernarus map which are identifiable on the real map. Press â€śAdd pointâ€ť (or Ctrl + A) to add points, and then click somewhere on the Chernarus map that you can recognize in the real map. It will ask you to enter map coordinates  if you press â€śFrom map canvasâ€ť you can now click on the corresponding point on the real map, and the coordinates will be filled out.</p>
<p><img src="/assets/20210901chernarusirl/entercoords.jpg" alt="GCP in LipovĂˇ" class="centerimage" /></p>
<p>This could e.g. be the Tjunction in Stary Sobor (LipovĂˇ)</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/gcpstary.jpg" alt="GCP in Stary Sobor" class="halfimage" />
<img src="/assets/20210901chernarusirl/gcpstaryirl.jpg" alt="GCP in LipovĂˇ" class="halfimage" /></p>
<p>Or the roads meeting at Topolka dam (just north of Povrly)</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/gcptopolka.jpg" alt="GCP near Topolka Dam" class="halfimage" />
<img src="/assets/20210901chernarusirl/gcptopolkairl.jpg" alt="GCP near Povrly" class="halfimage" /></p>
<p>Keep going until you have at least four points, preferably more  Iâ€™ve used 1015 points for a good fit, making sure to put points near the map edges as well.</p>
<p>Now open Transform Settings (yellow cogwheel) to set the parameters used for the georeferencing transform. After some experimenting I have chosen transformation type â€śPolynomial 2â€ť and resampling method â€śCubicâ€ť for the final transform, but feel free to see what works best for you. I suggest using â€śNearest Neighbourâ€ť resampling to begin with, as it is the fastest so you can do adustments more easily.</p>
<p><strong>If you want to use it on a Garmin device</strong> you should set â€śTarget SRSâ€ť to <code class="highlighterrouge">EPSG:4326  WGS 84</code> as this is the standard map projection used in these devices (and for GPS in general), unless you want to fiddle with your devices projection settings.</p>
<p><img src="/assets/20210901chernarusirl/transformationsettings.jpg" alt="Transformation settings" class="centerimage" /></p>
<p><a href="https://docs.qgis.org/3.16/en/docs/user_manual/working_with_raster/georeferencer.html">The Georeferencer documentation</a> describes in more detail how the transformations work, and says the following about the Polynomial transforms:</p>
<blockquote>
<p>The Polynomial algorithms 23 use more general 2nd or 3rd degree polynomials instead of just affine transformation. This allows them to account for curvature or other systematic warping of the image, for instance photographed maps with curving edges. At least 6 (respectively 10) GCPâ€™s are required. Angles and local scale are not preserved or treated uniformly across the image. In particular, straight lines may become curved, and there may be significant distortion introduced at the edges or far from any GCPs arising from extrapolating the datafitted polynomials too far.</p>
</blockquote>
<p>If you check â€śLoad in QGIS when doneâ€ť the georeferenced Chernarus map will automatically be added as a new layer in the project when done. The file will also be saved as a <a href="https://en.wikipedia.org/wiki/GeoTIFF">GeoTIFF</a>, here named <code class="highlighterrouge">chernarus_modified.tif</code>, which is our georeferenced and transformed map.</p>
<p>Once the transformation parameters have been set, QGIS will calculate a residual for each point. When calculating a polynomial transform, there are some mathematical constraints which means a point on the raster may not end up exactly on top of the point you referenced on the real map after the transform. The residual essentially tells you how far off each point will land from its target. Thus if one point has a much higher residual than the rest (I aim for less than 10 px in this case), you may want to correct it.</p>
<p><img src="/assets/20210901chernarusirl/gcpresiduals.jpg" alt="Residuals of GCPs" class="centerimage" /></p>
<p>Now press â€śStart georeferencingâ€ť (green arrow or Ctrl + G) and wait for it to finish (takes about a minute on my laptop). If everything worked out, the Chernarus map should now be visible on top of the area around ĂšstĂ nad Labem</p>
<p><img src="/assets/20210901chernarusirl/transformedmap.jpg" alt="Chernarus overlayed on area around ĂšstĂ nad Labem" class="centerimage" /></p>
<p>In order to see both the real map and Chernarus map at once, you can open the layer style panel (F7) and chose â€śBlending modeâ€ť â€śMultiplyâ€ť.
<img src="/assets/20210901chernarusirl/blendingmultiply.jpg" alt="Multiply blending for transparency" class="centerimage" /></p>
<p>While the fit will never be perfect, you should be able to see most roads fairly well aligned to the real life counterparts. Otherwise try adjusting the GCP points and transformation parameters
<img src="/assets/20210901chernarusirl/blendingstary.jpg" alt="Multiply blending for transparency" class="centerimage" /></p>
<p><img src="/assets/20210901chernarusirl/blendinggorka.jpg" alt="Multiply blending for transparency" class="centerimage" /></p>
<h2 id="addtohandheldgpsgarmingpsmap">Add to handheld GPS (Garmin GPSMap)</h2>
<p>If you have a Garmin GPS device you can add the georeferenced map if your device supports CustomMaps<sup id="fnref:devicesupport"><a href="#fn:devicesupport" class="footnote">4</a></sup>. Iâ€™m using a GPSMap 64s, but the steps should be similar for other devices.</p>
<p>These devices donâ€™t support GeoTIFFs directly, and we have to convert our newly made map into the Garmin CustomMap format. The format is a KMZ file, which uses raster map tiles with a <a href="https://en.wikipedia.org/wiki/Keyhole_Markup_Language">KML file</a> to keep track of their positions. This means we have to convert our image back into tiles! The difference is of course that we have adjusted the map and can specify proper coordinates for the tiles.</p>
<h3 id="installinggdal">Installing GDAL</h3>
<p>To convert the GeoTIFF I use the Python 3 script <a href="https://gitlab.com/kogens/gdal2custommap"><code class="highlighterrouge">gdal2custommap</code></a><sup id="fnref:fork"><a href="#fn:fork" class="footnote">5</a></sup>. You will need to install the <a href="https://gdal.org/">GDAL library</a> as well as the python bindings found in the <a href="https://pypi.org/project/GDAL/"><code class="highlighterrouge">GDAL</code></a> package.</p>
<p><strong>Linux</strong>: If you are running some variant of Debian/Ubuntu it can be installed system wide with <code class="highlighterrouge">sudo apt install python3gdal</code>, which will take care of the GDAL dependencies. If you are managing packages with e.g. <code class="highlighterrouge">pip</code> you can use <code class="highlighterrouge">python m pip install GDAL</code> (assuming <code class="highlighterrouge">python</code> points to <code class="highlighterrouge">python3</code>).</p>
<p><strong>Windows</strong>: Assuming you have a working Python 3 installation on your machine, you will need to install GDAL and the Python bindings manually. This can be done with binaries found at <a href="https://www.gisinternals.com/release.php">GISInternals</a>. Have a look at the python <a href="https://pypi.org/project/GDAL/#windows">GDAL package documentation</a> for more details. After this, install the python <code class="highlighterrouge">GDAL</code> package with Python Package manager of choice, e.g. <a href="https://docs.conda.io/en/latest/">Conda</a>.</p>
<h3 id="converttokmzcustommap">Convert to KMZ (CustomMap)</h3>
<p>If we have a georeferenced map of Chernarus named <code class="highlighterrouge">chernarus_modified.tif</code> we first convert it to tiles and an associated KML file with <code class="highlighterrouge">gdal2kml.py</code></p>
<div class="highlighterrouge"><pre class="highlight"><code>python gdal2kml.py chernarus_modified.tif chernarus.kml
</code></pre>
</div>
<p>It is possible to specify some options such as JPEG quality (defaults to 75), KML draw order etc, which is detailed in the <a href="https://gitlab.com/kogens/gdal2custommap/">readme</a> of the script. Next we use <code class="highlighterrouge">kml2kmz.py</code> to zip up all the files into a single KMZ file</p>
<div class="highlighterrouge"><pre class="highlight"><code>python kml2kmz.py chernarus.kml
</code></pre>
</div>
<p>Now you should have a <code class="highlighterrouge">chernarus.kmz</code> which is ready to put on your device.</p>
<h3 id="addkmzfiletogarmindevice">Add KMZ file to Garmin device</h3>
<p>Connect to the device storage with e.g. USB cable and transfer <code class="highlighterrouge">chernarus.kmz</code> onto the device under <code class="highlighterrouge">/Garmin/CustomMaps/chernarus.kmz</code>. If your device supports an SD card you can use this to store the map, just put it under the same directory structure as the internal memory. This is recommended for transferring large files as the SD card tends to be much faster if used directly in a card reader instead of through the USB interface. Next, disconnect the device, insert the SD card if used, and reboot it.</p>
<p>Now you need to enable the custom map to see it. For my GPSMap 64s i do the following: In the map view, press the MENU button, Setup Map and in the bottom under Map Information/Select Map, I can choose which maps to show.</p>
<p class="centerimages imborder"><img src="/assets/20210901chernarusirl/scrn/256.jpg" alt="Area near Ăšsti" />
<img src="/assets/20210901chernarusirl/scrn/181.jpg" alt="Setup Map" />
<img src="/assets/20210901chernarusirl/scrn/185.jpg" alt="Select Map" /></p>
<p>Find <code class="highlighterrouge">chernarus.kmz</code> under Custom Maps and enable it.</p>
<p class="centerimages imborder"><img src="/assets/20210901chernarusirl/scrn/191.jpg" alt="Custom Map" />
<img src="/assets/20210901chernarusirl/scrn/200.jpg" alt="Custom Map" />
<img src="/assets/20210901chernarusirl/scrn/206.jpg" alt="Custom Map" /></p>
<p>You can quickly check if it works by opening it again after enabling it, which should show a preview of the Chernarus map. If it doesnâ€™t show up, try disabling other map layers as they might cover the custom map, and make sure the CRS of the GeoTIFF is set to <code class="highlighterrouge">EPSG:4326</code></p>
<p class="centerimages imborder"><img src="/assets/20210901chernarusirl/scrn/195.jpg" alt="Custom Map" />
<img src="/assets/20210901chernarusirl/scrn/198.jpg" alt="Custom Map" /></p>
<p>If you havenâ€™t already, find the area around ĂšstĂ nad Labem and DÄ›ÄŤĂn and zoom in. You have to zoom quite close before the custom map renders, for me it shows only at the 500 m scale bar and closer.</p>
<p class="centerimages imborder"><img src="/assets/20210901chernarusirl/scrn/917.jpg" alt="Custom Map" />
<img src="/assets/20210901chernarusirl/scrn/920.jpg" alt="Custom Map" /></p>
<p>If you have other maps enabled, they might overlay or completely cover the Chernarus map. Hereâ€™s how it looks in LipovĂˇ with and without <a href="https://garmin.opentopomap.org/#czech_republic">OpenTopoMap for Garmin</a> enabled, where roads, town names and similar show as an overlay.</p>
<p class="centerimages imborder"><img src="/assets/20210901chernarusirl/scrn/409.jpg" alt="Custom Map" />
<img src="/assets/20210901chernarusirl/scrn/395.jpg" alt="Custom Map" /></p>
<p>You can experiment with the <code class="highlighterrouge">draworder</code> option for <code class="highlighterrouge">gdal2kml.py</code> if you want to use overlays, as the KML order determines â€śheightâ€ť of the layers, where layers with the lowest numbers are drawn on the top.</p>
<h2 id="visittonorthbohemia">Visit to North Bohemia</h2>
<p>The final step is of course to travel to the area in real life! ĂšstĂ can be reached <a href="https://en.mapy.cz/s/ferahakego">by direct train from Prague</a> in 70 minutes for 57 â‚¬ (tickets from <a href="https://oneticket.cz/#/search?from=57076&to=53179&fare=adult&adultnum=1">oneticket.cz</a> or directly from <a href="https://www.cd.cz/en/">ÄŚeskĂ© drĂˇhy</a>, the national rail service), and there are plenty of places to stay overnight in the area  I can warmly recommmend the combined hotelbrewery <a href="https://www.pivovarnarychte.cz/en/">Na RychtÄ›</a> both for eating at their restaurant or for staying a few nights in ĂšstĂ.</p>
<p>If you have played just a bit of Arma or Dayz in Chernaus, you will find the place quite familiar as you walk around, as the developers have included not just the road network but also several buildings in the game, as shown in <a href="https://imgur.com/a/Oeph3">this imgur album</a>. Below are a couple of the pictures I took myself</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_gorka_church.jpg" alt="Custom" class="halfimage" />
<img src="/assets/20210901chernarusirl/chernarus_gorka_church.jpg" alt="Custom" class="halfimage" />
Characteristic yellow church in Javory (â€śGorkaâ€ť).</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_altar_berezino.jpg" alt="Custom" />
<img src="/assets/20210901chernarusirl/chernarus_altar_berezino.jpg" alt="Custom" />
View from JavorskĂ˝ vrch (â€śAltarâ€ť) to DÄ›ÄŤĂn (â€śBerezinoâ€ť).</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_gps_stary.jpg" alt="Custom" class="halfimagesingle" />
View of GPS when entering LipovĂˇ (â€śStary Soborâ€ť).</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_stary_barn.jpg" alt="Custom" />
<img src="/assets/20210901chernarusirl/chernarus_stary_barn.jpg" alt="Custom" />
The infamous red metal barn in LipovĂˇ (â€śStary Soborâ€ť). It seems to have been repaired when comparing to older pictures.</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_barn_bukov.jpg" alt="Custom" />
<img src="/assets/20210901chernarusirl/chernarus_barn_bukov.jpg" alt="Custom" />
A farm building in ÄŚeskĂ˝ Bukov (â€śStaroyeâ€ť) used several places in the game.</p>
<p class="centerimages"><img src="/assets/20210901chernarusirl/irl_topolka_dam.jpg" alt="Custom" />
<img src="/assets/20210901chernarusirl/chernarus_topolka_dam.jpg" alt="Custom" />
View towards south of dam near Povrly (â€śTopolka Damâ€ť north of â€śElektrozavodskâ€ť).</p>
<h2 id="summary">Summary</h2>
<p>In this post we have obtained a highresolution raster map from a tileserver, georeferenced it in QGIS to the real world location it is based on, and then converted it to a Garmin compatible KMZ file for viewing on a handheld GPS device.</p>
<p>You can also grab the <a href="/assets/20210901chernarusirl/chernarus.kmz">Chernarus KMZ file</a> directly to put on your device and save some time.</p>
<p>The methods shown here can be applied to any map which is sufficiently similar to real world locations  It could be interesting to walk around your town using a historical map to get a sense of how things used to look, and how much it changed since the map was made, for example.</p>
<!
## TODO
 Add screenshots from Arma Chernarus
 Center multiple images with CSS?
 Add useful resources (OpenTopoMap Garmin, list of tileservers,...)
 Add GPX points and suggested routes?
 Give link to the full map, GCP points and KMZ file.
https://www.google.com/maps/d/viewer?mid=1EJNBRC6X6C2P6Q1MGrsOb8Zynt4&ll=50.719315505531675%2C14.126621773895067&z=13
>
<hr />
<div class="footnotes">
<ol>
<li id="fn:itsinalpha">
<p>DayZ standalone had quite a turbulent development history, being stuck in a paid â€śearly alphaâ€ť prerelease stage for five years from 2013 until its <a href="https://en.wikipedia.org/wiki/DayZ_(video_game)#Release">official release in December 2018</a>, but thatâ€™s a story for another timeâ€¦ <a href="#fnref:itsinalpha" class="reversefootnote">↩</a></p>
</li>
<li id="fn:montageerror">
<p>Ideally <code class="highlighterrouge">montage</code> can stitch all the images together in one command, but I had issues with <code class="highlighterrouge">too many open files</code> errors on my system when trying to stack them all, which is why it is split in two sections. <a href="#fnref:montageerror" class="reversefootnote">↩</a></p>
</li>
<li id="fn:jpginqgis">
<p>QGIS does support loading of JPEG raster images, but I encounter whenever I try: <code class="highlighterrouge">not a supported raster data source. (libjpeg: Wrong JPEG library version: library is 62, caller expects 80)</code>, which is why I convert to PNG. I didnâ€™t find a solution, let me know if you can get it to work! <a href="#fnref:jpginqgis" class="reversefootnote">↩</a></p>
</li>
<li id="fn:devicesupport">
<p>Thereâ€™s a list of supported Garmin devices on <a href="https://support.garmin.com/?faq=cVuMqGHWaM7wTFWMkPNLN9">this support page</a> in footnote 1, â€śClick Here for a List of Compatible Devicesâ€ť. <a href="#fnref:devicesupport" class="reversefootnote">↩</a></p>
</li>
<li id="fn:fork">
<p>This is a fork of the Python 2 <a href="https://github.com/tf198/gdal2custommap"><code class="highlighterrouge">gdal2custommap</code></a> scripts by Tris Forster, which I modified slightly to wok with Python 3 and iron out some bugs. <a href="#fnref:fork" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Wed, 15 Sep 2021 10:00:00 +0000
https://longcreek.me/blog/2021/chernarusirl
https://longcreek.me/blog/2021/chernarusirl
gis,
qgis,
chernarus,
arma,
dayz,
maps,
gps

Visualizing the magnetic field of nanoparticles with Python
<script type="math/tex; mode=display">\newcommand{\vr}{\mathrm{\mathbf{r}}}
\newcommand{\unit}[1]{\,\mathrm{#1}}
\newcommand{\vec}[1]{\mathrm{\mathbf{#1}}}
\newcommand{\fourier}[1]{\mathcal{F}\left[#1 \right]}
\newcommand{\vB}{\vec{B}}
\newcommand{\vq}{\vec{q}}
\newcommand{\Iholo}{I_\mathrm{holo}}
\newcommand{\Iside}{I_\mathrm{s.b.}}</script>
<p><em>Note: The following is a continuation on <a href="/blog/2021/holographyintro">this post on electron holography</a>, which I suggest reading first to get a sense of what is going on.</em></p>
<p>As you have probably seen before, magnets and their magnetic fields are often visualized as field lines like in the figure below<sup id="fnref:wikimediafieldlines"><a href="#fn:wikimediafieldlines" class="footnote">1</a></sup> which represent how the magnetic field extends outside a rectangular magnet</p>
<!
https://commons.wikimedia.org/wiki/File:VFPt_cylindermagnet_fieldrepresentations.svg
>
<p><img src="/assets/202106holopart2/magneticfieldlines2.svg" alt="Magnetic field lines" class="centerimage halfimage" /></p>
<p>This is a common illustration used for magnetic fields, where magnetic field lines are drawn along lines of constant field strength â€“ A sort of contour lines for the magnetic field strength. If we were to place a tiny compas at any position outside the magnet, the compass needle would align itself to the direction of the field lines at this position.</p>
<p>This is usually presented for magnets large enough hold in your hand, but it is actually possible to directly measure the magnetic field of nanoparticles in a <a href="https://en.wikipedia.org/wiki/Transmission_electron_microscopy">Transmission Electron Microscope</a> (TEM), just like this:</p>
<p><img src="/assets/202106holopart2/12hirescontouroverlay.png" alt="magnetic field lines around magnetite nanoparticle" class="centerimage" /></p>
<p>The TEM uses electrons as a probe to image particles, and electrons moving in a magnetic field will get affected by it. Information about the magnetic field of a sample can be saved using a technique known as Electron Holography which preserves the phase <script type="math/tex">\phi</script> of the electrons in the acquired image.</p>
<p>Reconstructing the above image from raw holograms involve a few different image processing techniques including Fourier analysis, frequency filtering and image matching to find the translational shift between two almost identical images, as well as other image processing techniques.</p>
<p>In this post I walk through all the Python code used to go from a raw hologram to a magnetic phase image such as the above.
You can see the full code in <a href="/assets/202106holopart2/holographypython.html">this Jupyter notebook</a>, which you can <a href="/assets/202106holopart2/holographypython.zip">download along with the holograms</a> if you want to play along yourself.</p>
<p>All the data shown was acquired as part of my bachelors thesis<sup id="fnref:bachelorthesis"><a href="#fn:bachelorthesis" class="footnote">2</a></sup> at <a href="https://www.nanolab.dtu.dk/">DTU Nanolab</a> (previously Center for Electron Nanoscopy) several years back and follows the procedure detailed in <a href="https://doi.org/10.5772/22366">this book chapter</a><sup id="fnref:holographybook"><a href="#fn:holographybook" class="footnote">3</a></sup>. I recently went back to revisit the subject to redo it in Python out of curiousity and decided do this writeup to share with anyone why might be interested.</p>
<!
A lot of terms used here were defined in my previous [post on electron holography fundamentals](/blog/2021/holographyintro) which you may want to read first to understand fully what we will be doing.
>
<h2 id="calculatingphaseimagefromahologram">Calculating phase image from a hologram</h2>
<h3 id="importingmicroscopedata">Importing microscope data</h3>
<p>We start out with some common packages for scientific Python, namely <code class="highlighterrouge">numpy</code> and <code class="highlighterrouge">matplotlib</code><sup id="fnref:requirements"><a href="#fn:requirements" class="footnote">4</a></sup>. We use pyplot for plotting and will set some default parameters with <code class="highlighterrouge">plt.rcParams</code> so we donâ€™t have to do manually for each plot.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Import numpy and fft functions directly</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">numpy.fft</span> <span class="kn">import</span> <span class="n">fft2</span><span class="p">,</span> <span class="n">ifft2</span><span class="p">,</span> <span class="n">fftshift</span>
<span class="c"># Set some default parameters for the plots</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span>
<span class="n">plt</span><span class="o">.</span><span class="n">rcParams</span><span class="p">[</span><span class="s">'figure.figsize'</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">16</span><span class="p">,</span> <span class="mi">8</span><span class="p">]</span>
<span class="n">plt</span><span class="o">.</span><span class="n">rcParams</span><span class="p">[</span><span class="s">'figure.frameon'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span>
<span class="n">plt</span><span class="o">.</span><span class="n">rcParams</span><span class="p">[</span><span class="s">'image.cmap'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'gray'</span></code></pre></figure>
<p>The raw data from the microscope is saved in <code class="highlighterrouge">dm3</code> files, a common format used in electron microscopy. It is usually opened with the proprietary <a href="https://www.gatan.com/products/temanalysis/gatanmicroscopysuitesoftware">Gatan Digital Micrograph</a> software or a program like <a href="https://imagej.nih.gov/ij/">ImageJ</a> which can be difficult to work with for more complex calculations, but luckily the <a href="https://openncem.readthedocs.io/en/latest/">OpenNCEM</a> project allows us to open it in Python (<a href="https://www.mathworks.com/matlabcentral/fileexchange/43005readdm3anddm4imagefiles">look here if using Matlab</a>).</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">ncempy.io.dm</span> <span class="kn">import</span> <span class="n">dmReader</span>
<span class="c"># Load dm3 files for a particle and reference</span>
<span class="n">dmdata1</span> <span class="o">=</span> <span class="n">dmReader</span><span class="p">(</span><span class="s">'data/h05 +30sat.dm3'</span><span class="p">)</span>
<span class="n">dmdata1r</span> <span class="o">=</span> <span class="n">dmReader</span><span class="p">(</span><span class="s">'data/h06r +30sat.dm3'</span><span class="p">)</span>
<span class="c"># Extract image data</span>
<span class="n">im1</span> <span class="o">=</span> <span class="n">dmdata1</span><span class="p">[</span><span class="s">'data'</span><span class="p">]</span>
<span class="n">im1r</span> <span class="o">=</span> <span class="n">dmdata1r</span><span class="p">[</span><span class="s">'data'</span><span class="p">]</span></code></pre></figure>
<p>The raw image in <code class="highlighterrouge">im1</code>, our hologram, looks like this</p>
<p><img src="/assets/202106holopart2/01particle1.png" alt="Raw hologram of magnetite nanoparticle" class="centerimage" /></p>
<p>Note that the numbered axes show pixel numbers instead of nanometers. We keep it this way for now as it helps illustrate the numerical calculalations we will be doing. However, it is still important to provide a scale when showing microscope images, and it is readily available from the metadata:</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="k">print</span><span class="p">(</span><span class="s">'Scale from metadata: 1 px = </span><span class="si">%</span><span class="s">s </span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span><span class="p">(</span><span class="n">dmdata1</span><span class="p">[</span><span class="s">'pixelSize'</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="n">dmdata1</span><span class="p">[</span><span class="s">'pixelUnit'</span><span class="p">][</span><span class="mi">0</span><span class="p">]))</span></code></pre></figure>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="gp">>> </span>Scale from metadata: 1 px <span class="o">=</span> 0.0037131761 Âµm</code></pre></figure>
<p>So 1 px = 0.0037 Âµm, <strong>but</strong>: the scale is off by a factor 10 in these specific images. This is because the objective lens (the one that forms the image) is the special Lorentz lens, and the correction has not been included correctly in the metadata. We will correct for this and convert to nanometers while weâ€™re at it:</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Redefine to nm and correct for Lorentz lens factor 10 error</span>
<span class="n">pxsize</span> <span class="o">=</span> <span class="n">dmdata1</span><span class="p">[</span><span class="s">'pixelSize'</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="mi">1000</span><span class="o">/</span><span class="mi">10</span>
<span class="n">pixel_unit</span> <span class="o">=</span> <span class="s">'nm'</span>
<span class="k">print</span><span class="p">(</span><span class="s">'Corrected scale: 1 px = </span><span class="si">%.4</span><span class="s">f </span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span><span class="p">(</span><span class="n">pxsize</span><span class="p">,</span> <span class="n">pixel_unit</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="s">'Field of view: </span><span class="si">%.2</span><span class="s">f nm x </span><span class="si">%.2</span><span class="s">f nm'</span> <span class="o">%</span><span class="p">(</span><span class="n">im1</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">*</span><span class="n">pxsize</span><span class="p">,</span> <span class="n">im1</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">*</span><span class="n">pxsize</span><span class="p">))</span></code></pre></figure>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="gp">>> </span>Corrected scale: 1 px <span class="o">=</span> 0.3713 nm
<span class="gp">>> </span>Field of view: 760.46 nm x 760.46 nm</code></pre></figure>
<p>The image is thus 760 nm from one side to the other, and the particle is about 90 nm wide.</p>
<p>There is a wealth of other information in the metadata such as acceleration voltage, magnification etc. which can be accessed with the <a href="https://openncem.readthedocs.io/en/latest/ncempy.io.html#ncempy.io.dm.fileDM"><code class="highlighterrouge">fileDM</code></a> method from <code class="highlighterrouge">ncempy</code> if needed.</p>
<h3 id="fouriertransformedhologramwithsidebands">Fourier Transformed hologram with sidebands</h3>
<p>We can now calculate the Fast Fourier Transform (FFT) on our hologram of the particle using <code class="highlighterrouge">fft2</code> from <code class="highlighterrouge">numpy</code>. We also use <code class="highlighterrouge">fftshift</code> which centers the main peak in the image, as it is otherwise split into the corners for numerical reasons.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Transform image, ready for visualization</span>
<span class="n">im1_fft</span> <span class="o">=</span> <span class="n">fftshift</span><span class="p">(</span><span class="n">fft2</span><span class="p">(</span><span class="n">im1</span><span class="p">))</span>
<span class="n">im1_fft_abslog</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">im1_fft</span><span class="p">))</span>
<span class="c"># Adjust colormap to increase contrast</span>
<span class="n">vmin_fft</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">amin</span><span class="p">(</span><span class="n">im1_fft_abslog</span><span class="p">)</span><span class="o">*</span><span class="mi">3</span>
<span class="n">vmax_fft</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">amax</span><span class="p">(</span><span class="n">im1_fft_abslog</span><span class="p">)</span><span class="o">*</span><span class="mf">0.8</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">im1_fft_abslog</span><span class="p">,</span> <span class="n">vmax</span><span class="o">=</span><span class="n">vmax_fft</span><span class="p">,</span> <span class="n">vmin</span><span class="o">=</span><span class="n">vmin_fft</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlim</span><span class="p">([</span><span class="mi">600</span><span class="p">,</span> <span class="mi">1448</span><span class="p">])</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylim</span><span class="p">([</span><span class="mi">1448</span><span class="p">,</span> <span class="mi">600</span><span class="p">])</span></code></pre></figure>
<p><img src="/assets/202106holopart2//02particle1fft.png" alt="FFT of particle" class="centerimage" /></p>
<p>Here we see the central peak with the two â€śsidebandsâ€ť off to the sides<sup id="fnref:fftabslog"><a href="#fn:fftabslog" class="footnote">5</a></sup>. Each sideband contains both the amplitude of the wave which passed through the sample and the reference wave <script type="math/tex">A_i(\vr)</script> and <script type="math/tex">A_r(\vr)</script> respectively, as well as the electron phase <script type="math/tex">\phi(\vr)</script>:</p>
<script type="math/tex; mode=display">\hat{I}_\mathrm{s.b.}(\vq) = \fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}}</script>
<p>Here <script type="math/tex">\vr = (x,y)</script> refers to a given point (or pixel) in the image. The next step in isolating the phase image <script type="math/tex">\phi(\vr)</script> is then to extract one of the sidebands.</p>
<h3 id="extractingasideband">Extracting a sideband</h3>
<p>The FFT image contains three significant peaks, namely the central one (by far most intense) and the two sidebands (similar intensity). Each sideband is almost identical, with the important difference being that they are complex conjugates of each other  This means a phase image calculated from one sideband will correspond exactly to the phase image from the other sideband with opposite sign.</p>
<p>To select a sideband, we will find coordinates of the peaks in the image using <code class="highlighterrouge">peak_local_max</code> from the <code class="highlighterrouge">scikitimage</code> package exactly like in <a href="https://scikitimage.org/docs/dev/auto_examples/segmentation/plot_peak_local_max.html">this example</a>. As the central peak is always strongest we simply find the second strongest peak and check if itâ€™s a positive or negative phase change, otherwise we take the third strongest. We write this into a function as weâ€™ll do it a few times</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">scipy.ndimage</span> <span class="kn">import</span> <span class="n">maximum_filter</span>
<span class="kn">from</span> <span class="nn">skimage.feature</span> <span class="kn">import</span> <span class="n">peak_local_max</span>
<span class="k">def</span> <span class="nf">peak_coordinate</span><span class="p">(</span><span class="n">image_fft</span><span class="p">,</span> <span class="n">peak_number</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
<span class="c"># Find the peaks</span>
<span class="n">im</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">image_fft</span><span class="p">)</span>
<span class="n">image_max</span> <span class="o">=</span> <span class="n">maximum_filter</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'constant'</span><span class="p">)</span>
<span class="n">coordinates</span> <span class="o">=</span> <span class="n">peak_local_max</span><span class="p">(</span><span class="n">im</span><span class="p">,</span> <span class="n">min_distance</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="c"># Get values of peaks and sort, get coordinates of second largest peak</span>
<span class="n">peaks</span> <span class="o">=</span> <span class="p">[</span><span class="n">image_fft</span><span class="p">[</span><span class="n">coordinate</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">coordinate</span><span class="p">[</span><span class="mi">1</span><span class="p">]]</span> <span class="k">for</span> <span class="n">coordinate</span> <span class="ow">in</span> <span class="n">coordinates</span><span class="p">]</span>
<span class="n">peaks_idx</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">peaks</span><span class="p">)</span>
<span class="n">coordinates_sorted</span> <span class="o">=</span> <span class="p">[</span><span class="n">coordinates</span><span class="p">[</span><span class="n">peak_idx</span><span class="p">]</span> <span class="k">for</span> <span class="n">peak_idx</span> <span class="ow">in</span> <span class="n">peaks_idx</span><span class="p">]</span>
<span class="n">sideband_idx</span> <span class="o">=</span> <span class="n">coordinates_sorted</span><span class="p">[</span><span class="n">peak_number</span><span class="p">]</span>
<span class="c"># Choose sideband on the right side of the image</span>
<span class="k">if</span> <span class="n">sideband_idx</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o"><</span> <span class="n">image_fft</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">//</span><span class="mi">2</span><span class="p">:</span>
<span class="n">sideband_idx</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span><span class="o">*</span><span class="p">(</span><span class="n">sideband_idx</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o"></span><span class="n">image_fft</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">//</span><span class="mi">2</span><span class="p">)</span>
<span class="n">sideband_idx</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span><span class="o">*</span><span class="p">(</span><span class="n">sideband_idx</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o"></span><span class="n">image_fft</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">//</span><span class="mi">2</span><span class="p">)</span>
<span class="k">return</span> <span class="n">sideband_idx</span></code></pre></figure>
<p>Weâ€™ll wrap this into a function to automatically extract a sideband from a given FFT image. Note that the image is stored as a matrix where rows (up and down) are indexed before columns (left and right), meaning <script type="math/tex">x</script> and <script type="math/tex">y</script> must be reversed since we want a horizontal xaxis and vertical yaxis.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="k">def</span> <span class="nf">extract_sideband</span><span class="p">(</span><span class="n">image_fft</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="mi">128</span><span class="p">):</span>
<span class="n">sideband_idx</span> <span class="o">=</span> <span class="n">peak_coordinate</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">image_fft</span><span class="p">))</span>
<span class="n">width</span> <span class="o">=</span> <span class="n">width</span><span class="o">//</span><span class="mi">2</span>
<span class="n">x1</span> <span class="o">=</span> <span class="n">sideband_idx</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o"></span><span class="n">width</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">sideband_idx</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">+</span><span class="n">width</span>
<span class="n">y1</span> <span class="o">=</span> <span class="n">sideband_idx</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o"></span><span class="n">width</span>
<span class="n">y2</span> <span class="o">=</span> <span class="n">sideband_idx</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">+</span><span class="n">width</span>
<span class="n">sideband_fft</span> <span class="o">=</span> <span class="n">image_fft</span><span class="p">[</span><span class="n">y1</span><span class="p">:</span><span class="n">y2</span><span class="p">,</span> <span class="n">x1</span><span class="p">:</span><span class="n">x2</span><span class="p">]</span>
<span class="k">return</span> <span class="n">sideband_fft</span></code></pre></figure>
<p>We can now apply our new function and take a closer look at the sideband.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="n">sideband_size</span> <span class="o">=</span> <span class="mi">200</span>
<span class="n">sideband1_fft</span> <span class="o">=</span> <span class="n">extract_sideband</span><span class="p">(</span><span class="n">im1_fft</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="n">sideband_size</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">sideband1_fft</span><span class="p">)))</span></code></pre></figure>
<p><img src="/assets/202106holopart2/
03particle1sideand.png" alt="Selected sideband of first hologram" class="centerimage" /></p>
<p>The vertical and horizontal streaks are artifacts which appear because our our original image is finite at the edges. When we decompose the image into its frequency components the sharp, abrupt change at the image have to be represented by many different frequencies, giving rise to these streaks. Later you will be able to see the the result of these artifacts in the reconstructed image along the very edges of the images but this will not affect our analysis otherwise.</p>
<h3 id="phaseandamplitudefromsideband">Phase and amplitude from sideband</h3>
<p>We are now ready to calculate the amplitude and phase of our image!</p>
<p>When performing a fourier transform of an image (real values), the FFT will contain complex values. Usually when the inverse fourier transform is applied we would get back the original image containing only real values. However, by cropping out the sideband we have redefined the center of the FFT image. Now when applying the inverse fourier transform to the sideband, the resulting image will contain complex values.</p>
<p>This is actually intended! We can find the â€śusualâ€ť intensity image from the absolute value of the complex number in each pixel as <script type="math/tex">I = \sqrt{\mathrm{Re}^2+\mathrm{Im}^2}</script> with <code class="highlighterrouge">np.abs</code> (like when visualizing the FFT). The phase image is found from the angle of the complex number <script type="math/tex">\varphi=\arctan\left(\frac{\mathrm{Im}}{\mathrm{Re}}\right)</script> using <code class="highlighterrouge">np.angle</code>.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Calculate amplitude and phase of image</span>
<span class="n">sideband1</span> <span class="o">=</span> <span class="n">ifft2</span><span class="p">(</span><span class="n">fftshift</span><span class="p">(</span><span class="n">sideband1_fft</span><span class="p">))</span>
<span class="n">sideband1_abs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">sideband1</span><span class="p">)</span>
<span class="n">sideband1_phase</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">angle</span><span class="p">(</span><span class="n">sideband1</span><span class="p">)</span></code></pre></figure>
<p>Hereâ€™s the amplitude image</p>
<p><img src="/assets/202106holopart2/04ap1amp.png" alt="Amplitude of magnetic particle hologram" class="centerimage" /></p>
<p>Note how the amplitude image is very similar to the original hologram, but in reduced resolution since we cropped it out as part of the full FFT image. The phase image looks like this</p>
<p><img src="/assets/202106holopart2/04bp1phase.png" alt="Phase of magnetic particle hologram with wrapping artifacts" class="centerimage" /></p>
<p>We immediately notice some odd artifacts in the form of sudden jumps in value. We calculated the phase (or angle) of the complex numbers with <a href="https://numpy.org/doc/stable/reference/generated/numpy.angle.html"><code class="highlighterrouge">np.angle</code></a> which is an implementation of <a href="https://en.wikipedia.org/wiki/Atan2">atan2</a>, a numerical version of the <script type="math/tex">\arctan</script> function which outputs an angle between <script type="math/tex">\pi</script> and <script type="math/tex">\pi</script>, and an angle is thus bound between these values.</p>
<p>This means that if a region of the phase image has a phase value extending above <script type="math/tex">\pi</script> for example, the value returned by <code class="highlighterrouge">atan2</code> will â€śwrap aroundâ€ť and return <script type="math/tex">\pi</script> as it is does not know that it should be continuing the values of the neighbouring pixels.</p>
<p>Fortunately â€śunwrappingâ€ť phase images is a common procedure in image processing and <code class="highlighterrouge">scikitimage</code> has it built into the <code class="highlighterrouge">unwrap_phase</code> method:</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">skimage.restoration</span> <span class="kn">import</span> <span class="n">unwrap_phase</span>
<span class="n">sideband1_phase</span> <span class="o">=</span> <span class="n">unwrap_phase</span><span class="p">(</span><span class="n">sideband1_phase</span><span class="p">)</span></code></pre></figure>
<p>Our phase image now looks like this</p>
<p><img src="/assets/202106holopart2/05p1unwrappedphase.png" alt="Phase image of particle" class="centerimage" /></p>
<p>So far so good, we have succesfully recovered the full phase shift of the electron beam after passing through a nanoparticle! We will improve it a bit by subtracting a background reference before.</p>
<h3 id="correctingphaseimagewithbackgroundreference">Correcting phase image with background reference</h3>
<p>First we load in a reference background image which was acquired in an area without any sample present to get a clean background reference. Using the functions defined earlier makes quick work of extracting the sideband</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="n">im1r_fft</span> <span class="o">=</span> <span class="n">fftshift</span><span class="p">(</span><span class="n">fft2</span><span class="p">(</span><span class="n">im1r</span><span class="p">))</span>
<span class="n">sideband1r_fft</span> <span class="o">=</span> <span class="n">extract_sideband</span><span class="p">(</span><span class="n">im1r_fft</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="n">sideband_size</span><span class="p">)</span>
<span class="n">fig</span><span class="p">,</span> <span class="p">(</span><span class="n">ax1</span><span class="p">,</span> <span class="n">ax2</span><span class="p">,</span> <span class="n">ax3</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">)</span>
<span class="n">ax1</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">im1r</span><span class="p">)</span>
<span class="n">ax2</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">im1r_fft</span><span class="p">)))</span>
<span class="n">ax3</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="nb">abs</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">sideband1r_fft</span><span class="p">)))</span></code></pre></figure>
<p class="centerimages noshadow"><img src="/assets/202106holopart2/06p1rextraction.png" alt="Reference phase image for background correction" /></p>
<p>Now we calculate the phase of the reference image</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="n">sideband1r</span> <span class="o">=</span> <span class="n">ifft2</span><span class="p">(</span><span class="n">fftshift</span><span class="p">(</span><span class="n">sideband1r_fft</span><span class="p">))</span>
<span class="n">sideband1r_phase</span> <span class="o">=</span> <span class="n">unwrap_phase</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">angle</span><span class="p">(</span><span class="n">sideband1r</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Background reference phase image'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">sideband1r_phase</span><span class="p">)</span></code></pre></figure>
<p><img src="/assets/202106holopart2/06p1rphase.png" alt="Reference background phase image" class="centerimage" /></p>
<p>Finally we subtract the reference image which leaves us with a much more even phase image</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="n">phi1</span> <span class="o">=</span> <span class="n">sideband1_phase</span><span class="o"></span><span class="n">sideband1r_phase</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Corrected phase image of particle'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi1</span><span class="p">)</span></code></pre></figure>
<p><img src="/assets/202106holopart2/08aphi1corrected.png" alt="Phase image with background reference subtracted" class="centerimage" /></p>
<p>The magnetic phase shift still not easily visible as the electric contribution dominates the phase image.</p>
<h3 id="hologramofparticlewithreversedmagnetization">Hologram of particle with reversed magnetization</h3>
<p>In order to isolate just the magnetic contribution <script type="math/tex">\phi_m</script> to the phase shift we need an image of the exact same particle where the particleâ€™s magnetization, and thus <script type="math/tex">\phi_m</script>, is reversed. Since <script type="math/tex">\phi_m</script> changes sign while <script type="math/tex">\phi_e</script> is unchanged it is a matter of simple subtraction</p>
<script type="math/tex; mode=display">\phi = (\phi_e+\phi_m)  (\phi_e  \phi_m) = 2\phi_m</script>
<p>Then we just divide by 2, giving us the pure magnetic phase image <script type="math/tex">\phi_m</script>.</p>
<p>The magnetic reversal is done by applying a strong magnetic field inside the microscope in the opposite direction of the previous used to magnetize the particle in the first place, using one of the magnetic lenses.</p>
<p>We load in the image and subtract a reference phase image just like before:</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Load data</span>
<span class="n">dmdata2</span> <span class="o">=</span> <span class="n">dmReader</span><span class="p">(</span><span class="s">'data/h19 30sat.dm3'</span><span class="p">)</span>
<span class="n">dmdata2r</span> <span class="o">=</span> <span class="n">dmReader</span><span class="p">(</span><span class="s">'data/h17r.dm3'</span><span class="p">)</span>
<span class="n">im2</span> <span class="o">=</span> <span class="n">dmdata2</span><span class="p">[</span><span class="s">'data'</span><span class="p">]</span>
<span class="n">im2r</span> <span class="o">=</span> <span class="n">dmdata2r</span><span class="p">[</span><span class="s">'data'</span><span class="p">]</span>
<span class="c"># Calculate FFT and extract sideband for image</span>
<span class="n">im2_fft</span> <span class="o">=</span> <span class="n">fftshift</span><span class="p">(</span><span class="n">fft2</span><span class="p">(</span><span class="n">im2</span><span class="p">))</span>
<span class="n">sideband2_fft</span> <span class="o">=</span> <span class="n">extract_sideband</span><span class="p">(</span><span class="n">im2_fft</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="n">sideband_size</span><span class="p">)</span>
<span class="n">sideband2</span> <span class="o">=</span> <span class="n">ifft2</span><span class="p">(</span><span class="n">fftshift</span><span class="p">(</span><span class="n">sideband2_fft</span><span class="p">))</span>
<span class="n">sideband2_phase</span> <span class="o">=</span> <span class="n">unwrap_phase</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">angle</span><span class="p">(</span><span class="n">sideband2</span><span class="p">))</span>
<span class="c"># Calculate FFT and extract sideband for reference image</span>
<span class="n">im2r_fft</span> <span class="o">=</span> <span class="n">fftshift</span><span class="p">(</span><span class="n">fft2</span><span class="p">(</span><span class="n">im2r</span><span class="p">))</span>
<span class="n">sideband2r_fft</span> <span class="o">=</span> <span class="n">extract_sideband</span><span class="p">(</span><span class="n">im2r_fft</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="n">sideband_size</span><span class="p">)</span>
<span class="n">sideband2r</span> <span class="o">=</span> <span class="n">ifft2</span><span class="p">(</span><span class="n">fftshift</span><span class="p">(</span><span class="n">sideband2r_fft</span><span class="p">))</span>
<span class="n">sideband2r_phase</span> <span class="o">=</span> <span class="n">unwrap_phase</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">angle</span><span class="p">(</span><span class="n">sideband2r</span><span class="p">))</span>
<span class="c"># Subtract reference phase shift</span>
<span class="n">phi2</span> <span class="o">=</span> <span class="n">sideband2_phase</span><span class="o"></span><span class="n">sideband2r_phase</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Corrected phase image of particle with reversed magnetization'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi2</span><span class="p">)</span></code></pre></figure>
<p><img src="/assets/202106holopart2/08bphi2corrected.png" alt="Phase image with reversed magnetization" class="centerimage" /></p>
<p>This phase image is almost identical to the first, but with two important differences: <script type="math/tex">\phi_m</script> is opposite of before, and the particle is in a slightly different position in the image. This means we cannot directly subtract the two images until we have found out exactly how much the particle position changed.</p>
<h2 id="matchingtwosimilarimagestoadjustfortranslation">Matching two similar images to adjust for translation</h2>
<p>There is a significant shift in x and y positions between the images, but only negligible rotation. This means we can use <a href="https://en.wikipedia.org/wiki/Phase_correlation">phase cross correlation</a> to get a good estimate of the translation between images as implemented in <a href="https://scikitimage.org/docs/dev/api/skimage.registration.html?skimage.registration.phase_cross_correlation#phasecrosscorrelation"><code class="highlighterrouge">phase_cross_correlation</code></a> in <code class="highlighterrouge">scikitimage</code><sup id="fnref:matchrotation"><a href="#fn:matchrotation" class="footnote">6</a></sup>. There is a decent <a href="https://scikitimage.org/docs/dev/auto_examples/registration/plot_register_translation.html#sphxglrautoexamplesregistrationplotregistertranslationpy">example with details here</a>. Knowing this shift we can correct for it and ensure the images have a common center so we can subtract them properly.</p>
<p>We start out by applying a bandpass with gaussian blur to remove some high and lowfrequency details which can detract from the image matching.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">skimage.filters</span> <span class="kn">import</span> <span class="n">window</span><span class="p">,</span> <span class="n">gaussian</span><span class="p">,</span> <span class="n">difference_of_gaussians</span>
<span class="kn">from</span> <span class="nn">skimage.registration</span> <span class="kn">import</span> <span class="n">phase_cross_correlation</span>
<span class="kn">from</span> <span class="nn">skimage</span> <span class="kn">import</span> <span class="n">transform</span>
<span class="c"># Bandpass with Gaussian blur</span>
<span class="n">sigma1</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">sigma2</span> <span class="o">=</span> <span class="mi">3</span>
<span class="n">phi1_filt</span> <span class="o">=</span> <span class="n">difference_of_gaussians</span><span class="p">(</span><span class="n">phi1</span><span class="p">,</span> <span class="n">sigma1</span><span class="p">,</span> <span class="n">sigma2</span><span class="p">)</span>
<span class="n">phi2_filt</span> <span class="o">=</span> <span class="n">difference_of_gaussians</span><span class="p">(</span><span class="n">phi2</span><span class="p">,</span> <span class="n">sigma1</span><span class="p">,</span> <span class="n">sigma2</span><span class="p">)</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi1_filt</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi2_filt</span><span class="p">)</span></code></pre></figure>
<p class="centerimages noshadow"><img src="/assets/202106holopart2/09aphi12blurred.png" alt="Gaussian blur for image matching" /></p>
<p>As noted when extracting the first sideband, there are significant artifacts along the edges of the image which can also cause issues â€“ image edges are generally troublesome when dealing with frequency decompositions. For this reason we will construct a â€świndowâ€ť and multiply with our images, which ensures all values go smoothly towards <script type="math/tex">0</script> at the edges.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Apply Hann window to avoid border issues</span>
<span class="n">window_type</span> <span class="o">=</span> <span class="s">'hann'</span>
<span class="n">window_image</span> <span class="o">=</span> <span class="n">window</span><span class="p">(</span><span class="n">window_type</span><span class="p">,</span> <span class="n">phi1_filt</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="n">phi1_windowed</span> <span class="o">=</span> <span class="n">phi1_filt</span><span class="o">*</span><span class="n">window_image</span>
<span class="n">phi2_windowed</span> <span class="o">=</span> <span class="n">phi2_filt</span><span class="o">*</span><span class="n">window_image</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">window_image</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi1_windowed</span><span class="p">)</span></code></pre></figure>
<p class="centerimages noshadow"><img src="/assets/202106holopart2/09bphiwindowing.png" alt="Hann window to avoid edge artifacts" /></p>
<p>With the bandpass filter and window applied to both images, we are now ready to calculate the shift in x and y</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Calculate the shift using phase cross correlation on filtered images</span>
<span class="n">shifts</span><span class="p">,</span> <span class="n">error</span><span class="p">,</span> <span class="n">phasediff</span> <span class="o">=</span> <span class="n">phase_cross_correlation</span><span class="p">(</span><span class="n">phi1_filt</span><span class="p">,</span> <span class="n">phi2_filt</span><span class="p">,</span> <span class="n">upsample_factor</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>
<span class="c"># Note that matrices are indexed as rows ("y") at [0] and columns ("x") at [1]</span>
<span class="n">dx</span> <span class="o">=</span> <span class="n">shifts</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">dy</span> <span class="o">=</span> <span class="n">shifts</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">print</span><span class="p">(</span><span class="s">'Translation: (dx, dy) = (</span><span class="si">%.2</span><span class="s">f, </span><span class="si">%.2</span><span class="s">f)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">))</span></code></pre></figure>
<figure class="highlight"><pre><code class="languagebash" datalang="bash"><span class="gp">>> </span>Translation: <span class="o">(</span>dx, dy<span class="o">)</span> <span class="o">=</span> <span class="o">(</span>15.04, 13.14<span class="o">)</span></code></pre></figure>
<p>The calculated translation is now applied as a geometrical transform to correct one of the unfiltered phase images. After this we can subtract one phase image from the other to isolate the magnetic phase shift (and remember to divide by 2).</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Translate the second unfiltered phase image for matching on top of the first</span>
<span class="n">tform</span> <span class="o">=</span> <span class="n">transform</span><span class="o">.</span><span class="n">EuclideanTransform</span><span class="p">(</span><span class="n">translation</span><span class="o">=</span><span class="p">(</span><span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">))</span>
<span class="n">phi2_tf</span> <span class="o">=</span> <span class="n">transform</span><span class="o">.</span><span class="n">warp</span><span class="p">(</span><span class="n">phi2</span><span class="p">,</span> <span class="n">tform</span><span class="o">.</span><span class="n">inverse</span><span class="p">)</span>
<span class="c"># Subtract unfiltered phase images, remember division by 2!</span>
<span class="n">phi_m_raw</span> <span class="o">=</span> <span class="p">(</span><span class="n">phi1</span><span class="o"></span><span class="n">phi2_tf</span><span class="p">)</span><span class="o">/</span><span class="mi">2</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Magnetic phase shift'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi_m_raw</span><span class="p">)</span></code></pre></figure>
<p><img src="/assets/202106holopart2/10phimraw.png" alt="Magnetic phase image without electric contribution!" class="centerimage" /></p>
<p>The magnetic phase shift has revealed itself! Note how there is a lighter region on one side of the particle and dark on the other. This is the magnetic contribution to the phase shift.</p>
<h3 id="magneticfieldlinesfromphaseimage">Magnetic field lines from phase image</h3>
<p>We can enhance the magnetic phase image by cutting out the relevant part where the images overlap and filtering some noise away.</p>
<p>First we make a function to cut off the parts of the image that without overlap after subtraction</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="k">def</span> <span class="nf">crop_from_shifts</span><span class="p">(</span><span class="n">image</span><span class="p">,</span> <span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
<span class="n">dx</span><span class="p">,</span> <span class="n">dy</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">dx</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">dy</span><span class="p">)</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">image</span><span class="p">[:,</span> <span class="n">dx</span><span class="p">:</span><span class="o"></span><span class="mi">1</span><span class="p">]</span> <span class="k">if</span> <span class="n">dx</span> <span class="o">>=</span> <span class="mi">0</span> <span class="k">else</span> <span class="n">image</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">:</span><span class="n">dx</span><span class="p">]</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">image</span><span class="p">[</span><span class="n">dy</span><span class="p">:</span><span class="o"></span><span class="mi">1</span><span class="p">,</span> <span class="p">:]</span> <span class="k">if</span> <span class="n">dy</span> <span class="o">>=</span> <span class="mi">0</span> <span class="k">else</span> <span class="n">image</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">dy</span><span class="p">,</span> <span class="p">:]</span>
<span class="n">border</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">border</span><span class="p">)</span>
<span class="k">if</span> <span class="n">border</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">image</span><span class="p">[</span><span class="n">border</span><span class="p">:</span><span class="o"></span><span class="n">border</span><span class="p">,</span> <span class="n">border</span><span class="p">:</span><span class="o"></span><span class="n">border</span><span class="p">]</span>
<span class="k">return</span> <span class="n">image</span></code></pre></figure>
<p>We can also filter the image a bit with a gaussian blur to reduce high frequency noise, since the magnetic field (and phase shift) only varies slowly across the image. We will also apply a scalebar to show the size of the features.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">matplotlib_scalebar.scalebar</span> <span class="kn">import</span> <span class="n">ScaleBar</span>
<span class="c"># Scale original pixel size to match resolution of phase image and add to scalebar</span>
<span class="n">pxsize_phase</span> <span class="o">=</span> <span class="n">im1</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">/</span><span class="n">sideband_size</span><span class="o">*</span><span class="n">pxsize</span>
<span class="n">scalebar</span> <span class="o">=</span> <span class="n">ScaleBar</span><span class="p">(</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">pixel_unit</span><span class="p">,</span> <span class="n">length_fraction</span><span class="o">=</span><span class="mf">0.25</span><span class="p">)</span>
<span class="c"># Blur before cropping to minimize edge effects</span>
<span class="n">phi_m_filt</span> <span class="o">=</span> <span class="n">gaussian</span><span class="p">(</span><span class="n">phi_m_raw</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="c"># Crop nonoverlapping part and a bit of border</span>
<span class="n">crop_border</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">phi_m</span> <span class="o">=</span> <span class="n">crop_from_shifts</span><span class="p">(</span><span class="n">phi_m_filt</span><span class="p">,</span> <span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="n">crop_border</span><span class="p">)</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">()</span>
<span class="n">ax</span><span class="o">.</span><span class="n">axis</span><span class="p">(</span><span class="s">'off'</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi_m</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">add_artist</span><span class="p">(</span><span class="n">scalebar</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Magnetic phase shift'</span><span class="p">)</span></code></pre></figure>
<p>The phase image now looks like this</p>
<p><img src="/assets/202106holopart2/10phimfiltered.png" alt="Visualization of magnetic field lines and phase shift on particle" class="centerimage" /></p>
<p>Contour lines representing the magnetic field lines can also be overlayed directly on top of the bright field image, letting us see the field lines emanating from the particle itself. Using a divergent colormap which shows the central values as white, while displaying values above and below that as blue and red respectively, we can enhance the readability of the phase image.</p>
<p>We just need to make sure we crop the bright field image in the same way so the centering is right.</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="c"># Apply a bit stronger blur to enhance contour lines</span>
<span class="n">phi_m_extrafilt</span> <span class="o">=</span> <span class="n">crop_from_shifts</span><span class="p">(</span><span class="n">gaussian</span><span class="p">(</span><span class="n">phi_m_raw</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="n">crop_border</span><span class="p">)</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
<span class="c"># Plot phase with divergent colormap, overlay contour lines</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi_m</span><span class="p">,</span> <span class="n">cmap</span><span class="o">=</span><span class="s">'RdBu'</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">contour</span><span class="p">(</span><span class="n">phi_m_extrafilt</span><span class="p">,</span> <span class="mi">15</span><span class="p">,</span> <span class="n">colors</span><span class="o">=</span><span class="s">'k'</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.5</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">axis</span><span class="p">(</span><span class="s">'off'</span><span class="p">)</span>
<span class="n">scalebar</span> <span class="o">=</span> <span class="n">ScaleBar</span><span class="p">(</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">pixel_unit</span><span class="p">,</span> <span class="n">length_fraction</span><span class="o">=</span><span class="mf">0.25</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">add_artist</span><span class="p">(</span><span class="n">scalebar</span><span class="p">)</span>
<span class="c"># Crop BF image with same parameters as the phase image was cropped to get same center</span>
<span class="n">bf_image</span> <span class="o">=</span> <span class="n">crop_from_shifts</span><span class="p">(</span><span class="n">sideband1_abs</span><span class="p">,</span> <span class="n">dx</span><span class="p">,</span> <span class="n">dy</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="n">crop_border</span><span class="p">)</span>
<span class="c"># Overlay contours onto bright field image</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">gaussian</span><span class="p">(</span><span class="n">bf_image</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">contour</span><span class="p">(</span><span class="n">phi_m_extrafilt</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="n">colors</span><span class="o">=</span><span class="s">'white'</span><span class="p">,</span> <span class="n">linewidths</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.8</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">axis</span><span class="p">(</span><span class="s">'off'</span><span class="p">)</span>
<span class="n">scalebar</span> <span class="o">=</span> <span class="n">ScaleBar</span><span class="p">(</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">pixel_unit</span><span class="p">,</span> <span class="n">length_fraction</span><span class="o">=</span><span class="mf">0.25</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">add_artist</span><span class="p">(</span><span class="n">scalebar</span><span class="p">)</span></code></pre></figure>
<p>By plotting the contour lines we are in essence displaying the magnetic field lines as they emanate from the particle:</p>
<p class="centerimages noshadow"><img src="/assets/202106holopart2/11phimcontouroverlay.png" alt="Alternative visualizations of magnetic field lines and phase shift on particle" /></p>
<p>Now that is looking like proper magnetic field lines from a magnetic nanoparticle!</p>
<!
## Magnetic field from phase shift
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">skimage.measure</span> <span class="kn">import</span> <span class="n">profile_line</span>
<span class="c"># Scale original pixel size to match resolution of phase image</span>
<span class="n">pxsize_phase</span> <span class="o">=</span> <span class="n">im1</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">/</span><span class="n">sideband_size</span><span class="o">*</span><span class="n">pxsize</span>
<span class="c"># Extract profile plot across particle</span>
<span class="n">x1</span><span class="p">,</span> <span class="n">y1</span> <span class="o">=</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">110</span>
<span class="n">x2</span><span class="p">,</span> <span class="n">y2</span> <span class="o">=</span> <span class="mi">160</span><span class="p">,</span> <span class="mi">60</span>
<span class="n">dist</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">((</span><span class="n">x2</span><span class="o"></span><span class="n">x1</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="p">(</span><span class="n">y2</span><span class="o"></span><span class="n">y1</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">pxsize_phase</span>
<span class="n">profile</span> <span class="o">=</span> <span class="n">profile_line</span><span class="p">(</span><span class="n">phi_m</span><span class="p">,</span> <span class="p">(</span><span class="n">y1</span><span class="p">,</span> <span class="n">x1</span><span class="p">),</span> <span class="p">(</span><span class="n">y2</span><span class="p">,</span> <span class="n">x2</span><span class="p">),</span> <span class="n">linewidth</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'nearest'</span><span class="p">)</span>
<span class="n">profile_dist</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">dist</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">profile</span><span class="p">))</span>
<span class="c"># Make axes of plot match the physical size</span>
<span class="n">width</span> <span class="o">=</span> <span class="n">phi_m</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">*</span><span class="n">pxsize_phase</span>
<span class="n">height</span> <span class="o">=</span> <span class="n">phi_m</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">*</span><span class="n">pxsize_phase</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">phi_m</span><span class="p">,</span> <span class="n">extent</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">width</span><span class="p">,</span> <span class="n">height</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">plot</span><span class="p">((</span><span class="n">x1</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">x2</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">),</span> <span class="p">(</span><span class="n">y1</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">y2</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">),</span> <span class="n">color</span><span class="o">=</span><span class="s">'r'</span><span class="p">,</span> <span class="n">linewidth</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">scatter</span><span class="p">(</span><span class="n">x1</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">y1</span><span class="o">*</span><span class="n">pxsize_phase</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s">'r'</span><span class="p">,</span> <span class="n">marker</span><span class="o">=</span><span class="s">'o'</span><span class="p">,</span> <span class="n">s</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">scatter</span><span class="p">(</span><span class="n">profile_dist</span><span class="p">,</span> <span class="n">profile</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s">'r'</span><span class="p">)</span>
<span class="n">ax</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">set_box_aspect</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></code></pre></figure>
![Profile plot of magnetic phase shift across particle](/assets/202106holopart2/11phimprofile.png){:.centerimage}
>
<p>To enhance the image a bit, we will overlay the contour lines on top of the original hologram image so we get much better contrast. We do this by rescaling the magnetic phase image used for contour lines and again cropping parts of the orginal hologram to get the same center</p>
<figure class="highlight"><pre><code class="languagepython" datalang="python"><span class="kn">from</span> <span class="nn">skimage.transform</span> <span class="kn">import</span> <span class="n">rescale</span>
<span class="c"># Increase resolution of phase image</span>
<span class="n">scaling</span> <span class="o">=</span> <span class="n">im1</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">/</span><span class="n">sideband_size</span>
<span class="n">phi_m_upscaled</span> <span class="o">=</span> <span class="n">rescale</span><span class="p">(</span><span class="n">gaussian</span><span class="p">(</span><span class="n">phi_m</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span> <span class="n">scaling</span><span class="p">)</span>
<span class="c"># Crop original image the same way as the sideband images were, correcting for scale</span>
<span class="n">im1_crop</span> <span class="o">=</span> <span class="n">crop_from_shifts</span><span class="p">(</span><span class="n">im1</span><span class="p">,</span> <span class="n">dx</span><span class="o">*</span><span class="n">scaling</span><span class="p">,</span> <span class="n">dy</span><span class="o">*</span><span class="n">scaling</span><span class="p">,</span> <span class="n">border</span><span class="o">=</span><span class="n">crop_border</span><span class="o">*</span><span class="n">scaling</span><span class="p">)</span>
<span class="n">im1_filt</span> <span class="o">=</span> <span class="n">gaussian</span><span class="p">(</span><span class="n">im1_crop</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">im1_filt</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">contour</span><span class="p">(</span><span class="n">phi_m_upscaled</span><span class="p">,</span> <span class="mi">15</span><span class="p">,</span> <span class="n">colors</span><span class="o">=</span><span class="s">'white'</span><span class="p">,</span> <span class="n">linewidths</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.8</span><span class="p">)</span></code></pre></figure>
<p><img src="/assets/202106holopart2/12hirescontouroverlay.png" alt="Magnetic contour lines overlayed on bright field image" class="centerimage" /></p>
<h2 id="finalthoughts">Final thoughts</h2>
<p>Congratulations for making it all the way through!</p>
<p>We have worked all the way from loading raw holograms taken in a Transmisison Electron Microscope into Python to extracting and visualizing a magnetic phase image by using various image processing techniques and a bit of Fourier transform magic.</p>
<p>There is still much that can be done with this data, such as calculations on the exact magnetization of the particle and how it compares to expected values from other sources, which I have skipped as this post is already running long. If you want to try the code yourself, make sure to download <a href="/assets/202106holopart2/holographypython.zip">the associed Jupyter Notebook</a> with data, where you can make your own calculations (<a href="/assets/202106holopart2/holographypython.html">preview of notebook</a>).</p>
<p>While I have omitted many details on this complex topic, I hope it can server as an interesting appetizer on some of the things that can be done in Transmission Electron Microscopes, and how information that is usually not obtainable can be brought forward with some clever techniques.</p>
<!
# Notes and Todo:
 ~~Show lineplot across particle~~
 ~~Fix axes to match physical size~~
 ~~**Pixel size is off, figure out right one (look in thesis or old matlab code)**. Is it just factor 10?~~
 ~~Add note on particle size~~
 ~~Relate phase to magnetic field strength (add to first article as well)~~
 ~~Check the thing about choosing negative/positive phase shift (which part of fourier image is it?)~~
 ~~Add classic magnetic field lines image to first post~~
 ~~overlay contour lines on full resolution image~~
 ~~Calculate and show gradient, relate to B field (strength and direction as vector map)~~ Naaahhh...
 ~~Draw flow chart of steps?~~ Naaah
 ~~**Make images with scale bars some places**~~
 Add note on profile plot about 2 pi phase being 4.14Ă—10^15 Tm2. of enclosed flux and how much we would expect from magnetite?
 ~~Finish section on phase being equivalent to field strength in first article~~
 Clean up notebook for publishing
 Export notebook to html, zip up files with data.
>
<hr />
<div class="footnotes">
<ol>
<li id="fn:wikimediafieldlines">
<p>Adapted from diagrams of the magnetic field around a bar magnet from <a href="https://commons.wikimedia.org/wiki/File:VFPt_cylindermagnet_fieldrepresentations.svg">Wikimedia Commons</a> <a href="#fnref:wikimediafieldlines" class="reversefootnote">↩</a></p>
</li>
<li id="fn:bachelorthesis">
<p>Supervisors: Takeshi Kasama, Marco Beleggia, Cathrine Frandsen. <a href="#fnref:bachelorthesis" class="reversefootnote">↩</a></p>
</li>
<li id="fn:holographybook">
<p>Takeshi Kasama, Rafal E. DuninBorkowski and Marco Beleggia (2011). <em>Electron Holography of Magnetic Materials</em>, Holography  Different Fields of Application, F. A. Ramirez (Ed.), IntechOpen, <a href="https://doi.org/10.5772/22366" target="_blank">DOI: 10.5772/22366</a> <a href="#fnref:holographybook" class="reversefootnote">↩</a></p>
</li>
<li id="fn:requirements">
<p>All packages used in the notebook are: <a href="https://pypi.org/project/numpy/"><code class="highlighterrouge">numpy</code></a>, <a href="https://pypi.org/project/matplotlib/"><code class="highlighterrouge">matplotlib</code></a>, <a href="https://pypi.org/project/ncempy/"><code class="highlighterrouge">ncenmpy</code></a>, <a href="https://pypi.org/project/scikitimage/"><code class="highlighterrouge">scikitimage</code></a> (imports as <code class="highlighterrouge">skimage</code>) and <a href="https://pypi.org/project/matplotlibscalebar/"><code class="highlighterrouge">matplotlib_scalebar</code></a>. <a href="#fnref:requirements" class="reversefootnote">↩</a></p>
</li>
<li id="fn:fftabslog">
<p>Remember that each pixel in the fourier transformed image <code class="highlighterrouge">im1_fft</code> consists of complex values with a real value <script type="math/tex">\mathrm{Re}</script> and imaginary value <script type="math/tex">\mathrm{Im}</script>. What we display as â€śthe FFTâ€ť is the amplitude as <script type="math/tex">r = \sqrt{\mathrm{Re}^2+\mathrm{Im}^2}</script> using <code class="highlighterrouge">np.abs</code>. Further, to avoid the central peak dominating the image I also calculate the logarithm of each pixel value with <code class="highlighterrouge">np.log</code> to enhance visibility of the smaller values in the sidebands relative to the peak, as well as adjusting <code class="highlighterrouge">vmin</code> and <code class="highlighterrouge">vmax</code>. <a href="#fnref:fftabslog" class="reversefootnote">↩</a></p>
</li>
<li id="fn:matchrotation">
<p>Even if the image is rotated and scaled this method can still be used as demonstrated in <a href="https://scikitimage.org/docs/dev/auto_examples/registration/plot_register_rotation.html#sphxglrautoexamplesregistrationplotregisterrotationpy">this example</a> â€“ essentially the matching is performed on the FFT of the image to obtain rotation, followed by the same method as here after correcting for rotation. <a href="#fnref:matchrotation" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Tue, 06 Jul 2021 10:00:00 +0000
https://longcreek.me/blog/2021/holographycode
https://longcreek.me/blog/2021/holographycode
holography
imaging
tem
microscopy
physics
magnetism
Python

Visualizing the magnetic field of nanoparticles  Fundamentals
<script type="math/tex; mode=display">\newcommand{\vr}{\mathrm{\mathbf{r}}}
\newcommand{\unit}[1]{\,\mathrm{#1}}
\newcommand{\vec}[1]{\mathrm{\mathbf{#1}}}
\newcommand{\fourier}[1]{\mathcal{F}\left[#1 \right]}
\newcommand{\vv}{\vec{v}}
\newcommand{\vB}{\vec{B}}
\newcommand{\vq}{\vec{q}}
\newcommand{\Iholo}{I_\mathrm{holo}}
\newcommand{\Iside}{I_\mathrm{s.b.}}</script>
<!
This post deals with the math relating to electron holography in preparation for a more visual approach coming later. Stick around the blog to see it applied to actual images! The math that follows is essentially a boiled down version of [this book chapter](https://www.intechopen.com/books/holographydifferentfieldsofapplication/electronholographyofmagneticmaterials){:target="_blank"}[^1] on electron holography of magnetic materials[^2].
The tiny wavelengths allows us to image cool stuff like this
![Rows of atoms in a particle](/assets/holography/HRTEM_brightfield.jpg){:.shadowimage .halfimage .centerimage}
This is a nanoparticle of platinum, and the dots you see are individual rows of atoms! That kind of "direct" imaging is known as Bright Field (BF) imaging and is one of the simplest operating modes in TEM and very analogous to an optical microscope.
While atomic resolution images are very interesting and useful, it only scratches the surface of the techniques possible in a TEM. One of my favourite techniques is one that allows you to *directly visualise the magnetic field of a magnetic nanoparticle*.
>
<!
Electron microscopes are really versatile things  Not only can they be used to see nanoparticles down to the atomic level, but because we are using electrons instead of visible light we get a lot of analytic capabilities "for free" as well. One of my favourites is the a method were we can directly "see" the magnetic field around nanoparticles.
>
<p><em>This is the first of two posts  the second on reconstructing holograms using Python can be found <a href="/blog/2021/holographycode">here</a>.</em></p>
<p>When learning about magnets and magnetic fields, you have probably seen visualizations similar to these<sup id="fnref:wikimediafieldlines"><a href="#fn:wikimediafieldlines" class="footnote">1</a></sup> which represent how the magnetic field extends outside a rectangular magnet</p>
<!
https://commons.wikimedia.org/wiki/File:VFPt_cylindermagnet_fieldrepresentations.svg
>
<p class="centerimages noshadow"><img src="/assets/202106holopart2/magneticfieldlines1.svg" alt="Magnetic field lines" /></p>
<p>The left represents how iron filings agglomerate when sprinkled around a magnet as they roughly align to the magnetic field, a common method to directly â€śseeâ€ť the magnetic field around larger magnets. In the middle the magnetic field is represented with a series of magnetic field lines which are drawn along lines of constant field strength, while the rightmost image shows how an array of freely rotating magnets (e.g. compasses) would align themselves in the same magnetic field.</p>
<p>This is usually presented for magnets large enough hold in your hand, but it is actually possible to directly measure the magnetic field all the way down at the nano scale in a <a href="https://en.wikipedia.org/wiki/Transmission_electron_microscopy">Transmission Electron Microscope</a> or <em>TEM</em>, just like this:</p>
<p><img src="/assets/202106holopart2/12hirescontouroverlay.png" alt="magnetic field lines around magnetite nanoparticle" class="centerimage" /></p>
<p>The above image shows a magnetite nanoparticle around 90 nm wide where the contour lines represent the magnetic field around the particle as measured directly in the microscope. Iron filings and tiny compasses work great for magnets large enough to handle, but what do we do when working on the nanoscale?</p>
<!
A TEM uses electrons as a probe and as luck would have it, electrons moving in a magnetic field will get affected by it, specifically by shifting the phase $$\phi$$ of the electrons. In usual imaging in the TEM the phase is not preserved, but by using a technique known as [Electron Holography](https://en.wikipedia.org/wiki/Electron_holography) we can preserve the phase and use it to extract information about the magnetic field (among other things).
>
<p>Below i will explain a method known as OffAxis Electron Holography and how it can be used to measure and visualize the magnetic field around magnetic nanoparticles in the TEM to make images like the above. Much of what follows is based on <a href="https://www.intechopen.com/books/holographydifferentfieldsofapplication/electronholographyofmagneticmaterials" target="_blank">this book chapter</a><sup id="fnref:1"><a href="#fn:1" class="footnote">2</a></sup> on electron holography of magnetic materials which I recommend reading if you are interested in the subject.</p>
<p>The data shown was acquired as part of my bachelors thesis<sup id="fnref:2"><a href="#fn:2" class="footnote">3</a></sup> at <a href="https://www.nanolab.dtu.dk/">DTU Nanolab</a> (previously Center for Electron Nanoscopy) several years back. I recently went back to revisit the subject and redo (and improve) in Python out of curiousity and decided do this writeup to share with anyone why might be interested.</p>
<p>This post is part one of two and covers some TEM fundamentals and relevant mathematics. For numerical calculations, take a look at part two where we <a href="/blog/2021/holographycode">reconstruct holograms using Python</a>.</p>
<h2 id="verybriefintrotothetem"><em>Very</em> brief intro to the TEM</h2>
<p>A Transmission Electron Microscope in conventional operation is very similar to a typical optical microscope, but instead of using light waves we use a beam of electrons. Since electron waves generated at high voltages have a <em>much</em> shorter wavelength than visible light, we can use it to see much smaller things. As an example, blue light has a wavelength around <script type="math/tex">400 \unit{nm}</script>, while an electron will have a wavelength around <script type="math/tex">0.0025 \unit{nm}</script> at <script type="math/tex">200 \unit{kV}</script> acceleration voltage â€“ for reference a hydrogen atom has a diameter of around <script type="math/tex">0.01 \unit{nm}</script><sup id="fnref:3"><a href="#fn:3" class="footnote">4</a></sup>. <!As a (very handwavy) rule of thumb, features need to be larger than your probing wavelength in order to affect the waves significantly.></p>
<p>The diagram below shows a simplified schematic of a TEM in bright field (BF) mode, where an electron gun at the top provides a beam of coherent electrons which are condensed to a uniform plane wavefront before passing through a sample. After this the beam is focused, various apertures can be inserted, and the image is formed in the image plane on the bottom, where we place 2D detector to save the image.</p>
<p class="centerimages noshadow"><img src="/assets/holography/temdiagram.png" alt="Simplified TEM diagram" /></p>
<p>Unlike in the optical microscope, the lenses are not made from glass or any physical material. Instead, we use magnetic lenses to focus the electron beam â€“ Essentially a symmetric bunch of coils with adjustable current going through them to create a magnetic field. The reason we can manipulate the electrons with magnetic fields is due to the <a href="https://en.wikipedia.org/wiki/Lorentz_force" target="_blank">Lorentz force</a>.</p>
<script type="math/tex; mode=display">\vec{F} = q \, \vec{v}\times\vec{B}</script>
<! This basically means that any charged particle (such as electrons) moving in a magnetic field will feel a force orthogonal to its direction of movement. >
<p>With a strong field of a certain shape we can use this to focus our electron beam. This is also the reason we are able to visualise the magnetic field of a sample when using electrons, as the field around a magnetic particle deflects the electrons slightly and change their phase.</p>
<p>I am barely scratching the surface of the subject for the sake of brevity  have a look at the very detailed and pedagogical <a href="https://www.doitpoms.ac.uk/tlplib/tem/index.php">DoITPoMS from the University of Cambridge</a> on the TEM if you want more details.</p>
<h3 id="electronbeamasawavefunction">Electron beam as a wavefunction</h3>
<p>Mathematically we can describe the electron beam at any point <script type="math/tex">\vr</script> in the image plane as a plane wave with amplitude <script type="math/tex">A(\vr)</script> and phase <script type="math/tex">\phi(\vr)</script> in a complex exponential phase term</p>
<script type="math/tex; mode=display">\psi(\vr) = A(\vr)e^{i \phi(\vr)}</script>
<p>In typical operating conditions, when recording the image on a detector the phase information is lost as the intensity distribution is the absolute square of the wavefunction, where the conjugated exponential terms cancel out</p>
<script type="math/tex; mode=display">I(\vr) = \left \psi(\vr) \right^2 = \left[A(\vr)e^{i \phi(\vr)}\right]\left[A(\vr)e^{i \phi(\vr)}\right]= A(\vr)^2</script>
<p>In other words weâ€™re only recording the square of the amplitude of the wave, which simply tells us how many electrons reached each pixel on the sensor, regardless of their phase. This is plenty for most uses, but in our case we actually need to know the phase of the electrons.</p>
<p>When an electron wave passes through a sample the phase <script type="math/tex">\phi</script> of the wave will experience a phase shift. For thin magnetic samples we can express the phase shift at a given point <script type="math/tex">\vr</script> after passing through a sample as the sum of an electric and magnetic contribution</p>
<script type="math/tex; mode=display">\phi(\vr) = \phi_e(\vr) + \phi_m(\vr)</script>
<p><script type="math/tex">\phi_e</script> arises from the electron beam intereacting with the electrons in the sample, while <script type="math/tex">\phi_m</script> depends on the inplane component of the magnetic field from the sample due to the Lorentz force. We note that since the force is proportional to the <em>cross product</em> <script type="math/tex">\vv\times\vB</script>, the moving charges are only affected by the components of the Bfield orthogonal to the movement direction.
Since the electrons are moving from the top and down into the sample they will thus only be affected by the inplane component of the Bfield. If we imagine a magnetic field pointing directly upwards from the sample, parallel to the movement of electrons there would be no phase shift, while a field perfectly orthogonal, along the sample plane, would show up fully in the phase shift.</p>
<p>The phase shift <script type="math/tex">\phi_m</script> is thus our key to imaging the magnetic field of our particles, and we need to figure out how to record it. As luck would have it there are workarounds for this problem.</p>
<h3 id="holographyintheelectronmicroscope">Holography in the electron microscope</h3>
<p>In case you are wondering, <em>holo graphy</em> comes from Greek and can be translated as <em>whole drawing</em>  in microscopy and this often refers to the fact that holographic methods preserve the phase. The â€śwholeâ€ť in cool shiny 3D holograms on old CDâ€™s, toys, credit cards etc. refers to the fact that they capture the â€śwholeâ€ť three dimensional scene.</p>
<p>The challenge for us is to find a way to make sure the phase information of our electron wave is not lost as we record it on the detector. Below is a very simplified diagram of a TEM in a setup used for a method called <em>OffAxis Electron Holography</em>.</p>
<p class="centerimages noshadow"><img src="/assets/holography/holography_schematic.png" alt="Holography schematic" /></p>
<p>The green part of the beam which passed trough the sample weâ€™ll call <script type="math/tex">\psi_s</script>, and the other half which goes unhindered through the vacuum is our reference wave, <script type="math/tex">\psi_r</script>.</p>
<p>Like before we generate a coherent plane wave of electrons, but now we only send part of it through the sample (green part). Further down, after the objective lens<sup id="fnref:lorentzlens"><a href="#fn:lorentzlens" class="footnote">5</a></sup>, is the key part of the setup: the biprism. It is simply a wire going all the way through the middle of the electron beam with a plate on each side along the length of the wire outside the beam. When applying a positive charge to the beam and negative charge to the plates the electrons are repelled by the plates and attracted to the wire.</p>
<p>So we have essentially split the whole wavefront in two parts which we can bend towards each other causing them to overlap, and this is where the magic happens. When two coherent (inphase) plane waves meet each other at an angle a sinusoidal interference pattern will appear as illustrated in the bottom of the diagram and can be recorded with out image sensor. The pattern is highly sensitive to small differences in phase between the two waves and will thus have information relating to <script type="math/tex">\phi_e</script> and <script type="math/tex">\phi_m</script> embedded in it.</p>
<p>So how do we extract the phase?</p>
<h2 id="mathematicsofthephaseretrieval">Mathematics of the phase retrieval</h2>
<p>Letâ€™s formalise a bit on what was described above.</p>
<h3 id="contributionstothephaseshift">Contributions to the phase shift</h3>
<p>For a sample with no external charge distributions or external electric fields, the electrostatic contribution to the phase shift can be expressed in terms of a mean inner potential <script type="math/tex">V_0</script> which is assumed constant throughout the sample.</p>
<script type="math/tex; mode=display">\phi_e(x,y) = C_E V_0 t(x,y)</script>
<p>where <script type="math/tex">C_E</script> is a constant dependent on the microscope acceleration voltage<sup id="fnref:CE"><a href="#fn:CE" class="footnote">6</a></sup> and <script type="math/tex">t(x,y)</script> the thickness profile of the sample projected in the beam direction.</p>
<p>The magnetic phase shift can be expressed as</p>
<script type="math/tex; mode=display">\phi_m(x,y) = \frac{e}{\hbar} \int A_z(x,y,z) \,dz</script>
<p>where <script type="math/tex">A_z</script> is the magnetic vector potential, specifically the component parallel to the electron beam direction. The gradient of the magnetic phase then gives us a direct relationship to the Bfield form the phase shift</p>
<script type="math/tex; mode=display">\nabla \phi_m(x,y) = \frac{e}{\hbar}\begin{bmatrix}B_y^p(x,y)\\\; B_x^p(x,y) \end{bmatrix}</script>
<p>where <script type="math/tex">B^p</script> is the total projection of the Bfield components perpendicular to electron beam direction through the sample area</p>
<script type="math/tex; mode=display">B^p(x,y) = \int B(x,y,z)\,dz</script>
<!
Between two points $$(x_1, y_1)$$ and $$(x_2, y_2)$$ in the sample plane, the magnetic phase shift can be expressed as
$$\Delta\phi_m=\phi_m(x_2, y_2)\phi_m(x_1, y_1)
= \frac{e}{\hbar}\left(\int A_z(x_2, y_2, z) \,dz + \int A_z(x_1,y_1,z) \,dz \right)$$
This can be rewritten as
$$\Delta \phi_m=\frac{e}{\hbar} \oint \vec{A}\cdot d\vec{l} $$
By Stoke's theorem
$$ \Delta\phi_m = \frac{e}{\hbar} \iint \vec{B}\cdot\hat{\vec{n}} \, dS$$
>
<p>As such, the phase shift of the electron beam after passing through a sample can be described as</p>
<script type="math/tex; mode=display">\phi(x,y) = \phi_e(x,y) + \phi_m(z,y)</script>
<p>Our goal now is to extract just the magnetic phase shift <script type="math/tex">\phi_m</script> of the electron beam and we need to get rid of <script type="math/tex">\phi_e</script> in some way.</p>
<p>If we can exactly reverse the magnetic field of our particle, the magnetic phase shift will also be reversed from <script type="math/tex">\phi_m</script> to <script type="math/tex">\phi_m</script> while <script type="math/tex">\phi_e</script> is unchanged. In the TEM this can be done by applying a strong magnetic field in one direction, record an image, then apply a strong field in the opposite direction to reverse the magnetisation of the particle, and record another image.</p>
<p>We can then record two images where the magnetisation of the particle is exactly reversed between them and then subtract the phase shifts. Then <script type="math/tex">\phi_e</script> cancels out and leaves just the magnetic information</p>
<script type="math/tex; mode=display">\phi_{\mathrm{rev}} = \phi_1  \phi_2 = (\phi_e + \phi_m)  (\phi_e  \phi_m) = 2\phi_m</script>
<p>Now we just remember to divide by 2 and we have our magnetic phase shift!<sup id="fnref:5"><a href="#fn:5" class="footnote">7</a></sup></p>
<h3 id="preservingthephaseinformation">Preserving the phase information</h3>
<p>Remembering that the electron beam at a given point <script type="math/tex">\vr</script> in the image plane can be described as a wavefunction <script type="math/tex">\psi_i(\vr) = A_i(\vr) e^{i\phi(\vr)}</script>, we now introduce a plane reference wave that only passes unhindered through vacuum</p>
<script type="math/tex; mode=display">\psi_r(\vr)=A_r(\vr)e^{i2\pi \vq_c\cdot \vr}</script>
<p>with <script type="math/tex">\vq_c</script> being a twodimenstional reciprocal space vector determining the tilt of the reference wave. This is what we do with the biprism in the offaxis holography setup, when we split the beam into two parts and make them overlap. In the overlap the wavefunction is simply a superposition of the two wavefunctions leading to the following intensity distribution</p>
<script type="math/tex; mode=display">\Iholo(\vr) = \left\psi_i(\vr)+\psi_r(\vr)\right^2 = \left A_i(\vr) e^{i\phi(\vr)} + A_r(\vr)e^{i2\pi \vq\cdot \vr}\right^2</script>
<p>This can be evaluated as</p>
<script type="math/tex; mode=display">I_{holo}(\vr) = A_i^2(\vr) + A_r^2(\vr) + A_i(\vr)A_r(\vr)\left(e^{i(2\pi\vq_c\cdotp\vr\phi(\vr))}+e^{i(2\pi\vq_c\cdotp\vr\phi(\vr))}\right)</script>
<!
$$I_{\mathrm{holo}}(\vr) = A_i^2(\vr) + A_r^2(\vr) + A_i(\vr)A_r(\vr) \cos\left[2\pi\vq_c\cdot\vr+\phi(\vr)\right]$$
>
<p>The phase information is now preserved directly in the intensity distribution on our image sensor! We have the amplitude of the wave passing through the sample <script type="math/tex">A_i(\vr)^2</script> (as before), the amplitude of the reference wave <script type="math/tex">A_r(\vr)^2</script>, and finally an oscillating term<sup id="fnref:Iholo"><a href="#fn:Iholo" class="footnote">8</a></sup> which contains the phase of the sample beam <script type="math/tex">\phi(\vr)</script>.</p>
<p>Letâ€™s see how it looks in practice. Below is a holographic image of a <a href="https://en.wikipedia.org/wiki/Magnetite" target="_blank">magnetite</a> nanoparticle which is about 90 nm wide. It is actually shaped like an <a href="https://en.wikipedia.org/wiki/Octahedron" target="_blank">octahedron</a> but as it is lying down the projection through the particle appears as a hexagon. The large round shape in the top left half of the image is a hole in a thin carbon substrate<sup id="fnref:holeycarbon"><a href="#fn:holeycarbon" class="footnote">9</a></sup>, and our particle is just barely holding on to the edge of this hole.</p>
<p><img src="/assets/holography/particlelines.png" alt="Magnetite particle with interference pattern" class="shadowimage centerimage" /></p>
<p>The thin lines all across the image is a direct result of the interference of the two beams. In areas with even background it closely matches a cosine function as expected. Note that the much larger parallel lines (top left) are an artifact from diffraction fringes at the edges of the biprism wire and not directly related to the phase. Letâ€™s look at a closeup of the particle</p>
<p><img src="/assets/holography/particlelinescloseup.png" alt="Closeup of interference pattern" class="shadowimage centerimage" /></p>
<p>Notice how the lines are not straight everywhere. If we follow one from top to bottom, we see it bends at the top end of the particle, continues in a mostly straight line and then bends back close to its original position.
This bending is directly due to the phase shift of the electron beam due to the particle  at the edges the thickness varies causing the bend, while the center of the particle is fairly even, giving straight lines. The phase shift is primarily from the electric interaction <script type="math/tex">\phi_e</script> as it tends to be stronger than <script type="math/tex">\phi_m</script>.</p>
<h3 id="recoveringjustthephase">Recovering just the phase</h3>
<!Now, as we're only interested in the phase information we don't care much about the amplitude terms which is essentially just a regular bright field image. Further, the image is now muddled by our reference wave which gives rise to the intereference lines seen.>
<p>To separate the phase term, we turn to the trusty Fourier transform, which I wrote a bit about <a href="/blog/2020/fourierfiltering" target="_blank">here</a>. Now, as the Fourier transform is a linear operator, the transform of a sum of terms is equivalent to the sum of individually transformed terms:</p>
<script type="math/tex; mode=display">\fourier{\Iholo} = \fourier{A_i(\vr)^2} + \fourier{A_r(\vr)^2} + \fourier{A_i(\vr)A_r(\vr)\left(e^{i(2\pi\vq_c\cdotp\vr\phi(\vr))}+e^{i(2\pi\vq_c\cdotp\vr\phi(\vr))}\right)}</script>
<p>With a bit of mathematical gymnastics the last term in the expression can expressed as a convolution with a delta function <script type="math/tex">\delta(\vq\pm\vq_c)</script></p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{align*}
\hat{I}_\mathrm{holo} =& \hat{A}_i(\vq)^2 + \hat{A}_r(\vq)^2 \\
&+ \delta(\vq+\vq_c)*\fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}} \\
&+ \delta(\vq\vq_c)*\fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}}
\end{align*} %]]></script>
<!
$$\begin{align*}
\hat{I}_\mathrm{holo} =& \fourier{A_i(\vr)^2} + \fourier{A_r(\vr)^2} \\
&+ \delta(\vq+\vq_c)*\fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}} \\
&+ \delta(\vq\vq_c)*\fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}}
\end{align*}$$
>
<p>where <script type="math/tex">*</script> denotes the <a href="https://en.wikipedia.org/wiki/Convolution">convolution operator</a>. The proof of the above is <a href="http://www.mathmatique.com/articles/leftexercisereader" target="_blank">left as an exercise to the reader.</a></p>
<p>It is slightly complex, but hereâ€™s what you need to take away from the above: When convolving a delta function with a fourier transform, it essentially becomes â€ścenteredâ€ť at the point where the argument of the delta function is zero; <script type="math/tex">\delta(\mathbf{0})</script>. Thus if convolving with <script type="math/tex">\delta(\vq+\vq_c)</script> the center will be at <script type="math/tex">\vq=\vq_c</script>.</p>
<p>So now we have two amplitude terms with no phase information, <script type="math/tex">\hat{A}_i(\vq)^2</script> and <script type="math/tex">\hat{A}_r(\vq)^2</script> centered around <script type="math/tex">\vq=(0,0)</script> in the middle of the reciprocal space in the fourier transformed image. They are more or less the regular bright field image you would get without the special technique. But now we also have two phase terms, which are the complex conjugates of each other, centered at <script type="math/tex">\vq=\vq_c</script> and <script type="math/tex">\vq=\vq_c</script> respectively. These will show up at opposite sides, away from the center of the fourier transformed image.</p>
<p>Visually it looks like this. Below is shown the 2D Fast Fourier Transform of the hologram image</p>
<p><img src="/assets/holography/particlelinesfft.png" alt="FFT with sidebands" class="shadowimage centerimage" /></p>
<p>A large peak in the center with two distinct peaks away from center at opposite sides. These are the sidebands mentioned before, and the <script type="math/tex">\vq_c</script> vector is the vector pointing from the center and out to these sidebands. The visible line from the center all the way to the sideband comes from the diffraction fringes from the biprism as mentioned earlier.</p>
<p>Note that the Fourier transform of a real valued image results in complex values, i.e. each pixel has a real part <script type="math/tex">\mathrm{Re}</script> and imaginary part <script type="math/tex">\mathrm{Im}</script>. When showing the image here, we are seeing the absolute value, or amplitude, of this complex image.</p>
<p>We can now select a sideband from the full image, i.e. we pick out just the term</p>
<script type="math/tex; mode=display">\delta(\vq+\vq_c)*\fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}}</script>
<p><img src="/assets/holography/fftsideband.png" alt="Sideband in the FFT" class="shadowimage centerimage" /></p>
<p>The horizontal and vertical streaks are artifacts from the edges of the image and will not affect our analysis. The center of this new image is now at <script type="math/tex">\vq=\vq_c</script> and the convolution with the delta function becomes <script type="math/tex">1</script> leaving us with</p>
<script type="math/tex; mode=display">\hat{I}_\mathrm{s.b.}(\vq) = \fourier{A_i(\vr)A_r(\vr)e^{i\phi(\vr)}}</script>
<p>By performing an inverse Fourier transform the image is now simply</p>
<script type="math/tex; mode=display">\Iside(\vr) = A_i(\vr)A_r(\vr)e^{i\phi(\vr)}</script>
<p>We are now liberated of the other terms and we are one important step closer to isolating <script type="math/tex">\phi(\vr)</script> itself. Exciting!</p>
<h3 id="amplitudeandphaseofacomplexnumber">Amplitude and phase of a complex number</h3>
<p>Our sideband <script type="math/tex">\Iside(\vr)</script> is now a complex function on the exponential or polar form <script type="math/tex">z = re^{i\varphi}</script> where <script type="math/tex">r</script> is the amplitude and <script type="math/tex">\varphi</script> the phase. Mathematically we can directly read off the amplitude as <script type="math/tex">r=A_iA_r</script> and the phase of our image as <script type="math/tex">\varphi=\phi</script> but numerically our data is stored in another form.</p>
<p>The (Inverse) Fast Fourier Transform used to numerically perform fourier transforms gives us an image where complex numbers are in the cartesian form <script type="math/tex">z=a+ib</script>, and thus each pixel has a real value <script type="math/tex">\mathrm{Re}(z) = a</script> and imaginary value <script type="math/tex">\mathrm{Im}(z) = b</script>. Luckily the conversion comes from basic trigonometry. The amplitude is simply the pythagorean length of the complex number</p>
<script type="math/tex; mode=display">r = \sqrt{a^2+b^2}</script>
<p>and the phase is the angle</p>
<script type="math/tex; mode=display">\varphi = \arctan\left(\frac{b}{a}\right)</script>
<p>Thatâ€™s it, we now have everything we need to recover the phase! The reconstructed phase of our particle looks like this</p>
<script type="math/tex; mode=display">\phi(\vr) = \phi_e(\vr) + \phi_m(\vr)</script>
<p><img src="/assets/holography/particle1phase.png" alt="Phase from sideband" class="centerimage" /></p>
<p>The above image (besides artifacts) is a direct image of the phase of the electron beam where it reached the detector in the microscope, reconstructed from the shown hologram. We would expect the magnetic field somewhere outside the particle, so where is it? The electric contribution <script type="math/tex">\phi_e</script> is much larger than the magnetic <script type="math/tex">\phi_m</script> so it gets a bit lost in the signal. To see it properly we still need to subtract an image of the same particle with reversed magnetic field, meaning we still have some work to do.</p>
<h2 id="summary">Summary</h2>
<p>Weâ€™ve gone briefly through the workings of a transmission electron microscope and seen how the electron beam can be described as a wave. With a special setup in the microscope which introduces a reference wave, the interference pattern in the overlap of these waves enables us to record the phase information of the beam passing through the sample. Using some Fourier transform magic we walked through the mathematics of extracting the phase from this pattern and demonstrated it for a single particle.</p>
<p>The remaining work to isolate only the magnetic contribution to the phase shift is mainly computational. I continue this in the post on <a href="/blog/2021/holographycode">reconstructing holograms using Python</a> where I walk through the code used to go from raw holograms to finished magnetic phase images.</p>
<hr />
<!
The electric controbution is directly proportinal to the path
$$\phi_e = C_E \int_{\infty}^{\infty}V(z) dz$$
>
<!
## TODO
 Describe magnetite particle and support
 Ensure TEM diagrams are matching in size, colors etc
 Find HRTEM BF image of atomic resolution I'm allowed to use, make sure it is not STEM.
>
<div class="footnotes">
<ol>
<li id="fn:wikimediafieldlines">
<p>Diagrams of the magnetic field around a bar magnet from <a href="https://commons.wikimedia.org/wiki/File:VFPt_cylindermagnet_fieldrepresentations.svg">Wikimedia Commons</a> <a href="#fnref:wikimediafieldlines" class="reversefootnote">↩</a></p>
</li>
<li id="fn:1">
<p>Takeshi Kasama, Rafal E. DuninBorkowski and Marco Beleggia (2011). <em>Electron Holography of Magnetic Materials</em>, Holography  Different Fields of Application, F. A. Ramirez (Ed.), IntechOpen, <a href="https://doi.org/10.5772/22366" target="_blank">DOI: 10.5772/22366</a> <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>Supervisors: Takeshi Kasama, Marco Beleggia, Cathrine Frandsen. On a personal note: Takeshi Kasama passed away a few years back at a much too young age. He did an amazing job as supervisor on my bachelorâ€™s project. He had a knack for teaching and for making me â€ślearn how to learnâ€ť, and I am grateful to have been under his guidance. He has definitely shaped my approach to science and physics and how I explain things to others, so if you like what you read send him a friendly thought! <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<p>Strictly speaking an atom has a fuzzy undefined size, but the diameter from the Bohr model gives a useful sense of how far from the center of a hydrogen atom you are likely to find its accompanying electron. <a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
<li id="fn:lorentzlens">
<p>Since we are trying to image the weak magnetic field of our nanoparticles, the objective lens is now a Lorentz lens, as this does not generate a magnetic field at the sample. <a href="#fnref:lorentzlens" class="reversefootnote">↩</a></p>
</li>
<li id="fn:CE">
<p>With acceleration voltage as <script type="math/tex">U=\frac{E}{e}</script> we have <script type="math/tex">C_E = \left(\frac{2\pi}{\lambda}\right) \left(\frac{E+E_0}{E(E+2E_0)}\right)</script> where <script type="math/tex">\lambda</script> is the relativistic electron wavelength and <script type="math/tex">E_0</script> the rest mass energy of the electron. <a href="#fnref:CE" class="reversefootnote">↩</a></p>
</li>
<li id="fn:5">
<p>There are also other ways of removing the electric phaseshift <script type="math/tex">\phi_e</script>, such as directly simulating <script type="math/tex">\phi_e</script> from detailed sample information and subtracting that. It can also be useful on its own, e.g. for accurate measurements of particle thickness, but we will not consider this for now. <a href="#fnref:5" class="reversefootnote">↩</a></p>
</li>
<li id="fn:Iholo">
<p>Using some identities of complex exponentials we can also write the phase term as a cosine term <script type="math/tex">A_i(\vr)A_r(\vr) \cos\left[2\pi\vq_c\cdot\vr+\phi(\vr)\right]</script>. These sinusoidal interference fringes are what we directly measure in the raw hologram. <a href="#fnref:Iholo" class="reversefootnote">↩</a></p>
</li>
<li id="fn:holeycarbon">
<p>Take a look at <a href="https://www.agarscientific.com/tem/supportfilmsholeycarbon" target="_blank">Agar scientific</a> for some example images. <a href="#fnref:holeycarbon" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Thu, 20 May 2021 10:14:00 +0000
https://longcreek.me/blog/2021/holographyintro
https://longcreek.me/blog/2021/holographyintro
holography
imaging
tem
microscopy
physics

Some visual intuition for 2D Fourier transforms
<h2 id="improvingbyblurring">Improving by blurring</h2>
<p>Letâ€™s say we have a image of a cat and we want to blur it a bit</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/cat.png" alt="Cute!" />
<img src="/assets/20201210fourierfiltering/cat_filt.png" alt="Softer cute!" /></p>
<p>Thatâ€™s pretty cute, but you may not need blurred cat images so often. However there are also more useful things we can do using the exact same method. This here is Constable Sinclair Tait of the Invernessshire Constabulary</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable.jpg" alt="Constable Tait in his original form" /></p>
<p>Many things can be said about Mr. Tait, which you can read a lot more about on <a href="https://www.flickr.com/photos/91779914@N00/8378619946">Dave Connerâ€™s Flickr</a>, but he certainly hasnâ€™t been preserved all that well. The interesting thing is that by using the exact same method as for blurring a cat we can bring some life back to our constable:</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable_filt.png" alt="Fixed it!" /></p>
<p>Thatâ€™s pretty neat, and, some would argue, more useful than blurred cats. So how does it work?</p>
<h2 id="thefouriertransform">The Fourier transform</h2>
<p>Both the cat blurring and constable reconstruction is done by applying a lowpass filter to the images, or, in other words, by removing high frequencies.</p>
<p>The way we go about is by a <a href="https://en.wikipedia.org/wiki/Fourier_transform">Fourier transformation</a> of our image. I will not go into too many details of exactly how or why it works in this post  for now all we need to know is that <strong>any<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup> function or signal can be expressed as a sum of sine and cosine functions</strong> to arbitrary precision.</p>
<p>How does this help us blur cats? Consider this illustration by <a href="https://en.wikipedia.org/wiki/File:Fourier_transform_time_and_frequency_domains_(small).gif">Lucas Barbosa</a> of a 1D function being decomposed into itâ€™s frequency components</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/fourer_transform_animation.gif" alt="Animation of Fourier decomposition" /></p>
<p>Here a square wave function <script type="math/tex">f</script> is decomposed into a range of sine and cosine functions each multiplied by a constant <script type="math/tex">a_n</script> or <script type="math/tex">b_n</script> (their amplitude) and each with a different frequency (here proportional to <script type="math/tex">n</script>)  if each of these individual sine and cosines were to be added together the result is again the original function <script type="math/tex">f</script>. We are essentially seeing <script type="math/tex">f</script> being split up into its frequency components, and the function <script type="math/tex">\hat{f}</script> shows us how large each component is.</p>
<p>This is essentially what the Fourier transform does  it tells us which frequencies a function or signal consists of and how strong they each are. In other words, if our signal is in â€śreal spaceâ€ť we can transform it to â€śfrequency spaceâ€ť.</p>
<h3 id="notationandanimportantproperty">Notation and an important property</h3>
<p>Applying the Fourier transform can be denoted as <script type="math/tex">\mathcal{F}\left\{f\right\} = \hat{f}</script>, where <script type="math/tex">f</script> is transformed into <script type="math/tex">\hat{f}</script>. Importantly we can directly recover our original function with the inverse Fourier transform, <script type="math/tex">\mathcal{F}^{1}\{\hat{f}\} = f</script>. For disctrete (digital) signals such as images we typically use a numerical version called a <a href="https://en.wikipedia.org/wiki/Fast_Fourier_transform">Fast Fourier Transform</a>. For our use in this blog post we consider them equivalent, and I will use â€śthe FFTâ€ť in some places to refer to the result of a transform on an image.</p>
<p>A very useful fact about the Fourier transformation is its <a href="https://en.wikipedia.org/wiki/Fourier_transform#Basic_properties">linearity</a>. This means that the Fourier transform of a sum of functions can be expressed as a sum of individual Fourier transforms on each function: <script type="math/tex">\mathcal{F}\left\{f_1 + f_2 \right\} = \mathcal{F}\left\{f_1\right\} + \mathcal{F}\left\{f_2\right\}</script></p>
<p>We are going to leverage this to our advantage.</p>
<h2 id="frequenciesinanimage">Frequencies in an image</h2>
<p>Letâ€™s experiment a bit and create a little 2D image. Here <script type="math/tex">x</script> and <script type="math/tex">y</script> goes from <script type="math/tex">0</script> to <script type="math/tex">2\pi</script> (i.e. one full period of a trigonometric function) and we plot the function <script type="math/tex">f_1 = \cos (4x)</script></p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/cos4x.png" alt="cos(4x)" class="halfimage" /></p>
<p>The FFT of this simple cosine looks like this</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/cos4x_fft.png" alt="FFT of cos(4x)" class="halfimage" /></p>
<p>Two dots spaced equally apart from the image center  these are the frequency components of our image, and since our image is a â€śpureâ€ť cosine the frequency is very well defined. The reason there are two dots instead of just one, despite having just one frequency, is beyond the scope of this post. Just know that a Fourier transform of a real valued signal gives a symmetric and complex valued result, meaning each pixel in our image is a complex number, <script type="math/tex">z=a+ib</script>. The two dots represent exactly the same frequency and are each others complex conjugate<sup id="fnref:4"><a href="#fn:4" class="footnote">2</a></sup>. When I â€śshow the fourier transformâ€ť what I am actually showing is the absolute value of each point <script type="math/tex">\lvert z \rvert</script><sup id="fnref:3"><a href="#fn:3" class="footnote">3</a></sup>.</p>
<p>Letâ€™s try with a higher frequency and see how <script type="math/tex">f_2 = \cos(8x)</script> and its fourier transform looks</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/cos8x.png" alt="cos(8x)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cos8x_fft.png" alt="FFT of cos(8x)" class="halfimage" /></p>
<p>Again two dots centered around the middle, but now they are further apart. The last bit is important; <strong>higher frequencies are further away from the center in the FFT image</strong>.</p>
<p>We can also add the two images together, so weâ€™re now seing <script type="math/tex">f_3 = \cos(4x) + \cos(16x)</script></p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/cos_z1+z2.png" alt="cos(4x)+cos(16x))" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cos_z1+z2_fft.png" alt="FFT of cos(4x)+cos(16x)" class="halfimage" /></p>
<p>Then we have two sets of dots, exactly corresponding to the two frequencies we are visualising. And importantly, even though everything is muddled together in â€śreal spaceâ€ť, we still se everything distinct in â€śfrequency spaceâ€ť.</p>
<p>If we turn the picture by plotting <script type="math/tex">f_4 = \cos(4y)</script> we also turn the arrangement of dots</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_cos4y.png" alt="cos(4y)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/zz_cos4y_fft.png" alt="FFT of cos(4y)" class="halfimage" /></p>
<p>Again we can add the pictures together, i.e. our image now shows <script type="math/tex">f_5 = \cos(4x)+\cos(8x)+ \cos(4y)</script></p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_mix.png" alt="cos(4y)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/zz_mix_fft.png" alt="FFT of cos(4y)" class="halfimage" /></p>
<p>And again we see two distinct peaks for each frequency in our image.</p>
<h2 id="filteringwithamask">Filtering with a mask</h2>
<p>Letâ€™s freshen things up a bit and try with a whole bunch of frequencies</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_hifreq.png" alt="cos(4y)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/zz_hifreq_fft.png" alt="FFT of cos(4y)" class="halfimage" /></p>
<p>If you look closely you can see that our previous image with <script type="math/tex">f_5</script> is actually contained in the one above. Itâ€™s quite difficult to see in real space but in the FFT the dots from before a still clearly distinct from all the new ones.</p>
<p>Letâ€™s recover them!</p>
<p>Since all the unwanted frequencies are higher than the ones we want, we can do so with a highpass filter. A simple way of doing this is to just set all frequencies above a certain treshold to <script type="math/tex">0</script>. Remember that in the FFT the frequency is proportional to the distance from the center. This means we will need a circular mask, where white pixels are <script type="math/tex">1</script> and black are <script type="math/tex">0</script>.</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_mask.png" alt="cos(4y)" class="halfimage" /></p>
<p>Now we can apply the mask to the FFT image by multiplying them together (both real and imaginary parts). This way each pixel with a <script type="math/tex">0</script> in the mask becomres <script type="math/tex">0</script> in the FFT, and <script type="math/tex">1</script>â€™s in the mask simply keep their value. This results in the following</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_hifreq_fft.png" alt="FFT of cos(4y)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/zz_hifreq_fft_filt.png" alt="cos(4y)" class="halfimage" /></p>
<p>That should seem familiar. The lines crossing in the center are artifacts from discretization of the higher frequencies, but they donâ€™t matter to omuch now. We can now do an inverse Fourier transform and recover our image!</p>
<p class="centerimages noshadow"><img src="/assets/20201210fourierfiltering/zz_mix.png" alt="cos(4y)" class="halfimage" />
<img src="/assets/20201210fourierfiltering/zz_hifreq_filt.png" alt="cos(4y)" class="halfimage" /></p>
<p>And weâ€™re back to how we started!</p>
<h1 id="blurringthedarncat">Blurring the darn cat</h1>
<p>This brings us back to the cat. Letâ€™s see it again along with its fourier transform<sup id="fnref:5"><a href="#fn:5" class="footnote">4</a></sup>.</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/cat.png" alt="Cute!" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cat_fft.png" alt="Softer cute!" class="halfimage" /></p>
<p>As there are not many well defined frequencies in the picture we donâ€™t have the same well defined spots as before. Letâ€™s do the same as we did to that previous image and cut off some higher frequencies (aka. use a low pass filter). As before we create a mask and apply it to our image</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/cat_fft_filt.png" alt="Cute!" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cat_filt.png" alt="Softer cute!" class="halfimage" /></p>
<p>The mask is made from a <a href="https://en.wikipedia.org/wiki/Window_function#Tukey_window">Tukey window</a> which has tapered edges to avoid edge artifacts which would appear from a sudden frequency cutoff. There are many types of window functions to use (you can even make your own!). I chose Tukey somewhat arbitrarily  weâ€™ll save the specifics for another post.</p>
<p>We can increase or decrease the diameter of our filter (or window) which will in turn increase or decrease the cut off frequency and thus the sharpness of our image:</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/cat_fft_filt2.png" alt="Cute!" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cat_filt2.png" alt="Softer cute!" class="halfimage" /></p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/cat_fft_filt3.png" alt="Cute!" class="halfimage" />
<img src="/assets/20201210fourierfiltering/cat_filt3.png" alt="Softer cute!" class="halfimage" /></p>
<h1 id="restoringtheconstable">Restoring the Constable</h1>
<p>Using what we have now (hopefully) learned, we return to Mr. Tait and his Fourier transform</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable.jpg" alt="Dotted constable" class="halfimage" />
<img src="/assets/20201210fourierfiltering/constable_fft.png" alt="Constable FFT" class="halfimage" /></p>
<p>The dots are back! Why <em>is</em> that? As we saw earlier, a well defined frequency in an image leads to a well defined point in the FFT. The dots which were used to print Mr. Taits portrait are very evenly spaced and can be apparently represented well by a limited, fairly well defined set of frequencies. This is good news for us if we want to restore his image as we then only need to block fairly specific frequencies.</p>
<p>We can begin with the easy way  just cut off all frequencies above a certain point which contains the dots:</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable_fft_filt.png" alt="Constable FFT filtered" class="halfimage" />
<img src="/assets/20201210fourierfiltering/constable_filt.png" alt="Constable filtered" class="halfimage" /></p>
<p>Not a bad start!</p>
<p>But we may want to keep some information in the higher frequencies. We can basically do whatever we want with the FFT image. Instead of simply cutting off everything above a certain frequency (lowpass filter) weâ€™ll try to just blot out the dots and keep everything else.</p>
<p>We create a new mask, this time much smaller than the picture and inverted compared to before</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/notch_window.png" alt="Notch window" class="halfimage" /></p>
<p>This time we have <script type="math/tex">0</script>â€™s in the center and <script type="math/tex">1</script>â€™s at the edge, so when we multiply this onto our image we can â€śeraseâ€ť certain points. From this we create a larger mask which we can apply to the FFT of the constable</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/notch_mask.png" alt="Notch mask" class="halfimage" />
<img src="/assets/20201210fourierfiltering/constable_fft_notch_filt.png" alt="Notch window" class="halfimage" /></p>
<p>The inverse transform gives us the filtered constable</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable_notchfilt.png" alt="Notch mask" /></p>
<p>Not perfect.</p>
<p>You may be thinking that the improvement was not very significant compared to just the lowpass filter, and you would actually be correct.</p>
<p>The reason is there cannot be any details â€śbetween the dotsâ€ť in the original image. In other words, as all information in the original image is encoded purely by the dots themselves, there shouldnâ€™t be any additional information with higher frequency than those, and when we cut away higher frequencies we donâ€™t lose image information. However there may be additional details from the digitalisation (dust, markings, etc.) which could be interesting for some applications. There are also some artifacts around the edges, which I have not accounted for, seen as vertical and horizontal streaks in the FFT, but that is maybe food for a future blog post.</p>
<p>Let us combine the two methods and do both a notch filter and lowpass filter</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable_fft_doublefilt.png" alt="Notch mask" /></p>
<p>Our constable has now reached his final form:</p>
<p class="centerimages"><img src="/assets/20201210fourierfiltering/constable_doublefilt.png" alt="Notch mask" /></p>
<p>There are many other techniques that can be used for this kind of image reconstruction, but it is at least interesting to see how much you can do with a fairly straight forward method.</p>
<p>The principles we have seen and the intuition gained from playing with transforming images can carry over and be of great help in many other areas of science and physics, including understanding of lenses, diffraction gratings, crystallography and many other (some of which I hope to write more about here!)</p>
<p>Hope you enjoyed reading!</p>
<hr />
<div class="footnotes">
<ol>
<li id="fn:1">
<p>There are exceptions, but most reasonably wellbehaved functions one might normally encounter in the physical world (and the mathematics describing it) has a Fourier transform. There is a bit more on that on <a href="https://math.stackexchange.com/questions/1378633/everyfunctioncanberepresentedasafourierseries">this Stackexchange</a> question. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:4">
<p><a href="https://en.wikipedia.org/wiki/Complex_conjugate">complex conjugation</a> is fancy speak for â€śthe <script type="math/tex">i</script> changes signâ€ť in a complex number. Thus the complex conjugate of <script type="math/tex">a + ib</script> is just <script type="math/tex">a  ib</script>. <a href="#fnref:4" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<script type="math/tex; mode=display">\lvert z\rvert = \sqrt{a^2 + (i b)^2} = \sqrt{a^2 + b^2}</script>
<p><a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
<li id="fn:5">
<p>I am now displaying the logarithm of the value of each pixel now, i.e. <script type="math/tex">\log\left(\lvert z \rvert\right)</script> for all pixels, as the main peak would dominate everything in the image. With the logarithm lower valued pixels are enhanced and easier to see. <a href="#fnref:5" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Thu, 10 Dec 2020 00:00:00 +0000
https://longcreek.me/blog/2020/fourierfiltering
https://longcreek.me/blog/2020/fourierfiltering
fourier
filter
frequency
transformation
imageprocessing
dsp
FFT
imageprocessing