Tip

Download this tutorial as a Jupyter notebook, or as a python script with code cells. We highly recommend using Visual Studio Code to execute this tutorial. Alternatively, you could run the Python script in a terminal with python basics.py from the folder where the file is located.

Tutorial 1: Basics

This tutorial will walk you through the basics of optimap.

optimap is a package that can be imported and used in Python scripts. Python is a freely available programming language that is also very popular in scientific computing. The simplest way to access the functionality of optimap is to import it at the beginning of a new Python script as follows:

import optimap as om

When running this command in a terminal / Python shell or as part of a Python script and optimap was installed correctly, there should be no further output. We now have access to all the functions in the optimap package, and it can then be accessed by typing om. followed by a specific function as shown below (e.g. om.load_video()). If the import produces an error then optimap was not installed correctly, see Installation Guide for further details.

optimap relies heavily on other open-source software packages and libraries, foremost NumPy, which is a numerical programming library, Matplotlib, which is a library for plotting data, and OpenCV, which is a library for computer vision. In this example, we will import numpy and matplotlib right after optimap as follows:

import numpy as np
import matplotlib.pyplot as plt

You do not necessarily have to import numpy and matplotlib in your analysis scripts as it will be imported internally by optimap. Here we imported these packages to be able to work directly with some of their functions in our script and write some custom code for analysis and plotting.

If you encounter any issues with the steps above please see the Installation Guide.

Loading a Video file

The following file formats are currently supported by optimap:

• .tif, .tiff (TIFF) image stacks

• Folder containing sequence of TIFF or .png (PNG) images

• .gsd, .gsh (SciMedia MiCAM 05)

• .rsh, .rsm, .rsd (SciMedia MiCAM ULTIMA)

• .dat (MultiRecorder)

• .npy (numpy array)

• .mat (MATLAB), loads the first field in the file

We can use the optimap.load_video() function to load a video file, see also Tutorial 13: Import / Export (IO). The code below will automatically download an example file from our website cardiacvision.ucsf.edu and load it into our workspace as a video. Alternatively, you could load your own file by replacing filepath with the filename of your video which you have stored somewhere on your computer. The example file shows a fibrillating, weakly contracting rabbit heart stained with voltage-sensitive dye (Di-4-ANEPPS) imaged using a Basler acA720-520um camera at 500fps. Due to the staining, the action potential wave is inverted, i.e. an upstroke is observed as a negative deflection. The data is from [Chowdhary et al., 2023] and we extracted a short part of the original recording and saved the otherwise unprocessed raw video data as a numpy file (.npy). Experimenters: Jan Lebert, Shrey Chowdhary & Jan Christoph (University of California, San Francisco, USA), 2023.

filepath = om.download_example_data("VF_Rabbit_1.npy")
# alternative if you downloaded the file to your desktop
# filepath = 'VF_Rabbit_1.npy'
video = om.load_video(filepath)

om.print_properties(video)

Downloading data from 'https://cardiacvision.ucsf.edu/sites/g/files/tkssra6821/f/optimap-VF_Rabbit_1.npy_.webm' to file '/home/runner/work/optimap/optimap/docs/tutorials/optimap_example_data/VF_Rabbit_1.npy'.

------------------------------------------------------------------------------------------
array with dimensions: (1000, 390, 300)
datatype of array: uint16
minimum value in entire array: 51
maximum value in entire array: 3884
------------------------------------------------------------------------------------------

optimap imports video data as three-dimensional NumPy array, where the first dimension is time and the other two dimensions are the x- and y-dimensions, respectively. This convention is used throughout the library. The function print_properties() displays the dimensions and maximal and minimal intensity values of a video. Our example file has 1040 video frames. See load_video() for additional arguments, e.g. to load only a subset of the frames or to use memory mapping to reduce memory usage.

video = om.load_video('Example.dat', start_frame=100, frames=1000, step=2, use_mmap=True)

For some file formats, optimap can also load the corresponding metadata using load_metadata(). For example, the following code loads the metadata of a MiCAM ULTIMA recording:

metadata = om.load_metadata('Example.rsh')

To crop, rotate, or flip a video see optimap.video for a list of available functions.

Playing Videos

Videos can be viewed using either:

1. the built-in viewer show_video() based on matplotlib

2. Monochrome, which is a more advanced and performant viewer with better interactive features

Using the built-in Viewer

om.show_video(video, skip_frame=3);

See API documentation for show_video() for a list of available arguments. For instance, it is possible to specify a title, a value range (vmin=0, vmax=1) and colormap:

om.show_video(video, title="Example video", vmin=0, vmax=1, cmap="gray", interval=20)

Using Monochrome

Monochrome needs to be installed separately. We will provide installation instructions soon.

import monochrome as mc
mc.show(video, "raw video")

See the monochrome documentation and monochrome.show() for more information. For example, click in the video to view time traces at the selected positions.

Viewing and Extracting Traces

Using optimap it is possible to quickly extract and display optical traces from any location in a video and display them using the built-in player. With the select_traces() it is possible to interactively select, extract and view an optical trace. Click on the video image on the left to select a single position or multiple positions. Right click to remove positions. Close the window to continue.

traces, positions = om.select_traces(video, size=5, fps=500)

Internally select_traces() uses extract_traces(), see below. Traces can be extracted from a single pixel or from a small window surrounding the pixel. The size parameter controls the dimensions of the window. By default, this window is a rectangle with dimensions (size, size), but it can also be set to 'disc' using the window parameter, which then sets the window to a circular region with diameter size around the position. Use optimap.trace.set_default_trace_window() to change the default window type (e.g. by calling it before select_traces(), extract_traces() or at the beginnig of the script with 'disc' as input parameter). The default size of the window is 5 by 5 pixels (rectangular) or a diameter of 5 pixels (disc). To get the exact pixel values without spatial averaging, set size=1. Note that the traces above include strong motion artifacts. If you would like to display the time axis in seconds rather than frames, use the fps (frames per second) parameter.

traces = om.extract_traces(video, positions, size=1, show=True, fps=500)

Note that extract_traces() uses positions as input which you already previously selected or defined, see extract_traces() for more information. Internally extract_traces() uses show_traces() to plot traces. In general, all plotting functions in optimap have an ax parameter which can be used to specify a custom matplotlib axes object. For example, we can create a figure with two subplots and show the positions on the first subplot and the traces on the second subplot with milliseconds as time unit:

fig, axs = plt.subplots(1, 2, figsize=(10,5))

om.trace.show_positions(positions, video[0], ax=axs[0])

x_axis_ms = (np.arange(video.shape[0]) / 500.0) * 1000
traces = om.extract_traces(video[:300],
                           positions,
                           x=x_axis_ms[:300],
                           size=5,
                           window='disc',
                           ax=axs[1])
axs[1].set_xlabel('Time [ms]')
plt.show()

Since the optical traces above were sampled from a raw optical mapping video that shows a fibrillating contracting heart (no Blebbistatin was used during the experiment), the traces contain strong motion artifacts. We will now use optimap to track the motion and to compensate these motion artifacts.

Motion Compensation

optimap provides automatic routines for tracking and stabilizing motion in fluorescence videos. Here, the motion-stabilized videos are interchangeably referred to as warped videos, see also Christoph and Luther[1], Lebert et al.[2], Christoph et al.[3], Kappadan et al.[4], Kappadan et al.[5], Christoph and Ripplinger[6] for further details. Tracking motion and creating a motion-stabilized or warped video is just a few lines of code with optimap. The fibrillating heart from our example contracts rapidly and moves slightly. Even though the motion is small, it can have a strong effect onto the quality of the optical traces and cause motion artifacts. Motion artifacts can in many cases prevent further analysis of the data. We can use optimap’s motion_compensate() function to compensate the motion:

video_warped = om.motion_compensate(video,
                                    contrast_kernel=5,
                                    presmooth_spatial=1,
                                    presmooth_temporal=1)

Downloading data from 'https://cardiacvision.ucsf.edu/sites/g/files/tkssra6821/f/optimap-VF_Rabbit_1_warped.npy_.webm' to file '/home/runner/work/optimap/optimap/docs/tutorials/optimap_example_data/VF_Rabbit_1_warped.npy'.

The video_warped is the motion-stabilized version of the original video. The other parameters are explained in more detail in Tutorial 4: Motion Compensation. You will also find further background information about the motion tracking and compensation routines in [Christoph and Luther, 2018] and [Lebert et al., 2022]. Let’s view the original video and the motion-compensated video side by side using optimap.show_video_pair():

om.show_video_pair(video,
                   video_warped,
                   title1="with motion",
                   title2="without motion",
                   skip_frame=3);

We can see the effect of the numerical motion-stabilization when plotting the same traces as above but extract them from the warped video:

traces = om.extract_traces(video_warped, positions, size=5, show=True, fps=500)

Overall, motion artifacts are significantly reduced: the strong baseline fluctuations are gone and action potential waves are less distorted. The residual motion artifacts vary and depend on factors such as contractile strength, fluorescent signal strength, and illumination, among others, see [Christoph and Luther, 2018] and [Lebert et al., 2022] for a more detailed discussion. It is possible to further reduce motion artifacts using ratiometric imaging, see Tutorial 4.

Saving and Rendering Videos

Let’s save the motion-compensated recording as a tiff stack and also render it to a .mp4 video file.

om.video.save_video('warped_recording.tiff', video_warped)
om.video.export_video('warped_recording.mp4', video_warped, fps=50)

saving video to tiff stack warped_recording.tiff

exporting video:   0%|          | 0/1000 [00:00<?, ?it/s]

exporting video: 100%|██████████| 1000/1000 [00:06<00:00, 149.43it/s]

video exported to warped_recording.mp4

The optimap video-player functions such as optimap.show_video_pair() can also be exported to a video file:

animation = om.show_video_pair(video, video_warped, title1='Raw', title2='Compensated')
animation.save('Example.mp4')

See matplotlib.animation.Animation.save() for more details and Tutorial 13 for more information on how to export video files.

Visualization of Action Potential Waves

In this example, we stained the heart with the voltage-sensitive fluorescent dye Di-4-ANEPPS. Together with an appropriate bandpass filter, Di-4-ANEPPS produces a slight decrease in the measured fluorescence when the tissue depolarizes. Accordingly, in order to visualize action potential waves, this small optical signal needs to be amplified (numerically). One way to achieve this is to normalize the optical traces ‘pixel-wise’: each time-series measured in a single pixel is normalized individually. In simple terms, when normalizing an optical trace one divides each value of the time-series by the maximal value found in the entire time-series and subtracts the minimum of the time-series. This removes the baseline of the time-series and all values subsequently fluctuate between 0 and 1. In our case, the depolarized phase of the action potential corresponds to values close to 0 and the diastolic interval to values close to 1. Correspondingly, the action potential wave darkens the video image. optimap has several built-in routines, which perform these normalization steps automatically. A more detailed explanation of visualizing action potential or calcium waves with different post-processing and normalization functions is given in Tutorial 2.

Sliding-window Normalization

We can compute a ‘pixel-wise normalized video’ using optimap’s rolling- or sliding-window normalization function (video.normalize_pixelwise_slidingwindow()), see Tutorial 2 for more information. This function is applied to the motion-stabilized or warped video. We slide a short temporal window over the time-series extracted from this video and perform the normalization sequentially for one value after another with the corresponding local maxima and minima. The window_size parameter controls the length of the sliding window, which needs to be adjusted to the period of the action potential or calcium waves. Here, we use a window size of 60 frames (120 ms).

video_warped_normalized = om.video.normalize_pixelwise_slidingwindow(video_warped, window_size=60)

Before we display the motion-stabilized warped video, we mask out regions that do not show tissue using optimap’s optimap.background_mask():

mask = om.background_mask(video_warped[0])
video_warped_normalized[:, mask] = 1.0

Creating mask with detected threshold 401.0

Using a grayscale colormap, action potential waves can be visualized as black/dark waves due to the pixel-wise normalization:

om.show_video(video_warped_normalized, interval=20);

In this tutorial, we demonstrated that one can visualize vortex-like rotating action potential waves across the surface of a moving fibrillating heart using optical mapping. The action potential waves were imaged on the (slightly) contracting, fibrillating heart surface in a co-moving frame of reference.

Further reading:

---

[1]

Jan Christoph and Stefan Luther. Marker-free tracking for motion artifact compensation and deformation measurements in optical mapping videos of contracting hearts. Frontiers in Physiology, 2018. doi:10.3389/fphys.2018.01483.

[2]

Jan Lebert, Namita Ravi, George Kensah, and Jan Christoph. Real-time optical mapping of contracting cardiac tissues with GPU-accelerated numerical motion tracking. Frontiers in Cardiovascular Medicine, 2022. doi:10.3389/fcvm.2022.787627.

[3]

Jan Christoph, M. Chebbok, C. Richter, J. Schröder-Schetelig, P. Bittihn, S. Stein, I. Uzelac, F. H. Fenton, G. Hasenfuß, R. F. Gilmour Jr., and S. Luther. Electromechanical vortex filaments during cardiac fibrillation. Nature, 555(7698):667–672, 2018. doi:10.1038/nature26001.

[4]

Vineesh Kappadan, Saba Telele, Ilija Uzelac, Flavio Fenton, Ulrich Parlitz, Stefan Luther, and Jan Christoph. High-resolution optical measurement of cardiac restitution, contraction, and fibrillation dynamics in beating vs. blebbistatin-uncoupled isolated rabbit hearts. Frontiers in Physiology, 2020. doi:10.3389/fphys.2020.00464.

[5]

Vineesh Kappadan, Anies Sohi, Ulrich Parlitz, Stefan Luther, Ilija Uzelac, Flavio Fenton, Nicholas S Peters, Jan Christoph, and Fu Siong Ng. Optical mapping of contracting hearts. The Journal of Physiology, 601(8):1353–1370, 2023. URL: https://physoc.onlinelibrary.wiley.com/doi/abs/10.1113/JP283683, doi:https://doi.org/10.1113/JP283683.

[6]

Jan Christoph and Crystal M. Ripplinger. Paralysis by analysis: overcoming cardiac contraction with computer vision. Proceedings of the National Academy of Sciences, 120(42):e2314448120, 2023. URL: https://www.pnas.org/doi/abs/10.1073/pnas.2314448120, doi:10.1073/pnas.2314448120.