Tip

Download this tutorial as a Jupyter notebook, or as a python script with code cells.

Tutorial 13: Import / Export (I/O)¶

Using optimap you can import, export or convert various video or image file formats acquired with different cameras and software packages. For instance, you can load .rsh videos acquired with SciMedia cameras and save them as Matlab files using the following few lines of code:

import optimap as om

video = om.load_video("filename_of_input_video.rsh")
om.save_video("video.mat", video)

Many more file formats can be imported, saved or exported. More detailed instructions are provided below. To test whether you can load your own example video file (e.g. an .rsh file), make sure that the video is located in the appropriate folder, copy and paste the code above into a python script, modify the filename to match your filename and execute the script.

Importing Videos¶

The following file formats can be imported with optimap:

.tif, .tiff (TIFF) image stacks
Folder containing sequence of TIFF or .png (PNG) images
.gsd, .gsh (SciMedia MiCAM 05)
.rsh, .rsm, .rsd (SciMedia MiCAM ULTIMA)
.dat (MultiRecorder)
.npy (NumPy array)
.mat (MATLAB), loads the first field in the file
.mp4, .avi, .mov, … (digital video files)

Additional file formats will be added in the future (and upon request). All files can be imported using the same load_video() function, with which it is also possible to load only a specific number of frames or range of the data (e.g. from a specific frame to another), see below.

In all cases, it is possible to import only parts of a video to reduce loading times using the following optional arguments:

start_frame: The first frame to load (default: 0)
frames: Number of frames to load (default: all)
step: Load every step-th frame (default: 1)

For instance, load 1000 frames and only every 2nd frame starting from frame 100:

video = om.load_video("filename.tif", frames=1000, step=2, start_frame=100)

This will load frames 100, 102, 104, 106, …, 2098.

Some formats can be loaded using memory mapping, which can be useful to reduce memory usage or loading times when working with large or numerous video files. This is enabled by setting the use_mmap argument to True:

video = om.load_video("example.npy", use_mmap=True)

This will load the video as a read-only memory-mapped array, which means that the data is not loaded into memory until it is accessed. use_mmap is disabled by default and only supported for TIFF stacks, MultiRecorder .dat files and NumPy .npy files.

Note

Imported videos retain the same data type as the original file. For instance, 16-bit TIFF images will be loaded as 16-bit arrays, and 8-bit images as 8-bit arrays.

Often it is useful to convert the data to 32-bit floating point arrays and to normalize the data to the range [0, 1]. This can be done using the video.normalize() method.

video = om.load_video("example.tif")
video = om.video.normalize(video)

This will convert the video to 32-bit floating point arrays and normalize the data to the range [0, 1] based on the minimum and maximum values in the video.

To specify a custom range for normalization to [0, 1], use the vmin and vmax arguments:

video = om.load_video("example.dat")
# 12-bit data, normalize to [0, 1]
video = om.video.normalize(video, vmin=0, vmax=4095)

Saving / Exporting Videos¶

The following file formats can be saved / exported with optimap:

videos as raw data (TIFF stacks, NumPy or Matlab files)
image sequences in a folder
rendered videos with or without colormap (e.g. .mp4 videos)
multiple videos rendered next to each other
video rendered with overlays

optimap distinguishes saving and exporting files. Saving files saves them as raw data (e.g. three-dimensional arrays), while exporting renders videos for visualization purposes (such as a .mp4 file). Saving videos prevents data loss (e.g. floating point or integer precision is preserved) and one can reload and continue to process the saved videos later. Exporting data is associated with data loss (e.g. loss of floating point precision) as the recording is rendered using a video encoder.

Saving Videos¶

The default mode in optimap is to save videos as arrays in numpy (.npy) or Matlab (.mat) file formats. This saves the data in its rawest form without information loss:

# NumPy NPY format
om.save_video('video.npy', video)
# TIFF image stack
om.save_video('video.tif', video)
# MATLAB
om.save_video('video.mat', video)

The videos can later be imported and further processed or used in other applications. The save_video() function preserves the data type (e.g. floating point or integer precision).

Another way to save video data is to save them as a TIFF or PNG image series in a folder (here called "my_folder"):

om.save_image_sequence(".", video)

Will create files frame_0000.png, frame_0001.png, … in the current working directory. The prefix argument can be used to specify a custom prefix for the filenames, and the format argument can be used to specify the file format (.png by default).

om.save_image_sequence(r"C:\example_directory", video, filepattern="frame-{:04d}", format=".tif")

The resulting images will have the same resolution as the video. If the videos are 8- or 16-bit, then the resulting .tif or .png images will automatically be 8- or 16-bit, correspondingly. It is recommended to save your video as 16-bit to prevent data loss. If the images are all black, all white, show salt and pepper noise or look weird in any way it is likely that your video was formatted incorrectly (e.g. is an integer video even though the original video was floating point). For instance, 8-bit videos can only contain integer values between 0-255. You can use the code snippet below to format your video data before saving it as an image series:

# Convert to uint8 data type with has value range [0-255]
video = om.video.normalize(video, dtype="uint8")
om.print_properties(video)

With the normalization (the subtraction of the minimum and the division by the maximum) in the code snippet above it is ensured that no values are below 0 or above 255 (or 65536, respectively).

Exporting Videos¶

The main purpose of exporting videos is to generate or render videos in a file format (.mp4) that can be played with an external video player application (e.g. Quicktime, VLC, Windows Media Player etc.) or be included in slideshows (e.g. Powerpoint). You can export videos in several ways:

a single grayscale video (e.g. showing the original or normalized video)
a processed video (e.g. phase maps) with a special colormap (e.g. hsv, jet, magma, etc.)
an overlay of 2 videos on top of each other (e.g. original grayscale video plus calcium waves highlighted in a particular color)
multiple videos rendered next to each other (e.g. original grayscale video plus motion-stabilized video or original grayscale video and processed video in a different colorcode)

The most straight-forward way to export a video is:

om.export_video('video.mp4', video)

exporting video:   0%|          | 0/90 [00:00<?, ?it/s]

exporting video: 100%|██████████| 90/90 [00:00<00:00, 197.36it/s]

video exported to video.mp4

This will generate a .mp4 video file containing the entire video data at the original resolution with a display framerate of 60fps (default). See video.export_video() for more options, e.g.:

om.export_video("video.mp4", video, fps=15, cmap="magma", vmin=0.5, vmax=0.9)

exporting video:   0%|          | 0/90 [00:00<?, ?it/s]

exporting video: 100%|██████████| 90/90 [00:00<00:00, 196.73it/s]

video exported to video.mp4

Here, the video is exported with a framerate of 15fps, vmin and vmax are used define the dynamic range of pixel values. The step parameter can be used to only export every n-th frame.

video.export_video_with_overlay() can be used to overlay two videos on top of each other. For instance, a pixel-wise normalized video which shows action potential or calcium waves, see Tutorial 2, on top of the original grayscale video:

overlay = om.video.normalize_pixelwise(video)
om.export_video_with_overlay("video.mp4", video, overlay=overlay, fps=15)

calculating flows (CPU):   0%|          | 0/90 [00:00<?, ?it/s]

calculating flows (CPU): 100%|██████████| 90/90 [00:04<00:00, 20.20it/s]

Creating mask with detected threshold 0.019291819291819293

exporting video:   0%|          | 0/90 [00:00<?, ?it/s]

exporting video: 100%|██████████| 90/90 [00:00<00:00, 181.89it/s]

video exported to video.mp4

To export videos side-by-side, use export_videos():

om.export_videos(
    "video.mp4",
    [video, overlay_motion, video_warped, overlay],
    fps=15,
    ncols=2,
    cmaps=['gray', 'Purples', 'gray', 'Purples'],
    vmins=0,
    vmaxs=1
)

exporting video:   0%|          | 0/90 [00:00<?, ?it/s]

exporting video: 100%|██████████| 90/90 [00:00<00:00, 109.54it/s]

Video exported to video.mp4

Lastly, you can export animated plots, such as two videos in two sublots next to each other, as follows:

animation = om.show_video_pair(
    video1=video,
    video2=overlay,
    title1="Original Recording",
    title2="Action Potential Wave Front",
    cmap2="Purples",
    vmin2=0,
    vmax2=1
)
animation.save("video.mp4", fps=15, hide_framecounter=True)

The output of the animation functions show_video(), show_videos(), and show_video_overlay() can be saved in the same fashion.

See video.InteractivePlayer.save() and matplotlib.animation.Animation.save() for further details.

Images and Masks¶

Importing Images & Masks¶

Individual images (.tif, .png, .jpg, .npy, …) can be imported using load_image():

image = om.load_image("example.tif")

the as_grey argument can be used to convert the image to a grayscale image (if it is not already).

Segmentation masks can be imported using load_mask():

mask = om.load_mask("mask.png")

See Tutorial 3: Masking / Segmentation for more details on how to load, create and use masks.

Saving or Exporting Images & Masks¶

As for videos, we differentiate between saving and exporting. The save functions aim to prevent any loss of precision such that the resulting file can be loaded with load_image(). The export function on the other hand support colormaps and should be used when the resulting file will be used for presentation.

Images can be saved using save_image():

om.save_image("example.png", image)

The following file formats and image data types are supported:

NumPy: .npy, all data types
PNG: .png, 8-bit or 16-bit unsigned per image channel
TIFF: .tif/.tiff, 8-bit unsigned, 16-bit unsigned, 32-bit float, or 64-bit float images
JPEG: .jpeg/.jpg, 8-bit unsigned
Windows bitmaps: .bmp, 8-bit unsigned

The image data is saved as it is, without any normalization or scaling.

Images appear black in another image viewer

Standard external image viewers can often not handle 16-bit unsigned or 32-bit float image data, which can make images saved with save_image() appear black. Use export_image() instead to save images in a format that can be viewed with standard image viewers if the image will not be further processed.

To export images use export_image():

om.export_image("example.png", image)

For grayscale images export_image() uses the minimum and maximum value of the image by default for the normalization to an 8-bit image (integer values 0-255). It is often advised to set the input data range manually using vmin and vmax:

om.export_image("example.png", image, vmin=0, vmax=1, cmap="gray")

In this example the value range [0, 1] is mapped to [0, 255].

Masks can be saved using save_mask():

om.save_mask("mask.png", mask)

Exporting Plots¶

All plotting functions such as show_image(), show_mask(), show_traces() etc are based on matplotlib. To export the output click the save icon in the plot window, or use matplotlib.pyplot.savefig():

import matplotlib.pyplot as plt
om.show_image(image)
plt.savefig("image.png", bbox_inches='tight')

or equivalently:

ax = om.show_image(image)
ax.figure.savefig("image.png", bbox_inches='tight')

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[11], line 1
----> 1 ax = om.show_image(image)
      2 ax.figure.savefig("image.png", bbox_inches='tight')

NameError: name 'image' is not defined

All plotting accept a matplotlib.axes.Axes as input to create subplots figures:

fig, axs = plt.subplots(2)
om.show_image(image, ax=axs[0])
om.show_traces(traces, ax=axs[1])
fig.savefig("image.png", dpi=300)