micro_dl.preprocessing module

Submodules

micro_dl.preprocessing.estimate_flat_field module

Estimate flat field images

class micro_dl.preprocessing.estimate_flat_field.FlatFieldEstimator2D(input_dir, output_dir, channel_ids, slice_ids, block_size=32)

Bases: object

Estimates flat field image

estimate_flat_field()

Estimates flat field image for given channel indices using a polynomial fit of sampled intensity medians from blocks/tiles across the image, assuming a smoothly varying flat field.

get_flat_field_dir()

Return flatfield directory :return str flat_field_dir: Flatfield directory

get_flatfield(im, order=2, normalize=True)

Combine sampling and polynomial surface fit for flatfield estimation. To flatfield correct an image, divide it by flatfield.

Parameters:

  • im (np.array) – 2D image

  • order (int) – Order of polynomial (default 2)

  • normalize (bool) – Normalize surface by dividing by its mean for flatfield correction (default True)

Return np.array flatfield:

Flatfield image

sample_block_medians(im)

Subdivide a 2D image in smaller blocks of size block_size and compute the median intensity value for each block. Any incomplete blocks (remainders of modulo operation) will be ignored.

Parameters:

im (np.array) – 2D image

Return np.array(float) sample_coords:

Image coordinates for block centers

Return np.array(float) sample_values:

Median intensity values for blocks

micro_dl.preprocessing.generate_masks module

Generate masks from sum of flurophore channels

class micro_dl.preprocessing.generate_masks.MaskProcessor(input_dir, output_dir, channel_ids, flat_field_dir=None, time_ids=-1, slice_ids=-1, pos_ids=-1, int2str_len=3, uniform_struct=True, num_workers=4, mask_type='otsu', mask_channel=None, mask_ext='.npy')

Bases: object

Appends Masks to zarr directories

generate_masks(str_elem_radius=5)

Generate masks from flat-field corrected flurophore images. The sum of flurophore channels is thresholded to generate a foreground mask.

Parameters:

str_elem_radius (int) – Radius of structuring element for morphological operations

get_channel_thr_df()
get_mask_channel()

Return mask channel :return int mask_channel: Assigned channel number for mask

get_mask_dir()

Return mask directory :return str mask_dir: Directory where masks are stored

micro_dl.preprocessing.resize_images module

Generate masks from sum of flurophore channels

class micro_dl.preprocessing.resize_images.ImageResizer(input_dir, output_dir, scale_factor, channel_ids, time_ids, slice_ids, pos_ids, int2str_len=3, num_workers=4, flat_field_dir=None, flat_field_channels=[])

Bases: object

Resize images for given indices

get_resize_dir()

Return directory with resized images :return str resize_dir: Directory where resized images are stored

resize_frames()

Resize frames for given indices.

resize_volumes(num_slices_subvolume=-1)

Down or up sample volumes

Overlap of one slice across subvolumes

Parameters:

num_slices_subvolume (int) – num of 2D slices to include in each volume. if -1, include all slices

micro_dl.preprocessing.tile_3d module

Tile 3D

class micro_dl.preprocessing.tile_3d.ImageTilerUniform3D(input_dir, output_dir, time_ids, channel_ids, slice_ids, pos_ids, normalize_channels, tile_size=[64, 64, 64], step_size=[32, 32, 32], depths=[1], hist_clip_limits=None, flat_field_dir=None, image_format='zyx', num_workers=4, int2str_len=3, normalize_im='stack', min_fraction=None, tile_3d=True)

Bases: ImageTilerUniform

Tiles all volumes in a dataset

micro_dl.preprocessing.tile_nonuniform_images module

class micro_dl.preprocessing.tile_nonuniform_images.ImageTilerNonUniform(input_dir, output_dir, time_ids, channel_ids, slice_ids, pos_ids, normalize_channels, tile_size=[256, 256], step_size=[64, 64], depths=1, hist_clip_limits=None, flat_field_dir=None, image_format='zyx', num_workers=4, int2str_len=3, tile_3d=False, tiles_exist=False)

Bases: ImageTilerUniform

Tiles all images images in a dataset

tile_first_channel(channel0_ids, channel0_depth, cur_mask_dir=None, is_mask=False)

Tile first channel or mask and use the tile indices for the rest

Tiles and saves the tiles, meta_df for each image in self.tile_dir/meta_dir. The list of meta_df for all images gets saved as frames_meta.csv

Parameters:

  • channel0_ids (dict) – [tp_idx][ch_idx][ch_dict] for first channel or mask channel

  • channel0_depth (int) – image depth for first channel or mask

  • cur_mask_dir (str) – mask dir if tiling mask channel else none

  • is_mask (bool) – Is mask channel

Return pd.DataFrame ch0_meta_df:

pd.Dataframe with ids, row_start and col_start

tile_mask_stack(mask_dir, mask_channel, mask_depth=1)

Tiles images in the specified channels assuming there are masks already created in mask_dir. Only tiles above a certain fraction of foreground in mask tile will be saved and added to metadata.

Saves a csv with columns [‘time_idx’, ‘channel_idx’, ‘pos_idx’, ‘slice_idx’, ‘file_name’] for all the tiles

Parameters:

  • mask_dir (str) – Directory containing masks

  • mask_channel (int) – Channel number assigned to mask

  • mask_depth (int) – Depth for mask channel

tile_remaining_channels(nested_id_dict, tiled_ch_id, cur_meta_df)

Tile remaining channels using tile indices of 1st channel / mask

Parameters:

  • nested_id_dict (dict) – nested dict with time, channel, pos and slice indices

  • tiled_ch_id (int) – self.channel_ids[0] or mask_channel

  • cur_meta_df (pd.DataFrame) – DF with meta for the already tiled channel

tile_stack()

Tiles images in the specified channels.

Assuming mask channel is not included in frames_meta in self.input_dir. Else this will cause an error as the filename = self.input_dir + file_name from frames_meta.csv. Masks are generally stored in a different folder.

Saves a csv with columns [‘time_idx’, ‘channel_idx’, ‘pos_idx’,’slice_idx’, ‘file_name’] for all the tiles

micro_dl.preprocessing.tile_uniform_images module

class micro_dl.preprocessing.tile_uniform_images.ImageTilerUniform(input_dir, output_dir, time_ids, channel_ids, slice_ids, pos_ids, normalize_channels, tile_size=[256, 256], step_size=[64, 64], depths=1, hist_clip_limits=None, flat_field_dir=None, image_format='zyx', num_workers=4, int2str_len=3, normalize_im='stack', min_fraction=None, tile_3d=False, tiles_exist=False)

Bases: object

Tiles all images in a dataset

get_crop_tile_args(channel_idx, time_idx, slice_idx, pos_idx, task_type, tile_indices=None, mask_dir=None)

Gather arguments for cropping or tiling

Parameters:

  • channel_idx (int) – channel index for current image

  • time_idx (int) – time index for current image

  • slice_idx (int) – slice index for current image

  • pos_idx (int) – position / sample index for current image

  • task_type (str) – crop or tile

  • tile_indices (list) – list of tile indices

  • mask_dir (str) – dir containing image level masks

Return list cur_args:

tuple of arguments for tiling list tile_indices: tile indices for current image

get_tile_dir()

Return directory containing tiles :return str tile_dir: Directory with tiles

tile_mask_stack(mask_dir, mask_channel, mask_depth=1)

Tiles images in the specified channels assuming there are masks already created in mask_dir. Only tiles above a certain fraction of foreground in mask tile will be saved and added to metadata.

Saves a csv with columns [‘time_idx’, ‘channel_idx’, ‘pos_idx’, ‘slice_idx’, ‘file_name’] for all the tiles

Parameters:

  • mask_dir (str) – Directory containing masks

  • mask_channel (int) – Channel number assigned to mask

  • mask_depth (int) – Depth for mask channel

tile_stack()

Tiles images in the specified channels.

https://research.wmz.ninja/articles/2018/03/ on-sharing-large-arrays-when-using-pythons-multiprocessing.html

Saves a csv with columns [‘time_idx’, ‘channel_idx’, ‘pos_idx’,’slice_idx’, ‘file_name’] for all the tiles

Module contents

Module for preprocessing