Definition of the classes and modules we use to simulate recordings given network outputs
%load_ext autoreload
%autoreload 2

os.environ["CUDA_VISIBLE_DEVICES"] = '1'

class Microscope[source]

Microscope(parametric_psf:List[Module]=None, empirical_psf:List[Module]=None, noise:Optional[Module]=None, scale:float=10000.0, psf_noise=None, clamp_mode='cp') :: Module

Mircoscope module takes  4 volumes 'locations', 'x_os_3d', 'y_os_3d', 'z_os_3d',
'ints_3d'  and applies point spread function:
1) Extracts values of intensities and offsets of given emitters
2) Applies parametric PSF if given
3) Applies empirical  PSF if given
4) Combine all PSF and multiply by intenseties
5) Normalize PSFs
6) Places point spread function on to sampled locations  to
generate 'x_sim' simulated image
7) Multiplies x_sim by multipl


Args:
    parametric_psf (torch.nn.Module): List of Paramateric PSF
    empirical_psf (torch.nn.Module): List of Emperical PSF
    noise (torch.nn.Module): Camera noise model
    scale(float): Paramter for scaling point spread functions
    multipl(float): Multiplication value that will be applied to xsim
Shape:
    -Input: locations: Tuple(BS, Frames, H, W, D)
            x_os_val: (N_emitters,)
            y_os_val: (N_emitters,)
            z_os_val: (N_emitters,)
            ints_val: (N_emitters,)
            output_shape: Shape Tuple(BS, Frames, H, W, D)

    -Output: xsim:    (BS, C, H, W, D)

place_psf[source]

place_psf(locations, psf_volume, output_shape)

Places point spread functions (psf_volume) in to corresponding locations.

Args:
    locations: torch.Tensor
    psf_volume: torch.Tensor
    output_shape: torch.Tensor

Shape:
    -Input: locations: Tuple(BS, Frames, H, W, D)
            psf: (Num_E, C, PSF_SZ_X, PSF_SZ_Y, PSF_SZ_Z) [
            Num_E-Number of Emitters, PSF_SZ_{X, Y, Z} - PSF filter size]
            output_shape: shape of the output volume (BS, Frames, H, W, D)
    -Output: placed_psf: (BS, Frames, H, W, D)

Returns:
    placed_psf

extractvalues[source]

extractvalues(locs:tensor, x_os:tensor, y_os:tensor, z_os:tensor, ints:tensor, dim:int=3)

Extracts Values of intensities and offsets of given emitters

 This function will take `locs`, `x_os`, `y_os`, `z_os`, `ints` all of the shapes,
 and will extract `coord` coordinate of locations where our emittors  are present.
 This `coord` will be used to extract values of `x`, `y`, `z`,
 offsets and intensities - `i` where the emitter is present

Args:
    locs: location
    x_os: X offset
    y_os: Y offset
    z_os: Z offset
    ints: Intenseties
    dim:  Dimension 2D or 3D

Shape:
    -Input: locs_3d: (BS, C, H, W, D)
            x_os_3d: (BS, C, H, W, D)
            y_os_3d: (BS, C, H, W, D)
            z_os_3d: (BS, C, H, W, D)
            ints_3d: (BS, C, H, W, D)

    -Output: :
            x_os_val: (Num_E, 1, 1, 1, 1)
            y_os_val: (Num_E, 1, 1, 1, 1)
            z_os_val: (Num_E, 1, 1, 1, 1)
            ints_val: (Num_E, 1, 1, 1, 1)
from decode_fish.engine.psf import LinearInterpolatedPSF
from decode_fish.engine.noise import sCMOS
from decode_fish.engine.point_process import PointProcessUniform
from decode_fish.funcs.plotting import plot_3d_projections
from decode_fish.funcs.file_io import get_gaussian_psf

psf = get_gaussian_psf([13,21,21],[1,1,1]).cuda()

noise = sCMOS()

micro = Microscope(parametric_psf=[psf], noise=noise, int_mu=2, int_scale=5, int_loc=1, psf_noise=0.0001).cuda()

point_process = PointProcessUniform(local_rate = torch.ones([2,1,48,48,48]).cuda()*.0001)
locs_3d, x_os_3d, y_os_3d, z_os_3d, ints_3d, output_shape = point_process.sample()

plot_3d_projections(psf.forward(torch.tensor([1]).cuda(),torch.tensor([2.5]).cuda(),torch.tensor([0]).cuda())[0,0])

array([<AxesSubplot:xlabel='x', ylabel='y'>,
       <AxesSubplot:xlabel='x', ylabel='z'>,
       <AxesSubplot:xlabel='y', ylabel='z'>], dtype=object)
xsim = micro(locs_3d, x_os_3d, y_os_3d, z_os_3d, ints_3d, output_shape)
plot_3d_projections(xsim[0,0])

tensor([[[[[1.0056]]]],



        [[[[1.0054]]]],



        [[[[1.0055]]]],



        [[[[1.0055]]]],



        [[[[1.0054]]]],



        [[[[1.0055]]]],



        [[[[1.0055]]]],



        [[[[1.0055]]]],



        [[[[1.0056]]]],



        [[[[1.0054]]]],



        [[[[1.0055]]]],



        [[[[1.0055]]]],



        [[[[1.0054]]]],



        [[[[1.0055]]]],



        [[[[1.0056]]]],



        [[[[1.0055]]]],



        [[[[1.0056]]]]], device='cuda:0', grad_fn=<SumBackward1>)

array([<AxesSubplot:xlabel='x', ylabel='y'>,
       <AxesSubplot:xlabel='x', ylabel='z'>,
       <AxesSubplot:xlabel='y', ylabel='z'>], dtype=object)
from decode_fish.funcs.file_io import get_gaussian_psf
cfg = OmegaConf.load('/groups/turaga/home/speisera/Mackebox/Artur/WorkDB/deepstorm/models/fishcod/N2_352/sweep_b1/psf_noise:0xnum_iter:10000/train.yaml')
psf = get_gaussian_psf(cfg.microscope.psf_extent_zyx, cfg.PSF.gauss_radii)
# psf.load_state_dict(torch.load(Path(cfg.output.save_dir)/'psf.pkl'))
noise = hydra.utils.instantiate(cfg.noise)
micro = Microscope(parametric_psf=[psf], noise=noise, multipl=cfg.microscope.multipl, psf_noise=2e-3, clamp_mode=cfg.microscope.clamp_mode).cuda()

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-74-e711ed475c01> in <module>
      4 # psf.load_state_dict(torch.load(Path(cfg.output.save_dir)/'psf.pkl'))
      5 noise = hydra.utils.instantiate(cfg.noise)
----> 6 micro = Microscope(parametric_psf=[psf], noise=noise, multipl=cfg.microscope.multipl, psf_noise=2e-3, clamp_mode=cfg.microscope.clamp_mode).cuda()

TypeError: __init__() got an unexpected keyword argument 'multipl'
get_gaussian_psf(cfg.microscope.psf_extent_zyx, cfg.PSF.gauss_radii).psf_volume.sum()/psf.psf_volume.sum()

tensor(1., grad_fn=<DivBackward0>)
xsim = micro(locs_3d, x_os_3d, y_os_3d, z_os_3d, ints_3d, output_shape)
print(xsim.sum())
plot_3d_projections(xsim[0,0])

divpsf tensor(22.7526, device='cuda:0', grad_fn=<SumBackward0>)
psf*i tensor(16.8434, device='cuda:0', grad_fn=<SumBackward0>)
placepsf tensor(16.6788, device='cuda:0', grad_fn=<SumBackward0>)
scaled tensor(6671519., device='cuda:0', grad_fn=<SumBackward0>)
clamped tensor(34994180., device='cuda:0', grad_fn=<SumBackward0>)
tensor(34994180., device='cuda:0', grad_fn=<SumBackward0>)

array([<AxesSubplot:xlabel='x', ylabel='y'>,
       <AxesSubplot:xlabel='x', ylabel='z'>,
       <AxesSubplot:xlabel='y', ylabel='z'>], dtype=object)
xsim = micro(locs_3d, x_os_3d, y_os_3d, z_os_3d, ints_3d, output_shape)
print(xsim.sum())
plot_3d_projections(xsim[0,0])

divpsf tensor(23.0567, device='cuda:0', grad_fn=<SumBackward0>)
psf*i tensor(17.0866, device='cuda:0', grad_fn=<SumBackward0>)
placepsf tensor(24.2850, device='cuda:0', grad_fn=<SumBackward0>)
scaled tensor(9713998., device='cuda:0', grad_fn=<SumBackward0>)
clamped tensor(23002324., device='cuda:0', grad_fn=<SumBackward0>)
tensor(23002324., device='cuda:0', grad_fn=<SumBackward0>)

array([<AxesSubplot:xlabel='x', ylabel='y'>,
       <AxesSubplot:xlabel='x', ylabel='z'>,
       <AxesSubplot:xlabel='y', ylabel='z'>], dtype=object)
plot_3d_projections(torch.clamp_min(psf.psf_volume[0],-5),'mean')

array([<AxesSubplot:xlabel='x', ylabel='y'>,
       <AxesSubplot:xlabel='x', ylabel='z'>,
       <AxesSubplot:xlabel='y', ylabel='z'>], dtype=object)
from decode_fish.funcs.file_io import *
from decode_fish.funcs.output_trafo import *
cfg = OmegaConf.load(default_conf)
gt_img = load_tiff_image('/groups/turaga/home/speisera/share_TUM/FishSIM/sim_1/mRNAlevel_200/cell3D/strong/w1_HelaKyoto_Gapdh_2597_p01_cy3__Cell_CP_14__cell3D__1.tif')

model_out = torch.load('../data/model_output.pt')
locs_ae, x_os_ae, y_os_ae, z_os_ae, ints_ae, output_shape_ae = model_output_to_micro_input(model_out, threshold=0.1)
ae_img = micro(locs_ae, x_os_ae, y_os_ae, z_os_ae, ints_ae, output_shape_ae)

log_p_x_given_z = - micro.noise(ae_img,model_out['background']).log_prob(gt_img[None,:,30:].cuda()).mean()
log_p_x_given_z.backward()

log_p_x_given_z

!nbdev_build_lib

Converted 00_models.ipynb.
Converted 01_psf.ipynb.
Converted 02_microscope.ipynb.
Converted 03_noise.ipynb.
Converted 04_pointsource.ipynb.
Converted 05_gmm_loss.ipynb.
Converted 06_plotting.ipynb.
Converted 07_file_io.ipynb.
Converted 08_dataset.ipynb.
Converted 09_output_trafo.ipynb.
Converted 10_evaluation.ipynb.
Converted 11_emitter_io.ipynb.
Converted 12_utils.ipynb.
Converted 13_train.ipynb.
Converted 15_fit_psf.ipynb.
Converted 16_visualization.ipynb.
Converted 17_eval_routines.ipynb.
Converted index.ipynb.