A Step-by-Step Tutorial for Radiologists on

Building a Neural Network for Segmentation

Masoom Haider, MD, FRCPC FSAR
Professor of Radiology, University of Toronto
Clinician Scientist, Ontario Institute of Cancer Research
Senior Scientist, Lunenfeld Tanenbaum Research Institute
Director Machine Learning and Radiomics Lab
Director Sinai Health System Research MRI
Co-Director – Artificial Intelligence Center
Joint Dept of Medical Imaging, UHN, SHS, WCH
[email protected]
2

Navigating this e-poster

• If clickable links in this presentation do not work please
download a pdf from this link:

https://www.haiderlab.ca

February 11, 2023

3

You want to “get under the hood” and build and

AI Segmentation Model

This is meant to be a
reference and a list of
resources for those taking
the journey to gain
competence in coding for
AI in medical imaging

February 11, 2023

4

Required Skills

• Python coding
• Basic linear algebra proficiency
• A basic knowledge of deep
neural network theory/design

February 11, 2023

5

Where can I get these skills online?

• Python
– Python for Beginners https://www.pythonforbeginners.com/
– Learn Python https://www.learnpython.org/
– Real Python https://realpython.com/
• Pytorch
– https://www.udemy.com/course/deep-learning-with-pytorch-for-
medical-image-analysis/
• Pytorch Lightning
– https://www.youtube.com/@PyTorchLightning
February 11, 2023
6

Where can I get these skills online?

• Github
– https://docs.github.com/en/get-started
• AI Fundamentals
– https://www.coursera.org/learn/ai-for-medical-diagnosis
• Radiology-specific AI certificate program by RSNA
• https://www.rsna.org/ai-certificate
• Annual tutorials/workshops at RSNA
• More education programs for our field are likely
forthcoming
February 11, 2023
7

Setting up your Development Environment

• Hardware
– PC – i7/i9 CPU and Nvidia GPU (more VRAM the better)
• Operating System
– Windows or Linux
• Linux greater compatibility with open source
– CUDA (11.7)
• https://developer.nvidia.com/cuda-11-7-0-download-archive
• Open source repository https://github.com/

February 11, 2023

8

Setting up your Development Environment

Software
Cloud options not covered
• Annotation tools
– 3D Slicer https://www.slicer.org/ standalone
– OHIF https://ohif.org/ server
– ITK Snap http://www.itksnap.org standalone

February 11, 2023

9

Setting up your Development Environment

Software Stack
• Environment manager
– MiniConda https://docs.conda.io/en/latest/miniconda.html
• Code editor
– VSCode https://code.visualstudio.com/Download
• Jupyter Notebook for prototyping/documentation
– Jupyter Notebooks

February 11, 2023

10

What is MONAI?
https://monai.io/

a set of open-source, freely available

collaborative frameworks built for
accelerating research and clinical
collaboration in Medical Imaging. The
goal is to accelerate the pace of
innovation and clinical translation by
building a robust software framework
that benefits nearly every level of
medical imaging, deep learning
research, and deployment.
February 11, 2023
11

Setting up your Development Environment

Software Stack
• Setup environment in Anaconda (Miniconda)
– See Conda cheat sheet
• Install software stack
– Pytorch
– Pytorch Lightning
– MONAI

February 11, 2023

12

Windows Batch Script

Create your environment in Anaconda
(.bat file)
SET ENV_NAME=monai10 the conda environment name you wish to assign
call conda create -n %ENV_NAME% python=3.8 –y
call conda activate %ENV_NAME%
call conda install jupyter –y
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia –y
call pip install pytorch-lightning
call pip install monai
call pip install -r requirements-dev.txt this file from the monai git and installs additional libraries
call conda install jupyter –y
February 11, 2023
13

Watch the MonAI Youtube Channel

• Basics 2023
– https://www.youtube.com/
watch?v=U77UPifZ1Uw&t
=1170s
• Bootcamp 2021
– https://www.youtube.com/
playlist?list=PLtoSVSQ2X
zyCobzE6NvwjNpITsQyP
Utfs
February 11, 2023
14

Clone the Monai Tutorial Git

• Download the spleen segmentation tutorial code
– https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/spleen_segmentation_3d_lightning.ipynb

• Save your prostate segmentations and source DICOM

data in nifti format using one of the segmentation software
tools (i.e. itksnap or 3d slicer)
• Customize the prepare_data function in the tutorial

February 11, 2023

15

Use Case
• Fully automated segmentation of the prostate on T2 MRI
• For use in PSA density calculation
• For use in TRUS/MRI fusion biopsy system
• Radiologist or technologist aid
• Input: Axial T2 images following PiRads standard
• Output: Segmentation mask of prostate

February 11, 2023

16

The UNET Model

• MONAI
provides basic
models
• UNET is
particularly
good for simple
segmentation
tasks

February 11, 2023

17

SOP for Data Storage

• Structure your data storage using a
standard so that you can use similar
DataLoader logic for many
applications
+---
AIPR_10000009_19010101_2ec9998ef7f6bb8
PROSTATRESEG | | adc.nii.gz
<examid> |
|
|
|
axt2.nii.gz
b0.nii.gz
<segid>_<pulseseq>_<reader initials>.nii.gz | | b1600.nii.gz
| | dce.nii.gz
<pulseseq>.nii.gz | | dwi.nii.gz
| | p_axt2_ig.nii.gz
| | p_b0_ig.nii.gz

February 11, 2023

18

Example of data_setup()
mSourceDir = r"D:\PSEG“ # directory containing data
seriesname=‘axt2’
segname='p’

def prepare_data(self):
self.filenames = []
mFolders = [ f.path for f in os.scandir(self.mSourceDir) if f.is_dir() ]
seriesfname = self.seriesname + ".nii.gz"
segfname = self.segname + "_" + self.seriesname+ "_*" + ".nii.gz"
for mFolder in mFolders:
# get prostate images and segmentations
mseriesfname = os.path.join(mFolder, seriesfname)
mseriesfname = glob.glob(mseriesfname)
msegfname = os.path.join(mFolder, segfname)
msegfname = glob.glob(msegfname)
if len(mseriesfname) > 0 and len(msegfname)>0:
self.filenames.append({self.fn_keys[0]: mseriesfname[0], self.fn_keys[1]: msegfname[0]})
# partition files
test = partition_dataset(self.filenames, ratios=(0.9,0.1), shuffle=True, seed=0, drop_last=False, even_divisible=False)
self.train_files = test[0]
self.val_files = test[1]
# define transforms # put your transforms below
…

February 11, 2023

19

• Run the remaining cells in the notebook

• Training may take a while
• Video from the MONAI workshops are particularly helpful
and the associated git tutorial discussion groups are
helpful resources

February 11, 2023

20

net1.eval()
device = torch.device("cuda:0")
Prediction Code
net1.to(device)
dl = net1.val_dataloader()
fn = net1.filenames
with torch.no_grad():
for i, pred_data in enumerate(dl):
roi_size = net1.roi_size
sw_batch_size = 1
pred_outputs = sliding_window_inference(pred_data["image"].to(device), roi_size, sw_batch_size, net1)
a = pred_data["image"]
sw_batch_size=a.shape[0]
for j in range(sw_batch_size):
print(f'Working on exam: {j}')
im = pred_data["image"][j,:,:,:,:]
seg = pred_data["label"][j,:,:,:,:]
segcalc = torch.argmax(pred_outputs, dim=1).detach().cpu()[j:j+1,:,:,:]
overlayim = blend_images(segcalc,seg, alpha=0.7, cmap='hsv', rescale_arrays=True)
matshow3d(overlayim,figsize=(100, 100), frame_dim=-1, show=True, channel_dim=0)

February 11, 2023

21

Prediction
{'image': 'D:\\PSEG\\AIPR_10000036_19010000_2ec9998ef7ff89f\\axt2.nii.gz’,
'label': 'D:\\PSEG\\AIPR_10000036_19010000_2ec9998ef7ff89f\\p_axt2_ig.nii.gz’}
Dice : 0.71546304

Overlay of radiologist and

predicted segmentation
22

Summary
• This is an overview of the journey in learning how to get
into the world of deep convolutional nnural network
coding.
• It requires a dedicated learning plan
• Tools continue to improve rapidly and it is expected that
things will become easier
• MONAI has greatly accelerated progress for those in the
medical imaging arena

February 11, 2023

23

[email protected]
https://www.haiderlab.ca