P4 Multispectral Image Processing Guide

v1.0 2020.07

I. Key EXIF and XMP information in images taken using the P4 Multispectral

【EXIF】MakerNotes
Key Value example Type Unit Meaning
Make DJI
Pitch +1.50 float degree Flight roll angle
Yaw +0.00 float degree Flight yaw angle
Roll +1.70 float degree Flight pitch angle
Camera Pitch -89.90 float degree Gimbal roll angle
Camera Yaw +0.00 float degree Gimbal yaw angle
Camera Roll +0.00 float degree Gimbal pitch angle

【XMP】drone-dji

Key Value example Type Unit Meaning

Absolute Absolute altitude of the camera based on

+50.00 float meter
Altitude the ellipsoid model used
Relative Relative altitude of the camera based on
+0.00 float meter
Altitude Home point
Gps Latitude 113.00 float - Latitude of camera position
Gps Longitude 22.00 float - Longitude of camera position
Gimbal Roll
+0.00 float degree Gimbal roll angle
Degree
Gimbal Yaw
+0.00 float degree Gimbal yaw angle
Degree
Gimbal Pitch
+0.00 float degree Gimbal pitch angle
Degree
Flight Roll
+0.00 float degree Flight roll angle
Degree
Flight Yaw
+0.00 float degree Flight yaw angle
Degree
Flight Pitch
+0.00 float degree Flight pitch angle
Degree
Flight X Speed +0.00 float m/s Ground speed northern vector
Flight Y Speed +0.00 float m/s Ground speed eastern vector
Flight Z Speed +0.00 float m/s Ground speed vertical

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 1
P4 Multispectral Image Processing Guide

Calibrated Focal
1913.3333 float pixel 5.74[mm] / 3.0[um/pixel] = 1913.3333..
Length
Calibrated X-axis coordinate of the designed position
800 float pixel
Optical Center X of optical center
Calibrated Y-axis coordinate of the designed position
650 float pixel
Optical Center Y of optical center
Rtk Flag 16 string - RTK status
Relative Optical Disparity on X direction relative to NIR
0.00 float pixel
Center X band
Relative Optical Disparity on Y direction relative to NIR
0.00 float pixel
Center Y band
2020-02-04;
1949.4499512,
1939.7700195,
Distortion calibration parameter
Dewarp Data 28, -17, string -
(calibrate_date;fx,fy,cx,cy,k1,k2,p1,p2,k3);
-0.4082, 0.3418,
0.00068, 0.00068,
-0.3540
0.000218235,
1.20722E-06,
-2.8676E-09, Coefficients for Vignetting calibration
Vignetting Data string -
5.1742E-12, ( k[0], k[1], k[2], k[3], k[4], k[5] )
-4.16853E-15,
1.36962E-18
Band Name NIR string - Name for each band
Band Freq 840(+/-26)nm string - The frequency for each band
Irradiance 2000.000 float - Calibrated sunlight sensor value
Sensor Gain 1 float - Image Sensor gain for each band
micro-
Exposure Time 1000 string Exposure time for each band
second
Sensor Gain Parameter for individual difference
1.002 float -
Adjustment correction (float),
Sensor index 5 integer - Blue:1, Green:2, Red:3, RE:4, NIR:5
CaptureUUID - UUID V1 - Unique label for one capture;
DroneID - string - Unique label for one drone, similar to S/N

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 2
P4 Multispectral Image Processing Guide

II. How to calculate NDVI values using images and spectral sunlight sensor values from the P4 Multispectral?

The general formula to calculate the Normalized Difference Vegetation Index (NDVI) is

𝑁𝐼𝑅 − 𝑅𝑒𝑑
NDVI = 𝑁𝐼𝑅𝑟𝑒𝑓 + 𝑅𝑒𝑑𝑟𝑒𝑓 (Eq. 1)
𝑟𝑒𝑓 𝑟𝑒𝑓

Where 𝑋𝑟𝑒𝑓 represents the reflectance value of the 𝑋 band, 𝑁𝐼𝑅𝑟𝑒𝑓 𝑎𝑛𝑑 𝑅𝑒𝑑𝑟𝑒𝑓 are the reflectance values of
the NIR and Red bands, respectively.

If we define 𝑋𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 and 𝑋𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡 as the reflected light and incident light of the 𝑋 band, then,

𝑁𝐼𝑅𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 𝑅𝑒𝑑𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑
𝑁𝐼𝑅𝑟𝑒𝑓 = , 𝑅𝑒𝑑𝑟𝑒𝑓 = .
𝑁𝐼𝑅𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡 𝑅𝑒𝑑𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡

When the sunlight (i.e. the incident light) is reflected off of the target, multispectral cameras capture the reflected
light of the target in the form of multispectral images, and the sunlight sensor captures the incident light to record
sunlight sensor signal values. Hence,

𝑁𝐼𝑅𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎
𝑁𝐼𝑅𝑟𝑒𝑓 = = × 𝜌𝑁𝐼𝑅 (Eq. 2)
𝑁𝐼𝑅𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡 𝑁𝐼𝑅𝐿𝑆

𝑅𝑒𝑑𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 𝑅𝑒𝑑𝑐𝑎𝑚𝑒𝑟𝑎
𝑅𝑒𝑑𝑟𝑒𝑓 = = × 𝜌𝑅𝑒𝑑 (Eq. 3)
𝑅𝑒𝑑𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡 𝑅𝑒𝑑𝐿𝑆

Here, 𝑋𝑐𝑎𝑚𝑒𝑟𝑎 is the signal value obtained from multispectral images of the 𝑋 band, while 𝑋𝐿𝑆 is the signal value
obtained from the sunlight sensor of the same band. 𝜌𝑥 is the conversion parameter between the camera and
sunlight sensor signal values. When converting between these two signal values, make sure that the reflected light
(i.e. the signal value of the camera images) and the incident light (i.e. the signal value of the spectral sunlight sensor)
are in the same unit. Also, the spectral sunlight sensor and cameras should have the same photosensitivity, which
means that the signal values of the multispectral images and sunlight sensor should be the same under the same
lighting conditions. The camera and the sunlight sensor values have a linear relationship, therefore they can be
converted from one to the other using 𝜌𝑥 .

In addition, because the sensitivity can be different for each camera within the array and between different sunlight
sensors, calibrations are required to ensure that cameras of different bands and different sunlight sensors have the
same signal value under the same lighting conditions. All bands are calibrated against the standard NIR band. The
𝑝𝐶𝑎𝑚𝑥
calibration parameters are 𝑝𝐶𝑎𝑚𝑥 and 𝑝𝐿𝑆𝑥 , respectively. Since 𝜌𝑥 = 𝜌𝑁𝐼𝑅 × 𝑝𝐿𝑆𝑥
, then,

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 3
P4 Multispectral Image Processing Guide

𝑁𝐼𝑅𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 𝑝𝐶𝑎𝑚𝑁𝐼𝑅 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 ×𝑝𝐶𝑎𝑚𝑁𝐼𝑅

𝑁𝐼𝑅𝑟𝑒𝑓 = = × 𝜌𝑁𝐼𝑅 × = × 𝜌𝑁𝐼𝑅 (Eq. 4)
𝑁𝐼𝑅𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡 𝑁𝐼𝑅𝐿𝑆 𝑝𝐿𝑆𝑁𝐼𝑅 𝑁𝐼𝑅𝐿𝑆 ×𝑝𝐿𝑆𝑁𝐼𝑅

𝑅𝑒𝑑_𝑟𝑒𝑓𝑙𝑒𝑐𝑡𝑒𝑑 𝑅𝑒𝑑𝑐𝑎𝑚𝑒𝑟𝑎 𝑝𝐶𝑎𝑚𝑅𝑒𝑑 𝑅𝑒𝑑𝑐𝑎𝑚𝑒𝑟𝑎 ×𝑝𝐶𝑎𝑚𝑅𝑒𝑑

𝑅𝑒𝑑𝑟𝑒𝑓 = 𝑅𝑒𝑑_𝑖𝑛𝑐𝑖𝑑𝑒𝑛𝑡
= 𝑅𝑒𝑑𝐿𝑆
× 𝜌𝑁𝐼𝑅 × 𝑝𝐿𝑆𝑅𝑒𝑑
= 𝑅𝑒𝑑𝐿𝑆 ×𝑝𝐿𝑆𝑅𝑒𝑑
× 𝜌𝑁𝐼𝑅 (Eq. 5)

Therefore, we can use Eq. 6 to calculate NDVI.

𝑁𝐼𝑅𝑟𝑒𝑓 − 𝑅𝑒𝑑𝑟𝑒𝑓
NDVI = =
𝑁𝐼𝑅𝑟𝑒𝑓 + 𝑅𝑒𝑑𝑟𝑒𝑓

𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 × 𝑝𝐶𝑎𝑚𝑁𝐼𝑅 𝑅𝑒𝑑𝑐𝑎𝑚𝑒𝑟𝑎 × 𝑝𝐶𝑎𝑚𝑅𝑒𝑑 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 × 𝑝𝐶𝑎𝑚𝑁𝐼𝑅 𝑅𝑒𝑑𝑐𝑎𝑚𝑒𝑟𝑎 × 𝑝𝐶𝑎𝑚𝑅𝑒𝑑

( − )⁄( + )
𝑁𝐼𝑅𝐿𝑆 × 𝑝𝐿𝑆𝑁𝐼𝑅 𝑅𝑒𝑑𝐿𝑆 × 𝑝𝐿𝑆𝑅𝑒𝑑 𝑁𝐼𝑅𝐿𝑆 × 𝑝𝐿𝑆𝑁𝐼𝑅 𝑅𝑒𝑑𝐿𝑆 × 𝑝𝐿𝑆𝑅𝑒𝑑

(Eq. 6)

The following section will explain how NDVI is calculated using multispectral images from the P4 Multispectral’s NIR
and Red bands.

Firstly, the multispectral images from the P4 Multispectral need to be aligned due to the slight difference in position
and exposure time between different bands. Here is how:

 Step 1: Alignment of the phase difference caused by different camera locations.

In the XMP[drone-dji] image metadata file (DJI****.TIF), find [Relative Optical Center X] and [Relative
Optical Center Y]. These two items represent the x direction offset and y direction offset of the physical
positions between the current band and the NIR band in pixels. If the current band is NIR, both values will
be 0. Doing so is sufficient in correcting any differences in position between images for different bands
captured in hover mode.

 Step 2: Alignment of the difference caused by different exposure times.

Before aligning, we recommend smoothing the images using a filter such as a histogram smoothing or a
Gaussian filter, etc.
Either of the two alignment methods outlined below would work:
o Method 1. Apply an edge detection filter (ex. Sobel filter) to detect edge lines from the two
images that need to be aligned. Then, apply an alignment algorithm such as the Enhanced
Correlation Coefficient (ECC) Maximization to the images. For more information on the ECC
maximization algorithm, please refer to the following URL
https://docs.opencv.org/3.0-
beta/modules/video/doc/motion_analysis_and_object_tracking.html
o Method 2. A traditional way for alignment includes feature point detection and matching.

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 4
P4 Multispectral Image Processing Guide

Feature point detection can be performed by using algorithms such as SIFT (Scaled Invariance
Feature Transform), AKAZE, etc. An alignment matrix can be computed by using several pairs of
matched feature points, and then applying the matrix to the to-be-aligned images.

NDVI can be calculated after aligning the NIR and RED images.

We will introduce how to obtain each factor in equation 6 using the NIR band as an example. Firstly, obtain two
camera related values: 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 and 𝑝𝐶𝑎𝑚𝑁𝐼𝑅 .

(𝐼𝑁𝐼𝑅 − 𝐼𝐵𝑙𝑎𝑐𝑘𝐿𝑒𝑣𝑒𝑙 ) × 𝐶𝑜𝑟𝑟𝑒𝑐𝑡𝑖𝑜𝑛

𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 = ⁄ 𝑁𝐼𝑅𝑒𝑡𝑖𝑚𝑒 (Eq. 7)
(𝑁𝐼𝑅𝑔𝑎𝑖𝑛 ∗ 1𝑒6 )

Here,
 𝐼𝑐𝑎𝑚𝑒𝑟𝑎 and 𝐼𝐵𝑙𝑎𝑐𝑘𝑙𝑒𝑣𝑒𝑙 are the normalized raw pixel value and normalized black level value, respectively.
Since the multispectral images of the P4 Multispectral are in 16-bit format, the normalization here is to
simply divide the original number by 65535. Original black level value is 4096, which can be found in [EXIF:
black level] in the metadata.

 Correction includes Vignetting correction and Distortion calibration. We apply the Vignetting correction
model shown in Eq. 8 to the input image 𝐼(𝑥,𝑦) .

𝐼(𝑥,𝑦) × (𝑘[5] ∙ 𝑟 6 + 𝑘[4] ∙ 𝑟 5 + ⋯ + 𝑘[0] ∙ 𝑟 + 1.0) (Eq. 8)

r is the distance between pixel (x, y) and the center of the vignette in pixels, which can be obtained by

𝑟 = √(𝑥 − Center𝑋)2 + (𝑦 − Center𝑌)2 (Eq. 9)

CenterX and CenterY are coordinates of center of the vignette, which can be found from the items
[Calibrated Optical Center X] and [Calibrated Optical Center Y] in [XMP: drone-dji] in the metadata.

Matrix k shows the polynomial coefficients for Vignetting correction, which can be found from [Vignetting
Data] in [XMP: drone-dji] in the metadata.

Distortion calibration is a regular process in image processing. If the NDVI orthomosaics are created using
structure from motion (SFM) technology, which optimizes for camera distortion, this step can be skipped.

The P4 Multispectral saves parameters for Distortion calibration of each individual camera into the
metadata, which can be found in [Dewarp Data] in [XMP: drone-dji]. [k1, k2, p1, p2, k3] are the polynomial

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 5
P4 Multispectral Image Processing Guide

coefficients for the calibration, and fx, fy, cx, cy are the intrinsic parameters of camera. These 4 intrinsic
parameters and the 2 parameters obtained in the Vignetting correction step above (CenterX, CenterY) make
up the camera matrix [(fx, 0, CenterX+cx), (0, fy, CenterY+cy), (0, 0, 1)] for Distortion calibration. For more
information on Distortion Calibration, please refer to the undistort() function in OpenCV.
https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_calib3d/py_calibration/py_calibration.html

 𝑣𝑎𝑙𝑔𝑎𝑖𝑛 is the sensor gain setting (similar to the sensor ISO) which can be found as [SensorGain] in [XMP:
drone-dji] in the metadata.

 𝑣𝑎𝑙𝑒𝑡𝑖𝑚𝑒 is the camera exposure time, which can be found as [ExposureTime] in [XMP: drone-dji] in the
metadata.

We can obtain the image signal value 𝑁𝐼𝑅𝑐𝑎𝑚𝑒𝑟𝑎 by following the steps above.

Further, parameter 𝑝𝐶𝑎𝑚𝑁𝐼𝑅 can be found in [S𝑒𝑛𝑠𝑜𝑟 𝐺𝑎𝑖𝑛 𝐴𝑑𝑗𝑢𝑠𝑡𝑚𝑒𝑛𝑡] in [XMP: drone-dji].

Then, we need to obtain signal values relevant to the sunlight sensor, 𝑁𝐼𝑅𝐿𝑆 and 𝑝𝐿𝑆𝑁𝐼𝑅 , and calculate their product
𝑁𝐼𝑅𝐿𝑆 × 𝑝𝐿𝑆𝑁𝐼𝑅 . The product of 𝑁𝐼𝑅𝐿𝑆 × 𝑝𝐿𝑆𝑁𝐼𝑅 is saved as [Irradiance] in [XMP: drone-dji] in the metadata, which
can be used in Eq. 6.

These are the steps for obtaining the desired information of the NIR band. The same steps can be used for the Red
band. Finally, NDVI can be calculated using Eq. 6.

https://www.dji.com/p4-multispectral
Copyright © 2020 DJI All Rights Reserved. 6