Real Time Image Warper using Digilent Spartan 3 FPGA
By Shane Agnew Supervisor: Dr. Fearghal Morgan Final Year Project
Oct 1, 2012
Contents
Initial aims / Specifications Introduction Camera Bitmaps VHDL Graphical User Interface Future Work Summary
Oct 1, 2012 2
Initial aims / Specifications
Generate image data Incorporate a communication protocol into the design Perform all required file formatting Implement digital signal processing on-board the FPGA Develop a real-time multi-functional video processing application
Oct 1, 2012
Introduction
8 - b it B M P
ra w 2b m p
R A W O u tp u t F ile
S p a rt a n 3 S ta r te r B o a r d
file R e a d
S p a rt a n 3 F P G A
U S B 2 .0 P e r ip h e r a B l C o m m u n ic a t io n s 1 M o d u le
D P IM R E F
3FFFFh E m p ty
file W r ite
3000h
D a t C t r lr IO C S R B lk
A p p lie d V H D L U S B S im p le V B G U I
R A W In p u t F ile
2000h
D S P B lk
M e m C t r lr
1000h
C a m e ra
2 4 - b it B M P
b m p C o n v e rt e r 2 4 to 8 B it
8 - b it B M P
b m p 2 ra w
0000h SRAM 0 SRAM 1
S R A M M e m o ry C o n te n t
Oct 1, 2012
Camera
Trust 120 Spacecam
Standard web camera 24-bit BMP Output Software provided
24-bit BMPs
Simple to use Uncompressed data Most web-cameras support them 1 byte for each of red, green and blue, hence RGB value
Oct 1, 2012
Bitmaps
8-bit BMPs
Limited SRAM memory available 3 times less communication overhead 3 times less file access Less awkward to process and store in SRAM Single byte represents a single pixel
Oct 1, 2012
Bitmaps Greyscale
Greyscale
256 colours available Black represents 0, white represented by 255 24-bit BMP converted to 8-bit by processing each pixels RGB value
Oct 1, 2012
Bitmaps Colours
Colours
256 Colours created using RGB values from 0% 0% 0% to 100% 100% 100%, e.g. 20% red 0% green 20% blue 24-bit pixels rounded to these boundaries Colours sorted by hue to give a natural colour progression
Oct 1, 2012 8
VHDL AppliedVHDLUSBSimple System Overview
ramDone txDat(7:0)
8
DPIMREF 1.0
Register1(7:0) mclk astb dstb pwr pdb(7:0) pwait Register0(7:0) rxDatValid txDatAck 8 8 8
mclk astb dstb pwr clk
pdb(7:0) pwait dspAddRange(15:0)
32
ramDone datFromRam(31:0) dspAddRange(15:0) 16
dspBlk 3.0
dspDone dspDat2Ram(31:0) 32
dspDone
dspFunc(7:0)
dspFunc(7:0) 8 dspActive
dspRamAdd(17:0) 18 dspRamRd dspRamWr dspDat2Ram(31:0) 32 18 dspRamAdd(17:0) dspRamRd dspRamWr dspActive
memCtrlr 5.0
ramDone datFromRam(31:0) ramAdd(17:0) ramIO(31:0) 32 18 32 CE1L UB1L LB1L WEL
rst
clk rst
ramAdd(17:0) ramIO(31:0) CE1L UB1L LB1L WEL OEL CE0L UB0L LB0L
8 32
IODat2Ram(31:0) IORamWr IORamRd datFromRam(31:0) 32 rxDat(7:0) 8 rxDatAck txDatValid ldByteCnt ldIODat2RamReg csrTask 8 csrOut(7:0) 8 IODat2Ram(31:0) 32 dPathDone txDat(7:0) 8
OEL CE0L UB0L LB0L
IOCSRBlk 4.0
rxDat(7:0) 8 rxDatValid txDatAck dspActive rxDatAck txDatValid ldByteCnt ldIODat2RamReg csrTask
datCtrlr 2.0
IORamAdd(17:0) 18
clk txDat(7:0)
dPathDone
dPathDone dspDone ramDone maxCount(15:0)
16
dPathDone dspDone ramDone maxCount(15:0)
csrOut(7:0) IORamWr IORamRd addrCnt(15:0) 16 CSR(7:0)
clk rst
clk rst
addrAndMaxCount 6.0
8 8 8 8 8 8
Oct 1, 2012
8 8
CSR7(7:0) : dspFunc(7:0) CSR6(7:0) : dspAddRange(7:0) CSR5(7:0) : dspAddRange(15:8) CSR4(7:0) : endIOAddr(7:0) CSR3(7:0) : endIOAddr(15:8) CSR2(7:0) : startIOAddr(7:0) CSR1(7:0) : startIOAddr(15:8) XXXXXX CSR0(1:0) : quadrant For Reference
16
& & &
8 16
dspFunc(7:0) dspAddRange(15:0)
maxCount(15:0)
16 16 2
&
18
VHDL DPIMREF
m em R dVa l 1.3
8 8 00000001
m em RdValid
m em W rVal 1.4
00000000
8 8
m em W rValid
m em RdValid m em W rValid N S & O utput D ecode
CS
d p im refF S M 1.1
txDatA ck rxD atValid
tx D atA ck rxD atVa lid pw ait
0 1
busEppO ut
pdb
8 8
m clk dstb astb pw r clk
clkM ain ctlEppDstb ctlEppAstb FS M C om b
NS ctlE ppW ait ctlE ppD ir FSM Synch ctlE pp Awr ctlE ppD wr
ctlE ppW r
d p im refR eg iste rR W 1.2
busEppIn 8 ctlEppAw r ctlEppD wr reg ister0 busEppD ata 8 regEppA dr 8
reg ister0
8
register1
8 ctlEppW r
register1
clk
Oct 1, 2012
10
VHDL DPIMREF
Sample provided by Digilent Provides VHDL side USB interface with USB 2.0 Peripheral Device Altered to generate rxDatValid and txDatAck signals These signals along with the rxDat and txDat input and output data lines allow communication with rest of the appliedVHDLUSBSimple system
Oct 1, 2012 11
VHDL IOCtrlr
allReadDone 4.1.3 maxCount
16 16
>
allRdDone
allWriteDone 4.1.4
+1 16 16 16
>
allWrDone
X0000" 16
addrCnt 4.1.2 Din Cnt(15:0) Ld CE
addrCnt(15:0)
16
ioCtrlrFSM 4.1.1
allRdDone allWrDone ldAddrCnt incAddrCnt ldIODat2RamReg
clk rst ldIODat2RamReg txDatValid rxDatAck ldByteCnt IORamWr IORamRd dspActive enCSRWr rxDat(6:4) rxDat(1)
3 1
txDatAck rxDatValid dPathDone dspDone ramDone rxDat(7:0)
8
txDatAck rxDatValid dPathDone dspDone FSM Comb ramDone
CS
txDatValid rxDatAck
NS & Output Decode
NS
ldByteCnt IORamWr IORamRd FSM Synch dspActive enCSRWr regCSRInfo clrCSRInfo
rxDat(3:0)
4
rxDat(3:0)
cmdReg 4.1.5
D0(2:0) D1 Ld clr Q0(2:0) Q1 3
clk rst
4
csrAdd(2:0) csrTask
clk rst
rxDat(1) & rxDat(6:4)
Oct 1, 2012
12
VHDL IOCtrlr
Responds to USB transfer signals, rxdatValid and txDatAck, generated by the DPIMREF Allows the system registers (CSRs) to be setup, both DSP and Input/Output Keeps track of the total number of bytes written or read from SRAM and increments the current SRAM address accordingly Controls DSP activation
Oct 1, 2012 13
VHDL DSPBlk
intDSPFunc (int)
A 32 B 32
B-A 32
dDatFromRam(31:0)
datFromRam(31:0)
32
32 0 32 32 32 32 32 1 2 3 4 32
This ANDing reverses the order of bytes -> backwards
dDatFromRam(31:24) dDatFromRam(23:16) dDatFromRam(15:8) dDatFromRam(7:0)
8 8 8 8
&
dspRamDat(31:0)
32
0 32 32 1 32
imageInvDSPWr
Conv_std_logic_vector
0 16 16 16 1 16
dspRamAdd(15:0)
cnt
(int)
%2
16
dspAddRange(15:0)
16
B-A
Oct 1, 2012
14
VHDL DSPBlk
Six DSP functions implemented:
Delta function Colour inversion Flip image over y-axis Distortion Morphing Colour change
Effective demonstrations of the Spartan 3s power but more complex DSP functions are possible
Oct 1, 2012 15
Graphical User Interface
Oct 1, 2012
16
Graphical User Interface
Allows a user to check the contents of any SRAM location Can activate the camera software to take pictures Image loading and conversion from 24-bit to 8-bit BMPs DSP options as mentioned previously Option to perform all operations in colour or greyscale
Oct 1, 2012
17
Future Work
Implement more complicated DSP, including:
Edge detection Object Extraction Object warping
Provide networking solution to allow Electronics department to host the project and make it accessible anywhere in the world
Oct 1, 2012
18
Summary
Initial aims / Specifications Introduction Camera Bitmaps VHDL Graphical User Interface Future Work Summary
Oct 1, 2012 19
