Visual Perception,
  Computation,
 and Geometry
             Jason Miller
 Associate Professor of Mathematics

      Truman State University
           12 September 2009
Outline
Outline
• a bit about me
Outline
• a bit about me
• computers & sight
Outline
• a bit about me
• computers & sight
• medical imaging and medialness
Outline
• a bit about me
• computers & sight
• medical imaging and medialness
• relative critical sets
Outline
• a bit about me
• computers & sight
• medical imaging and medialness
• relative critical sets
• subsequent work
Me
• B.A. in math from small, private liberal arts
  college
• Ph.D. in mathematics from University of
  North Carolina
• area = differentiable topology & singularity
  theory of René Thom
• “Relative Critical Sets in n-Space and their
  application to Image Analysis.”
The miracle of appropriateness of the language of
mathematics for the formulation of the laws of [science] is a
wonderful gift which we neither understand nor deserve.
We should be grateful for it, and hope that it will remain
valid for future research, and that it will extend, for better
or for worse, to our pleasure even though perhaps also to
our bafflement, to wide branches of learning.

                      — Eugene Wigner, The Unreasonable
                        Effectiveness of Mathematics
Computers & Sight
Computers & Sight


Semi-Autonomous Vehicles
Computers & Sight


Semi-Autonomous Vehicles     Descriptive and
                           Diagnostic Medicine
Computers & Sight


Semi-Autonomous Vehicles      Descriptive and
                            Diagnostic Medicine




  Automatic Annotation of
      Digital Content
Computers & Sight


Semi-Autonomous Vehicles      Descriptive and
                            Diagnostic Medicine




  Automatic Annotation of    Face Recognition,
      Digital Content       Motion Tracking, etc.
Computers & Sight

    The secret is …
Computers & Sight

     The secret is …


    They Suck at it!
Computers & Sight

       The secret is …


    They Suck at it!

   (they have no natural talent for sight)
Example: Captchas
Computers & Sight
Computers & Sight
Computers & Sight
Computers & Sight
Image Processing
• Challenges:
 Segmentation and
 Registration of Images

• Edge-based methods
• Medialness-based
 methods
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis
Medial Axis


    th
wid
Image Processing
Image Processing
Image Processing
Image Processing
Image Processing
Image Processing
• Digital images are
  collections of pixels

• Each pixel has an
  intensity



                               528 x 525 pixels
                          intensities: 0 ≤ I ≤ 255
Pixel intensity function
Pixel intensity function
Pixel intensity function
Pixel intensity function
Pixel intensity function
Pixel intensity function
Pixel intensity function




     nsity values
Inte
Image
shapes
Image     function
shapes   geometry
Image
shapes
         ←→    function
              geometry
Backstory: Why Me?
•   high-powered computer science research group!

•   they had algorithms computing medial axes of objects in
    medical images

•   dogged by some anomalous unexpected numerical
    problems

•   my advisor: “let’s figure out what should be happening”
Real            Mathematical
    World              World



  Assumptions         Mathematical
about Phenomena          Model




                  Logical Consequences
     Real           (Analyze Model)
     Data
Real                        Mathematical
    World                          World


                  translate
  Assumptions                     Mathematical
about Phenomena                      Model




                              Logical Consequences
     Real                       (Analyze Model)
     Data
Real                        Mathematical
    World                          World


                  translate
  Assumptions                     Mathematical
about Phenomena                      Model




                              Logical Consequences
     Real                       (Analyze Model)
     Data
Real                        Mathematical
    World                          World


                  translate
  Assumptions                     Mathematical
about Phenomena                      Model




                              Logical Consequences
     Real                       (Analyze Model)
     Data         compare
Real                             Mathematical
    World                               World


                       translate
  Assumptions                          Mathematical
about Phenomena                           Model

  adjust assumptions
      to improve

                                   Logical Consequences
      Real                           (Analyze Model)
      Data             compare
Relative Critical Sets
•   They extended the concept of local extrema where
                         I=0
    (vanishing derivative) to a higher dimensional set of
    points.

•   Let ei be the eigenvectors of the matrix of second
    partials of I , and λi ≤ λi+1 be the eigenvalues.

                    I · ei = 0 for i < n
                    λn−1 < 0
Image
shapes
Image     function
shapes   geometry
Image
shapes
         ←→    function
              geometry
Relative Critical Sets
 •   Used the following techniques to prove a
     structure theorem for the CS’s group’s
     medial axes

     •   wavelet theory (scale-space theory)

     •   Lie group actions

     •   transversality theorems

     •   semi-algebraic geometry

     •   combinatorics
Relative Critical Sets
 •   Used the following techniques to prove a
     structure theorem for the CS’s group’s
     medial axes

     •   wavelet theory (scale-space theory)
                                           abstract
     •   Lie group actions              mathematics in
                                          service of
     •   transversality theorems
                                        applied science
     •   semi-algebraic geometry

     •   combinatorics
Subsequent Work
•   Undergraduate Research Project on
    computing relative critical sets


•   Applied wavelets to bat echolocation project
    with Scott Burt (Biology)


•   Use medialness methods in vascular network
    project with Rob Baer (ATSU)
Subsequent Work
•   Undergraduate Research Project on
    computing relative critical sets          ramming
                            Mathem atica prog


•   Applied wavelets to bat echolocation project
    with Scott Burt (Biology)


•   Use medialness methods in vascular network
    project with Rob Baer (ATSU)
Subsequent Work
•   Undergraduate Research Project on
    computing relative critical sets           ramming
                             Mathem atica prog


•   Applied wavelets to bat echolocation project
    with Scott Burt (Biology)         assific ation and
                             sta tistical cl ethods
                                     cluster m
•   Use medialness methods in vascular network
    project with Rob Baer (ATSU)
Subsequent Work
•   Undergraduate Research Project on
    computing relative critical sets           ramming
                             Mathem atica prog


•   Applied wavelets to bat echolocation project
    with Scott Burt (Biology)         assific ation and
                             sta tistical cl ethods
                                     cluster m
•   Use medialness methods in vascular network
    project with Rob Baer (ATSU)
                                 grap h theor y
                                          ramming
                             M atlab prog

Computer Vision, Computation, and Geometry

  • 1.
    Visual Perception, Computation, and Geometry Jason Miller Associate Professor of Mathematics Truman State University 12 September 2009
  • 2.
  • 3.
  • 4.
    Outline • a bitabout me • computers & sight
  • 5.
    Outline • a bitabout me • computers & sight • medical imaging and medialness
  • 6.
    Outline • a bitabout me • computers & sight • medical imaging and medialness • relative critical sets
  • 7.
    Outline • a bitabout me • computers & sight • medical imaging and medialness • relative critical sets • subsequent work
  • 8.
    Me • B.A. inmath from small, private liberal arts college • Ph.D. in mathematics from University of North Carolina • area = differentiable topology & singularity theory of René Thom • “Relative Critical Sets in n-Space and their application to Image Analysis.”
  • 9.
    The miracle ofappropriateness of the language of mathematics for the formulation of the laws of [science] is a wonderful gift which we neither understand nor deserve. We should be grateful for it, and hope that it will remain valid for future research, and that it will extend, for better or for worse, to our pleasure even though perhaps also to our bafflement, to wide branches of learning. — Eugene Wigner, The Unreasonable Effectiveness of Mathematics
  • 10.
  • 11.
  • 12.
    Computers & Sight Semi-AutonomousVehicles Descriptive and Diagnostic Medicine
  • 13.
    Computers & Sight Semi-AutonomousVehicles Descriptive and Diagnostic Medicine Automatic Annotation of Digital Content
  • 14.
    Computers & Sight Semi-AutonomousVehicles Descriptive and Diagnostic Medicine Automatic Annotation of Face Recognition, Digital Content Motion Tracking, etc.
  • 15.
    Computers & Sight The secret is …
  • 16.
    Computers & Sight The secret is … They Suck at it!
  • 17.
    Computers & Sight The secret is … They Suck at it! (they have no natural talent for sight)
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
    Image Processing • Challenges: Segmentation and Registration of Images • Edge-based methods • Medialness-based methods
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
    Image Processing • Digitalimages are collections of pixels • Each pixel has an intensity 528 x 525 pixels intensities: 0 ≤ I ≤ 255
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
    Pixel intensity function nsity values Inte
  • 48.
  • 49.
    Image function shapes geometry
  • 50.
    Image shapes ←→ function geometry
  • 51.
    Backstory: Why Me? • high-powered computer science research group! • they had algorithms computing medial axes of objects in medical images • dogged by some anomalous unexpected numerical problems • my advisor: “let’s figure out what should be happening”
  • 52.
    Real Mathematical World World Assumptions Mathematical about Phenomena Model Logical Consequences Real (Analyze Model) Data
  • 53.
    Real Mathematical World World translate Assumptions Mathematical about Phenomena Model Logical Consequences Real (Analyze Model) Data
  • 54.
    Real Mathematical World World translate Assumptions Mathematical about Phenomena Model Logical Consequences Real (Analyze Model) Data
  • 55.
    Real Mathematical World World translate Assumptions Mathematical about Phenomena Model Logical Consequences Real (Analyze Model) Data compare
  • 56.
    Real Mathematical World World translate Assumptions Mathematical about Phenomena Model adjust assumptions to improve Logical Consequences Real (Analyze Model) Data compare
  • 57.
    Relative Critical Sets • They extended the concept of local extrema where I=0 (vanishing derivative) to a higher dimensional set of points. • Let ei be the eigenvectors of the matrix of second partials of I , and λi ≤ λi+1 be the eigenvalues. I · ei = 0 for i < n λn−1 < 0
  • 59.
  • 60.
    Image function shapes geometry
  • 61.
    Image shapes ←→ function geometry
  • 62.
    Relative Critical Sets • Used the following techniques to prove a structure theorem for the CS’s group’s medial axes • wavelet theory (scale-space theory) • Lie group actions • transversality theorems • semi-algebraic geometry • combinatorics
  • 63.
    Relative Critical Sets • Used the following techniques to prove a structure theorem for the CS’s group’s medial axes • wavelet theory (scale-space theory) abstract • Lie group actions mathematics in service of • transversality theorems applied science • semi-algebraic geometry • combinatorics
  • 64.
    Subsequent Work • Undergraduate Research Project on computing relative critical sets • Applied wavelets to bat echolocation project with Scott Burt (Biology) • Use medialness methods in vascular network project with Rob Baer (ATSU)
  • 65.
    Subsequent Work • Undergraduate Research Project on computing relative critical sets ramming Mathem atica prog • Applied wavelets to bat echolocation project with Scott Burt (Biology) • Use medialness methods in vascular network project with Rob Baer (ATSU)
  • 66.
    Subsequent Work • Undergraduate Research Project on computing relative critical sets ramming Mathem atica prog • Applied wavelets to bat echolocation project with Scott Burt (Biology) assific ation and sta tistical cl ethods cluster m • Use medialness methods in vascular network project with Rob Baer (ATSU)
  • 67.
    Subsequent Work • Undergraduate Research Project on computing relative critical sets ramming Mathem atica prog • Applied wavelets to bat echolocation project with Scott Burt (Biology) assific ation and sta tistical cl ethods cluster m • Use medialness methods in vascular network project with Rob Baer (ATSU) grap h theor y ramming M atlab prog

Editor's Notes

  • #21 digital pictures are messy object boundaries are not well defined
  • #22 digital pictures are messy object boundaries are not well defined
  • #23 digital pictures are messy object boundaries are not well defined
  • #24 big problems in computer vision
  • #56 differential calculus
  • #57 differential calculus
  • #58 differential calculus
  • #67 there are problems when the eigenvalues are equal or vanish (I put these here because a sophomore mathematics major can understand them)
  • #72 but mostly I just retool myself, learn new mathematical tools
  • #73 but mostly I just retool myself, learn new mathematical tools
  • #74 but mostly I just retool myself, learn new mathematical tools