Data Science, Data Curation, and Human-Data Interaction

Data Science, Data Curation, and Human-Data Interaction

• 1.
  Data Science, Data Curation, Human-DataInteraction Bill Howe, Ph.D. Associate Professor, Information School Adjunct Associate Professor, Computer Science & Engineering Associate Director and Senior Data Science Fellow, eScience Institute 7/26/2016 Bill Howe, UW 1
• 2.
  Dave Beck Director ofResearch, Life Sciences Ph.D. Medicinal Chemistry, Biomolecular Structure & Design Jake VanderPlas Director of Research, Physical Sciences Ph.D., Astronomy Valentina Staneva Data Scientist Ph.D., Applied Mathematics and Statistics Ariel Rokem Data Scientist Ph.D., Neuroscience Andrew Gartland Research Scientist Ph.D., Biostatistics Bryna Hazelton Research Scientist Ph.D., Physics Bernease Herman Data Scientist BS, Stats was SE at Amazon Vaughn Iverson Research Scientist Ph.D., Oceanography Rob Fatland Director of Cloud and Data Solutions Senior Data Science Fellow PhD Geophysics Joe Hellerstein Senior Data Science Fellow IBM Research, Microsoft Research, Google (ret.) Data Scientists Research Scientists Research Faculty Cyberinfrastructure Brittany Fiore-Gartland Ethnographer Ph.D Communication Dir. Ethnography http://escience.washington.edu
• 3.
  Time Amountofdataintheworld Time Processingpower What is therate-limiting step in data understanding? Processing power: Moore’s Law Amount of data in the world
• 4.
  Processingpower Time What is therate-limiting step in data understanding? Processing power: Moore’s Law Human cognitive capacity Idea adapted from “Less is More” by Bill Buxton (2001) Amount of data in the world slide src: Cecilia Aragon, UW HCDE
• 5.
  How much timedo you spend “handling data” as opposed to “doing science”? Mode answer: “90%” 7/26/2016 Bill Howe, UW 5
• 7.
  7/26/2016 Bill Howe,UW 8 Goal: Understand and optimize how people use and share quantitative information “Human-Data Interaction”
• 8.
  The SQLShare Corpus: Amulti-year log of hand-written SQL queries Queries 24275 Views 4535 Tables 3891 Users 591 SIGMOD 2016 Shrainik Jain https://uwescience.github.io/sqlshare
• 9.
  lifetime = daysbetween first and last access of table SIGMOD 2016 Shrainik Jain http://uwescience.github.io/sqlshare/ Data “Grazing”: Short dataset lifetimes
• 10.
  MYRIA: POLYSTORE MGMT Human-DataInteraction 7/26/2016 Bill Howe, UW 18
• 11.
  R A GK Modern Big Data Ecosystems many different platforms, complex analytics
• 12.
  Myria Algebra Tables KeyValArrays Graphs RACO: Relational Algebra COmpiler
• 13.
  Spark Accumulo CombBLASGraphX Parallel Algebra Logical Algebra RACO Relational Algebra COmpiler CombBLAS API Spark API Accumulo Graph API rewrite rules Array Algebra MyriaL Services: visualization, logging, discovery, history, browsing Orchestration https://github.com/uwescience/raco
• 14.
  7/26/2016 Bill Howe,UW 22 ISMIR 2016
• 15.
  Laser Microscope Objective Pine HoleLens Nozzle d1 d2 FSC (Forward scatter) Orange fluo Red fluo SeaFlow Francois Ribalet Jarred Swalwell Ginger Armbrust
• 16.
  7/26/2016 Bill Howe,UW 25
• 17.
  Ashes CAMHD http://novae.ocean.washington.edu/story/Ashes_CAMHD_Live
• 18.
  Extract synchronized slices Co-register (camerajitter, bad time synch) Separate fore- and back-ground Classify critters in the foreground Measure growth rate over time
• 19.
  “DEEP” CURATION Human-Data Interaction
• 20.
  Microarray experiments
• 21.
  7/26/2016 Bill Howe,UW 33 Microarray samples submitted to the Gene Expression Omnibus Curation is fast becoming the bottleneck to data sharing Maxim Gretchkin Hoifung Poon
• 22.
  color = labelssupplied as metadata clusters = 1st two PCA dimensions on the gene expression data itself Can we use the expression data directly to curate algorithmically? Maxim Gretchkin Hoifung Poon The expression data and the text labels appear to disagree
• 23.
  Maxim Gretchkin Hoifung Poon Better Tissue TypeLabels Domain knowledge (Ontology) Expression data Free-text Metadata 2 Deep Networks text expr SVM
• 24.
  Deep Curation Maxim Gretchkin HoifungPoon Distant supervision and co-learning between text-based classified and expression-based classifier: Both models improve by training on each others’ results. Free-text classifier Expression classifier
• 25.
  VIZIOMETRICS: COMPREHENDING VISUAL INFORMATION INTHE SCIENTIFIC LITERATURE Human-Data Interaction 7/26/2016 Bill Howe, UW 37
• 28.
  Observations • Figures inthe literature are the currency of scientific ideas • Almost entirely unexplored • Our thought: Mine patterns in the visual literature
• 29.
  Step 1: DismantlingComposite Figures Poshen Lee ICPRAM 2015
• 30.
  Step 2: Classification •Divide images into small patches • Take a random sample • Run k-means on samples (k = 200) • For each figure in training set, generate a length-200 feature vector by similarity to clusters. Train a model. • For each test image, create the vector and classify by the model
• 32.
  Do high-impact papershave fewer equations, as indicated by Fawcett and Higginson? (Yes) Poshen LeeJevin West high impact papers low impact papers
• 33.
  Do high-impact papershave more diagrams? (Yes) Poshen LeeJevin West
• 34.
  Do papers intop journals tend to involve more or less visual information? (More) Poshen LeeJevin West
• 37.
  7/26/2016 Poshen Lee,UW 52 viziometrics.org
• 38.
  7/26/2016 Poshen Lee,UW 53 Burrows-Wheeler Alignment Computation DNA Sequencing Citations: 7807 +11 since 2016 Eigenfactor: 0.0000574719 DNA Methylation Brain Cancer Chromosomal Aberrations Cancer Genome Atlas Citations: 2094 +7 since 2016 Eigenfactor: 0.0000279023 Memory-efficient Computation DNA Sequencing Citations: 7459 +17 since 2016 Eigenfactor: 0.0000875579 Molecular biology GeneticsGenomics DNA Citations: 3766 +15 since 2016 Eigenfactor: 0.0000183255 viziometrics.org
• 39.
  INFORMATION EXTRACTION FROM FIGURES Information-criticalfigures Metabolic pathway diagrams Phylogenetic heat maps Architecture diagrams
• 40.
  Sean Yang
• 41.
  Normalize Sean Yang
• 42.
  Corner Detection LineDetection
• 43.
  Extract Tree Structure SeanYang
• 44.
  VISUALIZATION RECOMMENDATION 7/26/2016 Bill Howe,UW 59
• 45.
  60
• 46.
  Example of aLearned Rule (1) low x-entropy => bad scatter plot 7/26/2016 Bill Howe, UW 61 bad scatter plotgood scatter plot
• 47.
  Example of aLearned Rule (3) 63 high x-periodicity => timeseries plot (periodicity = 1 / variance in gap length between successive values)
• 48.
  Voyager 7/26/2016 Bill Howe,UW 64 Kanit “Ham” Wongsuphasaw at Dominik Moritz InfoVis 15 Jeff Heer Jock Mackinlay Anushka Anand
• 49.
  SCALABLE GRAPH CLUSTERING 7/26/2016 BillHowe, UW 65
• 50.
  Seung-Hee BaeScalable Graph Clustering Version1 Parallelize Best-known Serial Algorithm ICDM 2013 Version 2 Free 30% improvement for any algorithm TKDD 2014 SC 2015 Version 3 Distributed approx. algorithm, 1.5B edges
• 51.
  Recap • “Human-Data Interaction”is the bottleneck! – SQLShare: Mining SQL logs to uncover user behavior – Myria/RACO: Polystore Optimization – Deep Curation: Zero-training labeling of scientific datasets – Viziometrics: Mining the scientific literature – Voyager: Visualization Recommendation – GossipMap: Scalable Graph Clustering
• 52.
  http://myria.cs.washington.edu http://uwescience.github.io/sqlshare/ https://github.com/vega/voyager Voyager @billghowe github: billhowe http://homes.cs.washington.edu/~billhowe/
• 53.
  • OCCs:Big Data/ Database researcher with broad impact and expertise in research data management, • Democratizing Data Science – Ourselves: Reduce overhead in attention-scarce regimes – Other fields: Reduce overhead of interdisciplinary research – The public: Reduce overhead of communicating with the public and policymakers • SQLShare – Why? What? Impact? – Key: RDM, NSF-funded, hundreds of users – Are these workloads any different than a typical database? • HaLoop – Why? What? Impact? – Key: Papers, new subfield in big data • Myria – Why? What? Impact? – Key: Funding • Viziometrics – Why? What? Impact? • Data Curation through an Algorithmic Lens – Why? What? Impact? – Volume, variety, velocity. Volume: tasks that scale with the number of records: movement, validation. Variety: tasks that scale with the number of datasets: metadata attachment, cataloging, metadata verfication. Velocity: tasks that scale with the time since release. Data journalism, legal cases – Example? Maxim’s work. Prevalence of missing and incorrect labels. – Is this dataset what it says it is? – Why? Reproducibility crisis – Is this fully automatic? No. Training data, computational steering
• 54.
  • http://www.urbanlibraries.org/living-voters- guide--librarians-as-fact-checkers-innovation- 722.php?page_id=167
• 55.
  • http://engage.cs.washington.edu/
• 56.
  • https://www.commerce.gov/datausability/
• 57.
  • Available – Canyou get it if you know where to look? • Discoverable – Can you get it if you don’t know where to look? • Manipulable – What can you do with it, besides download it? Can the structure be readily parsed and transformed? • Interpretable – Is the information internally consistent with respect to provenance, metadata, column names, etc.? • Contextualizable – Is the information externally consistent with respect to other related datasets? Can it be connected to other datasets through standards or conventions? Does it admit connections to other datasets
• 58.
  Services emphasizing discovery, citation,and preservation
• 59.
  Query, Viz, andAnalytics Services Google Fusion Tables
• 60.
  PredictDownload Query JoinVisualize url doi tags space and time ontologies standards
• 61.
  Server software, locallyinstalled
• 63.
  • ISMIR paper
• 64.
  • Allen Instituteexample: • flexibility gap between high level and low level – Domain-specific languages http://casestudies.brain- map.org/ggb#section_explorea http://blog.ibmjstart.net/2015/08/22/dynamic -dashboards-from-jupyter-notebooks/
• 65.
  Time Amountofdataintheworld Time Processingpower What is therate-limiting step in data understanding? Processing power: Moore’s Law Amount of data in the world
• 66.
  Processingpower Time What is therate-limiting step in data understanding? Processing power: Moore’s Law Human cognitive capacity Idea adapted from “Less is More” by Bill Buxton (2001) Amount of data in the world slide src: Cecilia Aragon, UW HCDE
• 67.
  A Typical DataScience Workflow 1) Preparing to run a model 2) Running the model 3) Interpreting the results Gathering, cleaning, integrating, restructuring, transforming, loading, filtering, deleting, combining, merging, verifying, extracting, shaping, massaging “80% of the work” -- Aaron Kimball “The other 80% of the work”
• 68.
  How much timedo you spend “handling data” as opposed to “doing science”? Mode answer: “90%” 7/26/2016 Bill Howe, UW 93