Geek Sync I Need for Speed: In-Memory Databases in Oracle and SQL Server

Geek Sync I Need for Speed: In-Memory Databases in Oracle and SQL Server

• 1.
  In Memory Databases in SQLServer and Oracle May 4, 2016
• 2.
  Who Are We BertScalzo • Senior PM with IDERA • Author (10 books), Speaker, Oracle subject matter expert Vicky Harp • Strategist with IDERA • Developer, Speaker, SQL Server subject matter expert
• 3.
  Agenda • Introductions • Whatare In Memory Databases • In Memory Databases in Oracle • In Memory Databases in SQL Server • Wrap-Up
• 4.
  In Memory Databases •Databases or table structures which reside entirely or primarily in main memory • Able to make efficient use of the huge RAM storage available on modern systems • Able to utilize structures and algorithms which would be inefficient on disk • May or may not be fully ACID compliant • May be 100s of times faster
• 5.
  In Memory forOracle
• 6.
  In Memory forOracle • In Memory Column Store (IMCS) • New Feature Oracle 12c 2nd release – 12.1.0.2.0 – July 214 – not in 12.1.0.1.0 – 2013 • The IMCS is a new static pool in the SGA. • Data in IMCS does not reside in traditional row format, instead, in a columnar format. • Each column is stored as a separate structure. • IMCS does not replace the buffer cache, but it acts as a supplement so data can be stored in memory in both row & columnar formats. • To enable IMCS, INMEMORY_SIZE init parameter must = a non-zero value. • The IMCS is included with the Oracle Enterprise Edition • IMCS is an option (i.e. not free, costs $$$ like partitioning).
• 7.
  IMCS Pros &Cons IMCS can apply to: • Column • Table • Materialized View • Tablespace • Partition IMCS is good for: • A query that scans a large number of rows and applies filters that use operators such as: =, <, >, and IN • A query that selects a small number of columns from a table or materialized view with a large number of columns, such as a query that selects five columns from a table with 100 columns (e.g. DW Dimension) • A query that joins a small table to a large table (e.g. DW Dimension) • A query that aggregates data (e.g. DW fact)
• 8.
  IMCS Pros &Cons Availability: • 12c 12.1.0.2 • Enterprise Edition • EE Option $$$ (see next slide) IMCS is not good for: • IM is NOT good for: • Queries with complex predicates • Queries that selects lots of columns • Queries that return lots of rows • Queries with multiple large table joins • Objects owned by SYS and stored in SYSTEM or SYSAUX tablespaces
• 9.
  Oracle Price Book
• 10.
  Oracle SGA As OracleDatabase populates data in the IM column store and converts this data to columnar format, the columnar version of the table gradually becomes available. If a query requires data that has not yet been populated in the IM column store, then the database reads the requested data from the buffer cache or disk while background processes asynchronously populate the missing data in the IM column store.
• 11.
  Oracle Parameter #1
• 12.
  Oracle Parameter #2
• 13.
  Oracle Parameter #3
• 14.
  Oracle Parameter #4
• 15.
  SGA Memory Considerations 15 Greenstar merely indicates What I chose for my tests
• 16.
  Compression Options
• 17.
  Loading Options When adatabase is restarted, all of the data for database objects with a priority level other than NONE is populated in the IM column store during startup.
• 18.
  Test Approach The TPCBenchmark™ H (TPC-H) is a decision support benchmark. It consists of a suite of business-oriented ad- hoc queries and concurrent data modifications. The queries and the data populating the database were chosen to have broad industry-wide relevance, while maintaining a sufficient degree of ease of implementation. This benchmark illustrates decision support systems that do the following: – Examine large volumes of data; – Execute queries with a high degree of complexity; – Give answers to critical business questions. http://www.tpc.org/tpc_documents_current_versions/pdf/tpch2.17.1.pdf
• 19.
  Example Queries National MarketShare Query (Q8) Volume Shipping Query (Q7)
• 20.
  Test Results Test # Feature Debut ConfigurationParameters Run Time (Mins:Secs) Compress Factor % Run Time Reduction 1 SGA_TARGET = 4G *** Nothing Else–Baseline *** 45:52 2 11g SGA_TARGET = 4G DB_FLASH_CACHE_SIZE = 16G DB_FLASH_CACHE_FILE = '/flash/flash01.dbf' 23:19 49% 3 SGA_TARGET = 20G *** Nothing Else – All memory buffer cache *** 19:50 57% 4 12c SGA_TARGET = 20G DB_BIG_TABLE_CACHE_PERCENT_TARGET = 80 PARALLEL_DEGREE_POLICY=auto ORA-600 5 12c SGA_TARGET = 20G INMEMORY_SIZE = 16G INMEMORY (default) PRIORITY CRITICAL (restart instance & wait for object to load) 05:01 3.63X 89% 6 12c SGA_TARGET = 20G INMEMORY_SIZE = 16G INMEMORY MEMCOMPRESS FOR QUERY HIGH PRIORITY CRITICAL (restart instance & wait for object to load) 04:56 4.68X 89%
• 21.
  Script #1 -- im_tabs.sql setlinesize 256 set pagesize 999 set verify off col OBJECT format a30 SELECT owner||'.'||table_name OBJECT, inmemory INMEMORY, inmemory_priority PRIORITY, inmemory_distribute DISTRIBUTE, inmemory_compression COMPRESSION, inmemory_duplicate DUPLICATE FROM all_tables where owner like upper('%&1%') ORDER BY inmemory, owner||'.'||table_name; SQL> @im_tabs BMF1 OBJECT INMEMORY PRIORITY DISTRIBUTE COMPRESSION DUPLICATE ------------------------------ -------- -------- --------------- ----------------- ------------ BMF1.H_CUSTOMER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_LINEITEM ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_NATION ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_ORDER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_PART ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_PARTSUPP ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_REGION ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE BMF1.H_SUPPLIER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE First, we need to know which tables for our test schema (BMF1) have been “in-memory” enabled and with which options.
• 22.
  Script #2 -- im_segs.sql setlinesize 256 set pagesize 999 set verify off col owner format a20 col segment_name format a30 select owner, segment_name, populate_status from v$im_segments; SQL> @im_segs OWNER SEGMENT_NAME POP STATU -------------------- ------------------------------ --------- BMF1 H_LINEITEM STARTED BMF1 H_SUPPLIER COMPLETED BMF1 H_PARTSUPP COMPLETED BMF1 H_ORDER COMPLETED BMF1 H_PART COMPLETED BMF1 H_CUSTOMER COMPLETED Second, we need to know when our “in-memory” enabled tables have been successfully loaded. In my tests, I waited until all such enabled tables were COMPLETED to more fully leverage the in- memory feature.
• 23.
  Script #3 -- im_pct.sql setlinesize 256 set pagesize 999 set verify off select segment_name,ROUND(SUM(BYTES)/1024/1024/1024,2) "ORIG GB", ROUND(SUM(INMEMORY_SIZE)/1024/1024/1024,2) "IN-MEM GB", ROUND(SUM(BYTES-BYTES_NOT_POPULATED)*100/SUM(BYTES),2) "% IN_MEM", ROUND(SUM(BYTES-BYTES_NOT_POPULATED)/SUM(INMEMORY_SIZE),2) "COMP RATIO" from V$IM_SEGMENTS group by owner,segment_name order by SUM(bytes) desc; SQL> @im_pct -- Try #1 - INMEMORY (default) SEGMENT_NAME ORIG GB IN-MEM GB % IN_MEM COMP RATIO ------------------------------ ---------- ---------- ---------- ---------- H_LINEITEM 8.42 2.32 100 3.63 H_ORDER 1.75 .89 100 1.97 H_PARTSUPP .6 .35 100 1.71 H_PART .29 .13 100 2.29 H_CUSTOMER .22 .19 100 1.19 H_SUPPLIER .01 .02 100 .91 SQL> @im_pct -- Try #2 - INMEMORY MEMCOMPRESS FOR QUERY HIGH SEGMENT_NAME ORIG GB IN-MEM GB % IN_MEM COMP RATIO ------------------------------ ---------- ---------- ---------- ---------- H_LINEITEM 8.42 1.77 98.17 4.68 H_ORDER 1.75 .63 100 2.78 H_PARTSUPP .6 .32 100 1.86 H_PART .29 .09 100 3.16 H_CUSTOMER .22 .15 100 1.52 H_SUPPLIER .01 .01 100 1.31 wait Third, we need to know how well our “in- memory” enabled tables compressed to confirm the Oracle statement that the in-memory feature does not add much overhead (20%).
• 24.
  In Memory forSQL Server
• 25.
  In Memory Databasesin SQL Server • First appearance in SQL 2014 “Hekaton” • Enterprise Edition feature • Increased functionality in SQL 2016 • Called by Many Names – In-Memory OLTP – Memory Optimized Tables – Hekaton (or just HK)
• 26.
  SQL Server In-MemoryOLTP • A distinct memory-optimized engine within SQL Server • Distinct storage and memory structures from the rest of SQL Server • Completely lock-less design eliminates blocking • ACID compliant with optional durability • Interacts with traditional TSQL or new natively-compiled stored procedures
• 27.
  SQL Server In-Memory Concepts •Memory Optimized Tables • Durability Options • T-SQL Interop • Natively Compiled Stored Procedures • Memory Optimized Table Types
• 28.
  Memory Optimized Tables •Created on a MEMORY_OPTIMIZED_DATA filegroup • Completely distinct physical structure than on-disk tables • Tables are compiled to a DLL on creation • Data and indexes fully resident in RAM • Fully optimistic (lockless) concurrency
• 29.
  Memory Optimized Table Structure •Data is written to and read from hash buckets in memory • Table data is persisted to the transaction log and periodically flushed to disk • Indexes are maintained only in memory and never persisted
• 30.
  Memory Optimized Table Limitations •Long list of miscellaneous unsupported features and commands – Computed columns, clustered indexes, sparse columns, DDL triggers, full text indexing, truncate, data compression, dbcc checkdb • Refer to documentation
• 31.
  Durability Options • SCHEMA_AND_DATA –Changes are logged in transaction log – Table structure and data are persisted on disk – ACID compliant • SCHEMA_ONLY – Changes are not logged – Table structure is persisted – Data is not persisted – Atomic, Consistent, and Isolated but not Durable
• 32.
  T-SQL Interop • AccessMemory-Optimized tables directly using regular T-SQL • Allows “cross-container” access • Most (but not all) T-SQL language constructs are supported • Only certain isolation levels supported – Snapshot (with caveats) – Repeatable Read – Serializable
• 33.
  Natively Compiled Stored Procedures •Stored procedures written in T-SQL and compiled directly to DLLs • Do not incur compilation overhead • Much, much faster • Many unsupported T-SQL features – LIKE, CONTAINS, PIVOT, EXCEPT, common table expressions, cursors, select into, temp tables, etc
• 34.
  Memory Optimized TableTypes • May be used as a temporary table but without making use of tempdb • May be desirable for temporary objects which can reside entirely in memory
• 35.
  Wrap Up
• 36.
  Wrap Up • Oracle(key points) – Point 1 – Point 2 – Point 3 • SQL Server Memory-Optimized Tables – Distinct engine and structure with lockless design – Optional table durability – Optional native compiled stored procedures
• 37.
  Thank you! • BertScalzo – [email protected] • Vicky Harp – [email protected] • http://community.idera.com