MySQL Conference & Expo 2011
MariaDB
Dynamic Columns
Oleksandr “Sanja” Byelkin
[email protected]Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
The (web) store problem:
All items need: ID, Type, Price, Country, Manufacturer)
A T-Shirt has the following additional properties:
Size, color...
A computer has the following additional properties:
CPU, MHz, memory, Watt...
There is no easy way to store many different types into a
relational database!
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
One common solutions to this is:
● Store all the 'extra' columns in a BLOB in some format
(HTML?)
● You need a lot of extra work to manipulate the blob
● Hard to access column data (usually done in client)
● Overhead in storage (especially when you use HTML)
● All values are 'text'
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
Another common solution:
● Create one table for all the 'extra' columns:
CREATE TABLE extra
(id int auto_increment, extra_column_id char(10),
value varchar(255));
INSERT INTO items set type=“t-shirt”, price=10;
INSERT INTO extra (NULL, LAST_INSERT_ID(), “color”,
“Blue”),(NULL, LAST_INSERT_ID(), “Size”, “M”);
The problems with this approach is:
● Every access to an extra column requires a key/row
lookup
● Slow performance (if database is not optimized for this)
● Big overhead in storage (especially in index)
● Risk for errors as data is not typed
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns
Dynamic columns is a bridge between relational
databases and non relational databases
● With dynamic columns all extra columns are stored in a
packed blob, maintained by the database.
● You can add more columns, remove or query them for a
row.
● You can access columns in the server or retrieve the full
blob to the client and manipulate it there.
● You can use virtual columns to create indexes on some
values.
● True indexes for dynamic columns is planned for later.
● Implemented through functions for use by ODBC, & etc.
● First implementation uses integer to access columns.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: supported
types
● unsigned int
● int
● char [character set <cs>]
● double
● decimal [(<len>, <frac>)]
● time
● date
● datetime
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Creating a table with a dynamic column for the store:
CREATE TABLE item (
ID int auto_increment primary key,
Type_id int,
Price decimal(7,2),
Country_id int,
Manufacturer_id int,
extra blob);
Where column 'extra' is dedicated to store dynamic
columns. It could be any column able carry text.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Creating/initializing a dynamic_column:
COLUMN_CREATE(column_nr, value [as type],
[column_nr, value [as type]], ...)
INSERT into item values
(NULL, 1 /* T-shirt */, 10, 1 /* Germany */, 1 /* Nike */,
COLUMN_CREATE(1 /* color */, "Blue", 2 /* Size */, "M"));
INSERT into item values
(NULL, 2 /* computer */, 1000, 1 /* Germany */, 2 /* intel
*/,
COLUMN_CREATE(3 /* cpu */, "T9400", 5 /* MHz */, 800 as
unsigned int));
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Updating a dynamic column:
COLUMN_ADD(string, column_nr, value [as
type],
column_nr, value [as type]]...)
UPDATE item SET extra=
COLUMN_ADD(extra, 6 /* Memory */, 2048)
WHERE id=2;
If the column already exists, it will be overwritten.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Deleting a dynamic column (if it exists):
COLUMN_DELETE(string, column_nr1,
column_nr2, ...);
UPDATE item SET extra=
COLUMN_DELETE(extra, 6)
WHERE id=2;
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Querying a dynamic column:
COLUMN_EXISTS(string, column_nr);
SELECT
ID, Type_id, Price, Country_id, Manufacturer_id from item
where COLUMN_EXISTS(extra, 3);
Querying which columns exist:
COLUMN_LIST(string);
SELECT COLUMN_LIST(extra) FROM item WHERE id=1;
→ “1,2”
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Retrieving a dynamic column:
COLUMN_GET(column_nr, string as type);
SELECT id, COLUMN_GET( 1 /* color*/, extra as char)
from item;
→ 1 Blue
→ 2 NULL
You can of course also do things like:
SELECT id, COLUMN_GET(1, extra as char) FROM item
where Type_id=1
order by COLUMN_GET(1, extra as char);
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: C library
C Library allows the same manipulations with dynamic
columns on the client side.
A description can be found in the worklog
http://askmonty.org/worklog/Server-Sprint/?tid=34 and in the
source.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: encoding
How is the dynamic column encoded?
Header:
<flag><number_of_columns>
Sorted index:
<column_nr1><offset/type><column_nr2><offset/type>
Each column is stored as:
<data1><data2>
Where 'offset' is offset from beginning of data part, 'type' is
3 bits in offset.
Length of the data could be calculated by offsets of 2
neighbor fields.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: data encoding
● Unsigned integer is just a variable integer field.
● Signed integer coded to make the variable size efficient:
0 → 0
-1 → 1
1 → 2
-2 → 3
2 → 4
...
● Double, date, time, and date time are fixed-size fields
● String stores collation number and the string
● Decimal stores sizes of parts before and after decimal point
and decimal in MySQL format.
● NULL – means removing the field.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: avaliability
When will dynamic columns be available?
First version is already pushed in separate tree for testing.
lp:~maria-captains/maria/5.3-mwl34
Should be available in main MariaDB 5.3 within several
weeks.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: plans
● Adding name directory
● Adding functional indices
● Supporting popular NoSQL data exchange formats
(for example, JSON or XML)
● Adding engine support for NoSQL databases as
HBase
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Thanks
Q&A
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
