Fixing The Worst DB in The World

Fixing the Worst Database in the World
Chris Saxon, @ChrisRSaxon & @SQLDaily

blogs.oracle.com/sql
youtube.com/c/TheMagicofSQL
asktom.oracle.com
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | blogs.oracle.com/sql www.youtube.com/c/TheMagicOfSQL @ChrisRSaxon
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Photo by Ken Treloar on Unsplash
Tables are the
of your database
Get them wrong
and…
…you need to
be a SQL
with attrs as (
select pr.productno, attrs.*
from products pr,
json_table (
productinfo , '$' null on error
columns (
description path '$.description'
,colour path '$.colour'
,weight path '$.weight'
,height path '$.dimensions.height'
,width path '$.dimensions.width'
,depth path '$.dimensions.depth'
)
) attrs
union all
select productno, description, colour,
weight, height, width, depth
from (
select pr.productno, pa.attrname, pa.attrvalue from (
select productno from products
) pr
left join (
select productno, attrname, attrvalue
from product_attributes
) pa
on pr.productno = pa.productno
)
pivot (
min(attrvalue) for attrname in (
1 height, 2 depth, 3 width,
4 weight, 5 colour, 6 description
)
)
), product_details as (
select productno, min(description) descr,
min(colour) clr, min(weight) wt,
min(height) ht, min(width) wd, min(depth) dp
from attrs
group by productno
)
select customers_ssn,
descr,
wt,
clr,
case
when ht like '%cm' then
to_number(regexp_substr(ht, '^[0-9\.]*')) / 100
when ht like '%m' then
to_number(regexp_substr(ht, '^[0-9\.]*'))
else to_number(ht) / 100
end *
case
when wd like '%cm' then
to_number(regexp_substr(wd, '^[0-9\.]*')) / 100
when wd like '%m' then
to_number(regexp_substr(wd, '^[0-9\.]*') )
else to_number(dp) / 100
end *
case
when dp like '%cm' then
to_number(regexp_substr(dp, '^[0-9\.]*')) / 100
when dp like '%m' then
to_number(regexp_substr(dp, '^[0-9\.]*') )
else to_number(dp) / 100
end volume,
sum ( qty * howmuchtheypay )
from "Order" o
join orderline ol
on o.orderno = ol.order_orderno
join product_details pd
on ol.products_productno = pd.productno
where ( to_date (
o."Date" default null on conversion error,
'DD-MON-YYYY'
) > add_months(sysdate, -1) or
to_date (
'MM/DD/YYYY'
) > add_months(sysdate, -1) or
to_date (
'YYYY/MM/DD'
) > add_months(sysdate, -1)
)
group by customers_ssn, descr, ht, wd, dp, wt, clr
DEMO
select customer_id,
descr, height, width, depth, weight, colour
sum ( quantity * unit_cost )
from orders o
join orderlines ol
on o.order_id = ol.order_id
join product_details pd
on ol.product_id = pd.product_id
where o.date_time >= add_months(sysdate, -1)
group by customer_id,
descr, height, width, depth, weight, colour
Bad Primary s
Bad Primary s
• Anything non-unique (duh…)

–if you have to ask, it (probably) isn't!
• Things you (may) need to update
–email addresses…
• Stuff which is "unknown" in some cases
Always surrogate?
• Need more joins; slower SQL?

• Still need to add unique constraint over natural key
• Can enforce complex business rules with "natural keys"
School Teacher Scheduling
Who is teaching what, when, and where?
Teacher only in one Only one class in

place at a time each room at a time
New Requirement!
You can only schedule approved teachers for a class
Surrogate key
create table class_teachers (
class_teacher_id integer,
teacher_id integer,
class_id integer,
primary key ( class_teacher_id ),
unique ( teacher_id, class_id )
);
Teachers must be approved for classes
No teacher_id!
How validate teacher only has one class at a time? ¯\_(ツ)_/¯
No surrogate: can implement rules!
Teacher only in Only one class in

one place at a time each room at a time
https://database-programmer.blogspot.co.uk/
Wrong Data Types
Wrong Data Types: Invalid Data
create table bad_data_types (
date_yyyymmdd integer
);
insert into bad_data_types values (20171121);

insert into bad_data_types values (20170000);
insert into bad_data_types values (-1);
alter table bad_data_types add constraint is_date check (
to_number (
to_date ( date_yyyymmdd , 'yyyymmdd' ) ,
'yyyymmdd' ) = date_yyyymmdd
);
ORA-01481: invalid number format model
alter table bad_data_types add constraint is_date check (
to_number (
to_date ( date_yyyymmdd , 'yyyymmdd' ) ,
'yyyymmdd' ) = date_yyyymmdd
) novalidate;
Stops new invalid dates

Existing data may still be dodgy!
Wrong Data Types: More Complex SQL
with dates as (
select to_date ( date_yyyymmdd, 'yyyymmdd' ) dt
from bad_data_types
where regexp_like (
date_yyyymmdd,
'[0-9]{4}(1[0-9]|0[1-9])(3[0-1]|[1-2][0-9]|0[1-9])'
)
)
select * from dates
where dt = :date_var
Still not quite right…
Defaults to client's NLS settings!
validate_conversion (
<expression> as <data type>,
[ <format mask> ],
[ <nls_parameters> ]
)
1 = Success!
0 = Failure 
select to_date(date_yyyymmdd, 'yyyymmdd')
from bad_data_types
where validate_conversion (
date_yyyymmdd as date, 'yyyymmdd'
) = 1;
this... ...must match this
cast (
<expression> as <data type>
[ default <value> on conversion error ],
[ <format mask> ],
[ <nls_parameters> ] to_date()
)
to_number()
Also...
to_timestamp()
etc.
select to_date(
date_yyyymmdd
default '99991231'
on conversion error,
'yyyymmdd'
)
from bad_data_types;
31-DEC-9999
Wrong Data Types: Performance
…how to screw up the optimizer's estimates
1-JAN-2018 - 31-DEC-2017 = 1 DAY

20180101 - 20171231 = 8,870
Flexible
• Easy to add new attributes

• Less time requirement gathering
Flexible Flexible
• Easy to add new attributes • Don't know contents until run-time

• Less time requirement gathering • More complex SQL
Functions (12.1.0.2)
• json_value – extract single value
• json_query – return JSON fragment
• json_table – convert JSON to relational table
Functions (12.1.0.2)
• json_value – extract single value
• json_query – return JSON fragment
• json_table – convert JSON to relational table
…but how do you know what to extract?
Data Guide
Data Guide (12.2.0.1)
alter table products add constraint
product_desc_is_json_c check (
product_description_json is json
);
create search index products_ji on products (

product_description_json
) for json parameters ( 'dataguide on' );
begin
DBMS_JSON.add_virtual_columns(
'PRODUCTS',
'product_description_json',
DBMS_JSON.get_index_dataguide('PRODUCTS',
DBMS_JSON.FORMAT_HIERARCHICAL)
);
end;
/
begin
DBMS_JSON.create_view (
'products_v',
'PRODUCTS',
DBMS_JSON.get_index_dataguide(
'PRODUCTS', 'product_description_json',
DBMS_JSON.FORMAT_HIERARCHICAL)
);
end;
Oracle Data Modeler Helps
You Build Better Databases
Use it!
thatjeffsmith.com
#MakeDataGreatAgain
Gratisography

Fixing The Worst DB in The World

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fixing The Worst DB in The World

Uploaded by

Copyright:

Available Formats

Fixing the Worst Database in the World

Chris Saxon, @ChrisRSaxon & @SQLDaily

• Anything non-unique (duh…)

• Need more joins; slower SQL?

Teacher only in one Only one class in

How validate teacher only has one class at a time? ¯\_(ツ)_/¯

Teacher only in Only one class in

insert into bad_data_types values (20171121);

ORA-01481: invalid number format model

Stops new invalid dates

1-JAN-2018 - 31-DEC-2017 = 1 DAY

• Easy to add new attributes

• Easy to add new attributes • Don't know contents until run-time

…but how do you know what to extract?

create search index products_ji on products (

You might also like