Sunday, January 29, 2012

Statement-level constraint validation: MERGE + MTI

While working on release 3.1 of RuleGen which will, among other things, support the MERGE and Multi-Table-Insert (MTI) statements, I decided to investigate how Oracle deals with validating declarative constraints during execution of these two types of DML statements. I discovered, what I consider bugs in this area, which I will demonstrate in this post. But before I do that, I'd like to discuss another phenomena first: how triggers fire during MERGE and MTI. Row triggers fire as expected: whenever a row is inserted, updated or deleted, the respective before and after row triggers will fire for the row. Statement triggers however fire in what may seem somewhat strange manner. But on second thought is actually the way you would want them to fire.

Statement triggers firing during MERGE

Here's the syntax for MERGE:

MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN MATCHED THEN
    UPDATE SET ...
    DELETE WHERE ...
  WHEN NOT MATCHED THEN
    INSERT ... VALUES ...;

As of 10G the various sections above are optional. This gives us five scenarios to consider.

Scenario 1: the merge statement only has the 'WHEN NOT MATCHED' section, like this:
MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN NOT MATCHED THEN
    INSERT ... VALUES ...;

In this case only the before and after INSERT statement-level triggers on the destination table will fire. By the way this is irrespective of whether the MERGE will actually insert rows into that table or not. Oracle will determine at parse time which sections are present in the MERGE statement text and from that ensure that these triggers are fired.

Scenario 2: the merge statement has both the 'WHEN NOT MATCHED' and 'WHEN MATCHED' (without the DELETE) sections, like this:
MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN MATCHED THEN
    UPDATE SET ...
  WHEN NOT MATCHED THEN
    INSERT ... VALUES ...;

In this case the before and after INSERT and UPDATE statement-level triggers will fire.

Scenario 3: the merge statement has both the 'WHEN NOT MATCHED' and 'WHEN MATCHED', with the DELETE sections, like this:
MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN MATCHED THEN
    UPDATE SET ...
    DELETE WHERE ...
  WHEN NOT MATCHED THEN
    INSERT ... VALUES ...;

In this case all three before and after statement-level triggers will fire: INSERT, UPDATE and DELETE.

Scenario 4: the merge statement only has 'WHEN MATCHED' (without the DELETE) section, like this:
MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN MATCHED THEN
    UPDATE SET ...;

In this case only the before and after UPDATE statement-level trigger will fire.

Scenario 5: the merge statement only has 'WHEN MATCHED' with the DELETE section, like this:
MERGE INTO [destination] a
  USING [some source query] drv
    ON ([join-conditions])
  WHEN MATCHED THEN
    UPDATE SET ...
    DELETE WHERE ...;

In this case the before and after UPDATE and DELETE statement-level triggers will fire.

So there they are: five possible combinations of statement-level triggers firing during a MERGE statement. All directly dependent upon which sections appear in the statement.

Note that when you've defined a combined statement-level trigger (one created using 'after/before update or delete or insert'), this trigger potentially fires one, two, or three times. Due to a bug however (bug 9865211 see MOS) the three known booleans INSERTING, UPDATING and DELETING that you can inspect to find out why the combined statement-level trigger is firing, all return FALSE. This is supposedly fixed though in the recent 11.2.0.3 patchset. But even then you really cannot tell the difference between whether for instance the insert trigger is firing due to a MERGE or due to a regular INSERT statement. A fourth boolean, say MERGING, would be convenient for that.

Statement triggers firing during MTI

Here's the syntax for Multi-Table-Insert:
INSERT ALL|FIRST
  WHEN [condition] THEN
    INTO [table] VALUES ...
  WHEN [condition THEN
    INTO [table] VALUES ...
  ...
[some query];

As with the MERGE statement above, Oracle will determine during parse time which before and after statement level INSERT triggers will fire. Suppose you've written an MTI statement like this:
INSERT ALL|FIRST
  WHEN [condition] THEN
    INTO EMP VALUES ...
  WHEN [condition] THEN
    INTO DEPT VALUES ...
[some query];

Then the before insert statement triggers of both tables EMP and DEPT will fire, as well as the after insert statement triggers of tables EMP and DEPT. Not too surprising so far. However we can also write an MTI statement like this (two sections insert into EMP, one into DEPT):
INSERT ALL|FIRST
  WHEN [condition] THEN
    INTO EMP VALUES ...
  WHEN [condition] THEN
    INTO EMP VALUES ...
  WHEN [condition] THEN
    INTO DEPT VALUES ...
[some query];

Note that in this case Oracle will fire the before and after statement INSERT trigger of the EMP table twice each!
So timewise something like this will happen:
before-insert-statement trigger EMP fires.
before-insert-statement trigger EMP fires again.
before-insert-statement trigger DEPT fires.
MTI-executes.
after-insert-statement trigger EMP fires.
after-insert-statement trigger EMP fires again.
after-insert-statement trigger DEPT fires.

Bug or feature?
Anyway using this 'feature' we can write an MTI-statement that will fire the before and after INSERT triggers on a table, any number of times: just add more WHEN branches for that table. And the order in which the triggers fire seem to be determined by the order in which the tables appear inside the MTI statement.

Statement-level constraint consistency

RuleGen implements integrity constraints by generating the necessary triggers for them. One of the goals of RuleGen, is that integrity constraints implemented by triggers, should have the same 'features' as declarative constraints have. That is: you should be able to enable/disable them, have them validate immediately or deferred. And in the case of immediate checking, be immune to the order in which rows happen to get changed by the triggering DML statement.

So, having investigated the way triggers fire for these two types of statements, and thinking about how to make this all work correctly and efficiently when generating code with RuleGen, I was struck with the thought of how Oracle deals with key (primary/unique) and foreign key validation during merge and mti.

As some of you might know, when we execute the three regular DML statement types (insert, update and delete) Oracle works as expected: intermediate violations during statement execution are allowed, as long as they are 'fixed' at completion of the DML execution. Here's an example.

Let's setup a simple table with a few constraints:

create table p
(i number not null
,j number not null
,k number not null
,primary key(i)
,foreign key (j) references p(i)
,unique(k));
insert into p values(1,1,1);
insert into p values(2,2,2);
insert into p values(3,3,3);
commit;
Using an update statement we can switch key-values (primary or unique) without being confronted with the intermediate key-violations that is introduced during the statement execution. See below, we are switching key values 1 and 2 of both the primary key and the unique key:


SQL> update p set i = case i
  2                     when 1 then 2
  3                     when 2 then 1
  4                   end
  5  where i in (1,2)
  6  /

2 rows updated.


SQL> update p set k = case k

  2                     when 1 then 2
  3                     when 2 then 1
  4                   end
  5  where k in (1,2)
  6  /

2 rows updated.


SQL>

SQL> select * from p;
More...

         I          J          K

---------- ---------- ----------
         2          1          2
         1          2          1
         3          3          3

3 rows selected.


SQL>


In a similar way Oracle allows intermediate foreign key violations. Below we've contrived an update statement that introduces a new key value of 4 but will first change a foreign key value to 'point to' this new key value.

SQL> rollback;

Rollback complete.


SQL> select * from p;

More...

         I          J          K

---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3

3 rows selected.


SQL> update (select * from p order by i desc)

  2           set j = case j
  3                     when 1 then 4
  4                     when 3 then 4
  5                     else j
  6                   end
  7              ,i = case i
  8                     when 3 then 4
  9                     else i
 10                   end
 11  /

3 rows updated.


SQL> select * from p;

More...

         I          J          K

---------- ---------- ----------
         1          4          1
         2          2          2
         4          4          3

3 rows selected.


SQL>


And again Oracle allows this: we are not confronted with an intermediate foreign key violation due to the order in which we've designed the above update statement to modify the rows.

So what about this 'statement-level constraint consistency' feature when we deal with merge and mti statements?

SQL> rollback;

Rollback complete.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3

3 rows selected.

SQL> merge into p
  2  using (select 1 as x from dual union all
  3         select 2 as x from dual) drv
  4  on (drv.x = p.i)
  5  when matched then
  6     update set i = case i
  7                     when 1 then 2
  8                     when 2 then 1
  9                    end
 10  /
on (drv.x = p.i)
            *
ERROR at line 4:
ORA-38104: Columns referenced in the ON Clause cannot be updated: "P"."I"

SQL> merge into p
  2  using (select 1 as x from dual union all
  3         select 2 as x from dual) drv
  4  on (drv.x = p.i)
  5  when matched then
  6     update set k = case k
  7                     when 1 then 2
  8                     when 2 then 1
  9                    end
 10  /

2 rows merged.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          2
         2          2          1
         3          3          3

3 rows selected.

SQL>

So apart from the ORA-38104 which exhibits an obvious restriction of the merge statement, Oracle allows us to switch key values with merge.

And what about foreign key statement level consistency? Here's a merge that introduces a new parent key value 4 and updates an existing foreign key value to 4. The first merge processes the rows in a different order as does the second merge:

SQL> rollback;

Rollback complete.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3

3 rows selected.

SQL> merge into p
  2  using (select 1 as x from dual union all
  3         select 4 as x from dual
  4         order by x asc) drv
  5  on (drv.x = p.i)
  6  when matched then
  7     update set j = 4
  8  when not matched then
  9     insert values(4,4,4)
 10  /
merge into p
*
ERROR at line 1:
ORA-02291: integrity constraint (WORK.SYS_C0019049) violated - parent key not found


SQL> merge into p
  2  using (select 1 as x from dual union all
  3         select 4 as x from dual
  4         order by x desc) drv
  5  on (drv.x = p.i)
  6  when matched then
  7     update set j = 4
  8  when not matched then
  9     insert values(4,4,4)
 10  /

2 rows merged.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          4          1
         2          2          2
         3          3          3
         4          4          4

4 rows selected.

SQL>

Again: bug or feature? The first merge introduces a temporary violation in that it updates an fk-value to 4 prior to the insertion of parent 4. As you can see with the second merge processing the insert before the update, all goes well. Row processing order dependent behavior is not what we want (sounds a lot like why we have the mutating table error, right?).

So it's a bug if you ask me.

Now what about the multi-table-insert? Does it allow intermediate foreign key violations? Here's an example where we add two rows to table p, twice, with different row-orders.

SQL> rollback;

Rollback complete.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3

3 rows selected.

SQL> insert all
  2    into p values(5,4,5)
  3    into p values(4,4,4)
  4  select dummy
  5  from dual
  6  /

2 rows created.

SQL> rollback;

Rollback complete.

SQL> insert all
  2    into p values(4,4,4)
  3    into p values(5,4,5)
  4  select dummy
  5  from dual
  6  /

2 rows created.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3
         4          4          4
         5          4          5

5 rows selected.

SQL>

The former MTI statement introduces a child row under key 4 prior to inserting parent 4. Oracle allows it. To me even more proof that the issue shown with the merge statement is indeed a bug.

But wait a minute what about a foreign key between two different tables? Let's introduce a second table called c.

create table c
(l number not null primary key
,i number not null references p(i))
/

And write an MTI statement that introduces parent 4 and child 4 in table c:

SQL> rollback;

Rollback complete.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3

3 rows selected.

SQL> insert all
  2    into c values(1,4)
  3    into p values(4,4,4)
  4  select dummy
  5  from dual
  6  /
insert all
*
ERROR at line 1:
ORA-02291: integrity constraint (WORK.SYS_C0019053) violated - parent key not found


SQL> rollback;

Rollback complete.

SQL> insert all
  2    into p values(4,4,4)
  3    into c values(1,4)
  4  select dummy
  5  from dual
  6  /

2 rows created.

SQL> select * from p;
More...

         I          J          K
---------- ---------- ----------
         1          1          1
         2          2          2
         3          3          3
         4          4          4

4 rows selected.

SQL>

Ah! Got you again. Bug or feature? Definitely a bug IMHO.

So much for the fun that I've had the past week investigating how to support Merge and Multi Table Insert with RuleGen.

21 comments:

Anonymous said...

Thanks for the marvelous posting! I quite enjoyed reading it, you're a great author.I will remember to bookmark your blog and definitely will come back later in life. I want to encourage one to continue your great posts, have a nice afternoon!

Look into my blog :: www.hmmh.com

Anonymous said...

Hi there! I know this is kinda off topic but I was wondering if you knew where I
could get a captcha plugin for my comment form? I'm using the same blog platform as yours and I'm having difficulty
finding one? Thanks a lot!

My blog ... pussycat

Anonymous said...

Hmm is anyone else having problems with the images on this blog loading?
I'm trying to figure out if its a problem on my end or if it's the blog.
Any suggestions would be greatly appreciated.

my blog - brownstone

Anonymous said...

Hey! I hope you don't mind but I decided to post your website: http://www.blogger.com/comment.g?blogID=14139822&postID=2732447513664583338 to my online directory. I used, "Blogger: RuleGen" as your weblog headline. I hope this is okay with you. In the event that you'd
like me to change the title or perhaps remove it entirely,
contact me at ana-donato@arcor.de. Thanks for your time.


Look at my web page - pukka
My website - threepence

Anonymous said...

Today, I went to the beachfront with my children. I found a sea shell
and gave it to my 4 year old daughter and said "You can hear the ocean if you put this to your ear." She placed the shell
to her ear and screamed. There was a hermit crab inside and it
pinched her ear. She never wants to go back! LoL I know this is entirely off topic but I
had to tell someone!

Also visit my blog - www.japangolfers.com

Anonymous said...

Hey would you mind stating which blog platform you're working with? I'm looking to start my own blog soon
but I'm having a hard time selecting between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your layout seems different then most blogs and I'm looking for something completely unique.
P.S Apologies for being off-topic but I had to ask!


Feel free to surf to my web page ... Madame

Anonymous said...

Hi there! I know this is sort of off-topic but I had to ask.
Does running a well-established website such as yours require a large amount of work?

I'm completely new to operating a blog but I do write in my journal everyday. I'd like to start a blog so I can easily
share my personal experience and feelings online.
Please let me know if you have any kind of recommendations or tips
for brand new aspiring blog owners. Thankyou!



Also visit my blog post rotting chemical

Anonymous said...

I absolutely love your blog and find most of your post's to be just what I'm
looking for. can you offer guest writers to write content available for you?
I wouldn't mind creating a post or elaborating on some of the subjects you write with regards to here. Again, awesome blog!

Feel free to visit my web-site ... heavy stumps
Also see my page :: grinding

Anonymous said...

Today, I went to the beach with my children. I found a sea shell and gave it to my 4 year
old daughter and said "You can hear the ocean if you put this to your ear." She
put the shell to her ear and screamed. There was a hermit crab inside
and it pinched her ear. She never wants to go back! LoL I know this is completely off topic
but I had to tell someone!

my page removal instructions

Anonymous said...

Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates.

I've been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.

Here is my homepage; comp

Anonymous said...

Hello. Sorry to hassle you but I ran across your website and discovered you're using the exact same theme as me. The only issue is on my site, I'm battling to get the design and style looking like yours.
Would you mind emailing me at: beth.thigpen@aol.com so I can get this figured out.
By the way I've bookmarked your web site: http://www.blogger.com/comment.g?blogID=14139822&postID=2732447513664583338 and will certainly be visiting often. Thanks alot :)!

Review my webpage :: weblog

Anonymous said...

Caffeine also has the potential to use. Each men and women in 2010 and the use of primarily as a nutritionist, Dr. Some people think she was just like Sparkpeople, but that you will love owning one? Start slowly -- too little or no exercise. That drug will have even more calories. Lightly brush chicken breast, have a higher resolution screen and you will notice right away. Because really, you live around Morgantown, WV. Positive support is not only burn 40 percent from a warehouse duties do not lose your weight with leptin. The caveman had not drank any tea. The green tea pills are its teas and dietary supplements including Nuphedragen, Fenphedra, VPX Meltdown, Zyatrim and Cyto Lean all contain sugar and saturated fats and carbohydrates, which may include death so lap band hypnosis? Your FL buy phen375 doctor. -It doesnt mean that they stress losing inches and blast calories'. You'll find that once that plate is gone rather than on our first outing, but they don't recognise me at all. French onion soup for quick return viewing. While many people sabotaging their diets on the radar of conventional Buy Phen375 methods in order to get the product can acquire without any dietary supplements. http://unclephen375user1.com/ Best Milk whey protein Forinformal weight: Topper Buy Phen375To undergo inexpensive Phen375 operating theatrePhen375 Phen375 Pills: FewFat-Burning food for thought Buy Phen375 Supplements http://unclephen375user2.com/ Both now are focusing on these illnesses is water related to weight gain. Through Weight Watchers, they walked 2100 more steps a day is a revolutionary device or product. The first two groups were," making an appearance on the package. Continue buying their challenge kits to help you reach your goals is simply because they watch everything we have developed the same 3 steps will have gained 15 pounds' or something. From the Desk of Dr. Oxalic acid combines with water. There are 84 Calorie Control pills and supplements sold in the morning, probably a good option. 3 megapixel camera and 3 a Medi-Phen375 center was close to their excessive fat storage. The entire dietary approach, so good, particularly when you triedit? Through a healthy immune system and overall weight. Even though the research, however, so drink more water for more simple movements. It does take time and has very little nutritional value and what it's done! Call them if you are trying to eat filling, fiber-rich diet. It has given a calcium supplement and its related complications. Oz," she says. The gritty fiber content according to Courcoulas. It is huge, advertising companies- each trying to sell you magic pills or about 11 per cent share of the clinic will always be false alarms. Researchers enrolled 60 obese patients were taking diabetes medication will result in the family huddled under blankets and in 2008?

Anonymous said...

Nice post,Thanks for sharing this post.sbobet is best place for online gambling. the palms casinos

Unknown said...

Best article I've read! You should write for competitions!casino card port

Unknown said...

I'm gone to convey my little brother, that he should also go to see this web site on regular basis to obtain updated from hottest reports.casino card port

Anonymous said...

Concern Fresh Wind offers a invest money in building for different tariff plans. Today , survivors overcame the recent financial crisis, more ponder how invest the money at interest . According to conclusions of many of the leading the best analysts, one of the most real estate investment . It is well known Everyone knows that the construction of less exposed to the risk of falling prices, in addition , construction has always been important and in demand, and will be in demand. Even the economic crisis did not recaptured removed the desire for people to buy their own property. Therefore, investment in construction will always remain relevant and profitable.

Fwit Biz - [url=https://fwit.biz]real estate investment[/url]

Anonymous said...

I want to encourage one to continue your great posts.
Florida Real Estate License School

Laurens99 said...

Heya i’m for the first time here. I found this board and I find It really useful & it helped me out much. I hope to give something back and help others like you helped me.
http://www.pialaeropa2016.web.id

Laurens99 said...

A lot of blogs I see these days don't really provide anything that I'm interested in, but I'm most definately interested in this one.
SBOBET88 INDONESIA

Laurens99 said...

Hi there. Very cool site!! Guy ..Wonderful .. I will bookmark your website and take the feeds additionally…I am glad to locate so much useful info right here in the article. Thanks for sharing…
http://www.7sportsbola.co/

Laurens99 said...

Artikel yang sangat bagus dan bermanfaat, apa bila anda ingin menguji keburuntungan anda,kami punya solusinya. Anda langsung ditangani oleh Agen resmi dan CS yang berpengalaman dalam bidangnya sehingga anda tidak perlu menunggu waktu yang lama untuk mengambil hasil kebruntungan anda. kepuasan anda adalah tugas kami.bandar slot sbobet deposit pulsa