Comments on: Trigger Perormance Issues when Journalling Started

By: FrankWM

FrankWM — Mon, 13 Aug 2012 08:08:12 +0000

And 5,000,000 rows in 25 minutes is a problem becauseBecause it only takes a little over 2 minutes without the trigger. Although, in practice you may be right. I need to do some analysis of just how many writes do take place. I'll do that later today.Interestingly, I have done some tests with record updates and they are quite acceptable. Something like 1.5 million updates. With no trigger 7 minutes and with the trigger, 16 minutes. I also have a lightweight version of the monitor file that does not include the before and after images (the thought is that I can write those to a separate, non-journaled file with same key). In that scenario I see blocking happening with the writes to the monitor file and it only takes 9 minutes; I haven't added the extra file for the images, so it will go up when I do

By: philpl1jb

philpl1jb — Sun, 12 Aug 2012 22:24:26 +0000

And 5,000,000 rows in 25 minutes is a problem because …

By: TomLiotta

TomLiotta — Sun, 12 Aug 2012 21:12:02 +0000

With 50 files, there could be a very large number of jobs all trying to write to the same (monitor) file. Contention could be excessive. Further, the extra journaling adds an additional layer of contention (for 50 files). And because the write to the journal must complete before the write can be allowed to the file, every point of contention has an extra delay added in.

All of it on top of the primary output to the 50 files and the journaling for them.

I’ve yet to understand why a “monitor” file is good for anything when journaling already exists. I don’t quite see the elegance yet.

Tom

By: FrankWM

FrankWM — Sun, 12 Aug 2012 21:05:04 +0000

What happened to my line-feeds? My previous message looks a mess

By: FrankWM

FrankWM — Sun, 12 Aug 2012 21:01:45 +0000

Doesn’t that wreak havoc on rollbacks?No, when STRCMTCTL issued, generally all files are open COMMITWhich modes are “wrong”?Yes - commit lock, if this indicates that the file needs to be open commit, but it isn't, then it is closed and opened commit and vice-versa.It has no effect on what? And are you saying that the timings you gave above were always outside of commitment control?Yes, my tests are outside of commitment control. 50% of real system transactions do not use commitment control and it was only for comparative purposes.The use journals questionThey were considered as an option but they also have their problems that are outside the scope of this.

By: philpl1jb

philpl1jb — Sun, 12 Aug 2012 20:05:32 +0000

So the trigger program only writes to the monitor file, which could have been created from the Journals. Why do this in real time?

By: TomLiotta

TomLiotta — Sun, 12 Aug 2012 10:43:57 +0000

We have mixture of processing, some in commitment control and some not.

Doesn’t that wreak havoc on rollbacks?

I close and open the monitor file if the flag in the passed parameter indicates I am open in the wrong mode.

Which modes are “wrong”? I assume the “parameter” is ‘Commit lock level’. How does opening/closing the file matter for that?

The Close/Open has no effect in my test as commitment control is always off.

It has no effect on what? And are you saying that the timings you gave above were always outside of commitment control?

Tom

By: FrankWM

FrankWM — Sun, 12 Aug 2012 10:02:21 +0000

Yes it does roll back the transaction if the transaction rolls back. That’s part of the elegance of the solution as I only want to know about changes that got committed. My trigger program is RPGIV. We have mixture of processing, some in commitment control and some not. I close and open the monitor file if the flag in the passed parameter indicates I am open in the wrong mode. The Close/Open has no effect in my test as commitment control is always off.

By: TomLiotta

TomLiotta — Sat, 11 Aug 2012 21:27:50 +0000

…it will roll back the inserted monitor file record if the transaction rolls back

Does it roll back if the transaction rolls back?

Tom

By: FrankWM

FrankWM — Sat, 11 Aug 2012 20:36:44 +0000

Thanks Phil. I have tried using a separate journal for the monitor file, but it makes no difference to the timings. Why journal the monitor? So it will roll back the inserted monitor file record if the transaction rolls back