Comments on: How to reduce the time taken to select the records from huge keyed physical file having 180 Million records

By: philpl1jb

philpl1jb — Mon, 08 Oct 2012 14:14:23 +0000

block of plutonium leading to truncation?

Certainly words to wake up homeland security.

By: graybeard52

graybeard52 — Sun, 07 Oct 2012 03:49:02 +0000

One more suggestion- do the query directly over the PF. Building a query over a LF is usually much slower.

By: TomLiotta

TomLiotta — Sat, 06 Oct 2012 10:45:48 +0000

(Previous comment was truncated on input…)

So far, the OP hasn’t told us anything useful about the table structure nor answered questions about the other three values. We don’t know what records look like. We haven’t even seen the OPNQRYF command yet.

The target program has a trivial description and sounds easy. It almost seems like it could be replaced by a single Qshell db2 statement. It’s hard to tell how it could be so difficult to change.

As far as we know, the OPNQRYF is already optimized to maximum possible degree. Maybe “20-30 minutes” is the best possible time on that system for that table.

Tom

By: TomLiotta

TomLiotta — Sat, 06 Oct 2012 10:33:59 +0000

The use of OPNQRYF can be optimized…

That’s true.

The use of plutonium for generating heat can also be optimized, but that doesn’t mean that a block of plutonium makes a good heater for my living room.

So far, the OP

By: nuccio

nuccio — Sat, 06 Oct 2012 09:16:51 +0000

Are you using OPNQRYF directly to read and write in printer file or CPYFRMQRYF ?The use of OPNQRYF can be optimized in various mode and it is particularly useful in programs because it allows you to use the normal definitions of files. The first thing to be done is reduce the number of fields in base to the real necessity, using a proper format, than use logical files only if correspond in arrangement with OPNQRYF, otherwise use physical file.

By: TomLiotta

TomLiotta — Sat, 06 Oct 2012 00:35:45 +0000

PLs suggest will it work.

It can be made to work, but it’s more difficult than simply indexing by the appropriate columns. It could also be significantly slower.

Tom

By: philpl1jb

philpl1jb — Fri, 05 Oct 2012 16:32:02 +0000

The fact that it’s a keyed file doesn’t help when the selection is not by the key or is a range or list. You may need to build this cl using different Logicals for different selection rules so the query engine doesn’t have to seach the entire data set.

By: rajeshece

rajeshece — Fri, 05 Oct 2012 04:10:51 +0000

Have 1 point. Create a logical File with multiple members. Arrange the date. you just select the correct member based on the User’s input. Read that Member and Filter the Records
Eg: Data should be arranged based on period and Country. So you can select the members which is related to res.country and period. then start filtering based on remaining 3 fields. PLs suggest will it work.

By: rajeshece

rajeshece — Fri, 05 Oct 2012 04:02:48 +0000

Using Sql is the best option. Or Create logical file based on some known selection creteria and use sql on that logical file

By: TomLiotta

TomLiotta — Fri, 05 Oct 2012 01:23:47 +0000

The period and country fields might be good candidates for EVIs. The other three fields are total unknowns, but any one of them might also be an EVI candidate. EVIs are potential ways to significantly speed up the process.

Of course, it’s not at all clear how OPNQRYF fits with “accessing keyed logical file” unless the LF is effectively being created by the OPNQRYF. An existing LF wouldn’t seem to be useful at all except potentially for the initial compile. Without seeing any programming, it’s pure guesswork.

As for the file size, I’ve seen much larger files for businesses smaller than $265M/yr. There are many reasons for larger files.

Tom