Need some performance tuning tips or suggestions.

pts.
Tags:
Backup & recovery
Hardware
i5
IBM
Installation
Integration/Connectivity
iSeries
Logical partitions
OS/400
PC/Windows Connectivity
Performance/Tuning
Printing
PTFs
Security
Server consolidation
Servers
System monitoring
tips and tricks
Tools
Web services
We currently have a 720-9406 model iSeries AS/400 system. We recently upgraded are OS to version V5R3M0. We are using the system mainly for our payroll processing which occurs weekly, as well as we have the system set up as a web server which we have employees accessing throughout the day. Since the upgrade, our performance has taken a hit on the system, especially the web service piece of the system. Our ASP is currently at 68%. CPU varies as always, but at certain points in the day, especially when we process batch jobs for payroll, the CPU takes a signficant hit. We sometimes have to end the web server so that we can finish the processing. Does anybody have any suggestions on how to increase performance so that we don't have to end jobs or put jobs on hold. Thanks for the help.

Answer Wiki

Thanks. We'll let you know when a new response is added.

Need a few answers, Webserver, iSeries Access for Web ? Webfacing, Hats ? Have you setup shared pools and tuned Java for the Webserver ?

Discuss This Question: 12  Replies

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • TomLiotta
    Since there's no activity after the first response, I might as well... The first asked some good beginning questions. At least two areas also need answers. First, you upgraded to V5R3 from V5R2 or V5R1? V5R3 has _significantly_ more needs than than V5R1 and more than V5R2. You could've been on the knee of the curve at V5R2. Second, has there ever been any decent tuning at all? E.g., are you running with the general default configuration as supplied by IBM? For example, default runs essentially every server function in *BASE. Pretty much _nothing_ should be running in *BASE -- that's the base memory pool that's used by OS/400 to move memory into pools that need it (hence the name, "*BASE"). If lots of jobs are using that memory, there's no rational way to do any useful tuning at all. Until you have your jobs running in separate pools so you can see pool activity, all you will ever see is the result of one job stepping on another. Without a little background, performance tips are limited to suggestions such as "Read the Work Management Guide and follow the recommendations."
    125,585 pointsBadges:
    report
  • Bkr0963c
    Tom, Thanks for the reply. I privately mailed the first response, so that me be the reason you did not see the answers to the questions. Anyways, the upgrade took place on a weekend before I started with the company. They upgraded from V5R1 to V5R2 to V5R3 all in one weekend. As far as tuning, they allow IBM's QPFRADJ program to dynamically adjust the system. QBASE is not the controlling subsystem. They have broken it up into different subsystems with different memory pools. However, again, the system does automatic performance tuning. I have downloaded IBM's work management guide and I am presently reading through it to get some understanding on this subject. Maybe after reading through this manual 700 times :o I will have some more specific questions to ask as well as some answers for myself. Thanks again Tom for the comments and any other info you would like to offer.
    0 pointsBadges:
    report
  • Sully35
    Can someone point me to where I can find the Work Management Guide? I have a 2 year old 810 that seems to slow to a crawl when a couple of users run big reports. It's my feeling that the consultant who installed this for us may have left the performance settings at IBM's default. Thanks for any help you can provide.
    0 pointsBadges:
    report
  • Bkr0963c
    http://publib.boulder.ibm.com/iseries/v5r1/ic2924/books/c4153063.pdf Type this in for your url. It is the work management guide for V4R5, however, it should give you some good concepts and understanding to take into another release. Hope that helps.
    0 pointsBadges:
    report
  • TomLiotta
    A couple clarifications... First, it's a good start to break away from QBASE as the controlling subsystem; but that's almost unrelated to the question of whether *BASE is used as the memory pool for server jobs and other similar activity. A review of subsystems, their pool assignments and both their routing entries and their pre-start job entries, would be the beginning to see where much of your system activity actually runs. Even under QCTL as the controlling subsystem, most AS/400s have _all_ default routing and prestart job entries still pointed to *BASE. Second, if *BASE is the memory pool for so many active functions, then QPFRADJ will have minimal effect. If telnet servers, FTP servers, NetServer, SQL servers and all kinds of servers are active over *BASE memory, then it takes a _lot_ more effort for QPFRADJ to get memory moved successfully through *BASE into the pools where it's needed. AFAIK, QPFRADJ _only_ moves memory (1) from *BASE to shared pools and (2) from shared pools to *BASE. It doesn't move memory _between_ shared pools. I.e., memory that's shifted from one shared pool to another must spend some time in *BASE. If jobs are active in *BASE memory, then that shifting memory can become in-use for those *BASE jobs. If it's been grabbed by a job in *BASE, then the probable need for paging that memory grows significantly. And that's where performance can start getting hit in that scenario. In IBM's defense, they have no idea what kind of configuration a customer will do best with; so they supply a generic default that "works" for everybody. (Actually, a couple of defaults -- QBASE and QCTL configurations.) OTOH, they also make money selling memory and faster processors. They do supply the Work Management Guide, but I'm sure they know that the vast majority of sites never change such things as default routing entries (even though they will recommend it if asked). There are a _lot_ of routing entries, etc., and few sites look at them without feeling intimidated.
    125,585 pointsBadges:
    report
  • Bkr0963c
    Tom, Let me say thanks first of all for your responses. You are opening up my mind in the areas of the things that I am studying. Now, let me apologize for the misunderstanding about the functions running in *BASE. I understand now that you mean the *BASE memory pool. So, based off of what you are saying, this is what I believe I need to do. 1. Look at the subsystem descriptions of each subsystem that is running and verify what memory pool that the routing entries, prestart jobs, and job entries that the jobs are pulling from as well as what memory pools are assigned to that subsystem. 2. If most of them are pulling from the *BASE memory pool, then I need to consider creating memory pools and changing the descriptions to have these jobs pull from the designated memory pools. 3. In the case of step 2 above, that would mean turning off QPFRADJ and manually tweaking the system myself, which requires establishing a baseline, gathering some performance collection data and interpreting the data (this I lack in, however, I will take the necessary steps to learn). Tom, I believe this is where you are leading. Confirm if you don't mind. Thanks again for your knowledge.
    0 pointsBadges:
    report
  • TomLiotta
    Okay, looks like you've made the first jump and OS/400 work management is starting to get some solid shape for you. Now, point (1), yes, that's a good starting point. And point (2) is the logical next step although I would personally go farther to the point of changing _every_ one away from *BASE. I have the idea that the only things I should have running in *BASE are (possibly) the subsystem monitor jobs themselves, I think they run in whatever the first memory pool assigned to a subsystem is, and jobs that the system moves into *BASE via the QTSEPOOL system value and similar things. As far as possible, allow OS/400 to decide when something should run in *BASE. Also, for the most part, you will be using shared pools not private pools. In that sense, you won't exactly be "creating" memory pools for this but simply allowing memory to be assigned to pools that already exist but aren't being used. You're currently using at least two shared pools known as *INTERACT and *SPOOL if you're running under the QCTL scheme. Now you'll want to activate a couple more -- *SHRPOOL1 and *SHRPOOL2 for example. I can generally see reason for assigning as many as six, *SHRPOOL1 through *SHRPOOL6, in the beginning. Which brings us to point (3) and a definite "No." In a short sense, the entire point of QPFRADJ is to allow the system to move memory between shared pools. Once you have your jobs out of *BASE _and separated into different shared pools_ you'll have a system where performance tuning becomes reasonably possible. QPFRADJ finally becomes useful _after_ separation into appropriate shared pools. (Shared, not private. Private pools have some useful specific purposes.) Say for example that you use a scheme of three *SHRPOOLs. Your production batch in *SHRPOOL1, host servers in *SHRPOOL2 and TCP/IP servers in *SHRPOOL3. (And no more stuff stomping around in *BASE.) As demand rises and lowers, QPFRADJ will make adjustments between shared pools. Maybe ODBC starts getting hit heavy and some heavy production batch is finishing -- QPFRADJ will start increasing *SHRPOOL3 at the expense of *SHRPOOL1 which doesn't need much now. The whole point, though, is that now you can _see_ it happen! If they were all still in *BASE, there would be no practical way for you to know that ODBC was taking over your memory, or that it _needed_ to have more; you couldn't know if it was ODBC or NetServer or whatever. Now when you see faulting or paging, you can narrow it down to a particular group of components. So then consider a six-pool setup -- *SHRPOOL1 & 2 to your batch, *SHRPOOL3 & 4 to host servers, *SHRPOOL5 & 6 to TCP/IP servers. Attach the pools to the appropriate subsystems and route everything into pools 1, 3 and 5. And then say you want to investigate _ONLY_ the File server host server or _ONLY_ ODBC. You now have the setup in place where you can change just a single routing entry (or prestart job entry) for just the component you want to isolate. Switch ODBC from *SHRPOOL5 to *SHRPOOL6 and you now _know_ that you're running some request that sends faulting through the roof because you _know_ what *SHRPOOL6 contains. OS/400 provides all the pieces to let you see almost everything you need to know. All it takes is putting the environment infrastructure in place. Once done, you can _know_ whether you're really short of memory or you're simply running conflicting jobs for example. You can buy more tools, but they really aren't that useful until you learn to do it yourself anyway.
    125,585 pointsBadges:
    report
  • Bkr0963c
    Tom, this is great. I am seeing a light at the end of the tunnel and it seems really bright. Just a some more scratching, digging, and pulling I think I will see things a lot more clearly. I thank you again in advance for your time and help. It is greatly appreciated. Now here is where I believe I stand. I have 5 memory pools. *MACHINE, *BASE, *SPOOL, *SHRPOOL1, and *INTERACT defined on the system in that order. I will give an example of a couple of subsystems that I have running and my thoughts on the possible changes that I need to make to maybe help performance. I have QBATCH which processes payroll jobs. It has *BASE and *SHRPOOL1 assigned to it in that order. QBATCH has three job queue entries (QBATCH - 3 MaxAct, QS36EVOKE - *NOMAX, QTXTSRCH - *NOMAX). QBATCH has four routing entries: SEQ PROG COMPARE MEMORY POOL ASSIGNED 15 QCMD 'QIGC' *BASE 300 QCMD 'QS36EVOKE' *SHRPOOL1 700 QCL 'QCMD38' *SHRPOOL1 9999 QCMD *ANY *SHRPOOL1 QHTTPSVR subsystem has only *BASE memory pool assigned to it. It has only one job queue entry which has *NOMAX assigned to it for active jobs, and it has only one routing entry seq - 10, program - QCMD, and compare value - "HTTPWWW" which obviously pulls its memory from *BASE. QBATCH handles my payroll jobs and program compiles, and maybe some queries here and there. Now, I believe I need to determine how these jobs are routed into the subsystem to make sure they are not coming in under the first SEQ number because they will use *BASE for their memory. Better yet, change the routing entry on the first SEQ to point to SHRPOOL1 so that these batch jobs run in their own memory pool. This way, I can see how much memory I need to allocate, or in this case, allow QPFRADJ move the appropriate memory allocation into that memory pool. The other thought I had in mind was since payroll only runs one job at a time, create a subsystem specifically for payroll along with its own memory pool, etc... so that I can seperate those jobs from program compiles and queries which I can choose to not allocate as much memory to at first for the program compiles and queries and set the MaxAct job for the payroll subsystem to 1 because only one job needs to run at a time for payroll. Now looking at QHTTPSVR, I need to create another memory pool for this subsystem to use to get away from *BASE. Then I can monitor how much memory and page faults that jobs running in this subsystem is getting. Most of these jobs are batch-immediate jobs. This way, I should also get direction on the question of "Do I have enough system resources to accomplish the work that I want accomplished on the system?" I am sure I could say a lot more, and I am sure there are other areas to look, but I hope I am starting to look in the right direction. Thanks again Tom for you insight. It is greatly appreciated. I hope that once I get the understanding that I will be able to return the favor to someone else that needs the help.
    0 pointsBadges:
    report
  • TomLiotta
    Yow! VERY good! (IMO) You've restated things back here in terms of your environment and added details that make good sense. AND you seemed to know _why_ they made sense. Stay aware that these steps _probably_ won't make significant difference in performance. BUT, they will start putting you in shape so that the _next_ steps can be determined and those steps will make a difference. Note that you actually already have all of the shared pools "defined". You can see their definitions with the WRKSHRPOOL command. What you don't have is (1) resources assigned to any of them except *INTERACT, *SPOOL and *SHRPOOL1 and (2) any associations between them and subsystems. There are also some existing routing entries, etc., that you've already seen might be changed to better use shared pool resources. By completing the configuration of one or a couple more *SHRPOOLs, you _might_ be able to make one very easy change in the direction you're going. Assign a basic configuration to, say, *SHRPOOL2 and associate it with QHTTPSVR (i.e., CHGSBSD QHTTPSVR POOLS((1 *BASE) (2 *SHRPOOL2)) ). Then do a CHGRTGE to the single routing entry to route into subsystem pool 2 which would then be *SHRPOOL2 (i.e., CHGRTGE QHTTPSVR SEQNBR(10) POOLID(2) ). That should get it going and QPFRADJ will begin to do a small amount of work for it. Those changes can be made while everything is up and running, but _might_ not be a good idea -- any response time in QHTTPSVR could slow noticeably until *SHRPOOL2 settles and its memory and activity levels get close to what's needed. If you learn a bit more about memory management, e.g., setting minimum memory, clearing pool memory with CLRPOOL, etc., and when why steps are done, you could prepare *SHRPOOL2 ahead of time to minimize the transition effect. Maybe it won't even matter; I can't judge. However, without also freeing *BASE so that it functions as intended to be the _base_ memory pool from which working pools draw memory as needed, QPFRADJ won't be very effective for QHTTPSVR. Still, WRKSHRPOOL gives you some direct access and DSPSYSSTS gives you visibility into effects. You might get basic experience with just QHTTPSVR and *SHRPOOL2 even with less than stellar results. Finally, _never_ lose sight of the fundamental possibility that just _maybe_ you really will need to buy more memory! But at least you'll be able to know when that's the reasonable answer.
    125,585 pointsBadges:
    report
  • TomLiotta
    Before anything can be suggested, we need to know an overview. Software environment was already requested. But even the most basic hardware resource description is also needed. Use DSPSYSSTS, WRKSBS and WRKDSKSTS to create three general views/lists. Paste them here to give us an idea of what you have to work with. WRKSBS only needs the first screen to show active subsystems and pools. DSPSYSSTS should show as ASTLVL(*INTERMED}. No statistics are needed yet, so just the initial numbers are fine. Tom
    125,585 pointsBadges:
    report
  • TomLiotta
    Whoa! Ignore my previous post. I entered that in the wrong thread. Tom
    125,585 pointsBadges:
    report
  • YuVa47
    There is a software in the market, which can help you. Send me a note to: yudavago at 013 dot net and I'll tell you what you could do. Thanks, YuVa
    1,300 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following