Do you have any VPN or firewall software running on the clients, especially with their services disabled? We had Checkpoint on some clients but disabled the services when logged on in the office and they exhibited these problems. They worked great when working remotley on the VPN! We swapped to Cisco and no longer have the problem. It may be worth just getting a basic client up and running that has no firewall or other network-type software and see if it replicates the problem.

Have you also considered your Global Catalogue? This is on a domain controller. If there is an issue there (load balancing / dodgy NIC / poor performance / dodgy file access / AD issues ….. the list could go on), Outlook grinds to a halt! You can check some network connectivity issues using the Outlook icon that appears in the system tray. Hold down CTRL as you right-click and you get an extra menu item appear – Connection Status. (Why do Microsoft hide this?). This may give you a hint as to which connections are failing consistently. You can expect some failures (although I don’t usually see any at all), but more than 10 – 15% may be indicative of an issue.

Other thoughts and comments:

You will need to run eseutil defrag from time to time, but I don’t think that is the cause of your problem unless a lot of emails get created an deleted frequently.

I would never run Outlook in cached mode – it confuses users amongst other things!

Running MS Exchange 2003 Best Practice Analyzer is essential. For 70 users with a 20GB IS, we run it monthly. It is very comprehensive and could show an issue.

I would also say that having more than 100Mb per mailbox is a risky thing! 100Gb for 40 users is asking for trouble in my limited experience. Exchange is not designed to be a storage utility. You need to archive / save emails / delete the rubbish or something. Force the users to manage their mailboxes. Set the limits to warn them at 100Mb and refuse to send after 120Mb.

Finally, get rid of Public Folders. In their wisdom, Microsoft have decided to get rid of them in the next version of Exchange. Create a mailbox and allow everyone to access it. Users can add it as an additional mailbox. Unfortunately, you can’ automatically delete stuff after a certain time.

Hope that helps. Let me know if you need me to expand on anything.

Cheers!

Dave.

100GB is a LOT of database to defragment even in terms of disk transfer rates.

I suggest schedule the whole weekend after your Friday night full backup should complete (leave an hour or two gap for safety).

As the article you quoted said – defragmentation will not take place during backup or any other heavy system activity. in fact you might need to schedule to boot users off during defragment time. I remember users who loved to leave their workstations logged (locked or maybe not) during the weekend despite company policy. If they have email open and google desktop indexing — online dragmentation will not happen.

I think your thoughts about reading thousands of headers in a folder are long known issues as is opening huge 10MB-100MB emails.

I suggest you think about online archiving standards for your users. You should focus on teaching users to
(1) use more small folders and
(2) configuring their search tools to work smarter.
(a) Reduce the frequency of desktop indexing tools –
maybe even limit full reindexing to a once per day
on staggered schedule (interesting scripting)
(b) Declare certain folders as archive folders to be reindexed only as after moving a new batch of emails to that archive folder — script or manually.

The main things to look for on a network are % bandwidth remaining and error rates. High error rates in server transmission can sometimes cause rereads on disk too. Also full bandwidth normally would effect all applications not just email…unless you have QoS priority on the other services and email is left with the whatever remains if anything.

Keep in mind that even with single user per port switches you probably won’t be able to get 100Mbps on a full 100Mbps Ethernet connection — 85-95% each way depending on card and switch. Much less if you have half duplex or multiple user hubs in the mix.

Now where should you look for bandwidth? Find the bottlenecks in the physical data flow. First look at the server NIC itself. Then look at the total flow rates into key switches or routers. You don’t need to be exact or track down minor flows. You just need a ballpark to see if you are overloading the switch backbone/backplane (see manual) or router packet or byte handling capabilites.

Worse some of the search tools are written poorly or more for the local .pst of home users — thus as long as the local machine seems idle they index and reindex.

You mentioned Lookout Search. I have about 5 people using it. Do you really think it impacts the server? I know when it indexes, it has to do a lot of talking to the server, but when it’s done, the indexes are local and should not hit the server, except when doing an actual lookup.

I know the disk configuration is not the best but not much I can do right now about that, but will definitely run the analyzer.

Outlook clients, have some using cached mode, some not, both have this issues, because most of the time when this happens it’s when they are accessing another users calendar or in the case of admins the exec’s contacts which are not cached anyway.

Will also look for desktop search engine users as some others have said.

Thanks again!

Good luck

Do you run these outlook clients in cached mode or not? Are users getting the pop-up (“Outlook is trying to retrieve data…”) when the click on things? One thing worth noting is that, depending on how the users interact with the server, you could jus be punishing the server with a lot of database and t-log hits. Perfmon is a great tool, it’s just not easy to understand what all to look for.

I learned a few things from the MS MOM 2005 Management pack for exchange, one of the more interesting was the average disk read/write latency averaged over some period of time. On my 55gb exchange server with dual 3.2ghz cpus, 2gb ram, RAID1 logs and store with only 25 not-so-heavy users, the disk latency could get up to almost 1sec where it should be below 20ms.

If users perform advanced find, google desktop email indexing, windows desktop/lookout indexing of mail files, it can really beat up the server. Even just one good search will spike the server. Exchange doesn’t index stuff so when you say to advanced search your 3gb inbox for a phrase, the database gets murdered.

Try scanning with the MS Exchange 2003 Best Practices Analyzer and I can perhaps recommend some perfmon objects from MOM that you can check (or install trial of MOM yourself, it’s a great tool!). I highly doubt the network is impacting the server. It’s more likely user behavior and lack of adequate disk resources (e.g. a raid 1 striped array for Logs and DB with a lot of disks in it).

Even memory is rarely a problem with exchange because most people know to give it at least 1-2gb. It’s usually a disk problem and rarely do people but the kind of disk into and exchange server that it needs. If you read some of the exchange benchmark testing that HP and others do to prove how badass their new server X is, they often use databases with hundreds of users and will literally run insane RAID1 arrays of 20 to 40 spindles striped over either physical or virtual disks.

You can easily have over 1 million emails on your IS. Imagine how a SQL database with several million unindexed rows in a table would respond if you did a full table scan or several smaller scans concurrently against it and you will understand how exchange feels.