IT Trenches:

diagnostics

Jun 29 2009   8:15PM GMT

Another threat to watch out for - Ants in the keyboard!



Posted by: Troy Tate
computer vulnerability, troubleshooting, hardware, hardware failure, diagnostics, research, computer peripheral, keyboard failure, keyboard

On Friday I posted a tongue-in-cheek type posting about a worm taking down a laptop. Not necessarily big news but something different to see as a risk to computing equipment.

Another similar story came to my attention today. It has to do with ants in a membrane switch type keyboard. Check out the article Ant Farm In The Keyboard. Hey… isn’t it about sharing troubleshooting information between peers? wink-wink - nudge-nudge

Thanks for reading. Have a great day and let’s continue to be good network citizens!

Feb 11 2009   8:08PM GMT

Tracking down that user/computer that locks AD accounts



Posted by: Troy Tate
Data security, administration, analysis, antivirus, anti-virus, diagnostics, howto, information security, malicious activity, malware, Microsoft, Microsoft Windows, Active Directory, AD, network security, Password, policy enforcement, reporting, risk, risks, scanning, search, Security, security notification, tools, troubleshooting, Windows, password management, account management

With an environment spanning 18+ sites and more than 3000 computers around the globe, you could understand how challenging it would be to track down what device/user might be locking user accounts. There are tools out there that you can pay for that can help do this. However, Microsoft has some free tools that with a little testing and use will permit you to quickly track down where the account is being locked and address the situation.

We had a situation recently where malicious software got onto a couple of machines and attempted to use the Administrator account to login. We have account lockout on our Windows 2003 AD domain, so after the appropriate number of invalid tries the Administrator account was locked out in the domain. This is because the machines were members of the domain and the malware did not distinguish the local administrator account from the domain administrator when attempting to elevate authority. Note that we use least user authority in our environment so the malware was not able to spread beyond these two machines. We suspect the machines became infected due to out of date antivirus signatures.

Unfortunately, the antivirus we use did not alert us to the situation. The way we were alerted was by our Microsoft Systems Center Operations Manager (SCOM) implementation. It notified the SCOM admin that the domain Administrator account was locked. The operations team was then tasked with tracking down what or who was locking this account. This is where the Microsoft Account Lockout and Management Tools came in use and helped isolate the cause. Continued »


Nov 17 2008   7:44PM GMT

Surviving Cisco Telephony - SRST



Posted by: Troy Tate
administration, Cisco, VoIP, unified communications, IP telephony, DataCenter, PSTN, design, risk, diagnostics, vulnerability

As you may have seen in some of my previous posts the company I work for has implemented VOIP/IP telephony at some of our locations.

VOIP - IPT - QOS - COS on and on - Oh My!

CampIT Enterprise VOIP conference

VOIP virtual panel discussion

Recently we had a phone system outage at the largest of these sites. This was a site with a clustered Cisco CallManager solution. This outage lasted 4+ hours. We were definitely surprised that both members of the cluster failed at the same time and how long it took to recover. Since that time we obviously are working with our support vendor to find a better method of providing uptime to the phone system at this site. I am also looking at making sure my other sites are prepared in the event of a similar outage.

The solution for providing a backup to the CallManager cluster is called Survivable Remote System Telephony (SRST). Think of this as CallManager light. A limited number of the phones still have connectivity and can make/receive calls. I say “limited” because the SRST function is dependent on the PSTN gateway hardware. A larger gateway can support more users. The current gateway we had was a Cisco 2821 series router. This would support 96 users. A Cisco 3825 will support 175 users.

One thing I understand though is that you cannot necessarily specify which phones will get serviced by SRST. The phones are serviced on a first-come-first-served basis. This could be an issue if there are phones that should be serviced and an outage is occurring. Unneeded phones would need to be disconnected from the network to provide capacity to support the critical phones.

Hopefully this will be the last of 4+ hour outage for the phone systems at this site and none will happen at my others. The Cisco solution has been very good for my organization and so far has been very reliable with the exception reported here.

Thanks for continuing to read my blog and hope you have a great day on the technology frontier wherever that may be for you!


Nov 11 2008   4:07PM GMT

Did you see this? - Microsoft SharePoint Toolkit



Posted by: Troy Tate
administration, Networking, tools, Microsoft Windows, Monitoring, Development, reporting, internet, WAN, LAN, debugging, performance monitoring, SharePoint, design, MOSS, troubleshooting, Performance, howto, network analysis, Metrics, awareness, diagnostics, toolkit, analysis

Many organizations are finding value in the Microsoft SharePoint technologies. Whether you use the free Windows SharePoint Services or the Microsoft Office SharePoint Server, your organization will gain a lot of value from using these services. To enhance your ability to manage these technologies, there is a project on Codeplex called the SharePoint Toolbox. Per the website, the purpose of this project is as follows:

This project includes powerful and useful tools and add-ons for SharePoint that help developers and IT pros implement SharePoint based solutions more quickly and managed them more effectively. Contributions will come from the Microsoft SharePoint Product Group, Microsoft SharePoint Online Services Group, Microsoft Information Technology Group, and Microsoft Consulting Services Group.

I have personally used the CopyTimer utility  to measure throughput from remote sites to a SharePoint server. It worked well and helped gather some excellent data about the site and global network performance.

Enjoy using these tools and give me some feedback on what you find useful and how SharePoint provides value to your organization.


Oct 6 2008   1:12PM GMT

Did you see this? - Process monitor now does TCP/UDP monitoring



Posted by: Troy Tate
administration, Networking, forensics, Security, tools, Microsoft Windows, Monitoring, reporting, internet, LAN, debugging, Data security, malware, performance monitoring, recovery, Microsoft, anti-virus, troubleshooting, Performance, howto, network analysis, Sandbox, packet capture, research, diagnostics, Sysinternals, toolkit, analysis

If you ever need to get under the covers of running Windows processes for investigating why a system is running slow, then the Sysinternals toolkit has an updated tool that will help you. Per the website:

Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity. It combines the features of two legacy Sysinternals utilities, Filemon and Regmon, and adds an extensive list of enhancements including rich and non-destructive filtering, comprehensive event properties such session IDs and user names, reliable process information, full thread stacks with integrated symbol support for each operation, simultaneous logging to a file, and much more. Its uniquely powerful features will make Process Monitor a core utility in your system troubleshooting and malware hunting toolkit.

Process Monitor runs on Windows 2000 SP4 with Update Rollup 1, Windows XP SP2, Windows Server 2003 SP1, and Windows Vista as well as x64 versions of Windows XP, Windows Server 2003 SP1 and Windows Vista.

I had previously talked about the Sysinternals Live website. This update to one of the excellent tools is well worth your time in investigating. Take a look at the updated tool here. The entire Sysinternals toolset can be found here.

If you have not used these tools yet, then you are definitely missing a critical item for being successful in your IT position. Check them out… it may save your reputation some time!


Oct 3 2008   7:59PM GMT

Did you see this? - Open Source Tools University



Posted by: Troy Tate
administration, Networking, Firewalls, forensics, Security, tools, Monitoring, reporting, internet, IT education, WAN, LAN, debugging, Data security, SSL, performance monitoring, blogging, design, anti-virus, troubleshooting, Performance, howto, network analysis, Sandbox, Metrics, wireshark, packet capture, research, blog, podcast, diagnostics, toolkit, analysis

If you are like me, you like those little goodie tools like nmap and wireshark that do something that is actually pretty complex but do it well and have a great following. I just came across this website that I am going to have to take some time to go through and find all of the nuggets it offers. Hope you get some use out of it too and let us know what you discover and how it made your job easier.

LoveMyTool

There are presentations on this site like the Wireshark IO Graph for Response Time Analysis (by Ray Tompkins).This should be a great online learning experience. You will find contributors like Sake Blok, a Wireshark Core Developer and Denny K Miu of StartupforLess.org - A Survival Guide for Bootstrapping Entrepreneurs


Oct 2 2008   12:00PM GMT

My server is hung! What do I do? - debugging resources



Posted by: Troy Tate
administration, tools, Microsoft Windows, reporting, debugging, performance monitoring, recovery, server, troubleshooting, Performance, howto, diagnostics, toolkit

Debugging a dump from a hung server may not be something you do every day, so you may want to engage with a Microsoft debug expert, however with this information as your guide you may find that you can narrow down a problem and save yourself a call.

My Server is hung - what do I do?

http://blogs.msdn.com/ntdebugging/archive/2008/09/12/red-alert-my-server-is-hung-what-do-i-do.aspx

If you need additional background on debugging, this article can get you started:

Basics of Debugging Windows

http://blogs.msdn.com/ntdebugging/archive/2008/08/28/basics-of-debugging-windows.aspx

 ·  Collect a kernel dump:  http://support.microsoft.com/kb/244139

·  Set up the debugger:  http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx

·  Know how to use the symbol server:  http://support.microsoft.com/kb/311503

Additional resources that you may find useful (including links to the tools, book recommendations, etc.):

·  Microsoft Debugging Tools

·  ADPlus – An automated way to use the cdb.exe to capture/create a usermode dump when a process hangs or crashes (more info - http://msdn.microsoft.com/en-us/library/cc265629.aspx or KB286350)

·  Public Symbols for Microsoft Operating Systems:

o Microsoft Public Symbol server : srv * DownstreamStore * http://msdl.microsoft.com/download/symbols

o   example: srv*c:\mysyms*http://msdl.microsoft.com/download/symbols

o   Microsoft Symbol packages http://www.microsoft.com/whdc/devtools/debugging/symbolpkg.mspx#d

·  Use !Analyze-v to gather additional information about the bugcheck and a bucket-id for your dump file.  The bucket-id can be submitted to Microsoft for review for similar crashes and resolutions.  Try using the Microsoft Online Crash Analysis to submit your crash dump bucket-id for possible follow up from Microsoft or for Microsoft to look for trends:  http://oca.microsoft.com/en/Welcome.aspx

·  For concepts, tools and information about the system architecture:  http://msdn.microsoft.com/en-us/default.aspx

·  Windows Internal 4th edition (by Mark E. Russinovich & David A. Solomon):  the whole book or Chapter 14 - Crash Dump Analysis

·  Advanced Windows Debugging (by Mario Hewardt & Daniel Pravat):  http://technet.microsoft.com/en-us/default.aspx

·  How to Access the User Mode Debugger from the Kernel Debugger

·  How can I find out why the Cluster Resource Monitor dumped – Access Violation

·  1394 Kernel Debugging Tips and Tricks [WinHEC 2004; 373 KB]

·  Debugging Windows Vista


Aug 18 2008   7:24PM GMT

Did you see this? - Online tools/tutorials - RingOfSaturn



Posted by: Troy Tate
administration, Networking, Storage, Security, tools, Monitoring, VoIP, web, reporting, internet, DataCenter, DataManagement, WWW, IT education, WAN, LAN, malware, design, website, troubleshooting, Performance, howto, network analysis, online identity, Metrics, wireshark, risk, packet capture, research, awareness, diagnostics

Ok, I admit it. I’m a network tool junkie. I constantly look for neat tools to perform tasks in the easiest manner possible and give me reliable information. This website from RingofSaturn.com is definitely one of the cooler online tool websites. Check out the browser sniffer tool if you are curious about what information your browser gives up while surfing the web. You might be surprised!

Check out the TCP/IP tutorial. It’s a quick easy read that you can share with those you are trying to explain how a network works.

Checkout this website. I guarantee that if networks are in your blood, you will find something of interest here.

Thanks for your time. Let’s be good network citizens together & practice safe networking!


Aug 18 2008   7:11PM GMT

Did you see this? - Boot CD tutorial



Posted by: Troy Tate
administration, forensics, Security, tools, Microsoft Windows, Monitoring, Mobile, DataCenter, DataManagement, antivirus, recovery, Microsoft, troubleshooting, Performance, howto, risk, packet capture, research, diagnostics, bootcd

How often have you needed to recover a Windows system or use some type of boot disk? It’s not easy to create a bootdisk in the current versions of Windows (XP or Vista). There’s still a need for this capability. One source of how-to information can be found on the BootCD.US website. I recommend that you check out this fine resource and test this capability before you are in need and don’t have a lot of time to wade through a lot of how-to documentation.

Thanks for your time. Let’s be good network citizens together & practice safe networking!


Jul 2 2008   1:57PM GMT

If no one is answering the front door - try the back door



Posted by: Troy Tate
Development, Browsers, web, CIO, DataManagement, WWW, customer service, blogging, design, website, troubleshooting, Performance, howto, Metrics, awareness, diagnostics

I recently went to Target and was going to look at my daughter’s wedding registry to see what she and her fiance had selected. When I got to the registry kiosks, there was a Target team member and a customer having problems getting into the service. The Target team member was on the phone apparently with another store or technical support. I heard things like “This is happening at all of the stores.” “We can’t get it to work.” “How do you reset this thing?”

Since there was another open kiosk, I thought I would try my luck and see what errors may appear. The main kiosk user page is intuitive and I immediately found the wedding registry icon and clicked it as any customer would. The application immediately responded with an error page describing some issues with scripting or something. Ahhhh… so I was receiving the same error as the other customer.

Well, the IT detective side came out in me and I started back over at the kiosk home page. Target designed this page with lots of options and ways to get to information that a customer may be looking for. Along the side of this page I found another link to get into the various registry areas, baby, wedding, etc. I clicked on that topic, navigated my way to the wedding registry and lo and behold… I was able to print out my daughter’s wedding registry while the other customer and the Target team members were still grumbling about the other kiosk.

I want to commend Target for providing multiple navigation means around their website. I would hope this experience would encourage more of the same for other vendors. I know, in IT, we like to restrict how many paths a user can go through an application to get to the same information, but in this case, Target did the right thing and provided good customer service.