It’s obviously critical to have a smoothly operating IT network, both wired and wireless. However, it’s even more critical to be able to quickly diagnose network and application problems with precise troubleshooting so that there is minimal downtime for everyone involved.

This kind of troubleshooting is what we do. So I have put together this case study about the problems one large client was having and how we resolved their network issues for them.

This client had a network of 1,600 virtual servers and 300 physical servers. It was a large-scale infrastructure that contained approximately 70,000-80,000 users in a metropolitan style configuration. There were 15 districts with 5 or 6 buildings in each.

They were experiencing a lot of network problems including huge delays in server performance. Unfortunately for them, they were flying completely blind and had no application or appliance of any kind to help them troubleshoot their massive network.

So they called us.

What Was Discovered

The first step was to collaborate with their IT team and dig deeper into the existing network.  We deployed our VisionPro managed IT solution which provides real-time network diagnostics.

Using our newly installed solution, we reviewed in detail the network and found a series of issues.

First, there was a bad network card. Then we found a server interface that was causing problems. Lastly, we found that there was an issue with the SQL database queries in the network. The servers were experiencing incredibly long 90 second response time delays.

We found these problems almost immediately with our VisionPro platform. Had we not deployed our solution, our clients might in some instances have never found these issues.

In the past, they would work to troubleshoot an issue using network “archeology.” This involves a network admin going to the data center or remote office and generating packet captures. Once this is done, it is then necessary to work through each capture in an attempt to identify the problem or problems.

Some of these packet captures can contain hundreds of MB of data, if not GBs, to churn through.

If the IT staff weren’t lucky and looking in the right place at the right time, it would be impossible to pinpoint the problem. So most issues using this method might take days to resolve if they could be found at all.

Going back to this 90-second response delay in the SQL database, their application team was looking at the network for the problem. However, once we identified the issue and provided the SQL team with the SQL queries that were related to the long delays, they responded back that the issue had been escalated to the manufacturer and that there would be a patch coming to fix the issue.

The upshot is that once the real problem was located it could be properly solved.

Precise Infrastructure Operation, Awareness, and Troubleshooting

In addition to getting the network up to speed, our troubleshooting efforts also discovered one amazing thing for the client. They were very excited about the fact that we had found a server on their system that nobody knew anything about.

Quite a thing to learn!

On the last day of our intervention, the traffic on the network at 6 am spiked over a period of five minutes to 490 million transitions. Usually, at the same time of day, there were 200-300 million transactions.

The client was surprised because they had no idea of what might be causing the traffic increase.  They had no way of knowing it was there before.

So it became a real highlight for them that through our platform they could now see that backups were running over the production network and that it could be impacting their infrastructure. They never before had a full picture of what that impact was.

As a result of our pointing out the network load, they were going to push the backup team to complete the backup migration to a dedicated network.

Client took over our troubleshooting platform

Our success in troubleshooting the network inspired the client to continue using our VisionPro platform and have us train up their own IT staff on its use.

So we did and they now carry the ball on monitoring their own network.

However, if they need it, we can still help them remotely, without leaving our head office in Dallas, to diagnose and resolve those network issues with which they are having difficulty.

The negative impact to their users from the lack of network performance was finally solved for them.

Needless to say, the client was very satisfied with the results of our work. They ended up with a unified network and application performance troubleshooting system that solves problems fast. It is simple, intelligent, and complete.