Page tree
Skip to end of metadata
Go to start of metadata

These are many, and can include:

  • The Chase server is not running on the local LAN and the internet connection or WAN connection to the server is unreliable or slow or overloaded
  • The server has hardware problems or software Operating System problems causing slow response
  • The server's network connection is unreliable or slow or overloaded
  • The server's hardware is too slow to keep up with the workload
  • Other software on the server is overloading the server
  • SQL server
  • Chase system is causing unnecessary load on the server
  • Chase data problem
  • Chase system problem.

     

 Top Level Server health check

To start with, you can run a health check on the Server. From the Resource Monitor on Windows, look at the following:

  • Memory
    High usage and page file swapping (Hard Page Faults per second). This will knock on to increasing disk usage. If memory usage is 95% without Hard Page Faults, it is probably not a problem, but a spike in memory usage could cause a sudden slow down of the server. 
  • Disk
    Beware of slow response times more than 30ms, or high disk usage stuck at 100% for long times. Try to see which files are being accessed repeatedly. Also look at the disk transfer speeds. If it seems to be stuck at very low speeds something may be wrong with the disk hardware, or the disks are shared with other servers and are being kept busy by other servers.
  • CPU
    High usage could be an issue, although single threaded processing will only max out one CPU core and will not show 100% CPU total usage. In fact the CPU is still battling to keep up with the load. It might help to look at the list of processes and sort them on CPU usage descending, if the server has 8 CPU cores, look out for a process that constantly consumes 12.5% CPU (100 divided by 8) this could be a problem. Pay attention to processes that consume an unusual amount of CPU, like anti-virus or server monitoring software. Some server CPUs are very slow at processing single threaded loads, significantly slower than modern laptop CPUs. Have a look at the speed benchmarks on https://www.cpubenchmark.net/.
  • Network usage
    Sometimes a 1 gigabit network adaptor can be overloaded by the number of packets rather than the amount of bandwidth utilised, but this is not usually a device that causes a slow down.

If the server is a Virtual Machine, it is possible that the host hardware is being kept busy by other VMs. This VM will be slow to respond, and not necessarily show as being overloaded.

It is preferable to run a health check on the host operating system.


 Network (TCP/IP)

TCP's job is reliable transfer of data, while finding the middle ground between a fast data transfer on the one hand and preventing network congestion on the other.

Two variables causing slow TCP performance is high Latency (pings) and high Packet Loss.


 Packet loss and Congestion

Flooding a network router with too much data will cause the router to drop the packets (one reason for packet loss) and these packets will need to be resent, causing the data transfer to slow down. So preventing congestion is important for TCP.

Packet loss is probably the most important indicator for TCP to slow down data transfer to avoid congestion, although sometimes packet loss is not caused by congestion. Packet loss caused by a network problem can cause the network to slow down unnecessarily.


 WiFi and interference

WiFi is known to be unpredictable. More WiFi devices in one area causes the radio signal noise to increase which causes packets to get dropped and re-sent. WiFi devices start transmitting at random times, and if 2 devices transmit at the same time both packets are lost and need to retry later. More devices cause this to happen more often and this is also called noise. This causes network slowdowns. Sudden spikes in WiFi network utilisation can also cause the network slowdowns that is impossible to reproduce. Bluetooth devices and Microwave ovens use the same 2.4GHz frequencies and these can also cause interference and network issues. The 5GHz WiFi channels are more reliable, but to remove this variable from testing, it is best to ALWAYS test using a LAN cable if possible.

Cabled LAN is not always free from problems such as packet loss, hardware issues and cable interference from power cables, or overloaded network devices. 

MacOS and even Windows is known to sometimes use the WiFi connection even when a LAN cable is connected. When testing WiFi versus cabled LAN, always switch off or disconnect the WiFi device to be sure it is not used.

If WiFi must be used, it might be helpful to run ping to the server in a separate window while testing and keep an eye on the ping results to make sure that there is no ping spikes or packet loss on the network.


 Long networks / High pings

On any new connection to a server, TCP starts off sending 2, or hopefully 10 packets. One packet being about 1450 bytes. TCP waits to hear back from the server confirming all packets were received, then tries to send more packets, increasing the speed until it detects that the network limit is reached. It means that a Chase server on a local LAN could take 15ms to download a 40kB file. From Cape Town to Johannesburg it would take 60ms to download the same file, and from the UK to Johannesburg it could take 480ms to download this file, no matter how fast the internet connection is. Keep in mind the browser only starts 2 or maybe more downloads at a time, and download files sequentially.

If the internet line is overloaded by people watching Youtube, while also trying to access hosted Chase on Azure 170ms away, the internet router will drop packets to tell the PCs to slow down. Then the local traffic will increase back up to a much faster speed, possibly increase speed every 5ms, while the traffic to Chase will only increase speed every 170ms, or ramp up +- 30 times slower, so by the time the Chase traffic tries to use its share of the available bandwidth, it is already taken by local servers. This is often confused with "international throttling" by the ISP. The end result is that Chase seems slow, but local internet seems fast, so the perception is that the internet line load cannot be causing Chase to be slow. In theory this can be corrected by creating router QOS rules to prioritise Chase traffic, but this tends to be very difficult to configure. This is the client's IT responsibility. 

Similarly, if a client has a Cape Town and Johannesburg office with their server running in Cape Town, but their global IT requirements state that WAN traffic is routed from Johannesburg to London (160ms) and then to Cape Town (additional 140ms), then users in Jhb will have a 300ms ping time to the server in Cape Town making the above problem even more obvious. In this case it is very important for the client's IT department to route the traffic efficiently between Jhb and Cape Town, which should be about 20ms instead.


 Ping

Ping can be used to test for a stable connection to the server. It is best to let it run for a few minutes and look at the packet loss percentage. The ping times should also preferably be low and consistent. Ping can be run with larger packet sizes to add some load to the network to test whether the network can handle additional load and still be stable.

Ping times for a Chase server running on the same LAN should be very low, 3ms or less.

Ping times between Johannesburg and Cape Town should be around 20ms, maybe 30ms.

Ping times from SA to Europe should be around 160-200ms.

If ping times seem unusually high for the distance, there might be a network problem.


 Trace Route

Trace route, using the Tracert tool, lists the routers or hops handling the traffic to the server with 3 ping times to each of those hops. This can be used to try and understand how the data is routed to the server, and this might show that networks are configured inefficiently.

Chrome extension ResponseTime Monitor will test a proxy server response if it exists, unlike ping and Tracert. This plugin can be installed on a user Chrome browser and configured to download a small icon file from the Chase server every 5 seconds. This will test the user PC and browser, the network to the Chase server and the performance of IIS serving small static files.


 Client Proxy servers

This is uncommon, but could be a problem. Clients can run Proxy servers with their internet routers to cache frequently downloaded files to speed up access to those files. If the Proxy server is under load it could slow down downloads from the Chase application on http and https connections, and confusingly ping replies will not show problems as these are not handled by the proxy server. Some badly configured networks can even run Chase traffic through the proxy server when the Chase server and users are in the same office. In this case you might find that accessing Chase while logged on to the server with IP address 127.0.0.1 is fast as it bypasses the proxy server. But usually proxy servers are fast enough not to cause concern, just keep it in mind.


 Local Chase servers

Access with IP address versus external URL Bad Routing or wrong IP address provided on LAN

Internet routers should handle accessing the Chase server using the URL used for external access from the internet. But sometimes the router sends the traffic out to the internet, just to receive it back again and pass it to the Chase server.

The DNS server should hand out the direct access IP address to the Chase server for users working on the LAN. You could use NSLOOKUP {ChaseURL} to find the IP address given - it should be the local LAN IP when tested from the LAN and the public IP when tested from the internet. It is worth asking the users to test using Chase with the LAN IP address if there is a chance that the network is configured wrong.


 DNS server issue

A slow DNS server can be an issue. Even if the DNS server provides the correct local IP address, the DNS server might still be slow to respond and this may also cause Chase to seem slow, In this case, if using the IP address instead of the Chase server name or Chase URL makes the system more responsive, then the DNS server may be causing Chase to be slow to respond. In theory the IP should be cached and not queried from the DNS server often, so this is unlikely to be an issue.



  • No labels