Disk Write Back Latency Issue with AWS Servers
Version: All
For the past two-plus years, Netcellent has migrated all of its in-house servers to AWS (Amazon Web Services). We also help some of our customers in implementing their servers on AWS. We can advise you that Elliott Business Software works in AWS. This article documents the "Disk Write Back Latency" issue we have seen in the AWS environment that may require attention. Even though we did briefly test Elliott Business Software in the Microsoft Azure environment as well and it works, our experience with Azure is limited. The same latency issues may be applicable to Azure as well, but we can't be sure.
Network Storage Is Separated from Your Server
First, you have to realize that in an AWS environment, your server and your storage are physically located in different boxes. The computer accesses the disk storage through the Ethernet network and, therefore, there is a latency. In many situations, the latency is not noticeable. But sometimes the latency is obvious and causes problems.
Network Storage Write Back Speed
The storage write back speed in most situations depends on your neighbors who share the same network storage. If they are also busy writing to that storage, your write back can take much longer. We noticed the write back speed can degrade significantly during the day while working fine at night time.
Mapped Network Drive vs. Local Drive vs C: Drive
- C: Drive: Typically, this is the fastest disk storage you can have and you won't notice latency.
- Other Local Drive: General speaking, these drive are not as fast as C: Drives, but you probably won't notice latency.
- Mapped Network Drive: These drives tend the be slow and you may notice network latency. Part of the issue has to do with how Windows treat network drives and anti-virus checking in Windows.
In our example, we save our source code in a mapped network drive G:. There are situations in which we save a file, say ABC.CBL to G:, and immediately proceed with compiling. Then, once in a while, we may receive a Network IO error message from the compiler. If we wait 30 seconds and compile it again, then it works. This is not a major issue because it does not happen often. If it does happen, we just wait a little bit and retry.
Example 2 - CSV Import Causing CS$ File Left Over
In another example, users placed the import file NEWITEM.CSV in the M:\Elliott8\Import folder, then proceeded with CSV import of a new item. The end result is that the interface will work the first time and the NEWITEM.CS$ will be left behind after the interface. Then, when the user tries to import the second time with NEWITEM.CSV, the system complains that NEWITEM.CS$ already exists and will not proceed. Just to explain how our CSV import works, this is what happens:
- In the above example, our import program will rename NEWITEM.CSV to NEWITEM.CS$ first. The import program does this to make sure no other users are currently appending new data to NEWITEM.CSV. If the rename is successful, then there's no sharing conflict.
- Then the import program will process NEWITEM.CS$.
- Upon successfully processing of NEWITEM.CS$, the system will close NEWITEM.CS$ and delete it. Here is where it may fail. We believe that the latency is the reason that the delete process fails and thus leaves the CS$ behind.
- When the user processes the next NEWITEM.CSV import, the system is not able to rename NEWITEM.CSV to NEWITEM.CS$ and hence gives the user an error message.
We solve this issue by asking the user to store the CSV file in a local folder like C:\Users\UserID\Import folder. Then the CSV import always works.
Example 3 - Copy & Paste Delay
Typically, we configure the AWS server as a remote desktop server. Users will login to the remote desktop server and run Elliott. In some situations, users will copy a file from a local computer and paste it into the remote desktop. For example, a user may choose to copy NEWITEM.CSV from his/her local computer and paste it into the M:\Elliott8\Import folder. The paste is done quickly and shows up in Windows Explorer. But if you navigate away from the M:\Elliott8\Import folder in Windows Explorer and come back quickly, you may notice the NEWITEM.CSV you just pasted into the M:\Elliott8\Import folder is not there. You can wait or press the F5 key to refresh. It may take 30 seconds before NEWITEM.CSV shows up in the folder. With the same principle, if this is an issue for you, we suggest that you to save the file in the C:\Users\UserID folder and you will not notice this kind of latency issue.
EMK