Missing files in transfer

[tpatil]

Hi,

I hope you are staying safe and healthy!

We have a few Ultimate licenses of BestSync application. One of that we are using to copy files from an S3 bucket to Azure SFTP. The application deals with mostly the audio files that are generally 1 to 15 MB (Not very large files). The frequency of the task is 5 min.

Issue:

In the source directory (S3 repository), files are being copied by some other program (at the client side).
While a file is being copied into the Source directory, at the same time, the BestSync runs and it misses to pick the file in that cycle.
We expect the file to be picked up in the next cycle but it never gets picked up. It is ignored until we restart the BestSync task manually.
e.g.
If a file is being copied to the source directory at 4:45 AM and at the same time BestSync starts running as scheduled, then BestSync cannot copy this file to the destination (I guess it must be locked by the program or BestSync). But it will not be picked up by BestSync in the next cycle too.

Could you please suggest any configuration changes to make it work correctly?

[RiseFly]

This is a timing problem.
Because Amazon S3 does not support setting the last modified timestamp of an upload file, BestSync scans and remembers the timestamp of the existing files in S3 after the synchronization. If files are updated by other software during BestSync synchronization, their timestamps are remember and BestSync thinks that these files are updated by itself.
At the next synchronization, BestSync compares the current timestamp and last remembered timestamp of a file to detect changed files. Because the remembered timestamp is update, so BestSync thinks the file is not changed, so it ignores the file.
We are try to find a solution of this problem.

[RiseFly]

To stop BestSync scanning and remembering the last updated timestamp of an uploaded file, you can uncheck the option "For reducing the scanning time, create "_Bestsync_Meta.dat_" file if necessary. " in "Miscellaneous" Tab of the task setting.
Please try this method to see if it can resolve this problem.

[tpatil]

Thanks for your response. I have made the change as described. It looks like the files are being transferred now. The scanning time has been increased by ~8 sec though but it can be ignored for now. Please let me know if you happen to resolve this issue by some way in the upcoming releases. I will update to the new version.

Thanks much

[RiseFly]

Thank you for the response, and glad to hear the problem can be solved.
To uncheck the option is the best solution in BestSync for the problem.