SFTP can sync 1021 files only

[dDIMA]

Hi.

I use BestSync to archive files from different SFTP servers. Best Sync can't download more than 1021 file. I need to restart task again to make sync complete. For example, if remote server has 2500 new files I need to restart task 3 times.
File #1022 and all next files have an error message in the log:

1==>2,\20150322065046-000598-001E.G.037.game.log.xz, 22.03.2015 10:05:01,3572,SFTP Error:
OpenFile:
sftpOpenFile:
remotePath: /...[truncate]... /20150322065046-000598-001E.G.037.game.log.xz
access: readOnly
createDisposition: openExisting,accessBackup
v3Flags: 0x1
Sen

All tasks have the same options:
- one way sync from 1 to 2.
- no backup, no include/exclude, no filters
- copy new empty folders, use intrermediate, verify file by the available methods
- retry 3 times
- no encryption, no renaming
- do nothing on delete
- run daily
- no speed control, no extra applications
- create _bestsync_meta.dat

Remote servers have vsftpd installed. Other SFTP clients download all files in the single session without any errors.
What can be wrong with either BestSync settings or with vsftpd settings?

[RiseFly]

BestSync has not file number limit.
Please confirm if you are using newest version (10.0.0.18), if not, please download and try the new version to see if the problem can be resolved.

[dDIMA]

RiseFly wrote:
> BestSync has not file number limit.
> Please confirm if you are using newest version (10.0.0.18), if not, please
> download and try the new version to see if the problem can be resolved.

I have version:
BestSync 2015
English Version Ver10.0.1.9(64Bit)

the problem is still exists.

[RiseFly]

We have used a new version of SFTP library, and the released version is BestSync 2015 (ver 10.0.2.0). We have confirmed by our SFTP server that the new version can upload more than 1021 files in one time synchronization.

[dDIMA]

RiseFly wrote:
> We have used a new version of SFTP library, and the released version is
> BestSync 2015 (ver 10.0.2.0). We have confirmed by our SFTP server that the
> new version can upload more than 1021 files in one time synchronization.

I have upgraded BestSynt to "English Version Ver10.0.2.0(64Bit)". The issue is still actual.

[RiseFly]

BestSync has not this kind of limit, and we have confirmed by our SFTP server that BestSync can upload/download more than 1021 files in one synchronization. Please check if your server has the limit or problem.

[dDIMA]

Hello

The issue is still actual.
It seems that BestSync keeps files opened until end of task.

This is a log from source FTP server while sync task is active
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
1440 0 2469363
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
1440 0 2469363
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
1488 0 2469363
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
1728 0 2469363
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
1837 0 2469363
1837 is a max value. Best Sync can't syncronize more files.

This is a log from source FTP server when BestSync task was aborted.
ddima@STATUS:~$ cat /proc/sys/fs/file-nr
816 0 2469363

Note 1837-816 is equal to 1021.
I can increase limits but I think it could be better to close already processed files immediately after sync.

[RiseFly]

We have added a closefile command after upload a file to SFTP server.
Please try the new version 10.0.3.2 to see if the problem can be resolved.

[dDIMA]

Yes. Now it works good.
Thank you very much. The issue is resolved.