scp - File too large

Issues related to applications and software problems
Post Reply
katumping
Posts: 3
Joined: 2018/06/12 03:45:59

scp - File too large

Post by katumping » 2018/06/12 03:57:29

hi, used 3 server. because, we using 3 different network, and must using bridge server to connected the other segmen
1. source server = 192.168.1.10 (SERVER1) (SOlaris 11)
2. Bridge Server = 192.168.1.11 (BRIDGE) / 172.16.12.10 (centos 7)
3. Destination Server = 172.162.12.100 (DESTServer) (solaris 10)

case:
1. Mounting Bridge Serve to Destination Server using IP 172.*/24 , using [mount //172.162.12.100/share /mnt/share]
2. from Source network, I try to SCP file to mounting at Bridge Server. My assumption the file will automated put on destination server
3. for file < 1 TB, the process copy always success. but When my file more than 1 TB, the error always said the file to large

ask:
how to set the SMB service will allow unlimited file, so I don't have any problem when to copy from source to Destination Server.

Thank you for your helping :oops:

User avatar
avij
Retired Moderator
Posts: 3046
Joined: 2010/12/01 19:25:52
Location: Helsinki, Finland
Contact:

Re: scp - File too large

Post by avij » 2018/06/12 07:02:18

Sounds like the underlying file system does not support files that big. Can you create a file bigger than 1TB directly on the destination server? On Linux this can be done with dd if=/dev/zero of=/tmp/bigfile bs=1048576 count=1200000, not sure about Solaris. This would create a 1.1TB file.

katumping
Posts: 3
Joined: 2018/06/12 03:45:59

Re: scp - File too large

Post by katumping » 2018/06/12 07:16:19

before its normal to copy even use >2 TB, from server 1(source) to server 3(destination). but in my case, will issue for our network, because terafile will take all traffic network. because that i used bridge server, with private.

for your advise,
dd if=/dev/zero of=/tmp/bigfile bs=1048576 count=1200000

is it ok, or not? if i only paste and execute without change anything?

thank you :oops:

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: scp - File too large

Post by tunk » 2018/06/12 11:07:13

You may want to replace /tmp/bigfile with either /share/bigfile or /mnt/share/bigfile (depending on where you want your original file).
The output of fdisk -l may also be helpful.

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: scp - File too large

Post by hunter86_bg » 2018/06/14 04:38:56

If the file system does not support files larger than 1 TB , you can always split the file on the solaris11 and then download the pieces to solaris10 and merge tjem with cat.

desertcat
Posts: 843
Joined: 2014/08/07 02:17:29
Location: Tucson, AZ

Re: scp - File too large

Post by desertcat » 2018/06/15 03:15:45

katumping wrote:
2018/06/12 03:57:29
hi, used 3 server. because, we using 3 different network, and must using bridge server to connected the other segmen
1. source server = 192.168.1.10 (SERVER1) (SOlaris 11)
2. Bridge Server = 192.168.1.11 (BRIDGE) / 172.16.12.10 (centos 7)
3. Destination Server = 172.162.12.100 (DESTServer) (solaris 10)

case:
1. Mounting Bridge Serve to Destination Server using IP 172.*/24 , using [mount //172.162.12.100/share /mnt/share]
2. from Source network, I try to SCP file to mounting at Bridge Server. My assumption the file will automated put on destination server
3. for file < 1 TB, the process copy always success. but When my file more than 1 TB, the error always said the file to large

ask:
how to set the SMB service will allow unlimited file, so I don't have any problem when to copy from source to Destination Server.

Thank you for your helping :oops:
If an option you could also go "Old School" : Copy the file to a 2TB drive and use "SneakerNet" to move it the the destination server, and then copy the file from the 2TB drive onto the destination drive. Some of these 2 TB drives are about the size of a a deck of playing cards, but slimmer and lighter. I use a 1 TB USB drive made by Seagate exactly for that purpose.

MartinR
Posts: 714
Joined: 2015/05/11 07:53:27
Location: UK

Re: scp - File too large

Post by MartinR » 2018/06/22 09:51:28

Further to desertcat's suggestion I remember some wise words from a tutor on my COBOL course back in the 1980s: "never underestimate the bandwidth of a bloke on a motorbike heading up the M1 with his panniers stuffed full of tapes". Networks may be faster and tapes/external drives/USB sticks have changed out of all recognition but the principle remains the same.

Post Reply