HSCP (Hybrid scp) Documentation

I am searching on internet for fast file transfer which help me in vmdk or vhdx file transfer which are 20 GB around so i think why we are not using UDP for fast file transfer rather than TCP. even though UDP connectionless but it faster than TCP. I have internet 512 kbps not too fast because of that i think why not we are using UDP to make connection fast and reliable. as of now UDP is not reliable but we can made such development to make UDP as Powerful as TCP . I am also trying with Tsunami and others if you have any tool/software such as please feel free.Most IMP thing thanks to :https://ccportal.ims.ac.jp/en/software/hscp

Description

HSCP (Hybrid scp) is developing to transmit the large size file at high speed on the long distance and wideband infrastructure. It has achieved the fast transfer by changing the file transfer part of scp into the UDP (using UDT)

For Setup HSCP need following things:

How to build:

  • The build is done with the static link excluding a standard library.
    The dependence libraries are openssh-5.4p1, and udt4.6p.
    Udt4.6p is a special library for hscp that adds the correction to original udt4.6.
    Udt4.6p is modified udt.h, core.h, core.cpp, and ccc.cpp in src.
    It is necessary to do the build with os=UNIX for SunOS.
    It is necessary to mask pthread_join() in cleanup() function in udt4/src/api.cpp for AIX
  • Create directory for hscp.
  • Store hscp-x.x.x.tar, openssh-5.4p1.tar, and udt.sdk.4.6p.tar in it.
  • Execute ”make libudt.a” in udt4p/src. (or “gmake libudt.a”).
  • Execute “./configure” and “make” in openssh-5.1p1.
    -lz is not found:  yum install zlib-devel.x86_64 (or ia32)
    -lcrypto is not found:  yum install openssl-devel.x86_64 (or ia32)
  • Execute “make” in hscp directory. (or “gmake”).
    When the error goes out by the signal function, two lines are added to openssh-5.4p1/openbsd-compat/bsd-misc.h.
  • /* wrapper for signal interface */
    #ifndef __cplusplus /* this line */
    typedef void (*mysig_t)(int);
    mysig_t mysignal(int sig, mysig_t act);
    #endif /* this line */

Run “make” again.

  • Successful build version for openssh, 5.1p1, 5.2p1, 5.3p1, 5.4p1.
  • Successful build version for UDT, 4.4p, 4.5p, 4.6p.
  • You have to do same step above from 1 to 8 to client as well as server machine.

How to install:

For use both the server and the client

  • HSCP is put on /usr/local/bin (It is necessary to set path).
  • File mode and owner of HSCP are changed as well as scp.
  • “hscp.conf” is preserved in /usr/local/etc.
  • File mode of “hscp.conf” is set 644 and owner is set to equally to scp.

For only use the client (when you install it in the system)

  • Installs them as well as 1-4 on.
  • “ServerAccept” parameter of “hscp.conf” is set 0 or mask.

 For only use the client (When it is not possible to install it in the system)

  • PATH is passed by arranging HSCP suitably ($HOME/bin etc.).
  • “hscp.conf” is put on $HOME/etc.
    If “hscp.conf” is not found or that cannot start HSCP from sshd, it doesn’t start as a server.
    Definition file “hscp.conf” gives priority and reads $HOME/etc.

How to setup:

Setup for client

  • Set the following values of “hscp.conf” appropriately.
  • Please set it with “SshPath” parameter when path is not “/usr/bin/ssh” of ssh (nether error).
    /usr/bin/ssh: No such file or directory
  • Please set it with “RshPath” parameter when path is not “/usr/bin/rsh” of rsh (nether error).
    /usr/bin/rsh: No such file or directory
  • When the error below is displayed when executing it, it is necessary to set “UDPRecvBufSize” paramter small.
    connect: Connection setup failure: unable to create/configure UDP socket: No buffer space available.
    However, it leads to deterioration in the transmission rate if the buffer is small.
    The reference value is equivalent with size ( MaxPktSize(1500) * 8192 = 12288000 ) of the buffer of UDT.
    There is an example that cannot be used so far if it doesn’t drop to about 768000 with AIX, SunOS, and MacOSX.
  • Client port number is assigned automatic.
    If you want to fix to range, please set bellow parameters.
    UDPClientStartPort: The first number of UDP port that stands by client
    UDPClientEndPort:   The last number of UDP port that stands by client
    The maximum range is limited to 256.

Setup for server

  • Set the following parameters of “/usr/local/etc/hscp.conf” appropriately.
    UDPStartPort: The first number of UDP port that stands by server
    UDPEndPort: The last number of UDP port that stands by server
    18001-18032 is set in default. The maximum range is limited to 256.
  • Firewall should be able to be passed UDP port within the range of the above-mentioned and the setting.
  • Not only the machine but also that when operating it on the site is necessary for setting firewall.

How to use:

A basic usage is the same as scp.
The usages other than a simple local<->remote forwarding are not verified.
The switch enhanced from scp is as follows.

-R   It executes it with rsh instead of ssh.
-I 1  The display mode is set by the numerical value of 1-3.

default same as scp

1    The display of the transmission rate etc. changes line and is displayed every second.

2    The same display as mode1 is displayed without changing line.

3    When the transmission ends, only the result of the transmission rate etc. is displayed.

Other supported switch (checked)
-l 500000     Limited bandwidth (unit for parameter is kbit/sec) supported 0.9.12 later

Example:

Create 1 GB file with dd command:

dd if=/dev/zero of=file.txt count=1024 bs=1048576

above command will create 1 GB file with zeros.

From client side run following command:

[root@localhost opt]# hscp -I 1 -P 9922 -l 1000000 file.txt root@ip-address:/tmp

root@ip-address’s password:

SndR(Mb/s)  RTT(ms)  CWnd  PktSendPd  RACK  RNAK  AvSndBfSz   Send(%)   Etime

183.848    2.656  5674    37.12us    62     3     -45000   3.458%    00:01

387.715    2.455  5433    34.23us    94     9    -351000   8.000%    00:02

349.757    2.277  4974    44.63us    93     4    1156500  11.899%    00:03

333.698    2.550  6143    33.37us    94     2    -310500  15.899%    00:04

303.765    2.193  5235    41.45us    95     4    -366000  19.493%    00:05

325.179    2.118  5028    45.64us    93     5     606000  23.188%    00:06

321.887    2.557  5424    35.12us    82     2    -208500  26.544%    00:07

285.786    2.066  5386    36.75us    93     4    -174000  29.866%    00:08

441.561    3.080  5420    26.73us    94     2    -106500  34.985%    00:09

355.423    2.471  5752    32.68us    94     7    -270000  39.121%    00:10

310.101    2.614  4901    38.91us    93     8    -333000  42.714%    00:11

316.160    4.133  5739    63.35us    74    14    -133500  46.206%    00:12

320.195    2.373  5440    33.81us    93     2    -201000  49.901%    00:13

365.481    3.256  5788    41.14us    94     5    -361500  54.172%    00:14

279.482    2.057  5434    39.51us    83     2    -112500  57.020%    00:15

523.115    4.168  5463    20.34us    94     1    -100500  63.088%    00:16

407.514    2.122  5329    37.10us    84    29     -33000  67.766%    00:17

293.100    1.952  5272    40.47us    94     4    -267000  71.224%    00:18

317.846    2.355  5627    38.04us    92     4    -316500  74.919%    00:19

348.136    2.776  5468    36.73us    94     4    -235500  78.953%    00:20

360.624    2.097  4868    35.18us    95     3     -48000  83.123%    00:21

328.812    3.512  6075    31.85us    85     3    -327000  86.615%    00:22

342.449    2.709  5540    34.11us    93     3    -343500  90.581%    00:23

378.278    2.921  5729    36.33us    95     4    -370500  95.022%    00:24

330.147    2.498  5342    41.23us    94     3    -358500  98.853%    00:25

274.490    2.371  5938    41.71us    34     1    -106500 100.000%    00:26

file.txt                             1073.742MB      42.197MB/s    25.446sec

[root@localhost opt]#

Here my server port is 9922 therefore -P 9922 is required

Advertisements
Posted in Linux

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: