Transfer files to/from MeSU Supercomputer

There are several ways for transferring data files from local machines to HPC systems for that you can use the following protocols:

For Unix system (Linux, MacOs):

  • ftp (File Transfer Protocol) with explicit SSL encryption.
  • scp (secure copy) : A scp program is part of recent releases of Unix-based systems (Linux and Apple OS X) requirements for users:
    • scp on the local machine
    • a connection to the SU campus network.
  • sftp (Secure Shell File Transfer software) requirements for users:
    •  SSH on the local machine
    • a connection to the UPMC campus network.

For Windows :

  • A SSH Secure File Transfer Utility is available at this address.

Transferring files Using sftp

To start the Secure File Transfer application, open a terminal and enter the command below. When prompted enter your Password.

sftp mesu.dsi.upmc.fr
Password:  Connected to mesu.dsi.upmc.fr

Once connected to mesu.dsi.upmc.fr. sftp>  you can use different commands for example:

  • To  navigate through the remote system’s file hierarchy :
pwd cd subdirectory
  • To print the local directory:
lpwd
Local working directory: /Users/demouser
  • To download files from MeSU, :
 get mesufile
  • To copy the remote file to a different name by specifying the name afterwards:
 get mesufile newlocalfile
  • To transfer files to MeSU use the put command:
 put localfile
  • To copy a local directory:
 put -r subdirectory
  • To return to the local session type:
 exit

Using scp

To start scp open a terminal and enter the command below, when prompted enter your Password.

Caution: Don’t forget to change to the directory where the files are located in your local machine (cd directory_containing_data_files).

scp filename username@mesu.dsi.upmc.fr:~/subdirectory/

Filename = The name of the file to be transferred,

Username= Your login name,

Subdirectory= The subdirectory in MeSU home directory to which files are being transferred.

Warning: username@ can be omitted if your login name on MeSU is the same as on your local machine.

 

Once connected to mesu.dsi.upmc.fr  you can use different commands for example:

  • To transfer multiple files from your local machine to your home directory on MeSU.
scp file1 file2 file3 username@mesu.dsi.upmc.fr:~/subdirectory/
  • Wildcards can also be used for multiple file transfer (e.g. all files with .txt extension):
scp *.txt username@mesu.dsi.upmc.fr:~subdirectory/
  • To transfer files from MeSU to your local machine, from your local machine, issue the command. The transferred file will be written in the current local directory.

Warning: username@ can be omitted if your login name on MeSU is the same as on your local machine.

scp username@mesu.dsi.upmc.fr:~/subdirectory/filename .
  • Wildcards can also be used for multiple file transfer (e.g. all files with .txt extension):
scp username@mesu.dsi.upmc.fr:~/subdirectory/\*.txt .
  • It is also possible to transfer a whole directory at once using the command:
scp -r local_subdirectory/ username@mesu.dsi.upmc.fr:~/subdirectory/

Download from internet to MeSU

The following example shows how to download FastaX software from internet directly to the cluster

wget https://github.com/Kitware/CMake/archive/master.zip