Using HSI with multiple HPSS systems

This section contains information on the following topics: 

Using HSI with multiple HPSS systems that are linked together via Junctions and Cross Cell authentication 

Using HSI to access multiple HPSS systems 

Opening connections to more than one HPSS system during a single HSI session 

Description of the Logical Drive pathname syntax that HSI uses to associate files with a connection during a session 

List of commands that are used to open, close, and switch between active sessions 

Examples of storing, retrieving, and copying files while connected to multiple HPSS systems. 



Using HSI to Access Multiple HPSS Systems


HSI has the ability to establish concurrent connections to more than one HPSS system at more than one site (or to the same HPSS site) within a single session. After the sessions are opened, normal HSI commands can be used to work with any of the sessions, as described below.  Files can be copied between the sites using one of the following mechanisms:


  • either 3rd-party transfers, which do not require the data to 

    flow through HSI memory

  • 1st party transfers, in cases where firewalls or other network connectivity problems prevent the movers from the source HPSS system from connecting to the HSI Gateway Server at the sink HPSS system.


The command that is used to copy files between sites is described below.


There are three requirements that must be met in order to use the Multi-HPSS feature:


  1. 1. you must have an account on the each of the HPSS systems that you wish to access
  2. 2. there must be network connectivity between the client system on which HSI is executing, and the HPSS site(s) that you wish to work with.
  3. 3. in order to transfer files using the 3rd-party mechanism, there must be network connectivity between the mover nodes on the source HPSS system, and the HSI Gateway server process on the sink HPSS system.

 


When working with multiple HPSS systems, HSI treats each connection as a "logical drive", borrowing from the familiar legacy of the early personal computers. In this environment, there are three key concepts: 

  1. 1. The notion of a current connection.
    This is initially established when the first connection is opened as HSI begins executing. It can change during execution as connections to other HPSS systems are opened, and as you switch between logical drives. (See below) Each connection has its own separate
    context, containing items such as:
           - The address of the HPSS server host
           - The logical drive letter that you (or HSI) assigned for the connection
           - Your home current directory
           - The current working directory
           - The classes of service that are available on the HPSS system 
  2. 2. The use of logical drive notation
    as a command prefix, and to reference files and directories on a particular HPSS system. Logical drive notation simply consists of a non-case-sensitive
    drive letter followed by the colon (:) character (with no intervening whitespace), for example:
           D:
    followed by the command or pathname, with no intervening spaces. 
  3. For example,
           d:ls would switch to connection D: and list the files in the current working directory.

           Ls C:*.c would list all of the files ending with the characters ".c" on the HPSS system associated with logical drive C:.

           get fromHPSS1 : D:file1 fromHPSS2 : F:file2 would fetch the file
    file1 from the HPSS system associated with drive D:, renaming it to the local file "fromHPSS1", and similarly would fetch the file file2 from the HPSS associated with drive F:, renaming it locally to ":fromHPSS2".

    Note that as of HSI version 2.4,the colon character (:) used to separate the local and HPSS pathnames, must be surrounded by whitespace, unlike earlier versions of HSI. This requirement was added in order to avoid ambiguity in the inclusion of colon characters in pathnames (see the Release Notes for more information) . You can tell which HSI version you are using by typing 
  4.        version 
  5. Drive letters are not case sensitive, so, the following are equivalent: 
    • f:get myfile 
    • F:get myfile 
  6. It is not necessary to use this notation; commands and pathnames without a drive prefix always reference the current connection. If the multi-HPSS feature of HSI is not used, commands and pathnames can be used just as in earlier versions of HSI. 
  7. Depending upon the context, the use of a standalone drive letter
           e.g. b: or Ls C:
    may be interpreted as either of the following: 
    • Set the current connection to the HPSS system associated with the logical drive (first example, above) or 
    • Reference the current working directory for the HPSS system associated with the logical drive (second example, above). 
  8. In commands where a filename is required, the use of a standalone drive letter will cause a usage error, for example:
           get P: 
  9. 3. Third-party copies.
    When copying files between HPSS systems, the default is to use a 3rd-party copy mechanism so that the data flows directly between HPSS systems, without having HSI in the middle of the transfer. See the CP command in Chapter 8 for more details. For some situations, such as copying files through a firewall, it may be necessary to use a store-and-forward mechanism to read data from either a local file or from an HPSS system inside the firewall, and then transfer it to an HPSS system outside the firewall. See the firewall command for more information on setting up store-and-forward transfers. 

Commands used for Multi-HPSS HSI Sessions

Commands used to access multiple HPSS systems are shown in the table below, and are also described in more detail in Chapter 8. The minimum abbreviation for each command is shown in the text preceding the "[" character.

  

Command 

Description 

open or connect

Establishes a new connection to an HPSS system. 

close

Closes an existing connection to an HPSS system. 

lscon[nections] or showc[on]

Shows a list of currently established connections, including the home directory and the current working directory for the connection 

lssi[tes]

Displays a list of site names that have been defined in the global .hsirc startup file, or in your private .hsirc file. 

drive:

As a standalone command, this changes the current active connection. 

drive:command

Changes the current active connection, then issues the specified command in the context of the new connection. 

setdrive

Changes the logical drive letter assigned to a connection 


Example Scenario for A Multi-HPSS Session


A typical interactive scenario for using HSI to communicate with multiple HPSS systems is described below. 


Set the prompt string so that we can tell which logical drive is our current active connection, and what our current working directory is within that connection. 

We want our prompt string to look like this:

      
drive_letter:[hostname]working_directory->


To accomplish this, the PS1 variable can be set in the hsirc file, or interactively as follows:

        PS1 = "%d[%H]%w3->"

which will yield a prompt sting similar to this:

      
C:[hpss]/home/gleicher-> 


The %w3 will cause long working directory pathnames to be displayed by truncating to at most 3 subdirectories on the right-hand part of the prompt string, and replacing characters in the middle of the pathname with the string "...", e.g. 

      
      S:[hpss05i]/users.../tools/scripts/backman->pwd
   pwd0: /users/sdsc/gleicher/HPSS/tools/scripts/backman
     

     

Establish initial connection. 


If the site administrator has set up the global hsirc file, or you have customized your private .hsirc file in your home directory, then all that is required is to startup HSI with no options (to connect to the site-defined default HPSS system), or
       hsi -s
sitename
to connect to the site called
sitename. Note that this is a case-sensitive option, which must exactly match the sitename contained in the hsirc stanza. 


If the -site option is not used on the command line, the following optional parameters may be used to connect to any HPSS system that you have an account on:
       hsi -h
hostname -p port -l login_name -A authentication method -k keytab filename 


The format of the login name varies, depending on the authentication method that you have chosen. For the combo method (-A combo)and the keytab method (-A keytab), the login name is your principal name on the HPSS system to which you are connecting. At most sites, this will be the same as your login name on any other machines at that site, but check with the HPSS administrator at the site to make sure. 

For kerberos authentication (-A kerberos),the login name is normally your principal name and the Kerberos Realm name, which is generally in uppercase, e.g.:


      
-A kerberos -l betty@UNM.EDU

(See "The 
HSI Startup File" for more information about the contents of the HSIRC file)


Normally, the site administrator will set up an HSI wrapper script containing all the information regarding hostnames, ports, and authentication mechanism(s) to be used, so that the optional parameters shown below are usually not needed. 


Establish connection to another site 


As above, if the site administrator has set up a global hsirc file, or if you have customized your private .hsirc file to add the appropriate information, the command:
        open -s
new_sitename
or
        open -d
drive_letter: 


can be used to establish a connection to the HPSS system designated by new_sitename. or drive_letter:


The lssites command can be useful when running interactively, if you need help remembering the exact sitename(s) and/or drive letter(s) that have been set up by the HPSS administrators. For example, at Gleicher Enterprises, the following sites are defined: 


  R:[hcdp01]/users/sdsc/gleicher->lssites 
  Site, Logical Drive....gel.62, "G:" 
  Site, Logical Drive....ornl.probe, "P:" 
  Site, Logical Drive....ornl.prod, "O:" 
  Site, Logical Drive....nersc.prod, "N:" 
  Site, Logical Drive....sdsc.prod, "R:" 
  Site, Logical Drive....sdsc.test, "T:" 
  Site, Logical Drive....iu.prod, "I:" 
     
Note that in this example, I am currently logged into HPSS at SDSC, and am going to open a connection to Indiana University:

  R: [hcdp01]/users/sdsc/gleicher->cd HPSS/tools/scripts/backman
 R:[hcdp01]/users/sdsc/gleicher->open -d I: -A combo
 Password: password
 I:[hpss]/home/gleicher->
      

Display the list of active sites after opening the second connection: 


 R:[hpss]/home/gleicher->lscon
 List of Currently Active Connections
 Current default connection handle: 3
       Han-    Remote          Remote            HSM  Fire- I/O    Ctl   Data Port  Cmd
Drive  dle    IP Address       Hostname         Level wall  Mode   Port   Min  Max Count
----------------------------------------------------------------------------------------
  R:   1 132.249.20.148   hcdp01.sdsc.edu        6.2.1   off normal  1217     0 65535 22
 HomeDir: /users/sdsc/gleicher
 pwd0: /users/sdsc/gleicher/HPSS/tools/scripts/backman
->I:   3 131.215.145.148 hpssb1.uits.indiana.edu 6.2.1  off normal  1218     0     0 13
 HomeDir: /home/gleicher
 pwd0: /home/gleicher

C:[hpss]/home/gleicher->
      

      

At this point, standard HSI commands can be used to list directories, store, retrieve and copy files using the logical drive pathname syntax, etc. 


In the next example, we want to store a copy of the local file "good_stuff" on one of the HPSS systems, and then replicate it on the other, in order to provide disaster recovery. We will do this by using the cp command, so that the data flows directly between the two HPSS systems, without having HSI in the middle of the transfer. 


First, store the file on logical drive R: in the directory big_project, which is an existing subdirectory directly underneath our home directory on both HPSS systems: 


 R:[hpss]/home/gleicher->put good_stuff : s:~/big_project/good_stuff
 put  good_stuff : /users/sdsc/gleicher/temp/good_stuff ( 1083916 bytes, 397.5 KBS (cos=6))
     

Next, copy the file from the S: drive to the I: drive: 

 R:[hpss]/home/gleicher->cp s:~/temp/good_stuff i:~/temp/good_stuff
 cp  R:/users/sdsc/gleicher/temp/good_stuff to I:/home/gleicher/temp/good_stuff (2001/09/16 23:36:18 1083916 bytes, 481.1 KBS (cos=12)
     

At this point, we are ready to close out the connection(s) and terminate the session(s). This can be done by simply exiting out of HSI. If we want to continue to work in one of the connections, but have no need to keep the connection open for the the other, then the close command can be used, e.g.:


   R:[hpss]/home/gleicher->close R: 

   I:[hpss05i]/home/gleicher-> 


Notice that when the current active connection is closed, HSI switches to one of the other active connections, in this case, the only other active connection was drive I:. If an attempt is made to close this connection, the following error message will be displayed: 


 I:[hpss05i]/home/gleicher/temp->close i:
 *** Cannot use <close> for the only open connection.
 Use  (or one of its aliases) to terminate HSI

Logging HSI Sessions

When HSI execution has terminated, a logfile containing a time-stamped record of all commands will be present in the hsilog file, which is normally written in the home directory.


This can be controlled by using the HSI "LOG" command, or by setting the HSI_LOGFILE environment variable as follows: 


If using CSH:
       setenv HSI_LOGFILE
somepath

If using KSH
       export HSI_LOGFILE=
somepath

If using SH (this will also work for KSH and BASH)
       export HSI_LOGFILE HSI_LOGFILE=
somepath 


somepath can be the reserved string none to disable automatic creation of a logfile. It can also contain metacharacters of the form %X as follows: 


  1. %H : is expanded to your local home directory pathname 

%N : is expanded to the non-canonical local hostname, i.e., the hostname without any domain name. " snuffles.sdsc.edu" would become "snuffles". 

%P : is expanded to the current HSI process ID 

%U : is expanded to the local login name. 


The default setting is usually determined by the HPSS administrator, and set in the HSI wrapper script. If the path already exists when HSI attempts to open the local logfile, HSI will try to append to it.