- Getting an account
- How do I login to Beocat?
- How do I transfer files to or from Beocat
- How do I compile my programs?
- How do I checkpoint jobs with Berkeley Lab Checkpoint/Restart (BLCR)?
- How do I submit jobs to the queue?
- How are the filesystems on Beocat set up?
- Additional tools
- Help! When I submit my jobs I get "Warning To stay compliant with standard unix behavior, there should be a valid #! line in your script i.e. #!/bin/tcsh"
- Help! When I submit my jobs I get "A #! line exists, but it is not pointing to an executable. Please fix. Job not submitted."
- Help! My jobs keep dying after 1 hour and I don't know why
- Help my error file has "Warning: no access to tty"
- Help! My job isn't going to finish in the time I specified. Can I change the time requirement?
- Help! My perl job runs fine on the head node, but only runs for a few seconds and then quits when submitted to the queue.
- Help! When using mpi I get 'CMA: no RDMA devices found' or 'A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces'
- How to get help
Getting an account
You will first need a Kansas State eID. You can request one here https://eid.k-state.edu if you do not have one already. After you have received your eID, login at https://account.beocat.cis.ksu.edu/user and fill out the requested information. Once your account is approved, you will get email from us indicating this along with some other helpful documents.
How do I login to Beocat?
When you have received login rights to Beocat with your eID, then you can proceed to log in to the system. Beocat currently only offers access through SSH (Secure Shell). The system only accepts your eID as the username, and your eID password as the password.
For logging in from Windows, I recommend using PuTTY. The download page will have many options, you want putty.exe. After downloading, there is no setup, simply double click on the exectuable. You will be greeted with the following settings window.
Enter beocat.cis.ksu.edu in the 'hostname' text box and make sure the 'SSH' radio button is checked. You may wish to save these settings under a name such as 'beocat' to save typing on future logins. After you have entered that information, hit 'Ok' and you will be prompted to save the host's key in the cache.
Click 'Yes' and you will arrive at the login screen.
Enter your login credentials here. Please note that your password will not show up at all, not even the traditional '*****' style.
You are now logged in to Beocat. Your login scripts may display different information.
If you are using OSX, Linux, or Unix, start your favorite terminal emulator. Use the program ssh for connecting to Beocat.
kuffs@[dr-orpheus] ~ % ssh beocat.cis.ksu.edu
or use firstname.lastname@example.org if your username on your local computer is different from your username on Beocat.
aaron@[dr-orpheus] ~ % ssh email@example.com Password for kuffs@W2K.CIS.KSU.EDU:
You will prompted for your password. Note that it will not be displayed, not even with '*'s.
After you are logged in via your client you will be greeted with a prompt similar to this.
Last login: Wed Mar 21 00:59:30 2007 from adsl-69-155-228-250.dsl.tpkaks.swbell.net kuffs@[loki] ~ %
You are now ready to start your work on Beocat.
How do I transfer files to or from Beocat
Once you have established how to log in to Beocat, it is likely that you have some files to transfer over from another system.
For Windows, I recommend WinSCP http://winscp.net. Download it and install it. When you first start it up you will be prompted with the window below.
Fill out the 'Host name' and 'User name' fields with the appropriate values. Optionally, you can also enter your password and save the connection as a bookmark. If you do not enter your password now, you will be prompted for it upon connecting. Click 'Login' and you will be prompted with a host key dialog similar to the one you were presented with in PuTTY.
Click 'Yes' on this one as well.
This is the working window for WinSCP. You can see that your local files are presented in the left-hand pane and your files on Beocat are presented in the right-hand pane. To transfer files or directories from one place to another, simply drag them from one side to the other.
For OSX, you can use an application called Cyberduck. When you first start it, you will see the following window:
Click on the "Open Connection" button, and you will be prompted with the following dialog box:
Click on the arrows next to "FTP (File Transfer Protocol)" and you will see this:
Choose "SFTP (SSH File Transfer Protocol)" and then enter "beocat.cis.ksu.edu" into the hostname field, your eID into the username field, and your eID password into the password field, as so:
Click connect, and you should connect to Beocat. It will look something like this, but with your Beocat files instead:
From here it is a simple drag and drop file interface that you should be used to.
For Linux, you can use an application called Filezilla. If you are using Ubuntu, install it with apt-get install filezilla. When you first start it, you will see the following window:
Click on the "Site Manager" button, and click "New Site." Name this site "Beocat." Like so:
Change "Servertype" to "SFTP - SSH File Transfer Protocol," Host to beocat.cis.ksu.edu, Port to 22, Logontype to "Normal," and enter your username and password, as so:
Click connect, and you may be prompted with the following window:
Click OK, and you should connect to Beocat. It will look something like this, but with your Beocat files instead:
From here it is a simple drag and drop file interface that you should be used to.
For OSX, Linux, or Unix use the program scp.
Open up a console or virtual terminal and run:
scp local_file beocat.cis.ksu.edu:~/
The same works for directories as well, with the -r flag
scp local_directory beocat.cis.ksu.edu:~/ -r
(As with regular ssh, you will need to include username@ if your Beocat username is not the same as your local username.)
To copy files from Beocat, use a syntax like this:
scp beocat.cis.ksu.edu:~/some_file .
Or for a directory:
scp beocat.cis.ksu.edu:/some_dir . -r
Read the scp manpage for more details.
How do I compile my programs?
ifort or gfortran
gcc and g++
mpif77 or mpif90
How do I checkpoint jobs with Berkeley Lab Checkpoint/Restart (BLCR)?
Information on checkpointing can be gained here: http://support.cis.ksu.edu/BeocatDocs/SunGridEngine#head-e5ead79b61d523530eeaf491746cbc23942f9817
Additional information on BLCR
See the BLCR Users Guide for more information
How do I submit jobs to the queue?
Job submission and other ways to interact with the queue can be found here: http://support.cis.ksu.edu/BeocatDocs/SunGridEngine#head-4f41b9904b0609bc97dd5e3921421a8f47f119df
How are the filesystems on Beocat set up?
Local / Shared
150TB total, quota'd *
glusterfs on top of xfs
Good enough for most jobs, will be used by default (will be faster on IB enabled nodes)
>30GB (varies per node)
Good for I/O intensive jobs
* Use 'quotacheck $USER' to check your quota limits
For most jobs you shouldn't need to worry, your default working directory is your homedir and it will be fast enough for most tasks. I/O intensive work should use /tmp, but you will need to remember to copy your files to and from this partition as part of your job script. This is made easier through the $TMPDIR environment variable in your jobs.
Example usage of $TMPDIR in a job script
1 #!/bin/bash 2 3 #copy our input file to $TMPDIR to make processing faster 4 cp ~/experiments/input.data $TMPDIR 5 6 #use the input file we copied over to the local system 7 #generate the output file in $TMPDIR as well 8 ~/bin/my_program --input-file=$TMPDIR/input.data --output-file=$TMPDIR/output.data 9 10 #copy the results back from $TMPDIR 11 cp $TMPDIR/output.data ~/experiments/results.$PBS_JOBID
You need to remember to copy over your data from $TMPDIR as part of your job. That directory and its contents are deleted when the job is complete.
GNU screen is a terminal multiplexer. It allows you to run multiple terminal sessions inside one. Not only that, but it allows you to disconnect and reconnect to your session. That means that if your network connection is interrupted, your work isn't.
GNU Screen is a somewhat complicated program. The following two links have decent writeups that will get you started on using it.
htop process manager
Htop is a better looking and more functional top. It displays processes in many different formats, with and without colorization. It also offers a menu system for selecting these options.
Nmon stands for "Nigel's Monitor". It is a handy program to monitor various information about the system, all easy selected when it's running.
This is the view you will get when you first start nmon.
As you can see, it has a list of keys to turn on the various monitoring subsystems. When you enable a monitor, its view pane will pop up in the program.
In this next screenshot, I have enabled the CPU, memory, and network monitoring systems.
Nmon also has a tool to create performance reports, called 'nmon analyser'. Check it out at http://www.ibm.com/developerworks/aix/library/au-nmon_analyser/
Help! When I submit my jobs I get "Warning To stay compliant with standard unix behavior, there should be a valid #! line in your script i.e. #!/bin/tcsh"
Job submission scripts are supposed to have a line similar to '#!/bin/bash' in them to start. I have recently started enforcing that these lines are valid, as I have had problems with people submitting jobs with invalid #! lines. When this happens the job fails and I have to manually clean it up. The warning message is there just to inform you that the job script should have a line in it, in most cases #!/bin/tcsh or #!/bin/bash, to indicate what program should be used to run the script. When the line is missing from a script, by default your default shell is used to execute the script (in your case /usr/local/bin/tcsh). This works in most cases, but may not be what you are wanting.
Help! When I submit my jobs I get "A #! line exists, but it is not pointing to an executable. Please fix. Job not submitted."
Like the above, error says you need a #!/bin/bash or similar line in your job script. This error says that while the line exists, the #! line isn't mentioning an executable file, thus the script will not be able to run. Most likely you wanted #!/bin/bash instead of something else.
Help! My jobs keep dying after 1 hour and I don't know why
Beocat has default runtime limit of 1 hour. If you need more than that, or need more memory, you'll want to look at the documentation here to see how to request it.
In short, when you run qsub for your job, you'll want to put something along the lines of '-l h_rt=10:00:00' before the job script if you want your job to run for 10 hours.
Help my error file has "Warning: no access to tty"
The warning message "Warning: no access to tty (Bad file descriptor)" is safe to ignore. It typically happens with the tcsh shell.
Help! My job isn't going to finish in the time I specified. Can I change the time requirement?
Generally speaking, no.
Jobs are scheduled based on execution times (among other things). If it were easy to change your time requirement, one could submit a job with a 15-minute run-time, get it scheduled quickly, and then say "whoops - I meant 15 weeks", effectively gaming the job scheduler. In fact, even the administrators cannot change the run-time requirement of a particular job. In extreme circumstances and depending on the job requirements, we may be able to manually intervene. This process prevents other users from using the node(s) you are currently using, so are not routinely approved. Contact Beocat support (below) if you feel your circumstances warrant special consideration.
Help! My perl job runs fine on the head node, but only runs for a few seconds and then quits when submitted to the queue.
Perl doesn't like getting called straight from the scheduler. However, there is a fairly easy workaround. Create a shell wrapper script that calls perl and its program.
For instance, I can create a script called runperl.sh that looks like this:
1 #!/bin/sh 2 #$ -l h_rt=1:00:00,mem=1G 3 /usr/bin/perl /path/to/my/perl_program.pl
Make this wrapper program executable:
chmod 755 runperl.sh
Then submit it with
Of course, the name of this script isn't important, as long as you change the corresponding chmod and qsub commands.
Help! When using mpi I get 'CMA: no RDMA devices found' or 'A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces'
This message simply means that some but not all nodes the job is running on have infiniband cards. The job will still run, but will not use the fastest interconnect we have available. This may or may not be an issue, depending on how message heavy your job is. If you would like to not see this warning, you may request infiniband as a resource when submitting your job. -l infinband=TRUE
How to get help
There are many sources of help for most Linux systems.
Unix man pages
Linux provides man pages (short for manual pages). These are simple enough to call, for example: if you need information on submitting jobs to Beocat, you can man qsub . This will bring up the manual for qsub.
GNU info system
Not all applications have "man pages." Most of the rest have what they call info pages. For example, if you needed information on finding a file you could use info find .
This documentation is very thoroughly researched, and has been painstakingly assembled for your benefit. Please use it.
Support can be contacted here. Please include detailed information about your problem, including the job number, applications you are trying to run, and the current directory that you are in.