Tuesday, August 14, 2012

What are useful .screenrc settings?

## gyaresu's .screenrc 2008-03-25
# http://delicious.com/search?p=screenrc
# Don't display the copyright page
startup_message off
# tab-completion flash in heading bar
vbell off
# keep scrollback n lines
defscrollback 1000
# Doesn't fix scrollback problem on xterm because if you scroll back
# all you see is the other terminals history.
# termcapinfo xterm|xterms|xs|rxvt ti@:te@
# These will let you use 
bind -c selectHighs 0 select 10 #these three commands are 
bind -c selectHighs 1 select 11 #added to the command-class
bind -c selectHighs 2 select 12 #selectHighs
bind -c selectHighs 3 select 13
bind -c selectHighs 4 select 14
bind -c selectHighs 5 select 15


bind - command -c selectHighs   #bind the hyphen to 
                                #command-class selectHighs 


screen -t rtorrent  0   rtorrent        #screen -t tunes        1       ncmpc --host=192.168.1.4 --port=6600 #was for connecting to MPD music server.
screen -t stuff         1
screen -t irssi         2       irssi
screen -t dancing   4       
screen -t python    5   python
screen -t giantfriend   6       these_are_ssh_to_server_scripts.sh
screen -t computerrescue    7   these_are_ssh_to_server_scripts.sh
screen -t BMon          8       bmon -p eth0
screen -t htop          9       htop
screen -t hellanzb  10  hellanzb
screen -t watching  3   #screen -t interactive.fiction  8
#screen -t hellahella   8       paster serve --daemon  /home/gyaresu/downloads/hellahella/hella.ini     

shelltitle "$ |bash"
# THIS IS THE PRETTY BIT
#change the hardstatus settings to give an window list at the bottom of the                                                                        
##screen, with the time and date and with the current window highlighted                                                                            
hardstatus             alwayslastline                                                                                                                          #hardstatus string '%{= mK}%-Lw%{= KW}%50>%n%f* %t%{= mK}%+Lw%< %{= kG}%-=%D %d %M %Y %c:%s%{-}'
hardstatus string '%{= kG}[ %{G}%H %{g}][%= %{= kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B} %d/%m %{W}%c %{g}]'


 # don't use the hardstatus line for system messages, use reverse video instead
 # (we'll be using it for the list of tab windows - see hardstatus alwayslastline
 # below)
 hardstatus off

 # use the caption line for the computer name, load, hstatus (as set by zsh), & time
 # the caption line gets repeated for each window being displayed (using :split),
 # so we'll use color cues to differentiate the caption of the current, active
 # window, and the others.
 #    always                  - display the caption continuously.  Since
 #                              hardstatus is 'alwayslastline', it will be on the
 #                              next to last line.
 #    "%?%F"                  - if (leading '%?') this region has focus ('%F') 
 #                              (e.g. it's the only region being displayed, or,
 #                              if in split-screen mode, it's the currently active
 #                              region)
 #      "%{= Kk}"               - set the colorscheme to blac[k] on grey (bright blac[K]),
 #                                with no other effects (standout, underline, etc.)
 #    "%:"                    - otherwise ('%:' between a pair of '%?'s)
 #      "%{=u kR}"              - set the colorscheme to [R]ed on blac[k], and
 #                                underline it, but no other effects (bold, standout, etc.) 
 #    "%?"                    - end if (trailing '%?')
 #    "  %h "                 - print two spaces, tthne the [h]ardstatus of the
 #                              current tab window (as set by zsh - see zshrc) and
 #                              then another space.
 #    "%-024="                - either pad (with spaces) or truncate the previous
 #                              text so that the rest of the caption string starts
 #                              24 characters ('024') from the right ('-') edge of
 #                              the caption line.
 #                              NOTE: omitting the '0' before the '24' would pad
 #                              or truncate the text so it would be 24% from the
 #                              right.
 #    "%{+b}                  - add ('+') [b]old to the current text effects, but
 #                              don't change the current colors.
 #    " %C:%s%a %D %d %M %Y"  - print the [C]urrent time, a colon, the [s]econds,
 #                              whether it's [a]m or pm, the [D]ay name, the [d]ay
 #                              of the month, the [M]onth, and the [Y]ear.
 #                              (this takes up 24 characters, so the previous
 #                              pad/truncate command makes sure the clock doesn't
 #                              get pushed off of the caption line)
 #    "%{= dd}"               - revert to the [d]efault background and [d]efault
 #                              foreground colors, respectively, with no ('= ')
 #                              other effects.
 #  other things that might be useful later are
 #    " %H"                   - print a space, then the [H]ostname.
 #    "(%{.K}%l%{-}):"        - print a '(', then change the text color to grey
 #                              (aka bright blac[K]), and print the current system
 #                              [l]oad.  Then revert to the previous colorscheme
 #                              ('%{-}') and print a close ')' and a colon.
 #                              NOTE: the load is only updated when some other
 #                              portion of the caption string needs to be changed
 #                              (like the seconds in the clock, or if there were a
 #                              backtick command)
 #    "%0`"                   - put the output of a backtick command in the line
 #    "%-024<"                - don't pad, just truncate if the string is past 24
 #                              characters from the right edge
 #    "%-="                   - pad (with spaces) the previous text text so that
 #                              the rest of the caption string is justified
 #                              against the right edge of the screen.
 #                              NOTE: doesn't appear to truncate previous text.
 caption always           "%?%F%{= Kk}%:%{=u kR}%?  %h %-024=%{+b} %C%a %D %d %M %Y%{= db}"
 # use the hardstatus line for the window list
 #    alwayslastline      - always display the hardstatus as the last line of the
 #                          terminal
 #    "%{= kR} %-Lw"      - change to a blac[k] background with bright [R]ed text,
 #                          and print all the tab [w]indow numbers and titles in
 #                          the [L]ong format (ie with flags) upto ('-') the
 #                          current tab window
 #    "%{=b Kk} %n%f %t " - change to grey (bright blac[K]) background with
 #                          [b]old blac[k] text, with no other effects, and print
 #                          the [n]umber of the current tab window, any [f]lags it
 #                          might have, and the [t]itle of the current tab window
 #                          (as set by zsh - see zshrc).
 #                          NOTE: the color match with the caption line makes it
 #                          appear as if a 'tab' is dropping down from the caption
 #                          line, highlighting the number & title of the current
 #                          tab window.  Nifty, ain't it)
 #    "%{-}%+Lw "         - revert to the previous color scheme (red on black)
 #                          and print all the tab [w]indow numbers and titles in
 #                          the [L]ong format (ie with flags) after ('+') the
 #                          current tab window.
 #    "%=%{= dd}"         - pad all the way to the right (since there is no text
 #                          that follows this) and revert to the [d]efault
 #                          background and [d]efault foreground colors, with no
 #                          ('= ') other effects.
 hardstatus alwayslastline "%{= kR} %-Lw%{=b Kk} %n%f %t %{-}%+Lw %=%{= dd}"



bindkey "^[Od" prev  # ctrl-left
bindkey "^[Oc" next  # ctrl-right


Wednesday, July 4, 2012

使程序在Linux下后台运行


使程序在Linux下后台运行

一、为什么要使程序在后台执行
我们计算的程序都是周期很长的,通常要几个小时甚至一个星期。我们用的环境是用putty远程连接到日本Linux服务器。所以使程序在后台跑有以下三个好处:
1:我们这边是否关机不影响日本那边的程序运行。(不会像以前那样,我们这网络一断开,或一关机,程序就断掉或找不到数据,跑了几天的程序只能重头再来,很是烦恼)
2:不影响计算效率
2:让程序在后台跑后,不会占据终端,我们可以用终端做别的事情。
二、怎么样使程序在后台执行
方法有很多,这里主要列举两种。假如我们有程序pso.cpp,通过编译后产生可执行文件pso,我们要使pso在linux服务器后台执行。当客户端关机后重新登入服务器后继续查看本来在终端输出的运行结果。(假设操作都在当前目录下)
方法1在终端输入命令:
 # ./pso > pso.file 2>&1 & 
解释:将pso直接放在后台运行,并把终端输出存放在当前目录下的pso.file文件中。
      当客户端关机后重新登陆服务器后,直接查看pso.file文件就可看执行结果(命
令:#cat pso.file )。
  
方法2在终端输入命令:
# nohup  ./pso > pso.file 2>&1 &
解释:nohup就是不挂起的意思,将pso直接放在后台运行,并把终端输出存放在当前
目录下的pso.file文件中。当客户端关机后重新登陆服务器后,直接查看pso.file
文件就可看执行结果(命令:#cat pso.file )。
三、常用任务管理命令
# jobs      //查看任务,返回任务编号n和进程号
# bg  %n   //将编号为n的任务转后台运行
# fg  %n   //将编号为n的任务转前台运行
# ctrl+z    //挂起当前任务
# ctrl+c    //结束当前任务

注:如果要使在前天执行任务放到后台运行,则先要用ctrl+z挂起该任务,然后用bg使之后台执行。



附:


在Linux中,如果要让进程在后台运行,一般情况下,我们在命令后面加上&即可,实际上,这样是将命令放入到一个作业队列中了:
$ ./test.sh &
[1] 17208

$ jobs -l
[1]+ 17208 Running                 ./test.sh &
对于已经在前台执行的命令,也可以重新放到后台执行,首先按ctrl+z暂停已经运行的进程,然后使用bg命令将停止的作业放到后台运行:
$ ./test.sh
[1]+  Stopped                 ./test.sh

$ bg %1
[1]+ ./test.sh &

$ jobs -l
[1]+ 22794 Running                 ./test.sh &
但是如上方到后台执行的进程,其父进程还是当前终端shell的进程,而一旦父进程退出,则会发送hangup信号给所有子进程,子进程收到hangup以后也会退出。如果我们要在退出shell的时候继续运行进程,则需要使用nohup忽略hangup信号,或者setsid将将父进程设为init进程(进程号为1)
$ echo $$
21734

$ nohup ./test.sh &
[1] 29016

$ ps -ef | grep test
515      29710 21734  0 11:47 pts/12   00:00:00 /bin/sh ./test.sh
515      29713 21734  0 11:47 pts/12   00:00:00 grep test
$ setsid ./test.sh &
[1] 409

$ ps -ef | grep test
515        410     1  0 11:49 ?        00:00:00 /bin/sh ./test.sh
515        413 21734  0 11:49 pts/12   00:00:00 grep test
上面的试验演示了使用nohup/setsid加上&使进程在后台运行,同时不受当前shell退出的影响。那么对于已经在后台运行的进程,该怎么办呢?可以使用disown命令:
$ ./test.sh &
[1] 2539

$ jobs -l
[1]+  2539 Running                 ./test.sh &

$ disown -h %1

$ ps -ef | grep test
515        410     1  0 11:49 ?        00:00:00 /bin/sh ./test.sh
515       2542 21734  0 11:52 pts/12   00:00:00 grep test
另外还有一种方法,即使将进程在一个subshell中执行,其实这和setsid异曲同工。方法很简单,将命令用括号() 括起来即可:
$ (./test.sh &)

$ ps -ef | grep test
515        410     1  0 11:49 ?        00:00:00 /bin/sh ./test.sh
515      12483 21734  0 11:59 pts/12   00:00:00 grep test
注:本文试验环境为Red Hat Enterprise Linux AS release 4 (Nahant Update 5),shell为/bin/bash,不同的OS和shell可能命令有些不一样。例如AIX的ksh,没有disown,但是可以使用nohup -p PID来获得disown同样的效果。
还有一种更加强大的方式是使用screen,首先创建一个断开模式的虚拟终端,然后用-r选项重新连接这个虚拟终端,在其中执行的任何命令,都能达到nohup的效果,这在有多个命令需要在后台连续执行的时候比较方便:
$ screen -dmS screen_test

$ screen -list
There is a screen on:
        27963.screen_test       (Detached)
1 Socket in /tmp/uscreens/S-jiangfeng.

$ screen -r screen_test

Tuesday, May 1, 2012

Mapping KEGG organism codes to NCBI RefSeq Ids

Mapping KEGG organism codes to NCBI RefSeq Ids

January 17th, 2012 by eric

I was working on a problem today that required matching up the 3-letter codes KEGG uses for identifying genomes to the NCBI RefSeq codes used for DNA molecules in the genomes I’m working with (e.g. NC_031415). Because I couldn’t find such a mapping file I had to do it myself. Here’s how.
First, I use linux command line tools to download the KEGG organism list and parse it to get the organism code and the NCBI ftp directory name:

wget http://www.genome.jp/kegg/catalog/org_list.html
awk '/Prokaryotes/,0' org_list.html | grep "show_organism?org" | cut -d'>' -f3 | cut -f1 -d'<' > org_a
awk '/Prokaryotes/,0' org_list.html | grep ftp | cut -d'/' -f6 > org_b
paste org_a org_b | sort -k2 > org_list

Then, I used lftp to find all of the RefSeq files within each directory. I just chose the ‘.rpt’ files, any of the unique files would do. I parsed the resulting file to get the ftp directory name and the RefSeq ID.

lftp -e 'open ftp://ftp.ncbi.nih.gov/genomes/Bacteria/; ls -d */*.rpt > refseq.rpt; bye'
awk '{print $9}' refseq.rpt | cut -f1 -d'.' | cut -f1 -d'/' > ref_a
awk '{print $9}' refseq.rpt | cut -f1 -d'.' | cut -f2 -d'/' > ref_b
paste ref_b ref_a | uniq -f1 | sort -k2 > ref_list

Then, I joined the two files on their common column: the NCBI ftp directory name. I don’t care about the directory name so then I make a new file without it.

jplace.pl refseqid refseqacclist > kegg2refseq.csvoin -j2 -t$'\t' org_list ref_list > kegg2refseq.txt
cat kegg2refseq.txt | cut -f2,3 > kegg_refseq.tmp
From here I’ve written a script (download: idreplacer) that uses the lproks_1.txt and lproks_2.txt files provided by NCBI to interconvert names and IDs.
./idreplacer.pl refseqacc refseqid kegg_refseq.tmp | ./idre

et voila! Here’s the final KEGG to RefSeq ID mapping file.
Postscript: Here is a database table from MicrobesOnline that does a similar task, mapping KEGG organism to NCBI taxonomy: KEGG2Taxonomy

Wednesday, April 18, 2012

rpy2 for Ubuntu

# 1. install all dependence packages
 
sudo apt-get build-dep r-base
 
# 2. Install R with shlib 
 
        make distclean
        ./configure --enable-R-shlib
        make
        make install 
 
# 3. install rpy2 
 
   sudo easy_install rpy2

Thursday, April 12, 2012

R install ubuntu

INSTALLATION

To obtain the latest R packages, add an entry like

  deb http://cran.cnr.Berkeley.edu/bin/linux/ubuntu lucid/

   sudo apt-get update
   sudo apt-get install r-base
 
public key trouble

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys <KEY NUMBER>

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9

Thursday, April 5, 2012

Bioperl Parsing_BLAST

http://www.bioperl.org/wiki/Parsing_BLAST_HSPs

http://www.bioperl.org/wiki/Parsing_BLAST_results_into_per-query_files


CPAN: Automatically install dependencies without confirmation 
perl -MCPAN -e shell 
o conf prerequisites_policy follow
o conf commit 
 

PREPARING TO INSTALL

This is optional, but regardless of your subsequent choice of installation method, it will help to carry out the following steps. They will increase the likelihood of installation success (especially of optional dependencies).
  • Upgrade CPAN:
>perl -MCPAN -e shell
cpan>install Bundle::CPAN
cpan>q
  • Install/upgrade Module::Build, and make it your preferred installer:
>cpan cpan>install Module::Build
cpan>o conf prefer_installer MB
cpan>o conf commit
cpan>q


>tar xvfz BioPerl-1.6.1.tar.gz
>cd BioPerl-1.6.1
Now issue the build commands:
>perl Build.PL
>./Build test 
>./Build install
  

Thursday, March 29, 2012