Mapping KEGG organism codes to NCBI RefSeq Ids

Mapping KEGG organism codes to NCBI RefSeq Ids

January 17th, 2012 by eric

I was working on a problem today that required matching up the 3-letter codes KEGG uses for identifying genomes to the NCBI RefSeq codes used for DNA molecules in the genomes I’m working with (e.g. NC_031415). Because I couldn’t find such a mapping file I had to do it myself. Here’s how.
First, I use linux command line tools to download the KEGG organism list and parse it to get the organism code and the NCBI ftp directory name:

wget http://www.genome.jp/kegg/catalog/org_list.html
awk '/Prokaryotes/,0' org_list.html | grep "show_organism?org" | cut -d'>' -f3 | cut -f1 -d'<' > org_a
awk '/Prokaryotes/,0' org_list.html | grep ftp | cut -d'/' -f6 > org_b
paste org_a org_b | sort -k2 > org_list

Then, I used lftp to find all of the RefSeq files within each directory. I just chose the ‘.rpt’ files, any of the unique files would do. I parsed the resulting file to get the ftp directory name and the RefSeq ID.

lftp -e 'open ftp://ftp.ncbi.nih.gov/genomes/Bacteria/; ls -d */*.rpt > refseq.rpt; bye'
awk '{print $9}' refseq.rpt | cut -f1 -d'.' | cut -f1 -d'/' > ref_a
awk '{print $9}' refseq.rpt | cut -f1 -d'.' | cut -f2 -d'/' > ref_b
paste ref_b ref_a | uniq -f1 | sort -k2 > ref_list

Then, I joined the two files on their common column: the NCBI ftp directory name. I don’t care about the directory name so then I make a new file without it.

jplace.pl refseqid refseqacclist > kegg2refseq.csvoin -j2 -t$'\t' org_list ref_list > kegg2refseq.txt
cat kegg2refseq.txt | cut -f2,3 > kegg_refseq.tmp
From here I’ve written a script (download: idreplacer) that uses the lproks_1.txt and lproks_2.txt files provided by NCBI to interconvert names and IDs.
./idreplacer.pl refseqacc refseqid kegg_refseq.tmp | ./idre

et voila! Here’s the final KEGG to RefSeq ID mapping file.
Postscript: Here is a database table from MicrobesOnline that does a similar task, mapping KEGG organism to NCBI taxonomy: KEGG2Taxonomy

rpy2 for Ubuntu

# 1. install all dependence packages

sudo apt-get build-dep r-base

# 2. Install R with shlib 

        make distclean
        ./configure --enable-R-shlib
        make
        make install 

# 3. install rpy2 

   sudo easy_install rpy2

R install ubuntu

INSTALLATION

To obtain the latest R packages, add an entry like

  deb http://cran.cnr.Berkeley.edu/bin/linux/ubuntu lucid/

   sudo apt-get update
   sudo apt-get install r-base

public key trouble

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys <KEY NUMBER>

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9

Bioperl Parsing_BLAST

http://www.bioperl.org/wiki/Parsing_BLAST_HSPs

http://www.bioperl.org/wiki/Parsing_BLAST_results_into_per-query_files


CPAN: Automatically install dependencies without confirmation 

perl -MCPAN -e shell 

o conf prerequisites_policy follow
o conf commit 

PREPARING TO INSTALL

This is optional, but regardless of your subsequent choice of installation method, it will help to carry out the following steps. They will increase the likelihood of installation success (especially of optional dependencies).

• Upgrade CPAN:

>perl -MCPAN -e shell
cpan>install Bundle::CPAN
cpan>q

• Install/upgrade Module::Build, and make it your preferred installer:

>cpan cpan>install Module::Build
cpan>o conf prefer_installer MB
cpan>o conf commit
cpan>q

>tar xvfz BioPerl-1.6.1.tar.gz
>cd BioPerl-1.6.1

Now issue the build commands:

>perl Build.PL
>./Build test 

>./Build install
  

Linux unique count

cat Galaxy6-\[Cut_on_data_5\].tabular | cut -f 1 | uniq | wc -l

Julia ?

我们需要有一门开源语言，这门语言在行业内是被公认许可的（基于许可证情况下）。我们希望这门语言能够有C语言一样的速度，Ruby一样得活力（dynamism）。我们需要像homoiconic一样的语言，它像Lisp一样有宏，但是也像Matlab一样有显而易见、熟悉的数学标记。

我们希望传统的编程语言像Python一样适用，像R语言一样适用于统计，像Perl一样适用于字符串处理，像线性代数Matlab一样强大，像 DOS命令一样擅长粘合程序。这似乎看起来简单易学，但是想要让黑客乐意去迎合它却不是简单之事。我们希望它具有互动性且能够被编译。

西游记杂感

孙悟空七十二变本事很大，是因为好学。因为当年下决心拜师学艺，师傅曾问他想学三十六变还是七十二变，他选择七十二变。

孙悟空闹上天宫，被如来收服，是为一山还有一山高，也可以理解为个人能耐再大，也斗不过组织。

悟空愿保唐僧西天取经，说明建功立业还是要靠加入组织，或者需要有团队合作精神。唐僧的前世是神仙，所以最好是选择有背景的团队。