ACEDB source code documentation: EMBL dumping

# EMBL dumping documentation

EMBL dump now gets information from a Method object, which controls what is dumped where. In fact, you can specify dump field information either in the Sequence object itself, or a method, or a method inherited from the method ..., i.e. you can have defaults that you override either in a specialised method or in the object itself.

You can now dump any sequence object for which DNA can be found, directly or indirectly (so you can dump links or subsequences).

You can also now dump from giface, including gifaceserver (see example below).

There is now code for dumping arbitrary features according to dump information in the feature method, plus possible #Feature_info data from the Feature lines. This also works on Homol lines now.

### Known bugs/incompletenesses:

1. I don't get arbitrary features from subsequences, and recurse on them. So in fact dumping links won't work. I realise.
2. I should use methods on subsequences when dumping them (partially done now; 980815).

## Using EMBL_dump_info and associated methods

The order of processing the header information is:

1. Needs to be a sequence object. Needs to find DNA.
2. Complains if no object under _Clone [clone]. This will be used in the context of Clone_left_end information (see below).
3. The ID entry is derived from _Database "EMBL" [id] [ac]. If it exists then use [id]. Else use ID_template from the dump_info, replacing %s by the sequence object name.
4. AC lines are calculated from _Database "EMBL" [id] [ac]. If that does not exist, look under _Ac_number [ac]. Else nothing.
5. DE lines come from DE_format in dump_info. %s is substituted by the clone name if one exists, else the sequence name.
6. KW lines come from _Keyword [keyword] entries.
7. OS line from OS_line in method.
8. OC lines from OC_lines in method. It does not rewrap these so you have to get the lines right.
9. References. Submission reference gets RA names from From_author in the object, then Previous_author in the object; both can be multiple. Fills RL line from RL_submission, relacing DD-MMM-YYYY with the date obtained from Submitted in the object if that exists, else the current date. Then gets further references from the EMBL_reference in the method, parsing fields out of the standard Paper model.
10. Gets CC lines from CC_line in the method. Replace %s in these by clone name, or object name if that does not exist. Each CC_line text will be wrapped and appear as a separate paragraph separated by a blank XX line from the next paragraph.
11. Writes standard CC lines. First set of these give left and right end information using Clone_left_end and Clone_right end info from the clone for this object, and for other clones with ends in this sequence. The second set gives overlap information with neighbouring sequences.
12. Finally, write CC lines for explicit DB_remark entries from the object itself. These are also wrapped and separated into paragraphs by XX lines.

The feature table is written in the following order:

1. A source feature is written. This uses source_organism from the method to fill in /organism. The clone name is given in /clone if that exists. The chromosome and map qualifiers are filled from EMBL_chromosome and EMBL_map fields in the #EMBL_dump_info of the sequence, if they are there. If not, and there is a Map object for the sequence, then it looks inside that for a Text entry following the tag EMBL_chromosome, and if that is found, writes a /chromosome entry containing it.
2. Next it does the subsequences:
   1. Prior to 980915 it looks for tags CDS, mRNA, and text following tRNA, snRNA, scRNA, misc_RNA and dumps accordingly. For the latter ones with text it produces a note containing the text, followed by "-tRNA" or "-RNA".
   2. From 980915, in order to be dumped, a subsequence must have a Method, and that method must have an EMBL_feature, which specifies the feature key used. For pseudogenes use the key "CDS" (required by EMBL rules). The RNA notes are filled in as before.
   3. For CDS checks if translatable without a stop codon, and if not then sets /pseudo qualifier and messerrors.
   4. Builds the location information, going across LINK boundaries. Checks for Start_not_found and End_not_found tags and uses greater than and less than signs in these cases (but not /partial). In these cases, it sets codon_start where necessary.
   5. If CDS_predicted_by a method, then dumps out /note="predicted using %s".
   6. If the name of the object ends in a lower case letter, then prints out /note="preliminary prediction". EBI recognises this, and puts the resulting peptide into REM-TREMBL, not SP-TREMBL.
   7. If a Locus attached then write /gene="{Locus}". If gene_from_name was set in the model, then it will also write out /gene="{object name}".
   8. If a DB_remark attached to the object, then dump that as a note. Else if Brief_identification, dump /note="similar to {brief_id}".
   9. For each sequence (normally an EST) attached via Matching_cDNA to the object, we print a line saying "cDNA EST EMBL:%s comes from this gene" or "cDNA EST %s comes from this gene" depending on whether we can find an accession number. 40 lines per /note qualifier.
   10. If find TSL_site followed by an integer, then write a line /note="Possible trans-spliced leader site at {position}". Very worm specific.
3. Next, Features and Homols are dumped as described in the text below, if their methods have EMBL_dump information.
4. Finally, the sequence itself is dumped as an SQ line followed by sequence.

### Model Changes

Create a new # (subobject) model:

    #EMBL_dump_info	EMBL_dump_method UNIQUE ?Method
    		ID_template UNIQUE Text
    		ID_division UNIQUE Text
    		DE_format UNIQUE Text
    		OS_line UNIQUE Text
    		OC_line Text
    		RL_submission Text
    		EMBL_reference ?Paper
    		CC_line Text
    		source_organism UNIQUE Text
    		gene_from_name
    		EMBL_chromosome UNIQUE Text
    		EMBL_map UNIQUE Text

This information is made accessible by adding to the Method model:

    	EMBL_dump_info #EMBL_dump_info

Add similarly to Sequence model:

    	  DB_info	...
    			EMBL_dump_info #EMBL_dump_info

The use of the shared subobject model makes things recursive. When looking for OS_line, for example, the first one that is found gets used, starting with information in the Sequence object, then in its EMBL_dump_method, then in its EMBL_dump_method...

Add to Map model:

    	EMBL_chromosome UNIQUE Text

This determines how a Map object is transformed to a /chromosome="xx" line under the "source" feature key.

### Models for dumping Features and Homols

    ?Sequence ...
    //	  Homol

## Example .ace file

    Method worm_EMBL-dump
    EMBL_dump_info ID_template "CE%s"
    EMBL_dump_info ID_division INV
    EMBL_dump_info DE_format "Caenorhabditis elegans cosmid %s"
    EMBL_dump_info OS_line "Caenorhabditis elegans (nematode)"
    EMBL_dump_info OC_line "Eukaryota; Animalia; Metazoa; Nematoda; Secernentea; Rhabditia;"
    EMBL_dump_info OC_line "Rhabditida; Rhabditina; Rhabditoidea; Rhabditidae."
    EMBL_dump_info RL_submission "Submitted (DD-MMM-YYYY) to the EMBL Data Library by:"
    EMBL_dump_info RL_submission "Nematode Sequencing Project, Sanger Centre, Hinxton, Cambridge"
    EMBL_dump_info RL_submission "CB10 1RQ, England and Department of Genetics, Washington"
    EMBL_dump_info RL_submission "University, St. Louis, MO 63110, USA."
    EMBL_dump_info RL_submission "E-mail: jes@sanger.ac.uk or rw@nematode.wustl.edu"
    EMBL_dump_info EMBL_reference seq-paper-2
    EMBL_dump_info CC_line "Current sequence finishing criteria for the C. elegans genome"
    EMBL_dump_info CC_line "sequencing consortium are that all bases are either sequenced"
    EMBL_dump_info CC_line "unambiguously on both strands, or on a single strand with both"
    EMBL_dump_info CC_line "a dye primer and dye terminator reaction, from distinct"
    EMBL_dump_info CC_line "subclones.  Exceptions are indicated by an explicit note.\n"
    EMBL_dump_info CC_line "Coding sequences below are predicted from computer analysis,"
    EMBL_dump_info CC_line "using predictions from Genefinder (P. Green, U. Washington),"
    EMBL_dump_info CC_line "and other available information.\n"
    EMBL_dump_info CC_line "IMPORTANT:  This sequence is NOT necessarily the entire insert"
    EMBL_dump_info CC_line "of the specified clone.  It may be shorter because we only"
    EMBL_dump_info CC_line "sequence overlapping sections once, or longer because we"
    EMBL_dump_info CC_line "arrange for a small overlap between neighbouring submissions.\n"
    EMBL_dump_info source_organism "Caenorhabditis elegans"

    Paper seq-paper-2
    Title "2.2 Mb of contiguous nucleotide sequence from chromosome III of C. elegans"
    Journal Nature
    Volume 368
    Page 32 38
    Year 1994
    Author "Wilson R"
    Author "Ainscough R"

    Map Sequence-I
    EMBL_chromosome I

    Map Sequence-II
    EMBL_chromosome II

    Sequence AH6
    EMBL_dump_info EMBL_dump_method worm_EMBL-dump

Note the "\n" at the end of comment lines to have them followed by blank lines.

## Example giface script

    giface < find sequence AH6
    acedb> gif
    acedb-gif> embl ah6
    acedb-gif> quit
    acedb> quit
    EOF

## Example Output

    ID   CEAH6      standard; DNA; INV; 37801 BP.
    XX
    AC   Z48009;
    XX
    KW   Zinc finger; Transposon; Guanylate cyclase.
    XX
    OS   Caenorhabditis elegans (nematode)
    OS   Eukaryota; Animalia; Metazoa; Nematoda; Secernentea; Rhabditia;
    XX
    RN   [1]
    RP   1-37801
    RA   Berks M.;
    RT   ;
    RL   Submitted (06-FEB-1995) to the EMBL Data Library by:
    RL   Nematode Sequencing Project, Sanger Centre, Hinxton, Cambridge
    RL   CB10 1RQ, England and Department of Genetics, Washington
    RL   University, St. Louis, MO 63110, USA.
    RL   E-mail: jes@sanger.ac.uk or rw@nematode.wustl.edu
    XX
    RN   [2]
    RA   Wilson R., Ainscough R., Anderson K., Baynes C., Berks M.,
    RA   Bonfield J., Burton J., Connell M., Copsey T., Cooper J.,
    RA   Coulson A., Craxton M., Dear S., Du Z., Durbin R., Favello A.,
    RA   Fulton L., Gardner A., Green P., Hawkins T., Hillier L., Jier M.,
    RA   Johnston L., Jones M., Kershaw J., Kirsten J., Laister N.,
    RA   Latreille P., Lightning J., Lloyd C., McMurray A., Mortimore B.,
    RA   O'Callaghan M., Parsons J., Percy C., Rifken L., Roopra A.,
    RA   Saunders D., Shownkeen R., Smaldon N., Smith A., Sonnhammer E.,
    RA   Staden R., Sulston J., Thierry-Mieg J., Thomas K., Vaudin M.,
    RA   Vaughan K., Waterston R., Watson A., Weinstock L.,
    RA   Wilkinson-Sproat J., Wohldman P.;
    RT   "2.2 Mb of contiguous nucleotide sequence from chromosome III of
    RT   C. elegans";
    RL   Nature 368:32-38 (1994).
    XX
    CC   Current sequence finishing criteria for the C. elegans genome
    CC   sequencing consortium are that all bases are either sequenced
    CC   unambiguously on both strands, or on a single strand with both
    CC   a dye primer and dye terminator reaction, from distinct
    CC   subclones.  Exceptions are indicated by an explicit note.
    CC
    CC   Coding sequences below are predicted from computer analysis,
    CC   using predictions from Genefinder (P. Green, U. Washington),
    CC   and other available information.
    CC
    CC   IMPORTANT:  This sequence is NOT necessarily the entire insert
    CC   of the specified clone.  It may be shorter because we only
    CC   sequence overlapping sections once, or longer because we
    CC   arrange for a small overlap between neighbouring submissions.
    CC
    CC   The true left end of clone AH6 is at 1 in this sequence.
    CC   The true right end of clone AH6 is at 3246 in
    CC   sequence Z36752.
    CC   The true left end of clone F35H8 is at 37686 in this sequence.
    CC   The true right end of clone R134 is at 30025 in this sequence.
    CC   The start of this sequence (1..101) overlaps with the end of
    CC   sequence Z48007.
    CC   The end of this sequence (37686..37801) overlaps with the start of
    CC   sequence Z36752.
    XX
    FH   Key             Location/Qualifiers
    FH
    FT   source          1..37801
    FT                   /organism="Caenorhabditis elegans"
    FT                   /clone="AH6"
    FT                   /chromosome="II"
    FT   CDS             complement(join(5054..5310,5380..5507,5551..5636,
    FT                   5683..5904,5960..6180,6233..6308))
    FT                   /product="AH6.2"
    FT   CDS             join(6579..6638,6872..7051,7096..7416,7464..7595)
    FT                   /product="AH6.3"
    FT   CDS             complement(join(7727..7885,7935..8121,8174..8446,
    FT                   8499..8644,8872..9102))
    FT                   /product="AH6.4"
    FT                   /gene="sra-1"
    FT   CDS             join(11523..11741,11789..12303,12357..12681,
    FT                   12731..12871,12923..13059,13109..13307)
    FT                   /product="AH6.5"
    FT                   /note="similar to zinc finger protein"
    FT                   /note="cDNA EST yk38b2.3 comes from this gene"
    FT                   /note="cDNA EST yk38b2.5 comes from this gene"
    FT                   /note="cDNA EST yk45f12.5 comes from this gene"
    FT   CDS             complement(join(29091..29243,29290..29476,29667..30316))
    FT                   /product="AH6.8"
    FT                   /gene="sra-4"
    FT   CDS             complement(join(33869..34021,34183..34375,34425..34816))
    FT                   /product="AH6.9"
    FT                   /gene="sra-5"
    FT   CDS             join(32117..32766,33253..33439,33487..33639)
    FT                   /product="AH6.11"
    FT                   /gene="sra-7"
    FT   CDS             join(17948..18189,18299..18441,18532..18617)
    FT                   /product="AH6.13"
    FT   CDS             join(19742..20391,20445..20637,21068..21220)
    FT                   /product="AH6.14"
    FT                   /gene="sra-9"
    FT   CDS             complement(join(36041..36193,36247..36433,36481..37130))
    FT                   /product="AH6.10"
    FT                   /gene="sra-6"
    FT   CDS             join(16211..16860,16906..17092,17141..17293)
    FT                   /product="AH6.6"
    FT                   /gene="sra-2"
    FT   CDS             join(25160..25809,26063..26249,26301..26453)
    FT                   /product="AH6.12"
    FT                   /gene="sra-8"
    FT   CDS             complement(join(26752..26904,26982..27168,27605..28254))
    FT                   /product="AH6.7"
    FT                   /gene="sra-3"
    FT   CDS             complement(21569..24025)
    FT                   /product="AH6.15"
    FT                   /pseudo
    FT                   /note="probably a transposon"
    FT   CDS             14448..15525
    FT                   /product="AH6.16"
    FT                   /pseudo
    FT   CDS             complement(join(Z48007:14051..Z48007:14230,102..337,
    FT                   397..500,544..951,1002..1179,1226..1526,1579..2235,
    FT                   2288..2418,2483..2621,2672..2797,2851..2948,2999..3151,
    FT                   3195..3424,3469..3711,3996..4126,4270..4368))
    FT                   /product="AH6.1"
    FT                   /note="similar to guanylate cyclase"
    XX
    SQ   Sequence  37801 BP;   12255 A; 6407 C; 6368 G; 12771 T; 0 other;
         ccatgagagc ttgatggatt tggaatccat ctatcgttgg ttactggtgg tgttgaccga
         ttactaatgc ttcttaactc ggttggttcc atttcaccaa atctgccgtg cacccaaaat
         gtttccataa ctccttttcc ttttataatt acttctcctc gggaactcgt ttcgtattga

#### Richard Durbin $Id: embl.doc.html,v 1.6 1999/01/25 16:53:25 edgrif Exp $