TEES 2.1 Automated Event Annotation Scheme Learning
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Gene_expression 619 ( 497) 600 ( 497) 80.29 82.83 81.54
Transcription 101 ( 30) 99 ( 30) 29.70 30.30 30.00
Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Localization 99 ( 50) 59 ( 50) 50.51 84.75 63.29
------------------------------------------------------------------------------------
=[SIMPLE ALL]= 833 ( 586) 776 ( 586) 70.35 75.52 72.84
------------------------------------------------------------------------------------
Binding 333 ( 141) 318 ( 141) 42.34 44.34 43.32
------------------------------------------------------------------------------------
Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 160 ( 122) 176 ( 122) 76.25 69.32 72.62
Ubiquitination 30 ( 0) 0 ( 0) 0.00 0.00 0.00
Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
------------------------------------------------------------------------------------
=[PROT-MOD ALL]= 191 ( 122) 176 ( 122) 63.87 69.32 66.49
------------------------------------------------------------------------------------
Regulation 288 ( 51) 165 ( 51) 17.71 30.91 22.52
Positive_regulation 1130 ( 340) 813 ( 340) 30.09 41.82 35.00
Negative_regulation 526 ( 203) 458 ( 203) 38.59 44.32 41.26
------------------------------------------------------------------------------------
=[REGULATION ALL]= 1944 ( 594) 1436 ( 594) 30.56 41.36 35.15
------------------------------------------------------------------------------------
==[EVENT TOTAL]== 3301 ( 1443) 2706 ( 1443) 43.71 53.33 48.04
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Gene_expression 619 ( 504) 600 ( 504) 81.42 84.00 82.69
Transcription 101 ( 55) 99 ( 55) 54.46 55.56 55.00
Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Localization 99 ( 50) 59 ( 50) 50.51 84.75 63.29
------------------------------------------------------------------------------------
=[SIMPLE ALL]= 833 ( 618) 776 ( 618) 74.19 79.64 76.82
------------------------------------------------------------------------------------
Binding 333 ( 141) 318 ( 141) 42.34 44.34 43.32
------------------------------------------------------------------------------------
Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 160 ( 122) 176 ( 122) 76.25 69.32 72.62
Ubiquitination 30 ( 0) 0 ( 0) 0.00 0.00 0.00
Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
------------------------------------------------------------------------------------
=[PROT-MOD ALL]= 191 ( 122) 176 ( 122) 63.87 69.32 66.49
------------------------------------------------------------------------------------
Regulation 288 ( 52) 165 ( 52) 18.06 31.52 22.96
Positive_regulation 1130 ( 376) 813 ( 376) 33.27 46.25 38.70
Negative_regulation 526 ( 215) 458 ( 215) 40.87 46.94 43.70
------------------------------------------------------------------------------------
=[REGULATION ALL]= 1944 ( 643) 1436 ( 643) 33.08 44.78 38.05
------------------------------------------------------------------------------------
==[EVENT TOTAL]== 3301 ( 1524) 2706 ( 1524) 46.17 56.32 50.74
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Gene_expression 619 ( 509) 598 ( 505) 82.23 84.45 83.32
Transcription 101 ( 56) 99 ( 56) 55.45 56.57 56.00
Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Localization 99 ( 53) 58 ( 50) 53.54 86.21 66.05
------------------------------------------------------------------------------------
=[SIMPLE ALL]= 833 ( 627) 773 ( 620) 75.27 80.21 77.66
------------------------------------------------------------------------------------
Binding 333 ( 146) 309 ( 145) 43.84 46.93 45.33
------------------------------------------------------------------------------------
Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Phosphorylation 160 ( 131) 176 ( 127) 81.88 72.16 76.71
Ubiquitination 30 ( 0) 0 ( 0) 0.00 0.00 0.00
Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
------------------------------------------------------------------------------------
=[PROT-MOD ALL]= 191 ( 131) 176 ( 127) 68.59 72.16 70.33
------------------------------------------------------------------------------------
Regulation 288 ( 53) 165 ( 52) 18.40 31.52 23.24
Positive_regulation 1130 ( 393) 804 ( 391) 34.78 48.63 40.55
Negative_regulation 526 ( 228) 457 ( 223) 43.35 48.80 45.91
------------------------------------------------------------------------------------
=[REGULATION ALL]= 1944 ( 674) 1426 ( 666) 34.67 46.70 39.80
------------------------------------------------------------------------------------
==[EVENT TOTAL]== 3301 ( 1578) 2684 ( 1558) 47.80 58.05 52.43
------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression 619 ( 497) 600 ( 497) 80.29 82.83 81.54
Theme-Transcription 101 ( 30) 99 ( 30) 29.70 30.30 30.00
Theme-Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Theme-Binding 418 ( 239) 400 ( 239) 57.18 59.75 58.44
Theme-Localization 99 ( 50) 59 ( 50) 50.51 84.75 63.29
Theme-Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Phosphorylation 160 ( 129) 173 ( 129) 80.62 74.57 77.48
Theme-Ubiquitination 25 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Regulation 257 ( 64) 158 ( 64) 24.90 40.51 30.84
Theme-Positive_regulation 1068 ( 393) 809 ( 393) 36.80 48.58 41.88
Theme-Negative_regulation 490 ( 249) 451 ( 249) 50.82 55.21 52.92
---------------------------------------------------------------------------------------
=[THEME ALL]= 3252 ( 1660) 2767 ( 1660) 51.05 59.99 55.16
---------------------------------------------------------------------------------------
Cause-Regulation 103 ( 20) 45 ( 20) 19.42 44.44 27.03
Cause-Positive_regulation 337 ( 89) 139 ( 89) 26.41 64.03 37.39
Cause-Negative_regulation 122 ( 29) 58 ( 29) 23.77 50.00 32.22
Cause-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Phosphorylation 2 ( 0) 4 ( 0) 0.00 0.00 0.00
Cause-Ubiquitination 8 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]= 572 ( 138) 246 ( 138) 24.13 56.10 33.74
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]== 3824 ( 1798) 3013 ( 1798) 47.02 59.67 52.60
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression 619 ( 504) 600 ( 504) 81.42 84.00 82.69
Theme-Transcription 101 ( 55) 99 ( 55) 54.46 55.56 55.00
Theme-Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Theme-Binding 418 ( 242) 400 ( 242) 57.89 60.50 59.17
Theme-Localization 99 ( 50) 59 ( 50) 50.51 84.75 63.29
Theme-Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Phosphorylation 160 ( 129) 173 ( 129) 80.62 74.57 77.48
Theme-Ubiquitination 25 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Regulation 257 ( 65) 158 ( 65) 25.29 41.14 31.33
Theme-Positive_regulation 1068 ( 429) 809 ( 429) 40.17 53.03 45.71
Theme-Negative_regulation 490 ( 265) 451 ( 265) 54.08 58.76 56.32
---------------------------------------------------------------------------------------
=[THEME ALL]= 3252 ( 1748) 2767 ( 1748) 53.75 63.17 58.08
---------------------------------------------------------------------------------------
Cause-Regulation 103 ( 21) 45 ( 21) 20.39 46.67 28.38
Cause-Positive_regulation 337 ( 90) 139 ( 90) 26.71 64.75 37.82
Cause-Negative_regulation 122 ( 30) 58 ( 30) 24.59 51.72 33.33
Cause-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Phosphorylation 2 ( 0) 4 ( 0) 0.00 0.00 0.00
Cause-Ubiquitination 8 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]= 572 ( 141) 246 ( 141) 24.65 57.32 34.47
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]== 3824 ( 1889) 3013 ( 1889) 49.40 62.69 55.26
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression 619 ( 509) 598 ( 505) 82.23 84.45 83.32
Theme-Transcription 101 ( 56) 99 ( 56) 55.45 56.57 56.00
Theme-Protein_catabolism 14 ( 9) 18 ( 9) 64.29 50.00 56.25
Theme-Binding 418 ( 274) 393 ( 261) 65.55 66.41 65.98
Theme-Localization 99 ( 53) 58 ( 50) 53.54 86.21 66.05
Theme-Protein_modification 1 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Phosphorylation 160 ( 138) 173 ( 134) 86.25 77.46 81.62
Theme-Ubiquitination 25 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Theme-Regulation 257 ( 66) 158 ( 65) 25.68 41.14 31.62
Theme-Positive_regulation 1068 ( 462) 791 ( 451) 43.26 57.02 49.19
Theme-Negative_regulation 490 ( 278) 447 ( 272) 56.73 60.85 58.72
---------------------------------------------------------------------------------------
=[THEME ALL]= 3252 ( 1845) 2735 ( 1803) 56.73 65.92 60.98
---------------------------------------------------------------------------------------
Cause-Regulation 103 ( 25) 45 ( 22) 24.27 48.89 32.44
Cause-Positive_regulation 337 ( 93) 139 ( 92) 27.60 66.19 38.95
Cause-Negative_regulation 122 ( 31) 58 ( 31) 25.41 53.45 34.44
Cause-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Phosphorylation 2 ( 0) 4 ( 0) 0.00 0.00 0.00
Cause-Ubiquitination 8 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Cause-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]= 572 ( 149) 246 ( 145) 26.05 58.94 36.13
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]== 3824 ( 1994) 2981 ( 1948) 52.14 65.35 58.00
---------------------------------------------------------------------------------------
Task 2
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Site-Binding 102 ( 20) 57 ( 20) 19.61 35.09 25.16
Site-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Phosphorylation 86 ( 30) 39 ( 30) 34.88 76.92 48.00
Site-Ubiquitination 7 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Regulation 28 ( 1) 4 ( 1) 3.57 25.00 6.25
Site-Positive_regulation 18 ( 0) 3 ( 0) 0.00 0.00 0.00
Site-Negative_regulation 34 ( 2) 9 ( 2) 5.88 22.22 9.30
CSite-Regulation 12 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Positive_regulation 35 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Negative_regulation 2 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[SITE ALL]= 324 ( 53) 112 ( 53) 16.36 47.32 24.31
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Loc-Localization 30 ( 11) 14 ( 11) 36.67 78.57 50.00
---------------------------------------------------------------------------------------
==[SECONDARY ARG TOTAL]== 354 ( 64) 126 ( 64) 18.08 50.79 26.67
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Site-Binding 102 ( 32) 57 ( 32) 31.37 56.14 40.25
Site-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Phosphorylation 86 ( 32) 39 ( 32) 37.21 82.05 51.20
Site-Ubiquitination 7 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Regulation 28 ( 1) 4 ( 1) 3.57 25.00 6.25
Site-Positive_regulation 18 ( 0) 3 ( 0) 0.00 0.00 0.00
Site-Negative_regulation 34 ( 2) 9 ( 2) 5.88 22.22 9.30
CSite-Regulation 12 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Positive_regulation 35 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Negative_regulation 2 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[SITE ALL]= 324 ( 67) 112 ( 67) 20.68 59.82 30.73
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Loc-Localization 30 ( 11) 14 ( 11) 36.67 78.57 50.00
---------------------------------------------------------------------------------------
==[SECONDARY ARG TOTAL]== 354 ( 78) 126 ( 78) 22.03 61.90 32.50
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Event-Arg Type gold (match) answer (match) recall prec. fscore
---------------------------------------------------------------------------------------
Site-Binding 102 ( 35) 57 ( 35) 34.31 61.40 44.03
Site-Protein_modification 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Phosphorylation 86 ( 33) 39 ( 33) 38.37 84.62 52.80
Site-Ubiquitination 7 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Acetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Deacetylation 0 ( 0) 0 ( 0) 0.00 0.00 0.00
Site-Regulation 28 ( 1) 4 ( 1) 3.57 25.00 6.25
Site-Positive_regulation 18 ( 1) 3 ( 1) 5.56 33.33 9.52
Site-Negative_regulation 34 ( 2) 9 ( 2) 5.88 22.22 9.30
CSite-Regulation 12 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Positive_regulation 35 ( 0) 0 ( 0) 0.00 0.00 0.00
CSite-Negative_regulation 2 ( 0) 0 ( 0) 0.00 0.00 0.00
---------------------------------------------------------------------------------------
=[SITE ALL]= 324 ( 72) 112 ( 72) 22.22 64.29 33.03
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Loc-Localization 30 ( 11) 14 ( 11) 36.67 78.57 50.00
---------------------------------------------------------------------------------------
==[SECONDARY ARG TOTAL]== 354 ( 83) 126 ( 83) 23.45 65.87 34.58
---------------------------------------------------------------------------------------
Task 3
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Negation 143 ( 24) 76 ( 24) 16.78 31.58 21.92
Speculation 195 ( 22) 110 ( 22) 11.28 20.00 14.43
==[MOD TOTAL]== 338 ( 46) 186 ( 46) 13.61 24.73 17.56
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Negation 143 ( 31) 76 ( 28) 21.68 36.84 27.30
Speculation 195 ( 36) 110 ( 36) 18.46 32.73 23.61
==[MOD TOTAL]== 338 ( 67) 186 ( 64) 19.82 34.41 25.15
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Negation 143 ( 31) 76 ( 28) 21.68 36.84 27.30
Speculation 195 ( 36) 110 ( 36) 18.46 32.73 23.61
==[MOD TOTAL]== 338 ( 67) 186 ( 64) 19.82 34.41 25.15
------------------------------------------------------------------------------------
Thank you for your participation!!