HDS for BioNLP

Task 1

Strict Matching

------------------------------------------------------------------------------------
       Event Class          gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------
     Gene_expression         619 (  453)      556 (  453)    73.18    81.47    77.11
      Transcription          101 (   58)      113 (   58)    57.43    51.33    54.21
   Protein_catabolism         14 (    7)       18 (    7)    50.00    38.89    43.75
      Localization            99 (   50)       66 (   50)    50.51    75.76    60.61
------------------------------------------------------------------------------------
     =[SIMPLE ALL]=          833 (  568)      753 (  568)    68.19    75.43    71.63
------------------------------------------------------------------------------------
         Binding             333 (  135)      412 (  135)    40.54    32.77    36.24
------------------------------------------------------------------------------------
  Protein_modification         1 (    0)        0 (    0)     0.00     0.00     0.00
     Phosphorylation         160 (  135)      178 (  135)    84.38    75.84    79.88
     Ubiquitination           30 (    0)        0 (    0)     0.00     0.00     0.00
       Acetylation             0 (    0)        0 (    0)     0.00     0.00     0.00
      Deacetylation            0 (    0)        0 (    0)     0.00     0.00     0.00
------------------------------------------------------------------------------------
    =[PROT-MOD ALL]=         191 (  135)      178 (  135)    70.68    75.84    73.17
------------------------------------------------------------------------------------
       Regulation            288 (   20)      140 (   20)     6.94    14.29     9.35
   Positive_regulation      1130 (  176)      587 (  176)    15.58    29.98    20.50
   Negative_regulation       526 (  120)      323 (  120)    22.81    37.15    28.27
------------------------------------------------------------------------------------
   =[REGULATION ALL]=       1944 (  316)     1050 (  316)    16.26    30.10    21.11
------------------------------------------------------------------------------------
    ==[EVENT TOTAL]==       3301 ( 1154)     2393 ( 1154)    34.96    48.22    40.53
------------------------------------------------------------------------------------

Approximate Span/Approximate Recursive

------------------------------------------------------------------------------------
       Event Class          gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------
     Gene_expression         619 (  501)      556 (  501)    80.94    90.11    85.28
      Transcription          101 (   66)      113 (   66)    65.35    58.41    61.68
   Protein_catabolism         14 (   10)       18 (   10)    71.43    55.56    62.50
      Localization            99 (   50)       66 (   50)    50.51    75.76    60.61
------------------------------------------------------------------------------------
     =[SIMPLE ALL]=          833 (  627)      753 (  627)    75.27    83.27    79.07
------------------------------------------------------------------------------------
         Binding             333 (  139)      412 (  139)    41.74    33.74    37.32
------------------------------------------------------------------------------------
  Protein_modification         1 (    0)        0 (    0)     0.00     0.00     0.00
     Phosphorylation         160 (  135)      178 (  135)    84.38    75.84    79.88
     Ubiquitination           30 (    0)        0 (    0)     0.00     0.00     0.00
       Acetylation             0 (    0)        0 (    0)     0.00     0.00     0.00
      Deacetylation            0 (    0)        0 (    0)     0.00     0.00     0.00
------------------------------------------------------------------------------------
    =[PROT-MOD ALL]=         191 (  135)      178 (  135)    70.68    75.84    73.17
------------------------------------------------------------------------------------
       Regulation            288 (   21)      140 (   21)     7.29    15.00     9.81
   Positive_regulation      1130 (  182)      587 (  182)    16.11    31.01    21.20
   Negative_regulation       526 (  121)      323 (  121)    23.00    37.46    28.50
------------------------------------------------------------------------------------
   =[REGULATION ALL]=       1944 (  324)     1050 (  324)    16.67    30.86    21.64
------------------------------------------------------------------------------------
    ==[EVENT TOTAL]==       3301 ( 1225)     2393 ( 1225)    37.11    51.19    43.03
------------------------------------------------------------------------------------

Approximate Span without Event Trigger Matching/Approximate Recursive

------------------------------------------------------------------------------------
       Event Class          gold (match)   answer (match)   recall    prec.   fscore
------------------------------------------------------------------------------------
     Gene_expression         619 (  511)      551 (  503)    82.55    91.29    86.70
      Transcription          101 (   67)      110 (   67)    66.34    60.91    63.51
   Protein_catabolism         14 (   10)       18 (   10)    71.43    55.56    62.50
      Localization            99 (   55)       66 (   52)    55.56    78.79    65.16
------------------------------------------------------------------------------------
     =[SIMPLE ALL]=          833 (  643)      745 (  632)    77.19    84.83    80.83
------------------------------------------------------------------------------------
         Binding             333 (  147)      397 (  146)    44.14    36.78    40.12
------------------------------------------------------------------------------------
  Protein_modification         1 (    0)        0 (    0)     0.00     0.00     0.00
     Phosphorylation         160 (  138)      176 (  135)    86.25    76.70    81.20
     Ubiquitination           30 (    0)        0 (    0)     0.00     0.00     0.00
       Acetylation             0 (    0)        0 (    0)     0.00     0.00     0.00
      Deacetylation            0 (    0)        0 (    0)     0.00     0.00     0.00
------------------------------------------------------------------------------------
    =[PROT-MOD ALL]=         191 (  138)      176 (  135)    72.25    76.70    74.41
------------------------------------------------------------------------------------
       Regulation            288 (   22)      139 (   21)     7.64    15.11    10.15
   Positive_regulation      1130 (  190)      568 (  189)    16.81    33.27    22.34
   Negative_regulation       526 (  127)      315 (  123)    24.14    39.05    29.84
------------------------------------------------------------------------------------
   =[REGULATION ALL]=       1944 (  339)     1022 (  333)    17.44    32.58    22.72
------------------------------------------------------------------------------------
    ==[EVENT TOTAL]==       3301 ( 1267)     2340 ( 1246)    38.38    53.25    44.61
------------------------------------------------------------------------------------

Strict Matching/Decomposition

---------------------------------------------------------------------------------------
     Event-Arg Type            gold (match)   answer (match)   recall    prec.   fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression           619 (  453)      556 (  453)    73.18    81.47    77.11
Theme-Transcription             101 (   58)      113 (   58)    57.43    51.33    54.21
Theme-Protein_catabolism         14 (    7)       18 (    7)    50.00    38.89    43.75
Theme-Binding                   418 (  243)      421 (  243)    58.13    57.72    57.93
Theme-Localization               99 (   50)       66 (   50)    50.51    75.76    60.61
Theme-Protein_modification        1 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Phosphorylation           160 (  139)      178 (  139)    86.88    78.09    82.25
Theme-Ubiquitination             25 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Regulation                257 (   25)      140 (   25)     9.73    17.86    12.59
Theme-Positive_regulation      1068 (  199)      584 (  199)    18.63    34.08    24.09
Theme-Negative_regulation       490 (  136)      321 (  136)    27.76    42.37    33.54
---------------------------------------------------------------------------------------
=[THEME ALL]=                  3252 ( 1310)     2397 ( 1310)    40.28    54.65    46.38
---------------------------------------------------------------------------------------
Cause-Regulation                103 (    0)        2 (    0)     0.00     0.00     0.00
Cause-Positive_regulation       337 (    8)       13 (    8)     2.37    61.54     4.57
Cause-Negative_regulation       122 (    5)       13 (    5)     4.10    38.46     7.41
Cause-Protein_modification        0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Phosphorylation             2 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Ubiquitination              8 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]=                   572 (   13)       28 (   13)     2.27    46.43     4.33
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]==        3824 ( 1323)     2425 ( 1323)    34.60    54.56    42.34
---------------------------------------------------------------------------------------

Approximate Span/Approximate Recursive/Decomposition

---------------------------------------------------------------------------------------
     Event-Arg Type            gold (match)   answer (match)   recall    prec.   fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression           619 (  501)      556 (  501)    80.94    90.11    85.28
Theme-Transcription             101 (   66)      113 (   66)    65.35    58.41    61.68
Theme-Protein_catabolism         14 (   10)       18 (   10)    71.43    55.56    62.50
Theme-Binding                   418 (  259)      421 (  259)    61.96    61.52    61.74
Theme-Localization               99 (   50)       66 (   50)    50.51    75.76    60.61
Theme-Protein_modification        1 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Phosphorylation           160 (  139)      178 (  139)    86.88    78.09    82.25
Theme-Ubiquitination             25 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Regulation                257 (   27)      140 (   27)    10.51    19.29    13.60
Theme-Positive_regulation      1068 (  205)      584 (  205)    19.19    35.10    24.82
Theme-Negative_regulation       490 (  137)      321 (  137)    27.96    42.68    33.79
---------------------------------------------------------------------------------------
=[THEME ALL]=                  3252 ( 1394)     2397 ( 1394)    42.87    58.16    49.35
---------------------------------------------------------------------------------------
Cause-Regulation                103 (    0)        2 (    0)     0.00     0.00     0.00
Cause-Positive_regulation       337 (    8)       13 (    8)     2.37    61.54     4.57
Cause-Negative_regulation       122 (    5)       13 (    5)     4.10    38.46     7.41
Cause-Protein_modification        0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Phosphorylation             2 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Ubiquitination              8 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]=                   572 (   13)       28 (   13)     2.27    46.43     4.33
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]==        3824 ( 1407)     2425 ( 1407)    36.79    58.02    45.03
---------------------------------------------------------------------------------------

Approximate Span without Event Trigger Matching/Approximate Recursive/Decomposition

---------------------------------------------------------------------------------------
     Event-Arg Type            gold (match)   answer (match)   recall    prec.   fscore
---------------------------------------------------------------------------------------
Theme-Gene_expression           619 (  511)      551 (  503)    82.55    91.29    86.70
Theme-Transcription             101 (   67)      110 (   67)    66.34    60.91    63.51
Theme-Protein_catabolism         14 (   10)       18 (   10)    71.43    55.56    62.50
Theme-Binding                   418 (  288)      404 (  283)    68.90    70.05    69.47
Theme-Localization               99 (   55)       66 (   52)    55.56    78.79    65.16
Theme-Protein_modification        1 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Phosphorylation           160 (  142)      176 (  139)    88.75    78.98    83.58
Theme-Ubiquitination             25 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
Theme-Regulation                257 (   28)      139 (   27)    10.89    19.42    13.96
Theme-Positive_regulation      1068 (  217)      560 (  215)    20.32    38.39    26.57
Theme-Negative_regulation       490 (  143)      312 (  138)    29.18    44.23    35.17
---------------------------------------------------------------------------------------
=[THEME ALL]=                  3252 ( 1461)     2336 ( 1434)    44.93    61.39    51.88
---------------------------------------------------------------------------------------
Cause-Regulation                103 (    0)        2 (    0)     0.00     0.00     0.00
Cause-Positive_regulation       337 (    8)       13 (    8)     2.37    61.54     4.57
Cause-Negative_regulation       122 (    5)       13 (    5)     4.10    38.46     7.41
Cause-Protein_modification        0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Phosphorylation             2 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Ubiquitination              8 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Acetylation                 0 (    0)        0 (    0)     0.00     0.00     0.00
Cause-Deacetylation               0 (    0)        0 (    0)     0.00     0.00     0.00
---------------------------------------------------------------------------------------
=[CAUSE ALL]=                   572 (   13)       28 (   13)     2.27    46.43     4.33
---------------------------------------------------------------------------------------
==[PRIMARY ARG TOTAL]==        3824 ( 1474)     2364 ( 1447)    38.55    61.21    47.30
---------------------------------------------------------------------------------------

Task 2

Strict Matching/Decomposition

N/A

Approximate Span/Approximate Recursive/Decomposition

N/A

Approximate Span without Event Trigger Matching/Approximate Recursive/Decomposition

N/A

Task 3

Strict Matching

N/A

Approximate Span/Approximate Recursive

N/A

Approximate Span without Event Trigger Matching/Approximate Recursive

N/A

Thank you for your participation!!