python - What's meaning of BOS and EOS in CRFSuite feature list and what is the role of them? -


in ner(named entity recognition) example in python-crf package website see function feature generator:

def word2features(sent, i): word = sent[i][0] postag = sent[i][1] features = [     'bias',     'word.lower=' + word.lower(),     'word[-3:]=' + word[-3:],     'word[-2:]=' + word[-2:],     'word.isupper=%s' % word.isupper(),     'word.istitle=%s' % word.istitle(),     'word.isdigit=%s' % word.isdigit(),     'postag=' + postag,     'postag[:2]=' + postag[:2], ] if > 0:     word1 = sent[i-1][0]     postag1 = sent[i-1][1]     features.extend([         '-1:word.lower=' + word1.lower(),         '-1:word.istitle=%s' % word1.istitle(),         '-1:word.isupper=%s' % word1.isupper(),         '-1:postag=' + postag1,         '-1:postag[:2]=' + postag1[:2],     ]) else:     features.append('bos')  if < len(sent)-1:     word1 = sent[i+1][0]     postag1 = sent[i+1][1]     features.extend([         '+1:word.lower=' + word1.lower(),         '+1:word.istitle=%s' % word1.istitle(),         '+1:word.isupper=%s' % word1.isupper(),         '+1:postag=' + postag1,         '+1:postag[:2]=' + postag1[:2],     ]) else:     features.append('eos')  return features 

you can see completed tutorial there: python-crfsuite ner example

as see after appending meaningful features - word.lower , ...- 2 features has appended.

features.append('eos') 

and

features.append('bos') 

my question "what's meaning of bos , eos , role of them?"

these stand "beginning of sentence" , "end of sentence". used in place of "previous word" , "next word" features words not have previous/next words.


Comments