in ner(named entity recognition) example in python-crf package website see function feature generator:
def word2features(sent, i): word = sent[i][0] postag = sent[i][1] features = [ 'bias', 'word.lower=' + word.lower(), 'word[-3:]=' + word[-3:], 'word[-2:]=' + word[-2:], 'word.isupper=%s' % word.isupper(), 'word.istitle=%s' % word.istitle(), 'word.isdigit=%s' % word.isdigit(), 'postag=' + postag, 'postag[:2]=' + postag[:2], ] if > 0: word1 = sent[i-1][0] postag1 = sent[i-1][1] features.extend([ '-1:word.lower=' + word1.lower(), '-1:word.istitle=%s' % word1.istitle(), '-1:word.isupper=%s' % word1.isupper(), '-1:postag=' + postag1, '-1:postag[:2]=' + postag1[:2], ]) else: features.append('bos') if < len(sent)-1: word1 = sent[i+1][0] postag1 = sent[i+1][1] features.extend([ '+1:word.lower=' + word1.lower(), '+1:word.istitle=%s' % word1.istitle(), '+1:word.isupper=%s' % word1.isupper(), '+1:postag=' + postag1, '+1:postag[:2]=' + postag1[:2], ]) else: features.append('eos') return features you can see completed tutorial there: python-crfsuite ner example
as see after appending meaningful features - word.lower , ...- 2 features has appended.
features.append('eos') and
features.append('bos') my question "what's meaning of bos , eos , role of them?"
these stand "beginning of sentence" , "end of sentence". used in place of "previous word" , "next word" features words not have previous/next words.
Comments
Post a Comment