先秦漢語自動分詞及詞性標注研究.pdf

1、南京師范大學碩士學位論文先秦漢語自動分詞及詞性標注研究姓名：石民申請學位級別：碩士專業(yè)：語言學及應用語言學指導教師：陳小荷20100509AbstrctAbstractAutomaticwordsegmentationandPOStaggingarethebasisofChineseinformationprocessinghavetwotothreedecadesofhistoryRemarkableachievementshavem

2、adeinthefieldofmodemChinese，butLiRIeresearchhasbeencarriedinthefieldofancientChinese，therearemanyissuestoberesolved，suchasthejudgementofPairsofsyllableswords，efficientancientChinesecorpusprocessingsoRwareandSOonThispaper

3、mainlydiscussedwordsegmentationandPOStaggingmethodsinthefieldofpreQinChinesefield，usingthetechnologiesofChineseinformationprocessingObservationandstatisticsoncorpusarethestartingpointforChineseinformationprocessingCorpus

4、basedstatisticsstrategyisthemajortechnologyusedinthisarticleCombinglargescalecorpus，wedidadetailedlinguisticstudyon‘‘ZuoZhuan’text，countedtheaveragewordlength，actualuseofthehighfrequencycharactersandwords，inordertosupply

5、datasupportforbuildingWSandPOSrulesofPreQinChineseOnthebasisoftraditionallinguisticsresearch，wesummedupthepreQinChinesepartsofspeechsystem，improvedwordnorms，thengavewordsprocessing(wordsegmentationandPOStagging)andanalys

6、edon“ZuoZhuan”textIntheendwecarriedoncomparativeexperimentsforautomaticwordsegmentation，POStaggingjointwordsegmentationandPOStaggingusingconditionalrandomfieldsmodel(CRF)InmepartofWSweusedthetheorythatwordiscomprisedofch

7、aracters，anddidcharacterbasedtaggingexperimentThetemplatecontainstwokinds：simpleliteralinformationandincreasingthecomplexcharacterfeaturesInordertovenfywhetherthereisanerrorspreadissueontheancientChinesecorpususingthetra

8、ditional“two—step’’approachwhichispostmarkedafterwordsegmentationthissectionspeciallydesignedtwocomparativeexperiments，thatisPOStaggingonthebestcuutingtextandjointWS／POStaggingexperiments，makingfulluseofpartofspeechinfor

9、mationWehavealsodonespeechtaggingexperimentsbasedonthestandardwordtextTheoretically，thewordaccuracyoftheBestSegtext，multipliedthePOStaggingaccuracyofthestandardtext，wecangetmeforecastvalueofPOStaggingontheBestSegtext，ast

10、hebestvaluationofthetestcorpusStudieshaveshownthattheintegratedapproachnotonlyincreasedthewordaccuracy，POStaggingresulthasalsobeensignificantlyimprovedThehighestFvaluesofwordandmarkopentestsepratelyreachedto9460％，8965％Oi