Notalllanguages,e.g.Chinese,havedelimitersforwords.Toextractwordsfromasentenceintheselanguages,weusuallyrelyonadictionaryforknownwords.Forunknownwords,someapproachesrelyonadomainspecificdictionaryoratailor-madelearningdataset.However,thisinformationmaynotbeavailable.Anotherdirectionistouseunsupervisedmethods.Thesemethodsrelyonagoodnessmeasuretoevaluatehowlikelythewordsaremeaningfulbasedonastatisticalargumentonthegive
2023/9/7 3:51:43
512KB
研究论文
1