from gensim.models import Word2Vec sentences [[cat, say, meow], [dog, say, woof]] model Word2Vec(sentences, min_count1)用Gensim 的 Word2Vec做词向量训练把单词转换成稠密数字向量语义相近的词向量会接近。1. from gensim.models import Word2Vec导入 gensim 库里面 Word2Vec 模型类Word2Vec 是经典词嵌入算法分 CBOW 和 Skip‑gram。2. sentences [[cat, say, meow], [dog, say, woof]]训练语料格式是嵌套列表外层一条条句子内层句子已经分词好的单词列表第一句cat say meow第二句dog say woof3. model Word2Vec(sentences, min_count1)开始训练 Word2Vec 模型sentences传入分词后的训练文本min_count1词最少出现1 次就纳入词汇表默认min_count5出现少于 5 次的词直接丢弃这里数据集很小必须设为 1不然 cat、dog 等低频词直接被删掉。训练完后能干什么model.wv→WordVectors词向量对象对象含义能干什么model完整训练模型还保留训练相关全部信息只能用来继续接着训练model.wv纯词向量集合WordVectors拿向量、算相似度、找近义词日常 99% 操作都用它wvmodel.wv print(wv) # 查看词向量 print(wv[cat]) # 找语义最相似的词 print(wv.most_similar(dog))输出结果为KeyedVectorsvector_size100, 5 keys array([-0.00713902, 0.00124103, -0.00717672, -0.00224462, 0.0037193 , 0.00583312, 0.00119818, 0.00210273, -0.00411039, 0.00722533, -0.00630704, 0.00464721, -0.00821997, 0.00203647, -0.00497705, -0.00424769, -0.00310899, 0.00565521, 0.0057984 , -0.00497465, 0.00077333, -0.00849578, 0.00780981, 0.00925729, -0.00274233, 0.00080022, 0.00074665, 0.00547788, -0.00860608, 0.00058445, 0.00686942, 0.00223159, 0.00112468, -0.00932216, 0.00848237, -0.00626413, -0.00299237, 0.00349379, -0.00077263, 0.00141129, 0.00178199, -0.0068289 , -0.00972481, 0.00904058, 0.00619805, -0.00691293, 0.00340348, 0.00020606, 0.00475374, -0.00711994, 0.00402695, 0.00434743, 0.00995737, -0.00447374, -0.00138927, -0.00731732, -0.00969783, -0.00908026, -0.00102276, -0.00650329, 0.00484973, -0.00616403, 0.00251919, 0.00073944, -0.00339216, -0.00097922, 0.00997912, 0.00914589, -0.00446183, 0.00908303, -0.00564176, 0.00593092, -0.00309722, 0.00343175, 0.00301723, 0.00690046, -0.00237388, 0.00877504, 0.00758943, -0.00954765, -0.00800821, -0.0076379 , 0.00292326, -0.00279472, -0.00692952, -0.00812826, 0.00830918, 0.00199049, -0.00932802, -0.00479272, 0.00313674, -0.00471321, 0.00528084, -0.00423344, 0.00264179, -0.00804569, 0.00620989, 0.00481889, 0.00078719, 0.00301345], dtypefloat32) [(cat, 0.17018887400627136), (meow, -0.013514958322048187), (woof, -0.023671690374612808), (say, -0.05234676972031593)] 学科发展前沿—第二次实验
from gensim.models import Word2Vec sentences [[cat, say, meow], [dog, say, woof]] model Word2Vec(sentences, min_count1)用Gensim 的 Word2Vec做词向量训练把单词转换成稠密数字向量语义相近的词向量会接近。1. from gensim.models import Word2Vec导入 gensim 库里面 Word2Vec 模型类Word2Vec 是经典词嵌入算法分 CBOW 和 Skip‑gram。2. sentences [[cat, say, meow], [dog, say, woof]]训练语料格式是嵌套列表外层一条条句子内层句子已经分词好的单词列表第一句cat say meow第二句dog say woof3. model Word2Vec(sentences, min_count1)开始训练 Word2Vec 模型sentences传入分词后的训练文本min_count1词最少出现1 次就纳入词汇表默认min_count5出现少于 5 次的词直接丢弃这里数据集很小必须设为 1不然 cat、dog 等低频词直接被删掉。训练完后能干什么model.wv→WordVectors词向量对象对象含义能干什么model完整训练模型还保留训练相关全部信息只能用来继续接着训练model.wv纯词向量集合WordVectors拿向量、算相似度、找近义词日常 99% 操作都用它wvmodel.wv print(wv) # 查看词向量 print(wv[cat]) # 找语义最相似的词 print(wv.most_similar(dog))输出结果为KeyedVectorsvector_size100, 5 keys array([-0.00713902, 0.00124103, -0.00717672, -0.00224462, 0.0037193 , 0.00583312, 0.00119818, 0.00210273, -0.00411039, 0.00722533, -0.00630704, 0.00464721, -0.00821997, 0.00203647, -0.00497705, -0.00424769, -0.00310899, 0.00565521, 0.0057984 , -0.00497465, 0.00077333, -0.00849578, 0.00780981, 0.00925729, -0.00274233, 0.00080022, 0.00074665, 0.00547788, -0.00860608, 0.00058445, 0.00686942, 0.00223159, 0.00112468, -0.00932216, 0.00848237, -0.00626413, -0.00299237, 0.00349379, -0.00077263, 0.00141129, 0.00178199, -0.0068289 , -0.00972481, 0.00904058, 0.00619805, -0.00691293, 0.00340348, 0.00020606, 0.00475374, -0.00711994, 0.00402695, 0.00434743, 0.00995737, -0.00447374, -0.00138927, -0.00731732, -0.00969783, -0.00908026, -0.00102276, -0.00650329, 0.00484973, -0.00616403, 0.00251919, 0.00073944, -0.00339216, -0.00097922, 0.00997912, 0.00914589, -0.00446183, 0.00908303, -0.00564176, 0.00593092, -0.00309722, 0.00343175, 0.00301723, 0.00690046, -0.00237388, 0.00877504, 0.00758943, -0.00954765, -0.00800821, -0.0076379 , 0.00292326, -0.00279472, -0.00692952, -0.00812826, 0.00830918, 0.00199049, -0.00932802, -0.00479272, 0.00313674, -0.00471321, 0.00528084, -0.00423344, 0.00264179, -0.00804569, 0.00620989, 0.00481889, 0.00078719, 0.00301345], dtypefloat32) [(cat, 0.17018887400627136), (meow, -0.013514958322048187), (woof, -0.023671690374612808), (say, -0.05234676972031593)]