8000 fastText的label有上限 · Issue #44 · jimichan/mynlp · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
fastText的label有上限 #44
New issue
New issue
Open
Open
fastText的label有上限#44
@yuyoyth

Description

@yuyoyth
yuyoyth
opened on May 18, 2022

这是打印读取train的结果

Number of words:  2148
Number of labels: 185898
Max threshold count: 2`
Number of wordHash2Id: 250728

可看到读取上限为185898，而train中我提供的label数为1300000+，为了排除数据问题，我将原本train以150000分割为9个文件，依次进行读取测试，结果均能正常返回label读取数，基本可排除是数据文件的问题
fastText是确定的设置了这个上限吗还是文件读取量有上限？原train文件有480MB大小，分割后最大为52MB

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0