8000 Week 10: Finish the Implementation of Regularized Radom Forest Framework, Tune Parameters and if possible implement Pruning of the Trees · Issue #20 · azmfaridee/mothur · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Week 10: Finish the Implementation of Regularized Radom Forest Framework, Tune Parameters and if possible implement Pruning of the Trees #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
azmfaridee opened this issue Jun 29, 2012 · 2 comments
Open

Week 10: Finish the Implementation of Regularized Radom Forest Framework, Tune Parameters and if possible implement Pruning of the Trees #20

azmfaridee opened this issue Jun 29, 2012 · 2 comments
Assignees
@azmfaridee
Labels
major task

Comments

@azmfaridee
Copy link
Owner

Related Issues: #3, #14, #15, #16, #17, #19

As per issue #19 I have been experimenting with outcomes after we have been running the algorithm on some real life data. Some of the new concerns are tree pruning for better performance, dealing with over-fitting data as well as the scheduled implementation of Regularized Random Forest Algorithm.

In this week I'd try to follow up with all these ideas that have been swirling around

End of Week Deliverable

  • Complete the implementation of Regularized Random Forest and compare performance/accuracy with normal Random Forest
  • Possible improved Performance with pruned trees
  • Investigate other ways of performance turing
@ghost ghost assigned azmfaridee Jun 29, 2012
azmfaridee added a commit that referenced this issue Jul 1, 2012
@azmfaridee
Issue #20: removed some wrong import
86c14ed
azmfaridee added a commit that referenced this issue Jul 1, 2012
@azmfaridee
Issue #20: created fileReaderFactory to handle two type of files.
aa3c1d2
azmfaridee added a commit that referenced this issue Jul 1, 2012
@azmfaridee
Issue #20: OptimumFeatureSubsetSelector class implemented, the number…
e2e48d7 
… of features selected in each split now can be fine tuned between log2 and square root of number of features
azmfaridee added a commit that referenced this issue Jul 1, 2012
@azmfaridee
Issue #20: getGlobalDiscardedFeatureIndices() function moved to Rando…
9fe31b0 
…mForest class
azmfaridee added a commit that referenced this issue Jul 3, 2012
@azmfaridee
Issue #20: RegularizedRandomForest class is being extended from Abstr…
50d44ed 
…actRandomForest
azmfaridee added a commit that referenced this issue Jul 3, 2012
@azmfaridee
Issue #20: incorporated AbstractDecisionTree so that we can easily bu…
0fbf638 
…ild RegularizedDecisionTree on top of it.
azmfaridee added a commit that referenced this issue Jul 3, 2012
@azmfaridee
Issue #20: some minor rearrangements.
64b9d76
@azmfaridee
Copy link
Owner Author

End of Week Update:

  • The implementation of Regularized Radom Forest is note 100% complete, there are a lot of classes that needs to be recycled, so I need to be extra careful to not to break existing code.
  • As for performance tuning, I've been reading the literature again and discovered that we need to use Information Gain Ratio than using basic information gain, that would boost accuracy a bit more.
  • Optimum number of selected feature in each split can be fine tuned between Log2 and SquareRoot based measure.
  • As I did not get sufficient learning materials (the books that I mentioned), I could not implement pruning.

@azmfaridee azmfaridee mentioned this issue Jul 3, 2012
Closed
azmfaridee added a commit that referenced this issue Jul 5, 2012
@azmfaridee
Issue #20: just copying some old codes to new class as placeholder
6efdaf3
azmfaridee added a commit that referenced this issue Jul 5, 2012
@azmfaridee
Issue #20: getStandardDeviation() function has been moved to Utils cl…
eb276bb 
…ass as static method
azmfaridee added a commit that referenced this issue Jul 7, 2012
@azmfaridee
Issue #20: removed already done TODO list
6555d18
azmfaridee added a commit that referenced this issue Jul 7, 2012
@azmfaridee
Issue #20: splitInformationGain added in TreeNode
f4818dd
azmfaridee added a commit that referenced this issue Jul 8, 2012
@azmfaridee
Issue #20: Integrating an almost working prototype of RegularizedRand…
d3ceade 
…omForest implementation, there are still some parts that are not working as expected.
@azmfaridee azmfaridee mentioned this issue Jul 11, 2012
Closed
@azmfaridee azmfaridee mentioned this issue Jul 24, 2012
Closed
@azmfaridee
Copy link
Owner Author

@kdiverson I've uploaded a pdf titled Pruning Decision Trees and Lists.pdf in Dropbox in the Pruning Folder. It seems to be a PhD thesis paper, and has some good examples of popular pruning methods. Take a look.

azmfaridee added a commit that referenced this issue Aug 8, 2012
@azmfaridee
Issue #20: [python] updateOutputClassOfNode() function implemented, i…
d77ea2e 
…t will help in pruning algorithm
azmfaridee added a commit that referenced this issue Aug 9, 2012
@azmfaridee
Issue #20: [python] testSampleMisclassificationCount printed in the o…
242f542 
…utput + some function call parameter change
azmfaridee added a commit that referenced this issue Aug 9, 2012
@azmfaridee
Issue #20: [python] updateMisclassificationCountRecursively() impleme…
22b12e5 
…nted
azmfaridee added a commit that referenced this issue Aug 9, 2012
@azmfaridee
Issue #20: [python] pruneRecursively() function implemented
8b53945
azmfaridee added a commit that referenced this issue Aug 10, 2012
@azmfaridee
Issue #20: [python] implemented pruneAggressiveness parameter
0614c00
azmfaridee added a commit that referenced this issue Aug 10, 2012
@azmfaridee
Issue #20: [python] default pruneAggressiveness is set to 0.9. this s…
9a71fcc 
…eems to be giving more consistant results than more aggressive value of 1
azmfaridee added a commit that referenced this issue Aug 10, 2012
@azmfaridee
Issue #20: [python] pruning criteria can be defined at the constructo…
3fd2fec 
…r of RandomForest
azmfaridee added a commit that referenced this issue Aug 10, 2012
@azmfaridee
Issue #20: [python] implemented a mechanism where highly erroneous tr…
8938925 
…ess are automatically discarded (error threshold defaults to 40% now).
azmfaridee added a commit that referenced this issue Sep 8, 2012
@azmfaridee
Issue #20: [c++] [integration] other random forest parameters now wor…
598e71e 
…k inside mothur, plus added mothurOut class in all of the RF classes to easy debug message anywhere
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@azmfaridee azmfaridee

Labels
major task
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@azmfaridee
0