8000 POC integration with DemDis by patcon · Pull Request #9 · polis-community/red-dwarf · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

POC integration with DemDis #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

POC integration with DemDis #9

wants to merge 10 commits into from

Conversation

patcon
Copy link
Member
@patcon patcon commented Feb 25, 2025 •
edited
Loading

This is a messy first-pass at providing output that DemDis could quickly swap into their system and compare.

The only changes I made in the function definition is adding a temporary skip_remap arg for testing. By skipping the remapping of votes from DemDis shape to the internal shape, this allows me to pass in Polis vote data, and avoid using votes with DemDis-specific keys and vote values (Demdis: agree/skip/disagree strings. Polis: 1/0/-1 ints that Red-Dwarf expects).

cc: @beowulf11 @denis-capkovic-demdis @Sketchead63

Todos

  • create basic run_clustering() function with existing functionality from red-dwarf
  • remap demdis vote style to red-dwarf internal style
  • add ability for force a number of clusters k
  • add ability to pass in initial cluster centers
    • update docstrings
    • add unit tests for this
  • implement statement_boost arg
  • implement calculation of StatementConversationMetric data
  • implement calculation of ClusteredStatement data
    • implement cluster_defining_{pos,neg,skip}_coefficient
  • refactor to easily return last_vote_at in results
  • clean up code
  • document run_clustering()
  • create helper to render visually (plotly?)
  • create jupyter notebook allow easy demonstration

@patcon
Copy link
Member Author
patcon commented Feb 26, 2025 •
edited
Loading

Questions for DemDis folks:

  1. Would you be open to sharing a dataset of raw vote data for a public conversation? I could use that to check if our calculations are working out the same. This one would work, but ideally one with more than 2 clusters if available
  2. Is cluster name (in all the places if appears), just "A", "B", "C" etc? (I'm reasonably certain of this via API)
  3. StatementConversationMetric
    1. mean_agreement_percentage is just the average agreement across all clustered participants? or across all participants, even ones filtered out of clustering algo?
    2. consensus_points. what is this specifically?
    3. polarization_measurement. what is this specifically?
  4. ClusteredStatement.
    1. Is agreement_percentage just (agreement_count / group_participant_count) * 100? Or does the denominator ignore unseen votes?
    2. What are cluster_defining_{pos,neg,skip}_coefficient?

@patcon patcon changed the title Create a run_clustering implementation that matches DemDis implementation Create POC integration of red-dwarf into DemDis Mar 2, 2025
@patcon patcon changed the title Create POC integration of red-dwarf into DemDis POC integration with DemDis Mar 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
Red-Dwarf Roadmap
Status: No status
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@patcon
0