Enhancements to Magic Cluster: Improved Accuracy and Multilingual Support

Avatar of Liam Scanlon
Liam Scanlon
September 05, 2024

Reading time: 1 minute

We've enhanced magic cluster 🪄 Hey Dovetail Champions! The team have made some improvements to increase theme accuracy by 2x and reduce outliers. Now supporting multilingual highlights and small datasets for sharper, more relevant insights. Let us know what you think!

  • Avatar of Josh
    Josh
    ·
    ·

    Is there anywhere we can read up on what "accuracy" means here?

  • Avatar of Jeremie Gluckman
    Jeremie Gluckman
    ·
    ·

    Hi Josh Somehow we missed this. Here's all of our documentation about the AI models and terms we use.

  • Avatar of Josh
    Josh
    ·
    ·

    Neither of those articles provide a definition for accuracy. The latter just says Dovetail has not verified the accuracy of their AI outputs.

  • Avatar of Jeremie Gluckman
    Jeremie Gluckman
    ·
    ·

    Thanks Josh. I can ping our security team and it would be helpful to understand the job you're completing and how clustering accuracy is working for you today. We're always open to feedback.

  • Avatar of Josh
    Josh
    ·
    ·

    There's not any particular thing I'm doing now, and I'm not criticizing the accuracy. I'm just trying to understand what the Dovetail team means when they say the model is twice as accurate. Knowing what "accuracy" means helps identify what the model is good at (and by extension, what it isn't good at).

  • Avatar of Jeremie Gluckman
    Jeremie Gluckman
    ·
    ·

    Josh Accuracy = whether highlights are assigned by AI into the same cluster that humans would, based on test data.

  • Avatar of Josh
    Josh
    ·
    ·

    But is there any documentation on how this was studied? Assessing that is a really complicated scientific problem, and there's lots of questions to unpack there (like what was the research question posed to the AI and researchers? what type of data? were clusters pre-defined? and what was the quantitative measurement? these are just a few examples) Please don't see this as trying to be a pain, but I have to work with an AI governance team that really wants to understand the appropriate use case to prevent misuse.

  • Avatar of Jeremie Gluckman
    Jeremie Gluckman
    ·
    ·

    These are great questions. The team is planning to create more blogs taking folks under the hood. I'll relay these thoughts to them.

  • Avatar of Josh
    Josh
    ·
    ·

    Thanks, I appreciate you!