RE: Text analytics reveal thirty two percent of comments on hive are not unique and at least ten percent add no value to discussion

[vatman]

I’m really looking forward to a post where you break down the process a bit more. I’m still kind of new to programming and data stuff, but I pick things up pretty fast, and this kind of analysis is super interesting to me.

Also, I was thinking — have you ever considered using ChatGPT or the GPT API to help analyze comments? Not sure what your take on it is, but I’ve used it a lot to help me understand topics that I’ve always struggled with. I just tell it to explain things like I’m five, and honestly, it helps a lot.

Obviously it’s not a replacement for real thinking or research, but as a tool, it can be surprisingly useful.

Comments

[holoz0r]

There's a lot of stuff that pre-dates LLM analytics which is pretty good for text. The whole area of NLP (Natural Language Processing) is probably very foundational to the existence of LLMs like GPT.

I have an idea that I can run a local model to identify some trends on the comments. That is a longer term goal. The problem with doing such analysis is that there's a random seed generated each time you ask a LLM a question, so each time, you'll get a slightly different answer.

As a result, you can't really run tests with different data over different time periods and expect it to treat the analytical process in exactly the same manner, even if you're using the same prompt in the same session, with a different data set.

There's a few commercially available models that focus on repeatable, more empirical analysis, I trialed a few at my old job, but there's a lot of stuff you can do to exhaust the data before you get to "AI" giving you new insights or new paths to go down.

I plan to exercise all those traditional methods before I dive headlong into whatever stuff I can cook up for a LLM to do these data sets.

It will be able to, at a guess - that would be more difficult in traditional NLP methods, if not normally possible:

1. Identify the propensity for content to not be human generated (ie, a confidence score)
2. Probably translate stuff into one language so it can all be analysed in one go :P
3. Tag the languages present in the content. I've seen English, Spanish and Cantonese / Mandarin / Korean / Japanese characters in the text.
4. Perform rapid sentiment analysis. Are users positive, negative, are there any trends that are commonly spoken about in the comments?
5. Probably use the comments as prompts to generate pointless images

I am not formally qualified in any of this data science or data reporting stuff. I've just worked with it for years at my old job, and hope to find a new job (I was made redundant about 6 weeks ago) where I can use these skills to make people do things with more integrity, ethics, respect, or.... worst case, y'know capitalism, cos a man like me loves pizza.