Oh, this is pretty much all I know about how to use python.
I don't have any formal data qualifications. I learned everything I do now on the job.
I thought I was pretty good at excel, with formulas, pivot tables, and vlookups and all that junk.
Then, during the start of the pandemic, I started using "KNIME" to automate the crap I'd do in spreadsheets all day. It's basically a visual coding tool that lets you string together different operations on data sets and do things you wouldn't believe.
I learned by basically saying "right, I how do I do {thing} in excel?" let me try and reproduce it in Knime. As a result, the only time I typically use excel now is for my household budget, and to track upcoming personal tax liabilities (though, I'd rather be throwing up in the street).
Then, I was introduced (by necessity) to PowerBI and Power Query as I moved into a temporary role at my org as a Reporting and Insights specialist.
Now all I do is extract data, transform it (and clean it up) and try to get it be represented in a way in which business stakeholders can easily see "green number good, red number bad, we blame filthy {blame recipient}"
The thing about data is the same thing about programming, I've found (and I've only done python stuff for less than a year) - is you are literally this guy:
Get a whole bunch of things which don't seem to have an obvious link, or connection to one another, and make them work together.
That's awesome man. I'm hardly a programmer. I think I'm honestly a hack. I'm just out here winging it. I admire your way of breaking into your industry. I didn't realize how far actions could take someone. I got accepted into academic publication for some blockchain stuff and it was mostly because I just did some foot work. No one ever accused me of being the smartest guy ever. Maybe you can still help me though, I'd have to give you some background info on DNA though.
Oh, KNIME can do chemistry and molecule analysis-y stuff in there too.
Lots of stuff in there that I can't remotely begin to understand. I reccomend checking it out. I wrote a different tutorial showing off some knime function in concert with some splinterlands stuff that is probably horribly outdated by this point, but it's useful, I guess.
https://peakd.com/hive-13323/@holoz0r/tutorial-using-the-splinterlands-api-and-knime-to-determine-what-cards-you-require-to-complete-your-untamed-collection
I'll have to take a look. Some of the stuff I'm doing would be modifying an existing machine learning paradigm. I don't have the slightest idea of how ML works other than number go in, number come out. We'll see. I'll do some digging and see if I can find some tangentially related shit between what you've said and what I need to work on.
What about the "pretty" data visualization stuff? (heat maps, {insert fancy graphs here}, etc)
Yeah, that's a part of Power BI.
Great thing about Knime is that it lets you build your own machine learning models, with blackjack, and hookers.
But in all seriousness, it has full support for python, custom nodes, and a whole bunch of other community plug ins. I'm surprised it is open source, and free.
If you want to automate stuff on a commercial / corproate level, the enterprise licencing makes sense (just to automate dataflows, and such, but its still an absolute steal compared to what other vendors would charge for software half as useful)
(I almost signed that off like a work email, it's fucking Sunday, and I don't want to go back to work tomorrow)