This is the full transcription of podcast 'GNU World Order Linux Cast' - gnuWorldOrder_553.
#Podcast #Transcription #ReadAlong #KnowledgeUnlocked
You are viewing a single comment's thread from:
This is the full transcription of podcast 'GNU World Order Linux Cast' - gnuWorldOrder_553.
#Podcast #Transcription #ReadAlong #KnowledgeUnlocked
Let's go through every single package installed with a Linux install image. I'm going through the software included with Slackware, but these are all open source applications and libraries, so whether you're running Slackware like me or Fedora, Debian, BSD, or even Mac or Windows, you can probably download, install, and try these on your computer. So chances are, you'll be able to learn something from this podcast. Let's get started. This is the final episode, I hope, of the L section. We'll see if that works out. I'm gonna try to do this. T1lib. That's Type 1 Font Rasterizer Library. It's a library donated by IBM a long time ago to the X11 project, and it takes Adobe Type 1 fonts and generates character and string glyphs from that specification. It's why you can just drop in a Type 1 font, whatever that is, TTF, I don't know, and have a font appear on your screen. That is exactly why. So that's an exciting thing. Next is Taglib. It is an audio metadata library. It reads and edits (1/45)
metadata of several popular formats, including, of course, MP3, but also Ogg, Vorbis, and Flack, and it does ID3 version 1 and ID3 version 2. Of course, this is a library. It is not a user-facing binary. There's Taglib-config and Taglib-extras-config or something, and I think those are associated with this, are they? Yeah, they are. Well, Taglib-config is associated with this. That does ship with this package, but you can't really do anything with it. If you're looking to tag an audio file, you're not going to use Taglib. Taglib is a, as its name implies, library, and so you're going to be looking at this. If you're programming, there are a bunch of header files, and a lot of them are kind of almost self-explanatory. There's like MP4 file.h. There's Flack. There's MP3, maybe? I don't actually see that one. There's Ogg. There's MPEG. There's MPEG. Opus, Speaks, a bunch of different things that you're going to, a lot of header files there that are kind of obvious if you're writing an (2/45)
application where you also need to read the metadata of files that your user is providing, then Taglib is one of those libraries you're going to use. Taglib-extras is the next package. That one does include Taglib-extra-config, and this was split off from the Amaroq project, apparently, to deliver support for reading and editing even more audio formats, like ASF, MP4v2, RMFF, and WAV files. Amaroq, of course, was a very, very popular music player for a while. It was one of those applications that I feel like it was kind of a killer app on Linux, really. I mean, it was quite popular. A lot of people quite enjoyed Amaroq, myself included. That was my music player for a very long time. It had a pretty good following, lots of people were into it, lots of extra things being developed for it, and then suddenly, like, it just kind of, I don't know, I don't know where it went. It just kind of disappeared. And when I say it disappeared, I mean, it, you know, I lost sight of it because it wasn't (3/45)
being included in Slackware anymore. Like if it's in Slackware, I know it exists, and if it's not, then it may as well not exist, practically. There was a release, I guess, an alpha release back in 2021, and they acknowledge in the release notes you read correctly after a long hiatus and all developers busy with real life, the Qt 5 port has had to wait. But finally, thanks to the dedicated work of a few hearty developers, Pedro and Heiko, we are proud to present the technical preview of the next Amaroq 2.9.71, which is also known as Amaroq 3.0 Alpha. That was in 2021, and I don't believe that they've had progress, at least to their website, since then. And yeah, I don't know. I mean, with that kind of delay, when a project does that, I feel like it's difficult to get back into it, because it's like, well, 2021, so when's your next one going to be? And maybe 2021 turned out to be, you know, that's all it took. Like, dang, it's done. It's better now. But I don't know. It's hard to kind (4/45)
of get back into it after sort of that breach. I don't want to sound overdramatic. I was going to say breach of confidence, but it's not a breach of confidence. It's just, you know, you're just not sure about the future of the project, and so you kind of hesitate to sort of throw yourself into it again. But I would love for Amaroq to come back full swing. I really would. I think that would be great. I'm using Elisa now, as I've said before, E-L-I-S-A. It's a nice little QML, I guess, based player. It's fine. It more or less is laid out the way I laid out Amaroq anyway. So well, actually, it's a little bit inverted. But yeah, it's got all the components I used in Amaroq. It fits the peculiar way I choose to consume my music. So it's fine. It's just not great. And Amaroq was kind of great. So I would love to see it come back. But not today, I guess. And that's fine. There's no judgment here. That's fine. I understand developers get busy. Projects have to wait sometimes. Luckily, there (5/45)
are other projects to kind of take the place. But I would not mind seeing Amaroq come back. So TAlloc. That's the Memory Pool System Library. It's a hierarchical reference-counted memory pool system with destructors. There you go. This is used by the Samba project to allocate memory, as you can imagine. And part of allocating memory is then sort of deallocating memory. I mean, you've got garbage collections and things like that. So maybe you don't have to worry about that so much. But it is a nice feature to have. The main goal of TAlloc is to remove the needs of creating a cleanup function for every complex structure. And the way that it does that is that it allocates memory in a hierarchical structure of TAlloc contexts such that deallocating it, like one context, recursively frees up all of its descendants as well. That's such a brilliant design. Which doesn't surprise me. The Samba project is one of the coolest projects, I think. They do amazing work, consistently amazing work. And (6/45)
it's so cool that this exists. And I don't know, to me, it sounds like, oh my gosh, well, why isn't everyone using TAlloc? Why would you use anything other than that? And I don't know. I don't know the subtleties involved in choosing your memory allocation library. But there's one that you might want to look at, TAlloc. Next up is the Tango icon theme, which I assume is included here, probably for XFCE, I would think. Well, I mean, you could use it for other desktops as well. Like for GTK applications, I guess. The Tango project was great. It was, or is great. I don't know if it's still around, to be honest. But it was, is the, I guess the official sort of look and feel, in a way, at least in terms of, you know, as much as an icon defines a look and feel, of the FreeDesktop.org project. If you go to Tango.FreeDesktop.org these days, it's not found, it doesn't exist. So I don't know if it's still sort of a thing for the FreeDesktop.org or not, but Tango was the official icon set for, (7/45)
really, the GNOME desktop for a very long time, I think. And it just kind of provided a reliably consistent, attractive, consistent design for the look of files in a file manager, essentially. I mean, that's, you know, what more do you want? More applications in an application menu. And it had a color scheme, and it had guidelines. You know, it had all these kind of strict rules around what an icon was allowed to look like, or at least a Tango icon was allowed to look like. And when you do that, you get consistency. So that's Tango icon theme. It's really, really nice. I have a lot of admiration for Tango as a concept. Like that kind of open source project with those kinds of, you know, rules maybe sounds too strict, but like just a scope or a constraint of what it can include. When you do that, I mean, yes, you're limiting user creativity and that sort of thing, or contributor creativity, but sometimes that's what a project is, is the limitation of a thing. That's why it's a project. (8/45)
You can do whatever you want in life, but if you want to do this specific thing that fits into this box, then you need to follow these rules. And I think that's a really great way of structuring things. And the result speaks for itself. Like Tango is a really attractive, I mean, look, I don't use Tango icons. So in a way it doesn't, you know, it's not that attractive, I guess, right? I use Breeze. But it's still, I still acknowledge that it's an attract, you know, it is a consistent and attractive looking sort of professional and yet fun kind of looking icon theme. And that's sometimes all you need, really. You just want all your icons to look more or less the same. And it's, it's hard. That's a lot of icons. So Tango existing and giving you a reliable set of icons for practically everything that you have on your computer is really useful. There's the next one, Tango icon theme extras, which has more icons like iPods and Dell Pocket DJ DAP. I don't know what that is. Probably another (9/45)
media player. Yeah. So, so more icons in, in Tango icon theme extras. After that one, there's TDB. This is the Trivial database library. It's a, it's a database API is what this is. And it was developed again, just like T.Alloc by the Samba team for Samba. And that's why it's included in Slackware. And the, the idea was that, that a lot of times you have like databases in a project and maybe there are different databases for different aspects of that project or, or whatever, or maybe even some ad hoc databasing going on. And so the Samba team realized what, what they needed was a Trivial database API so that you could just make calls to a database without having to learn which database this was using or the structure of that database and so on. It's just simple, simple database calls to some database TDB. Another T library is next, as you might expect, we're in the T section of the L section. But the, the T event is also from Samba and it is based on T.Alloc, the allocation, the memory (10/45)
allocation library. And this is the, this is the event system for Samba. So when something happens within Samba and you want another part of Samba to know about it, you're going to be using T event. Okay, next up is tidy HTML or is it, no, yeah, tidy dash HTML five. That's the name of the package. What that contains is a command called tidy. And this is a, it's essentially a linter for HTML five and XML. So if we do Emacs dot, let's do index dot HTML, I guess would work. And we do, well, let's just do a paragraph tag. Hello world, close paragraph tag. Now hopefully tidy won't be happy with that. So I'm going to do tidy of H of tidy space index dot HTML. And it says that it's found three warnings and zero errors, which is a lot nicer than I would have expected. And it spits it out for me with the header information. So doc type HTML, HTML head, meta name generator content equals HTML tidy for HTML five, title, no title head, close head, open body, P hello world, close paragraph, body, (11/45)
HTML. So it actually gave me the header and the footer or what I would think of as a header and the footer. That is everything up to the body opening tag and everything from the body closing tag gave me that for free. So that's kind of cool to know about, to be honest, because that means what you can do as a user, you could just write the HTML part that you care about and then have the top matter and the bottom matter auto-generated for you. So that's kind of cool. Let's go in, do tidy HTML. Oops, I didn't mean that. Let's do emacs index dot HTML and I'll do, let's make world strong, but then let's close it with emphasis. So we open tag of strong close with emphasis tag. Wait, emphasis, is that a, that's not a tag. EM, that's what it is. Slash EM. Yeah. Okay. Getting my XML and HTML confused. Now this time it found four warnings and zero errors. Wow. This is nice. This is so nice. And it, so it auto detected that there was the incorrect tag closing tag for EM. It says warning in line (12/45)
one, column 10 warning, replacing unexpected EM with slash close EM. I don't know what that means. It's not replacing EM with EM. It replaced it with hello strong world close strong. So it actually corrected the error that it did find. I don't know what it would take for this thing to find an error because it's correcting everything I throw at it. It's really, really nice. So this is tidy. Well it's tidy and yeah, you should use it. Like this is really, really nice. Like this is a great little linter. I should use it. Heck. Let's do the thing that I don't want to do and wget gnuworldorder.info. There's index.html now, a new one. And now I'm going to do tidy on it. Ooh boy, that's a lot of output. 1084 warnings and one actual, actual error. I can't find the error in this output though. And it needs me to fix the error before it will auto fix the warnings. That seems like a problem. Well I can supposedly send the errors out to a file with dash dash file or just dash f. So tidy index.html (13/45)
dash f errors dot log. Let's do that. Okay. That still spat out all of the output into my terminal and it hasn't given me my prompt back. It's a bit weird. Okay. Maybe I'll control c out of that and then let's look. Let's do a cat of errors dot log. It's empty file. Okay. So that did not work. Let's do a man tidy real quick. Is there a man tidy or is it just, yeah. Um, HTML, common use of HTML supports two different kinds. A dash, oh it's just a dash, it's a single dash for file, but still dash f should have worked. And there's also a dash o for output, dash output or dash o. Write output to the specified file. I don't know what the difference between output and, oh output is probably the corrected, the corrected thing. I don't know. It's not, it is not doing what it's saying that it's going to do. It just, it doesn't give me my prompt back. Okay. Well what if I pipe all of this, forget the output, pipe it, pipe the output, whatever to grep and grep for error. Wow. That did not work (14/45)
either. I don't know what they're, I don't know what they're putting into, I don't know where this stuff is getting output to. Okay, there. I've just done a two greater than errors dot log. Now let's look at errors dot log. There we go. That did it. But boy was that a, their options are not working. So let's do a search for the word error. Here it is. Error is in line 6772 column 16. Standard less than stdio dot h greater than is not recognized. So this is finding a little bit of a code sample in my show notes and it's mistaking it for, for HTML because I didn't escape the less than and greater than. So I'll go in and do that really quick manually. Ampersand hash less than semicolon. Oh wait, is that, no, you don't put the hash in HTML, right? That's different. That's greater than, I think, I don't know. You know what? I don't know the HTML. I'll just do the XML instead. So I'll do ampersand hash 60 semicolon for the less than and greater than hash 62 semicolon for the greater than (15/45)
symbol. And I'll return, I'll save and quit that. And now if I run tidy index, there it goes. Okay, so now it has dumped the corrected output into a file. Now I don't, interestingly, I don't trust the corrections enough to just accept this. So I'll have to work on this. I know that the name, a name warning can be easily fixed with a ID, but if I fix it here, that's not going to do me any good until next week because next week it'll get auto-generated again and it'll have names. So I have to go and fix podwrite as well. So yeah, that was interesting. That was informative, actually. That's really useful. I don't know that their application is working the way that it's advertising it's working, but I guess close enough is good enough. And that's useful. So if you do HTML or XML, don't overlook tidy. I didn't know about it. Okay, next up is UTF-8-proc. This is a Unicode processing library. It's for C. People need this for obvious reasons. UTF-8 is the wave of the future, coming way too (16/45)
late to the game, but it's here. We have UTF-8. Luckily, it's great. Unicode is wonderful. ASCII is not all that great, as it turns out. I mean, not for the scope of computing that we have today. So this is a library that provides Unicode normalization, case folding, and other things that you might need for UTF-8 encoding in C. And it's maintained by the Julia developers. That's interesting. So, I mean, it didn't start with the Julia developers, but apparently they took over maintainership for that. That's kind of cool. All right, next up is V4. So we're out of the Ts. We skipped past the whatever's before. I think it's a U, right? TUV. Yeah. So we skipped past the Us. There are no... No, there was a U. What was I talking about? UTF. We just did that. Okay, T. Okay, that was the Ts. U, UTF-8-proc. Okay, that's done. Now we're in the V section. That's V4L. That's video for Linux-UDLES. And this is... This... Well, the dash-UDLES, I guess I should just look really quick at the varlog (17/45)
packages. V4L-UDLES. What is this? 1.22 or something. Yeah, 1.22.1. So this has a bunch of UDEV rules, which is related to what I'm going to talk about momentarily. It also has a couple of include files, quite a few include files, and some .desktop files for some of the default applications, like Qvid, Capture, and so on. So let's back up a little bit. V4L, that's video for Linux. That's the project that provides drivers to the kernel for video capture devices. And this is important because when you plug in something with video capture capability, you want it to show up as a video capture device, meaning that it shows up under slash dev slash video something, video zero, video one, video two, or whatever, unless, of course, it's a video blank data that is like a device used for closed captioning or teletext. That would be a slash dev slash VBI something, zero, one, two, and so on. And then there are others. There are other ways, like there are touch sensors, V4L-touch zero or one or (18/45)
whatever. There are sub-devices and radio tuners and modulators and lots of different things that apparently fall into the same purview. I don't know how all of those go together, but apparently they do, or at least they're close enough for the V4L project to find that it makes sense for them to take that on. And once something's logged as a dev, as a device under the slash dev file tree, of course, it is in theory somewhat available to your users. Now if you're a programmer and you're doing cool things with a video device, you're not going to be tapping in necessarily to slash dev slash video directly. You're going to just be, you're just going to query a video 4L library and ask what kind of devices are here and where are they on the system and can I list them in this drop down menu so my user can select between which of the devices they want to use, that kind of thing. So there are some header files, lots of header files in here, to kind of help you do those kinds of extra (19/45)
activities in your code. There are some UDEV files as well, definitions, to make some of those devices, depending on what they are, useful to the user. So it kind of depends on what they are, obviously. I mean there's lots of different ones in there. There's Xbox DVD, there's Vega, a TeraTek, a TiVo, you know, all of these different sort of things. And one of the main features of the v4l.util package specifically is not just the v4l utilities but also the DVB, the digital video broadcasting stuff, which is kind of important for really the new way of broadcasting television. Very few televisions these days receive, well does anyone receive signal over air at this point? But yeah, certainly all the serious mainstream stuff, that's all digital broadcasting now. So having access for that if you're building your own digital video on demand type system, Myth TV, that sort of thing, the TiVo alike, which is funny because no one does any of that anymore, and yet people do still. There is (20/45)
television and you can record it and you can watch it later, and so if you're building your own system to let yourself do that, you're gonna want all of these utilities. DVR, that was the other, DVR, Myth TV is DVR. So yeah, this is really useful for that. V4l, big deal for just getting video input into your Linux box. Now you can do a quick little test of a video device if you have a webcam plugged in or built in, then you can do a QV, no what is it, Qvidcap, and that'll open up your, that'll just show you your, I think whatever's on video zero. I haven't actually tried it with more than one video capture device. I don't know how it would choose between different devices. So yeah, Qvidcap, that ships with V4l, it's kind of a quick and easy test of just like, okay, is the video camera recognized by my system and working? If you see yourself when you do Qvidcap in real time, then that's working. Whether or not the application you're trying to also make see the video camera is working is (21/45)
another question, but in terms of troubleshooting your webcam, Qvidcap is a great one. Hey, it's time for a coffee break. Go get yourself a cup of coffee. Come back. And we're back. I've got coffee. I'm still on this Brazil coffee from the southernmost dry goods store in the world. So, um, there is, uh, the next one, we're, we're, we're very close here to the end. Like we were very, very close. We've got maybe five more packages to go and then we're done the libraries. We're into the networking. So that's exciting. But the next one up that we have to deal with right now is vid.stab and vid.stab is a video stability, um, library for use with like ffmpeg. And I used to install this manually myself and then I would compile it into ffmpeg. But I think as of, I think it's been as of 15, I could be wrong, but I think as of 15 Slack, uh, Slackware has included vid.stab in it. So you don't even have to recompile. You can like, if you do, um, ffmpeg-F and then look through the output, capital F (22/45)
as in formats. Um, Oh, that doesn't, that's not, that's not a valid, it just ignored, it just ignored it and, um, gave me the exact output I wanted anyway. How about ffmpeg-v as in a version? Um, if you look through there and look for enable, dash dash enable dash lib vid.stab, which you will see, I believe, unless I've, I don't think I've recompiled this. Um, then you'll know, I mean, surely it will be there because that's why vid.stab is here. So yeah, lib vid.stab, that is the thing that, that you would be using. To use vid.stab because this is a library, right? I mean, it's not, there's no launchable application here. So you can use it directly from ffmpeg. And the way that you do that is you use a video filter in, within ffmpeg. So that is ffmpeg-i foo.mp4, let's say, so that your input is foo.mp4 and then you do a dash f, no dash vf as in video filter and then vid.stab.detect. That's the, the first filter that you need to use is, is the detection filter. And you can detect a (23/45)
couple of different things. You can look on the documentation, um, to, to, to get all, you know, to get everything that you can detect. But the most common one, I think, as its name kind of implies, video stability, is shakiness. So vid.stab.detect equals shakiness, s-h-a-k-i-n-e-s-s, equals 10 colon accuracy equals 15 colon result equals quote transforms.trf close quote. Okay, great. So now you've got a file, you know, you've run ffmpeg on, on this file foo.mp4 and nothing happens to the file, but it is analyzed. It's it's the, the video stab detect filter is run over the video and that results in a file called transforms.trf. You can use that data to then transform your video based on what vid.stab has found. That would be ffmpeg-i foo.mp4 space dash vf, another video filter, vid.stab.transform equals zoom equals 5 colon input equals quote my transforms.trf close quote. And then your output is foo stabilized, let's say, dot mp4. And that adjusts your video according to whatever (24/45)
detection vid.stab found on your first run. So it's a two step process. You have to detect first and then transform afterwards, but it works quite well. I've done it. Um, it works. I mean, when you're doing that, a lot of times you get sort of clipping in the corners or something like that. So you have to kind of account for that, but, um, it can stabilize video. Okay. So next up is the VTE, which is the terminal emulator widget. I'm assuming VTE much, but probably stands for like virtual terminal emulator, maybe. I'm not sure. Anyway, it's a, it's a widget. So it's a, it's a panel, a pane, uh, that you put into a window so that you can embed a terminal into a GTK application. So for instance, uh, I could imagine Glade for instance, using the VTE. It might not, I haven't looked at the source code of Glade lately, but it might, it could use that for, for the little terminal, terminal emulator down at the bottom of the, of the interface. Am I thinking of Glade or am I thinking of (25/45)
something different? I think it's a Glade. Anyway, um, this specifically VTE is used by XFCE for stuff. So that's why it's here in Slackware. It is akin to in the plasma desktop. If you press, if you, if you go into a dolphin window and then press F4, but by default anyway, you get a little terminal in the bottom of the, of your window, which mirrors where you are in your GUI and your graphical view. It's quite, quite cool. I use it a lot. Um, so that's, that's kind, that's the kind of thing you might see VTE being used for. After VTE is WAVpak, W A V P A C K. This is an open source audio compression format doing lossless and high quality lossy compression. I have never used WAVpak. I've, I always compile it into FFmpeg. When I used to compile FFmpeg all the time, now it's just, everything's in there anyway. Um, and, or at least everything reasonably everything. Um, and, and I just, I've never used it. I don't know why. I've just never needed it. I guess I don't use WAV files that (26/45)
often. I use them typically, if anything, as a temporary file format when one encoder can't take another format, you know, I'll just dump it out to WAV because that's pretty much taken by everything. So that's really all I use it for. Now WAVpak might be of interest to someone if they were using a WAV file and wanted to compress the WAV file. I don't, honestly, I don't, I can't think of the use case, but if you use WAV files a lot, there is a cool feature with WAVpak where you can create a, what's called a correction file. And this is a file containing basically the discarded parts of, of your lossless file, of your lossy file. So in other words, imagine taking a full quality digital photograph, a TIFF or a TARGA or EXR or whatever, and, and you make a JPEG out of it, but you get a correction file as well. And that correction file contains all the bits and pieces that are all the bits that were tossed out in order to get that full quality EXR down to a little tiny JPEG and through a (27/45)
WAV unpack command, you could then stitch those two things together. That's what happens with WAV un, with WAVpak. Or at least that's an option. You don't have to, you can just say, make this a WAVpak, we're done, that's all I want, forever. But if you want to make it a lossy compressed file, but keep the, the DIFF, you can do that. It's the dash C option, create correction file, which is a dot WVC file. And that, that provides sort of this hybrid mode that lets you have both a lossy compressed file with the option to stitch it back together with the differential so that you have a lossless file again. I don't exactly, like I said, I can't actually think of the use case for that now that I'm saying it out loud. It reads like a cool option and maybe it is a cool option. I'm just not sure what that option is. Like I don't know why that's a thing, but it is clever. Either way, that's a pretty cool little feature. So that's WAVpak and I've actually never used it. I doubt if I will use it. (28/45)
I mean, I've known about it for such a long time now and it's just never been something that I've thought, oh, I need, I need WAVpak. It just doesn't, I use FLAC for my, for my lossless file storage because I mean the file size is just so much better with FLAC. It really is. All right. So the next page here, W-O-F-F-2, W-O-F-2, is a font compression library. It is popularized by Google for web use. I have to say that fonts on the web used to be pretty difficult to manage, honestly. It was essentially as I recall, I believe that the, your options were, I think literally just like a set of Microsoft fonts. I could be, I could be making that up, but I believe that it was just like a set of fonts that sort of shipped really with your browser and, and that was kind of what the internet could have on it because the internet didn't really have a font. It would instead assume that you had a specific set of fonts installed, Arial and Helvetica and Times and things like that. And then it would, (29/45)
it would in, in its style sheet, which wasn't, you know, until CSS came along, the style sheet practically was just in HTML. But even after CSS came along, it would put in a style that the font was Arial or whatever, whatever sans serif font you happen to have on your system. Or it would be Times or whatever serif font you had, you happen to have on your system and mono, mono space or whatever. So it was, it was kind of this sort of just assume, assume that the user has configured their computer like the developer did. And that's a little bit, that, that can be a little bit quirky. Now I think in, in practice for most things that probably actually would work just fine. Like basically, you know, what are you, what are you doing with your webpage that you need the exact same font to exist? Well, as it turns out, people started kind of looking at web pages more as an artwork rather than a way to convey text. And they wanted the design to look a certain way. They wanted the spacing to (30/45)
appear a certain way. They wanted sort of a kind of a sense of certainty that the way that they've designed the page is also going to be the way that the user consumes it, which on one hand I get, I understand that. Like I do, I sympathize. On the other hand, that is awfully presumptuous, isn't it? Like, I mean, if, if, if someone doesn't like your font, then they're going to change your font. There's nothing you can do about it and they should be able to change your font. In worst case scenario, I guess they could select all and copy and paste it. And that's, I guess that's another thing. As much as you want your page to be consumed a certain way, I mean, someone might go to your page, select all, copy the text, paste it into a text file, and then go offline and read your, your, uh, text, you know, on the, on the subway or whatever. So it's, I think it was a little bit of an illusion that that would work out, but ultimately people wanted some kind of assurance that under ideal (31/45)
circumstances, when no one else disagrees, they could design a page that looked a specific way on most, basically all other devices, which I mean, it's, it's an impossible task. And I think people who are operating under these, the, under the illusion that, that that's true, I think are fooling themselves because I mean, it's just ultimately people have a lot of different devices out there and, and there's just, there aren't that many ways that there are lots of different ways to, to, to get that information. So anyway, people still want the illusion. And so in order for people to ensure that you had a font available to view their website in, basically started packaging fonts with their webpage, not, not that you would download, but instead the, the browsers started obtaining the ability to look at a file, a font file on the internet and use its specifications when rendering the glyphs. But the, another problem with this, once that started happening was now, now we're, we're sending (32/45)
both a webpage and a big old font file, potentially several font files along with each webpage. And of course the internet's about speed. I mean, you got to render fast or people are going to navigate away. So there was this need to define fonts as they have been defined, but smaller. The WOFF2 does exactly that. It compresses your TTF font so that you can send it over the internet quicker than you could just send a TTF file. You're probably consuming WOFF2 fonts without knowing it. They're, they, they come, you know, sort of, again, bundled along with a webpage. And so you don't, we don't really have to think about it. The web browser is doing everything that it needs to do. WOFF2 is useful though, if you're literally, if you're, if you want to take a TTF font and make it, you know, compress it and then upload it to your server and then reference it in a webpage or rather, you know, reference it in the style pay is style sheet of your webpage, thereby using the font on causing the (33/45)
font to appear on your webpage. You can, thanks to WOFF2. Next up is Zappian Core. I'm assuming that's a Zappian Core. It's X-A-P-I-A-N dash core. And this is, well, it's a search engine. And by that, I mean, it's, it's a, it's a method by which things are indexed into a database and then searched and, and, and returned based on some definition of relevance. And that's, that's the trick right there. Relevance. I mean, remember two episodes ago or one episode ago, maybe I was talking about, um, Nipomuk and the semantic social desktop. And then that kind of made me think about sort of just how a computer knows that you care about a certain thing and how context matters and how, how that, how a computer is able to determine that sort of thing. Maybe I wasn't. I thought about that at that time. And Zappian Core is a, is what they call a probabilistic search engine. So it, it is designed specifically to, to ideally like, you know, mathematically determine what a relevant result is based on (34/45)
your query. And that's a huge task. And they, they acknowledge that in the documentation and they list a bunch of different reading resources, in case you want to find out more about the literal maths behind their, their query solution or their query resolution. And it is really, really fascinating and way, way beyond me, but all of the usual search engine stuff I actually used to work for a search engine company, believe it or not. Not, not, not the big one. Um, and, and so you have like a database and then you've got your queries, which returns a document, meaning a document is anything returned by that database. And then you have terms that describe the document and you may have lots of terms describing lots of different documents. And that's what Zappian does, but it evaluates sort of the, the, the, the apparent relevance of a document based on your query and then tries to return the best results. How does it do? I don't know. Have you ever used Zappian core directly? Maybe I have, (35/45)
I'm not sure, but, um, that's, that's what it does. So searching is, is one thing and, and Zappian does a lot of multi-threading because it doesn't maintain state. So you can pretty much spam it with requests. It doesn't care. Um, you know, it's multi-threads. Um, but so, so it's, it's supposed to be pretty fast. Um, and, and, and, and then on top of that, it, you know, there's this layer of like, well, does, does this apply to what this person needs right now? Which is a huge question. Um, there is an Omega internet search engine as well, uh, that they, that they've written on based on Zappian core. So if you go to zappian.org slash docs slash Omega, you can get an overview of that. The docs for Zappian itself are well worth reading though. Like really digging into, into the technology is, is absolutely fascinating and just mind blowing. I mean, it's just, they've put so much thought into it. It's very, very cool. So check that out if you're at all interested in that sort of thing. (36/45)
Um, or even if you're not, it's more interesting than you might think. It's really cool. So that's, um, Zappian core. I don't know what, what, I wonder what's using Zappian core really. Cause I mean, I do kind of wonder something must be using it. Um, and there's a bunch of commands as well for like database, you know, maintenance and stuff like that. Um, let's try to find the actual library bundled in this thing. The actual library is libzappian and yeah, LDD cycling through everything in user bin anyway real quickly confirms that it's kind of what I suspect at Akanadi. So Akanadi the, the component of the personal information management suite PIM for KDE that apparently is using Zappian core. So that, that kind of makes sense. I figured it had to be either that or Baloo and does Baloo use it? Where is Baloo anyway? There it is Baloo underscore. No let's do Baloo search. Um, and then do a grep for Zappian I guess. No that does not use Zappian. I think Baloo must use, I don't know (37/45)
Baloo or something or some other component of Baloo does, but Baloo search does not. Um, yeah, it's using libkf5 Baloo engine. That could be what it's using. Who knows? I don't know. But um, anyway, Zappian being used by Akanadi. XX hash is next. This is an extremely fast hashing algorithm, ships with a library, XX hash or libxxhash.0.8.1. And a little demo application, xxh sum, which you can use to get, you know, hashes, hash results from, from files much like you would with sha, sha um 256 sum or what is it sha? No MD5 sum, sum, yeah, MD5 sum. You could, you can do that with xxh sum. So xxh sum dash h is your algorithm selection. So you've got 0, 1, 2, 3, 32, 64 or 128. So I'll put in 128 and then I'll just get a hash for this, um, this hello world file that I've just generated. And there you go. That was fast I guess. I mean, I don't think it's going to feel any less fast or more fast than sha 256 sum for instance or MD5 sum from a, from a terminal. I don't think that's where (38/45)
they're trying to optimize the process. I mean, maybe they are, but I think probably it's more, more targeting like the actual library functionality so that if in a, um, in an application you're generating hashes several times, you're not slowing things down by using some less slow library. I haven't done the benchmark tests myself, so I don't know. It could, could be true, could be false. They do have nice graphs on their webpage though. You can go look and see a bunch of sort of comparisons between x x h 64 versus x x h 128 and so on. So if you want to see how they compare to each other, you can get kind of a sort of a benchmark that way there's a table with benchmarks based on different algorithms. So yeah, there's, there's information there for benchmarking. Um, so yeah, that's, that's a potential source of information. If you're concerned about the speed of your hashing algorithms, the penultimate package in the L section is zlib. It's a general purpose thread safe compression (39/45)
library. So this is, um, kind of made famous for its very, very permissive license. Like you can just throw zlib in. I think it has its own license, but it has been, I think it has served as the basis for several other sort of really, really sort of, you know, don't worry about giving us credit or giving us code in return, just use it in whatever way you want to kind of license. So this is a compression library with, with everything written from scratch. You know, there's, so there's, there's no sort of patent infringement or, or threats of patent infringement on some other compression library, which I mean, you know, it's so hard to even conceptualize of how this could be a problem in the modern day and age where there are so many other problems to worry about, but you know, there's, um, there are people who care about that sort of thing, or companies rather, who care about the sort of thing. Um, and, and basically if you need a zip file, z-i-p, then, then, then zlib is one way to (40/45)
generate a zip file and, and it's, it's unencumbered by any patents because it is, is written from scratch. I feel like it's, um, it's just hard to imagine how, yeah, like compression could be something contentious, but there, there you go. That's, that's, it's, that's how things operate still. Uh, zlib is described in RFC, uh, I think 1950 and the, I'm not doing this from memory. I'm just looking, scrubbing across a page and, and it is, um, so it's, it's an open spec. I mean, it's something that you can, you can re-implement if you want to. I mean, it's a very, very open source library, so you wouldn't really need to, but you can see it all there. And then it, with zlib, of course, you can then generate zip files within your application. And as I've said before, I think on other zipping, uh, compression libraries, it's very, very common and kind of a convenient way for an application or for a developer, if you want to create your own file format, you just zip it up, call it whatever (41/45)
you want, but just dump it into a zip container. And that way all of the assets that you need for that save file are all in one place. That's not always the best way for all applications. I mean, it would be really, really annoying if Caden Live, for instance, did that because I want my assets outside of any kind of, some kind of container. I, I need access to that across several projects. I wouldn't want it to be bundled up. But other, other applications, it just makes good sense. You know, like document creation, you would want all of the assets that you believe are in your document to really be in your document, even if it's just copied over into a zip file, which is of course what the LibreOffice does, that the, the ODT format is a zip file with a bunch of XML and any images that you've dragged and dropped into your document and so on. It's all in there. It's all in that zip container. And the final package of the library set is Z standard. Z standard is another compression (42/45)
algorithm, believe it or not. That's what Zs are, right? That's what Z stands for in, in computers, compression. So Z standard does, sort of excels at high compression, high compression ratios for small data. It implements something called dictionary compression. The problem that this is trying to solve is that with small data set or with, no, rather with small pieces of data, the algorithm is not able to typically to learn from what's already been compressed because there's no, by the time you've compressed one thing and you're onto the next, now you're not, you're compressing something new. You're not learning from the past. Z standard apparently has a training sort of method or a training mode where it looks at the data, all the little small bits of data and kind of makes some rules, some general rules over what can be compressed and so on. I don't know enough about compression or dictionary compression or Z standard to say how all of that magic actually happens, but that's kind of (43/45)
more or less what they do. And the numbers that they provide on the webpage is, they're pretty good. There are lots of differences in good places. The webpage is facebook.github.io slash zsd. It's Z standard and yes, Facebook was the developer. They open sourced it in 2016 and it is now used all over the place. There's even graphical front ends to it by the 7-Zip folks, by the P-Zip project and a couple of other ones. So it's, I guess, a pretty effective little compression algorithm. It's one of those things where it's like, I mean, if it is being used, it was developed and being used by Facebook. Love them or hate them. They do have a lot of users and they do process a lot of data. So if they're using Z standard as their real-time compression algorithm, then there's probably some value to the library, if not to the company that created it. So that's it. That's the libraries. Those are all of them. We're out of the L directory now. The next one for the next episode is the N as in (44/45)
networking directory. I have no idea what we're about to embark upon here. This could be a bunch of libraries that we can't use directly or they might be a bunch of interesting applications. I have no clue. So we will find out together. Thanks for listening. Talk to you next time. Thanks for listening. My name is Klaatu. You can reach me anytime over email with feedback or comments, tips, or just to say hi. My email address is klaatu at slackermedia.info. You can also reach me on the Mastodon network, not klaatu at mastodon.xyz. The show's intro and outro music is by Fat Chance Lester. You can find their music on bandcamp.com or on gnuworldorder.info. In the archive, you'll find a music directory containing the album from which this music has been extracted. Until next time, thanks for listening and keep the source open. (45/45)