Digital Decluttering: Music Collection (Premium)

I am a lifelong music fan, and while there were many amazing advances to the ways we enjoy, curate, and share music over my lifetime, the advent of digital music in the late 1990s changed everything.

Well, not overnight: at the time, my friends and I were aware of the MP3 audio format, but computer hard drives were still too small to even dream about ripping what we, at the time, believed were the perfect digital recordings on our audio CDs. Indeed, my first hard drive, which I had purchased for my Amiga 500, had just 30 MB (not GB) of storage, so using 5 to 10 MB of that for just one song simply didn’t make sense. Even by 1998, I was more taken with the inclusion of Deluxe CD Player in Plus! for Windows 98, as it downloaded the artist, album, song, and other information about each music CD you used and displayed it in the app. Useful!

But things change. And as hard drive capacities grew, we enjoyed a nice evolution of the tools and file formats we could use to not just digitize our music collections, but store and manage them entirely on PCs, and sync some subset of those songs to portable media player devices. The rise—and fall—of Napster, competing formats like Windows Media Audio (WMA) and Advanced Audio Coding (AAC), iTunes and then the iTunes Music Store and its 99 cent singles, the iPod, the iPhone and the smartphone revolution, and streaming music services were all major milestones that defined the intervening 25 years for me and other music fans.

And when it comes to digital music, devices, and services, I’ve done it all, and more. I used every single music service to come down the pike, both pre- and post-streaming, dozens of different MP3 players, and almost every single iPod model ever made, and I ripped and re-ripped my audio CD collections more times than I can count or remember, in various formats and quality levels, over the years. As digital music stores came and went, I purchased more and more of my music online. And when the music industry finally gave up on DRM (digital rights management) copy protection controls, I made sure to download the unprotected tracks I had purchased everywhere and I added them to the local copies of my music collection.

As an obsessive ADHD type, I curated and managed these collections with great diligence, and I carefully copied them from PC to PC back when that was necessary so that I could enjoy my music on the road and sync it with whatever device I was using at the time. In using almost every Windows Home Server (WHS) server ever made, then some servers based on Windows Server Essentials, and finally, my NAS, this collection has always come forward with me.

But it’s also sat largely unused in the many years since I moved to streaming music. Here, things are a little fuzzy, but I actively used Zune Music Pass, which became Xbox Music Pass and then Groove Music Pass, and then Google Play Music, which became YouTube Music, all while experimenting with other streaming services like Spotify, Amazon Music, Apple Music, and many others. And thanks to my age and experience on both sides of the digital music revolution, I have always been keen to combine music from my own collection with that of whichever streaming services I used.

And that is interesting on two levels. First, this was necessary (to me) in the early days because the music services all had incomplete collections of their own, so my music could fill in the gaps. And second, the number of services that allow you to mix and match your private music collection with their cloud-based collections has dwindled over the years. Today, this need is less urgent, even for me, as the music services now all have massive collections that meet most needs. But there are now fewer choices than ever for those who wish to do this, and some are difficult to use.

If this does matter to you, the best choice today is YouTube Music, and for two reasons: it lets you upload 100,000 of your own songs for free, and it integrates with the YouTube video service, which has an incredible collection of unique music videos, including many live shows, that aren’t available anywhere else; you can add  YouTube videos to your YouTube Music collection and playlists and listen to them alongside other songs. Not surprisingly, this is what I use.

When I started this process, I wasn’t sure exactly what I had uploaded to Google Play Music/YouTube Music over the years or when—the “uploads” section in the web player doesn’t provide song or album counts, or provide much data of any kind, really—but I do know that it’s become less necessary to my daily listening. I have also created hundreds of playlists over the years, and almost all of it is comprised of cloud-hosted music from Google’s servers, I’m sure. This is what music is to me these days.

And there’s nothing wrong with that. This is, after all, the way that most music fans, casual or not, enjoy music these days, by streaming it from a paid or ad-supported streaming service of some kind to their phones, PCs, homes, cars, and elsewhere. Even an activity like downloading content locally to a phone so that it can be enjoyed offline—as when flying—is likely very uncommon now, as many, especially younger people, seem to just rely on always-available Internet connectivity. And yet it’s something I also do routinely: I would never fly anywhere with at least several downloaded playlists.

In any event, a few of my recent initiatives have collided in recent weeks, causing me to take another look at my old music collection on the NAS. The first was this year’s crazy spike in subscription services pricing, which triggered a lot of introspection and the perhaps overdue culling of those services we don’t use enough to justify the cost. The second was my digital decluttering push this past month and a half, which was so successful that I’ve expanded it past documents and photos and am now doing similar work with my home videos, music collection, videos collection, and more.

So what does all that mean? Why even bother with a local music collection when I’m doing everything in the cloud with YouTube Music?

There are reasons. First, the collection exists, and while I hadn’t looked at it in a while, it made sense to examine it again as part of my other digital decluttering work and see if I could cut some fat and save some storage space. Second, I suspected there was music in there we’d not listened to in a long time, and I wondered about adding some of that to our current playlists (and that was very much true, as I did just that). And I may not always want to pay for YouTube Music: ensuring that I have my full collection on the NAS as well as in YouTube Music means I will be able to access a large body of music from anywhere going forward, even if I don’t pay for a service. It’s a hedge, of sorts.

Plus, I didn’t think this process would be as difficult as my recent documents and photos archiving.

That previous decluttering work happened in a very linear way, and I was able to clean up and organize my documents (work and personal) and photo archives over five weeks recently and then ensure that they were duplicated in at least two places across cloud services and my NAS. But this new work, which includes more paper scanning, videos, and other content, is happening all at once, and so I’m now moving back and forth between various tasks on different PCs and laptops. In this article, of course, I’ll just focus on the music. Which I just finished archiving.

When I first examined my classic, NAS-based music collection the other day, it consisted of roughly 6,000 music files and took up 41 GB of disk space. But I also kept two parts of the full collection separate: instrumental music (which is mostly “new age” music but also lots of music from soundtracks), which took up 3.88 GB of disk space, and Christmas Music (3 GB of space). And so the total collection was about 48 GB, a more manageable size than my documents and photos archives, which were many times bigger. But the workflow was the same, and it started the same way: copy everything over the network to a laptop so I could work on the collection locally.

While that was happening, I examined the other folders in the Music share on the NAS. One was called “Google Play Music archive,” and with 10 seconds of research—l literally searched my own website—I was able to confirm that this is what I downloaded using Google Takeout on August 18, 2020 during the transition from Google Play Music to YouTube Music. It is the same 6173 files that I downloaded then, and it takes up 48.5 GB of disk space. These closely mirror the actual collection, so the two were likely nearly identical. But I copied that to the laptop too, so I could be sure.

Once I had all that music on the laptop, I got to work.

My music collection has consisted of three folders—Christmas Music, Instrumental, and Main, the latter of which contained everything that is not Christmas or instrumental music—and so the first task was to consider the organizational scheme. But this split has always made sense to me because Christmas and instrumental music are so different from the rock and pop music I usually listen to, and so I kept that as-is.

Next, I organized each of the folders. For the most part, I like to store music in the standard way—with top-level artist folders, each of which contains album folders, each of which contains songs—and my Main and Instrumental folders use this organizational scheme. Christmas does not, in part because there are only ~30 albums, and going through the process of making it consistent seemed too obsessive, even for me, in part because we only listen to this kind of music for one month a year and hadn’t accessed it in many years anyway. So I kept that in its original scheme, too, with top-level folder names in mostly in an “Artist name – Album name” format.

With documents and photos, I used WinDirStat to find the biggest files, but that’s not an issue with music. And so I instead decided to move the contents of Main into my user account’s Music folder on the laptop. Then, I opened the Windows 11 Media Player app—the modern replacement for what used to be Xbox Music and then Groove Music)—and configured it to only look for content in that one Music folder. And then I waited while it indexed the contents and displayed that part of my collection.

Media Player is not a great app, but it is a simple app, and it was semi-ideal for my needs here. Which was to simply find any music that did not belong in the main collection—for some reason, some Christmas and instrumental music had found its way in there—and to correct any obvious mistakes in individual files. So I scrolled through the entire (Main) collection several times, in the Albums view, and when I found a Christmas or instrumental album, I located the folder in File Explorer and moved it into the local Christmas Music or Instrumental folders (which were on my desktop), organizing them properly as I went.

Media Player also collected some improperly tagged files in an Unknown album entry (and a related Unknown artist entry in the Artists view). And so I took several minutes to identify each correctly, using File Explorer, and tag them correctly. In short order, there were no errant entries, thanks I’m sure to my compulsive music organizational work in the past. (This could have been a lot uglier.)

Then I had to deal with that Google Play Music archive. Ugh: another 6173 files, taking up 48.5 GB of disk space. Worse, it was unorganized: the Takeout dump was just that, with all 6173 of those files in a single folder.

This work was similar to what I had done previously with the documents and photos archives. That is, most of this mess was redundant, duplicating what was in the organized Music/Main, Christmas Music, and Instrumental folders. But I was betting that some of it was not, that if I carefully sifted through this mess of a folder, comparing it to the three organized folders, that I would find some music that was not in the latter.

And, of course, I did. And that meant that I couldn’t simply ignore the Google Play Music archive. I would need to step through all of it, comparing as I went, until I had deleted all the duplicates and moved the unique files into the local version of the collection.

This went slowly.

Part of the problem is that File Explorer doesn’t handle folders with several thousand files all that well. I would select big sections of files, type SHIFT + DEL to delete them permanently, confirm that decision, and then wait while File Explorer churned slowly through a task that should have taken 1 or 2 seconds. And it would often never return to normal, in that it would display deleted files until I exited and then reentered the folder. Which would cause File Explorer to ponderously load its content again, triggering another wait.

So I switched to Directory Opus, one of the heroes of my documents and photos archiving work, and used its side-by-side view to compare the Google Play Music archive with the Music/Main folder. (I’d bring up the much smaller Christmas Music and Instrumental in File Explorer as needed.) This worked better, but it was still slow going. I was down to about 3900 files and 31 GB after an afternoon of work. Surely there was a better—faster—way.

I decided to reorganize this massive folder of files into a more organized folder scheme that would mimic the way most music collections are organized (with folders of artists, each containing folders of albums, each containing some number of music files). And to automate that, I used Mp3tag, a popular audio file metadata utility. This was simple enough: I opened the folder in Mp3tag, selected all the files, chose Convert > Tag – Filename from the menu, and then entered %artist%\%album%\%title% as the format string. Bam! When I returned to File Explorer, my mess of 3900+ files was organized into just over 375 top-level artist folders. (I simplified this description a bit, as I had mistakenly added a filter on the first pass, but I got the whole folder organized in two passes.)

With this done, I used another hero from my documents and photos archive work, WinMerge, to compare the archive with the clean version of the (Main) collection in Music. They were obviously going to be very different, but I was looking for files in the “left” folder (the archive) that were not found in the “right” folder (Music/Main). And so I sorted the view by “Comparison result” and looked at the block of “Left only” entries first.

This let me do some pruning: there was some Christmas and instrumental music in there, so I deleted all that from within WinMerge, leaving me with 51 “Left only” folders, each representing an artist. Getting through that was easy enough, and I identified three or four artists that were not represented in the main collection in Music and moved them over. Then, I used WinMerge to delete the remaining “Left only” entries.

There was no need to deal with the “Right only” entries in WinMerge, as those represented things that were only in the main collection. But I did need to look at the “Folders are different” entries. And there were 189 of them (again, each representing the top of an artist folder structure). Ah boy.

For this, I moved back to File Explorer because the number of files wasn’t problematic, everything was local, and I wanted to compare folders side-by-side. This didn’t take too long, maybe an hour, and I moved another several albums into the main collection, which helped justify the time and work.

With this done, I evaluated what I had. The Main folder (in the Music folder) was now 34.1 GB, a bit smaller than the version on the NAS. And Christmas Music (4.3 GB) and Instrumental (7.2 GB) were both bigger. But the total for the music collection was a bit smaller at about 45.7 GB, probably due to me removing duplicates. But even better, the music collection was in a good place, as organized as it’s ever going to be. And it was time to wrap this up and move on to the next decluttering project.

Or was it?

While browsing through my tech feed earlier yesterday, I was suddenly reminded of iTunes Match. Some time ago, I had pushed my music collection into this $25-per-year service, which I treat as a sort of backup. And iTunes Match is interesting on a few levels. Key among them is that it compares your songs at upload time to the songs in its cloud-based collection, and if Apple already has any of the songs you own, it will just give you their version in nice, high-quality 256 Kbps AAC format instead of wasting storage space on duplicates. And that means you can download these high-quality song versions to your PCs and devices anytime you want.

This means it’s possible that iTunes Match was storing better-quality versions of some of the songs in my collection, making me wish I had thought of this earlier and perhaps started with the iTunes Match download as the basis for my new core music collection. But I had already done so much work, and most of my collection was already of reasonably high quality (for MP3/AAC files).

But it was worth checking.

First, how to download this collection? A bit of research reminded me that doing this on a PC would require me to install iTunes or Apple Music in Windows. But since these apps are terrible, I decided I’d just do this on my Mac, which already has Apple Music installed.

The tedious process I endured while repeatedly signing in, authorizing and deauthorizing devices, and failing for no obvious reason, all while being forced to manually type my full Apple password multiple times—only Apple could get away with this—is barely worth discussing. But I finally downloaded by iTunes Match archive on the Mac, all 49.5 GB worth, and copied it to a USB3 hard drive so I could move the files onto the laptop I’d been using for the rest of this process.

And … it wasn’t worth replacing any songs. I don’t think I made a single change.

The Apple archive was at least correctly organized (with folders of artists, each containing folders of albums, each containing some number of music files), which I appreciated. But it was also full of those superfluous additional “._” files that Macs always add: I was able to remove them in one whack with Directory Opus. After that, I compared the folders, but most of the filenames were different, which made automatic replacements of individual song files impossible or at least difficult. After a bit of work, I eventually just gave up: the collection is in good shape and there’s no need to overthink this any further.

With that matter settled, I connected to the NAS, moved the old music folders into a temporary “TO DELETE” folder that I will eventually remove, and uploaded the Christmas, Instrumental, and Main folders to the Music share on the NAS. Archive complete.

I will be writing soon about a related transition of my personal data from my Google Workspace account to my personal Gmail account, but related to this, I also started uploading the contents of Instrumental and Main to the YouTube Music service that is associated with my personal Gmail account.

I hadn’t used YouTube Music with this account before, and I’m not yet paying for YouTube Music Premium there, though doing so is part of the transition I’ll be making. Remember: YouTube Music lets anyone upload up to 100,000 songs for free. It’s not a horrible way to back up a music collection to the cloud and make it available from anywhere, even if you don’t subscribe.

More soon.