Your music cataloging "system"

[jcam_44]

2 hours ago, John L said:

I just visited the Collectorz online service.  Wow, does it suck!    It is just a list of of albums and songs by artist like iTunes.   All of the useful information from my database - musicians on albums, listening notes, etc. - vanished when I supposedly transferred by database there.   Now I am stuck in a dilemma and stand to lose what I have built.  

I just got a generic response from them when I inquired about comparability. Really disappointed. They didn’t even answer my question. The lack of full transfer to the cloud from the desktop platform is cause for me to stop using them. If the cloud platform was the same as the desktop I would stick with them. 

[JSngry]

23 hours ago, jcam_44 said:

I just want to input on my computer and transfer it to the mobile app.

The only way to really do this is to make your own file on your own database, save, and then back it up on a cloud drive.

Anything else and you will be, as you are discovering, at the mercy of a 3rd party, which is really convenient, until its not.

I feel your pain, but with data, you can step up to the table and roll the dice until you lose, or keep your cash in your pocket. And even then...

[John L]

1 hour ago, JSngry said:

The only way to really do this is to make your own file on your own database, save, and then back it up on a cloud drive.

Anything else and you will be, as you are discovering, at the mercy of a 3rd party, which is really convenient, until its not.

I feel your pain, but with data, you can step up to the table and roll the dice until you lose, or keep your cash in your pocket. And even then...

The question is what program to use for the database.   If you write your own computer programs, that might work.  I don't.   So we download special software.  But a third party needs to ensure that it remains compatible with new computer operating systems.  That is the problem.   There are programs like Excel that you can be pretty sure will always allow you to upgrade your files to a new operating system.  But Excel is cumbersome for a full music database.    

[jcam_44]

1 hour ago, JSngry said:

The only way to really do this is to make your own file on your own database, save, and then back it up on a cloud drive.

Anything else and you will be, as you are discovering, at the mercy of a 3rd party, which is really convenient, until its not.

I feel your pain, but with data, you can step up to the table and roll the dice until you lose, or keep your cash in your pocket. And even then...

Yeah and if i teach myself to drill for oil and refine it into gas and I can stop going to the pump. Thanks for the insight

[JSngry]

Figuring out Excel is easier than drilling for oil. And you don have to change clothes.

[jcam_44]

I’m well versed in Excel. That is not what I want in a database. 

[JSngry]

Right, Excel is not really a database.

But my point is more to not being dependent on a 3rd party service to store your data for you. As a backup, yeah. But just giving it to them....this is what happens.

Mike Weil talks about his database that he uses for his discographies. imo that's the type of thing to use to keep your data under your roof and not in somebody's cloud.

[BeBop]

Are you ready?  My collection, about 10,000 items, is cataloged on 3x5 cards.  Every person appearing on any recording (leader, sideman) has a card.  On that card, I've listed every recording that that individual plays on.  I've color-coded things: LPs are written in black; 78s in green, 10" and other less-common (in my collection) formats in red; CDs in blue (just so I know what part of the house to search).

[John L]

2 hours ago, BeBop said:

Are you ready?  My collection, about 10,000 items, is cataloged on 3x5 cards.  Every person appearing on any recording (leader, sideman) has a card.  On that card, I've listed every recording that that individual plays on.  I've color-coded things: LPs are written in black; 78s in green, 10" and other less-common (in my collection) formats in red; CDs in blue (just so I know what part of the house to search).

Wow.  Impressive.  

[jcam_44]

3 hours ago, BeBop said:

Are you ready?  My collection, about 10,000 items, is cataloged on 3x5 cards.  Every person appearing on any recording (leader, sideman) has a card.  On that card, I've listed every recording that that individual plays on.  I've color-coded things: LPs are written in black; 78s in green, 10" and other less-common (in my collection) formats in red; CDs in blue (just so I know what part of the house to search).

That’s insanely impressive. I could see doing that if I started from the beginning of collecting. I liked collectorz because I could add studio/venue, musicians and it was searchable. I also added the from and rear images to releases. The search for a new database has begun. 

[Teasing the Korean]

I don't catalog my LPs.  I simply arrange them alphabetically by artist within my own idiosyncratically defined genres.

I back up my CDs using XLD, and, in theory at least, they are all backed up.  I file these in folders arranged by my own idiosyncratically defined genres, but I don't change the tags.  So if a space-age bachelor pad record shows up as "jazz," I leave it that way. 

[Daniel A]

I am almost done cataloging my LPs on Discogs. It has the functionality I need. When I started buying CDs in the late 80s I created physical cards for every item. Then I switched to a paper based list, just to document where and when I purchased them and how much I paid. 20 years ago, I switched to Excel, though I might eventually go all Discogs. I rarely buy CDs these days. And (luckily) I never kept track on how much I have spent on LPs! 

[Stonewall15]

22 hours ago, BeBop said:

Are you ready?  My collection, about 10,000 items, is cataloged on 3x5 cards.  Every person appearing on any recording (leader, sideman) has a card.  On that card, I've listed every recording that that individual plays on.  I've color-coded things: LPs are written in black; 78s in green, 10" and other less-common (in my collection) formats in red; CDs in blue (just so I know what part of the house to search).

How do you search for a label? or for a song?

[BeBop]

1 hour ago, Stonewall15 said:

How do you search for a label? or for a song?

I go to the internet.  Yeah, there are some limits to the 3x5 cards.  Frankly, it isn't that often that I (just speaking for myself) am looking to hear every version of "all the Things You Are" that I have.  If I'm looking for, say, James Moody's "Darben the Redd Foxx", I have a good first place to look, and I can go to the interwebs for others. 

I did, basically, start with the 3x5 cards on my first record purchase back in the 60s.  Funny to look back: first LP was "Way Out Wardell", a Wardell Gray album with almost completely wrong discographical notes on the cover.

[porcy62]

I loaded all my items on discogs and I am fine with it. Anyway I haven't got a huge collection.

[hopkins]

I started this thread as I was curious to find out how some of you manage your music collection. Most of you probably do not see the need for a "computer based system" to organize and document your music collection, as you are so familiar with its content. Documentation is tedious, and the benefits may not be obvious.

Truth be told, I have been working on my own music collection "system" for roughly 5 years now, and would like to explain here what I have done. I did mention my work on an "audiophile" forum, but got no feedback, and was hesitant to share it here.

This is NOT A COMMERCIAL project, and it never will be. I build this system for my own use, and have zero interest in turning it into a commercial project.

The system I build was inspired by someone else's work, documented here (there are several aspects to his project, only the section on "metadata" is relevant here): https://medium.com/p/5ac042c40a8e. The developer of this application (who I have exchanged with at length) has build an incredibly sophisticated (yet simple to use) solution, which is far more ambitious than mine. He was unable to attract investors, abandoned the project for some years, but recently started working on it again. 

Here are some of the reasons I have set up this system in the first place:

- I developed a passion for Jazz at a relatively late stage (around the age of 40 - am now 56). I do not have all this music imprinted in my brain as most of you do (!),  and I soon found out that I needed to have some sort of system to find music and keep track of useful information ("best tracks", who plays on what, etc...). 

- 99% of my music is now stored on a computer, and I do not have access to the "physical copies". I do not like to read the tiny print on CD booklets, or even struggle with finding them and getting them out of the jewel boxes. I rip the CDs that I purchase, then store them, or sell some of them back (with the exception of Mosaic box sets!). I do not have the space to keep a large physical media collection. I also purchase music online (and will admit to sharing between friends as well..). I do have some LPs, but only for those albums which are not available in digital format at all. I have digitalized a few LPs. I do subscribe to a streaming service as well, but I prefer having my own files to play (with), for a number of reasons I do not need to get into here. The point is: digital music is very convenient, but you lose access to all the related content (line notes, credits..). I wanted to have the best of both worlds!

I had limited programming experience, and basically taught myself web programming to do all this.

So what is the point of sharing all this here ?

- I believe some of the content could be of interest to others. None of the material is "original", however, and most of it is available online: discographies (a big thanks to those of you who contribute to these), the internet archive, books, etc... Some of it is probably copyrighted. I have included in my album notes a lot of commentary from Jan Evensmo's solographies, and asked him if he was ok for me to do so. I assume that liner notes available on the Internet Archive can also be used. But I have also included lengthy extracts of books (ex: Eddie Lambert's "Duke Ellington - A Listener's Guide"), which are copyrighted, and also Mosaic liner notes, for example - is that an issue ? I don't know. 

- I would also like to benefit from content that other people may have, and that I am missing, but I do not know how this content could be shared, and on what "platform".

I have put a large part of my collection online: https://paulstephane.github.io/collection

This is a rough version with limited functionality (the online search, for example, does not work - yet). The albums are presented by recording year, but otherwise without any specific order. 

The music tracks cannot be played from this online version. I could store the tracks online as well, and offer the ability to play samples, but I hesitate to do so essentially because it would be costly, and because I believe samples have limited interest. I have started including links to the albums on the Internet Archives, as they offer samples or links to streaming services. YouTube is a great source of music, and I could link some of my albums to the YouTube version, but that is a lot of work. I do realize that being able to play the music online would probably be the most interesting aspect. I spend many hours myself finding these albums. Copyrights for music extends 100 years - all this will be freely available to future generations, but not for the time being.

At home, I have my own complete version (on a home network web server) - I will describe in my blog some of the technical aspects. In summary, I put all my albums on my computer, have a program that scans them and adds web pages for each album (and saves the album art online as well).  I can then edit the albums online (through the same we pages that are included above). The tracks, which are added automatically when the album is scanned, can be easily reorganized by session if needed.

In my home "version", I have the ability to play the tracks (whole album, or selected tracks from the album) to my stereo or desktop computer speakers, from the web pages. This is very convenient.

As of now, all the information is presented by "album". The point of the system is NOT to build "discographies" (BRIAN, for example, is a great tool for that). The purpose is to have accurate credits and recording dates (as much as possible) associated with the albums, as well as liner notes and other interesting comments.

Having the data entered by album does have one obvious limitation: duplicates of sessions across albums, that require copying the same data from album to album. I try not to have too many duplicates, but some are unavoidable (ex: Teddy Wilson sessions, included without Billie Holiday in the Mosaic box set, and only with the Billy Holiday vocal tracks on the Lady Day compilation, etc...).

The session information is entered, for each album, in a simple text editor (either in the web pages, or using another text editor), and formatted for display using the "markdown" syntax, which is very simple to use.

Here is the web page for Duke Ellington's "Side by Side":

https://paulstephane.github.io/album#1567008896998

And the corresponding text file:

https://storage.googleapis.com/cloudplayer/markdown/1567008896998.md

Because the data is entered in plain text (with the exception of the additional markdown syntax), it can easily be shared.

When the page is saved, the system interprets the data and indexes all the credits/dates/tracks which can then be used to search my collection. 

After reading Jan Evensmo's last solography on Clifford Brown, I spend a few days reorganizing and completing my collection of his albums, including Jan's comments, liner notes, etc... Here is the result of a search for "Clifford Brown" organized by recording year:

If I search for two musicians (ex: Clifford Brown and Harold Land) it will display those albums which have recording sessions with both musicians. This is quite useful, especially when trying to find whether I already own a given session.

I added another "view" to display search results not by album but by session. I find that I rarely use it.

Entering the information for an album can take anytime from a few minutes to half a day in the case of large box sets. It takes time, but I do it gradually and will often add information to existing albums when I find other interesting information. The time I spend working on this is also an opportunity to make time to listen to the music.

Because I have developed all this myself, I tried not to be too ambitious and only include information which I thought would be most useful. I regret not entering labels (Verve, Prestige, etc...) and am considering adding this. I am also considering adding artist pages, links to booklets in PDF format.

So if you can spare a few minutes of your time, please have a look at the online version, and let me know if you have any comments and suggestions!

Thanks,

Stephane (from Paris)

 

 

 

 

 

 

 

 

Edited March 8, 2023 by hopkins

[mikeweil]

On 2/1/2022 at 3:06 PM, JSngry said:

Mike Weil talks about his database that he uses for his discographies. imo that's the type of thing to use to keep your data under your roof and not in somebody's cloud.

BRIAN is very elaborate. For just cataloging your collection, Steve Albin offers Beyond Category, a simpler database.

[hopkins]

Hi,

Before embarking on my own project I did take a look at a few existing solutions, including Beyond Category. There is a similar program called Collectorz (https://www.collectorz.com/music/music-collection).These are nice solutions, but I wanted something that was more flexible, simpler to use, that provided visually appealing results, and that I could use to play my music files as well.

All these aforementioned programs require that data be entered in a set of "database fields". What I developed does not use "fields" but instead lets you edit all the information as free, (practically) unstructured text. What does this mean ?

- any type of content can be added - plain text, links, images, videos... - and I am not limited to a set number of fields.

- the tracks can be grouped by "session" (by simple copy/paste) and all the information (credits, comments) can be entered by "session" (without repeating the same information track by track)

- the text can be formatted using the markdown syntax (https://en.wikipedia.org/wiki/Markdown) which is very simple to use

Once the data is entered and saved, the text content is analyzed to extract specific information that is then stored in a database, but all this happens "behind the scenes".

I do use music file tags to edit basic information about the album - before "importing" it in my system. But when you rip a CD, or purchase music files, most of this basic data is already available in the tracks. I simply review it and edit it if required, using one of the many available music tagging software (I use this one, there are plenty others - https://www.mp3tag.de/en/). The only data that I use from these file tags are: album title, album artist(s) (basically whoever appears on the cover of the album), release year, genre, track title and duration, track composer(s) (when available).

All the other information, I fill in after the album has been added to my web pages. For each album, all the added information is kept in plain text format so that it can be easily edited, copy/pasted. For each album, the information is stored automatically online, and also stored automatically in a text file in each album's "folder" on my hard drive. 

I will make a short video of the editing process.

The limitation of my approach (as opposed to a database style entry), is that the information entered is not "checked" against list of allowed values. For example, I do not have a pre-defined "list of artists" or "list of instruments", etc...

Lists are useful to ensure consistency, but they are tedious to maintain, and I have chosen to do without them. How do I avoid inconsistencies across all my albums (ex: is it "Harold Baker", or "Shorty Baker", or "Harold Shorty Baker") ? For the time being, when I search for albums using artist names, I have an autofill feature that lists artists found in my album notes, and I can check if the list contains multiple syntaxes for the same artist. This is sufficient for me.

Anyway, I am not trying to "sell" my solution! I was curious to see if the information I have entered could be of any use to others, and if so how some of it could be shared. 

I am also curious to know if having the information available online to others raises any copyright issues.

I have spend a lot of time developing all this, and I like using it. The system is unique, and I think it is pretty cool  I have tried to keep things as simple as I could. There are some obvious limitations to my approach, but it works well for me, and I plan on using it to continue building up my collection in the (hopefully many) years to come.

 

 

 

 

Edited March 8, 2023 by hopkins

[mikeweil]

Can you search for sideman appearances in your system?

Edited March 8, 2023 by mikeweil

[hopkins]

36 minutes ago, mikeweil said:

Can you search for sideman appearances in your system?

Yes, that was one of my main objectives. All the credits entered for each session are parsed and indexed. In the example above - Side by Side - searching for Ben Webster will list the album, but searching for Ben Webster and Duke Ellington won’t because they are not playing together on either session of the album. 

One thing i did not do was to identify 'leader' roles. The leader is generally identified as text in the session ’header’, but i cannot specifically search for albums where a given artist is a leader. I do have  the album artist tag from the music files, but that does noy always include the leader, and there could be multiple sessions with multiple leaders on any given album.

The concept of ’leadership’ is not always clear and significant - often,as you know, leadership is only assigned for contractual reasons. But it is sometimes useful... 

 

 

Edited March 8, 2023 by hopkins

[mikeweil]

In BRIAN you can specify as many leaders as you want in a session, or noone, as in the case of all star sessions.

Edited March 8, 2023 by mikeweil

[hopkins]

6 minutes ago, mikeweil said:

In BRIAN you can specify as many leaders as you want in a session, or noone, as in the case of all star sessions.

BRIAN is the best discography tool available, IMO. In an ideal world, BRIAN discographies would be available in an online shared database, and my application would link to it to collect discographical information associated to my albums. I had in fact briefly discussed this with them, but got deeply involved in my project and never followed through (ashamed to say). 

Jazz enthusiasts, collectors, researchers, are terribly handicapped, in my opinion, by the lack of a common, curated, repository for session information. There are multiple sources. Tom Lord's discography is unreliable, paying, and there is no curating process to my knowledge. 

So I plowed ahead with my own solution, but would be more than happy to participate in discussions of other options... 

[hopkins]

Columbia's J-Disc was also an interesting initiative: https://www.jazzstudiesonline.org/content/j-disc-online-jazz-discography

I  contacted them a few years ago but never got an answer back. The link to their online discography is now dead... 

[hopkins]

Removed

Edited March 11, 2023 by hopkins

[hopkins]

Here is an ”artist” page presenting annotated albums: https://paulstephane.github.io/2023/03/11/Clifford-Brown.html

I’m not sure this is of any interest to others (aside for my own, to keep track of some references). I’ll see whether I continue with some additional examples. The purpose is not to have full discographies. There are some YouTube links in the album pages, but i’m not sure how persistent these will be.

Edited March 11, 2023 by hopkins