Music AI and how it can lead us to better understanding ourselves

I love listening to music in the background when I work, whether that be creativity sketching, coding, making a watches, keyboards or just building ikea furniture. I'm even learning to play music to help me better understand the technology we have but mainly cause it's fun.

This morning I wanted to bang on some tunes while I was working on something that I’m excited to demo to a group of underserved individuals.

So I paused and wondered how I would really like it to be.

I just want to press play, because if I was visually impaired, that’s the fastest thing I’d want to do. Still focussing on my impairment core principle in my AI work. To help me understand what would an AI ask to satisfy a need. But the way in questions can be asked are restricted or limited. Almost like a way to design an AI QA script.

I've been asking "Hey siri" play some music for quite a while, it's recommenedations to start were good but then realized they are the same all the time, just different order. Perhaps I would start with, sure how are you feeling? (why? So Do I play sad or happy music). How much energy you have? (why? fast vs slow) Have you eaten or drank anything? (why? I have yet to have my coffee this morning, only brushed my teeth X (2nd tier of How much energy do you have)). These inputs can then be used to query the best result, just like search indexes would.

Side note: I wonder sometimes if AI is just search and we need to refer to product research there. I was fortunate to concept, ideate and build a media search index service for the financial compliance industry (Complinet 2006). Using a trained named entity extractor using a Java stanford library, trained on global name lists I could find. Then we created a td-idf map using java lucene, with a vector distance weighting on the results. Allowing us to perform searches on the names but specifically for a question. Was that individual named alongside another that is sanctioned, or heck could've been used for much more than just that. Celebrity mentions, I wonder if the tech went on to be used in that way.

Back to Music. With that search input (AI questions), we can then compare that with what you had learnt about the individual, which to be fair isn’t much in the music world. We only store playlists, hit a heart button on a song, but never are storing any rich information about when you listened to that song.

Let me explain.

To better recommend music, I believe we need to capture a lot more context against the types of music we’re listening to. LastFM was great to start this movement, but we need to more today because of the possibilities of compute (I'm not saying AI modelling because it might not be the right fit to certain problems).

Let's just pick one data variable from the many in musicology, rhythm.

We could better capture rhythm based on how you’re typing on the keyboard or moving between different devices. Perhaps the pressure that you're hitting the mechanical keys. Specifically for my audience of impaired individuals, what would their tone of voice be, was it slow, shaky, sad/happy. That could then be better taken in to suggest the music. Or even just the query itself for other assistant types of features.

My example.
I woke this morning at 6:45 naturally (no alarm), felt like going for a run, have a bounce in my step almost for some reason. What music can enhance that energy and sustain it for me.

Why is this important? Because if we can better understand how we assist and enhance what a 'unique' individual is attempting to do. Then as designers we can build a better future for all, not one that is restricted by a pre-determined design or restriction. Ones today being the phones, operating sytems, and devices we use.

With rhythm, we do this already today in gym training. Bpm running or cycling. But I can't ever seem to find the right bpm for the pace I train with. Just playlists.

There used to a be a great music recommendation service that categorized music by mood called Songza, loved it for evening parties. I think Google acquired them. I would judge the audience of the room, the body language they'd be giving off, and as the DJ, pick the track they'd want or queue it up. Music should be categorized this way. I'm starting to design how to structure my music samples/loops in this way.

The reason I’m writing this up, is because I don’t think I can prototype this one easily because the data is not open. And I think music is going to be the hardest category to innovate AI on because of this. Copyright issues, as I witnessed with my twitch stream on UX usability restriction.

And as of now, I’m still not playing any music in the background while I’m doing work and just realized I’m hungry now, hmm oats. But first run. And oh I chose a song, don't change, Coldplay - Higher Power.