Music

AI Generated Music from Audio Wave Data

Image of Generated Audio Wave

Image of Generated Audio Wave (source: Michael Wehar)

Introduction

In this article we explore how AI can be used to analyze audio waves and automatically generate music.

My Question

I love music. While growing up, I took drum lessons throughout high school and college. I also make music on my computer using FL Studio and Virtual DJ.  Over the years, I have tried to learn a few songs on violin, but I have never had much success which has led me to the following question:

Can a computer play an instrument like a skilled musician?

In order to answer this question, we first need to explore the topic of computer audio.

Computer Audio

I am new to audio engineering and computer generated music, but from a few weeks of studying, I have learned a lot. Here are some important concepts that I learned about.

  1. A wave file (.wav) stores byte data that can be used to represent audio.
  2. In the PCM format, the byte data represents a sequence of signed numeric amplitude values.
  3. The positions in the sequence are associated with points in time and changes in the amplitude values lead to vibrations in your speakers.
  4. With a two byte per frame format (or 16-bit format), each numeric amplitude value is represented as a pair of bytes. This allows for amplitude values between -32768 and 32767.
  5. Using the little-endian order, the second byte contains the most significant bits and the first byte contains the least significant bits.
  6. Stereo has two audio channels (Ex: left & right ear) and mono has one audio channel.

I was able to apply this knowledge to programmatically read, write, play, copy, and blend audio data from multiple wave files.  I was also able to programmatically perform a few simple audio transformations.

Audio Transformations

Audio can be transformed by manipulating the associated sequence of amplitude values.

For example:

  1. To adjust volume, we multiply the amplitude values by a constant factor.
  2. To combine two sound clips, we sum together the amplitude values from their respective sequences.

Also Read: AI Generated Digital Painting from Start to Finish

My Solution for Music Generation

For my side-project, I made a computer AI that reads in a song and tries to recreate it using a given instrument.

If the instrument is a violin, then the program first reads in violin samples. Next, the program copies, transforms, and blends together these samples to recreate the original song.

You can watch an example of the recreation process using violin here.

Note: The generated violin plays towards the end of the video.

You can also listen to the generated audio here.

The recreation process is based on an intuitive idea. The violin samples are repeatedly played at differing intervals in many ways to create sounds that match segments of the original song.  The AI determines how to effectively carry out this process over a series of rounds.  As the number of rounds increases, the generated audio sounds more and more like music.

Existing Solutions

It turns out that there are existing software solutions that convert directly from Audio to MIDI.  This allows you to extract the melody from an audio file and transcribe it into a composition.  For future work, I hope to investigate various approaches for Audio to MIDI conversion.

Here are some helpful links that I found.

http://akoff.com/audio-to-midi.html

https://www.upf.edu/web/mtg/melodia

http://www.justinsalamon.com/melody-extraction.html

Final Remarks

Thank you very much for reading!  I gladly welcome any thoughts, feedback, or comments.

This was my first serious attempt at AI generated music.  In the future, I hope to learn more about existing technologies for music production and audio engineering to refine my approach for music generation.

Credits

I greatly appreciate all of the music artists who shared their songs and samples.

DJBA – https://spot1techno.newgrounds.com

Carlos Vaquero – https://freesound.org/people/Carlos_Vaquero

Also Read: AI Search Prediction for Online Dictionaries

Michael Wehar

I am a passionate researcher, mentor, and teacher. I am always happy to make new connections and explore possible collaborations. Please feel welcome to contact me anytime.

8 comments on “AI Generated Music from Audio Wave Data

  1. I was suggested this blog via my cousin. I’m now not positive whether this put up is written by way of him as no one
    else recognise such distinctive about my difficulty.
    You’re amazing! Thank you!

  2. Wow, superb blog layout! How long have you been blogging for?
    you make blogging look easy. The overall
    look of your web site is great, let alone the content!

  3. I’m really enjoying the design and layout of your site. It’s a very
    easy on the eyes which makes it much more pleasant for me to come here
    and visit more often. Did you hire out a developer to create your theme?
    Excellent work!

  4. I’m really inspired with your writing abilities
    and also with the layout in your blog. Is this a
    paid subject matter or did you modify it your self? Either way keep up the nice high
    quality writing, it is uncommon to see a nice weblog
    like this one these days..

  5. […] Also Read: AI Generated Music from Audio Wave Data […]

  6. Simply wanna tell that this is invaluable , Thanks for taking your time to write this. Dave Marich

  7. Good way of describing, and good article to take data about my presentation subject, which i am going to convey in school. Xavier Yendell

Leave a Reply