AI Generated Music from Audio Wave Data

Image of Generated Audio Wave

Image of Generated Audio Wave (source: Michael Wehar)


In this article we explore how AI can be used to analyze audio waves and automatically generate music.

My Question

I love music. While growing up, I took drum lessons throughout high school and college. I also make music on my computer using FL Studio and Virtual DJ.  Over the years, I have tried to learn a few songs on violin, but I have never had much success which has led me to the following question:

Can a computer play an instrument like a skilled musician?

In order to answer this question, we first need to explore the topic of computer audio.

Computer Audio

I am new to audio engineering and computer generated music, but from a few weeks of studying, I have learned a lot. Here are some important concepts that I learned about.

  1. A wave file (.wav) stores byte data that can be used to represent audio.
  2. In the PCM format, the byte data represents a sequence of signed numeric amplitude values.
  3. The positions in the sequence are associated with points in time and changes in the amplitude values lead to vibrations in your speakers.
  4. With a two byte per frame format (or 16-bit format), each numeric amplitude value is represented as a pair of bytes. This allows for amplitude values between -32768 and 32767.
  5. Using the little-endian order, the second byte contains the most significant bits and the first byte contains the least significant bits.
  6. Stereo has two audio channels (Ex: left & right ear) and mono has one audio channel.

I was able to apply this knowledge to programmatically read, write, play, copy, and blend audio data from multiple wave files.  I was also able to programmatically perform a few simple audio transformations.

Audio Transformations

Audio can be transformed by manipulating the associated sequence of amplitude values.

For example:

  1. To adjust volume, we multiply the amplitude values by a constant factor.
  2. To combine two sound clips, we sum together the amplitude values from their respective sequences.

Also Read: AI Generated Digital Painting from Start to Finish

My Solution for Music Generation

For my side-project, I made a computer AI that reads in a song and tries to recreate it using a given instrument.

If the instrument is a violin, then the program first reads in violin samples. Next, the program copies, transforms, and blends together these samples to recreate the original song.

You can watch an example of the recreation process using violin here.

Note: The generated violin plays towards the end of the video.

You can also listen to the generated audio here.

The recreation process is based on an intuitive idea. The violin samples are repeatedly played at differing intervals in many ways to create sounds that match segments of the original song.  The AI determines how to effectively carry out this process over a series of rounds.  As the number of rounds increases, the generated audio sounds more and more like music.

Existing Solutions

It turns out that there are existing software solutions that convert directly from Audio to MIDI.  This allows you to extract the melody from an audio file and transcribe it into a composition.  For future work, I hope to investigate various approaches for Audio to MIDI conversion.

Here are some helpful links that I found.

Final Remarks

Thank you very much for reading!  I gladly welcome any thoughts, feedback, or comments.

This was my first serious attempt at AI generated music.  In the future, I hope to learn more about existing technologies for music production and audio engineering to refine my approach for music generation.


I greatly appreciate all of the music artists who shared their songs and samples.


Carlos Vaquero –

Also Read: AI Search Prediction for Online Dictionaries

Michael Wehar

I am a passionate researcher, mentor, and teacher. I am always happy to make new connections and explore possible collaborations. Please feel welcome to contact me anytime.

52 comments on “AI Generated Music from Audio Wave Data

  1. Just wish to say your article is as astonishing.
    The clarity for your post is just nice and that i can assume you are knowledgeable on this subject.
    Fine along with your permission let me to snatch your feed to
    stay updated with impending post. Thank you a million and please keep
    up the gratifying work.

  2. Outstanding story there. What occurred after? Thanks!

  3. I was suggested this blog via my cousin. I’m now not positive whether this put up is written by way of him as no one
    else recognise such distinctive about my difficulty.
    You’re amazing! Thank you!

  4. Heya i’m for the first time here. I found this board and I find It really
    useful & it helped me out a lot. I hope to give something back and aid others like
    you helped me.

  5. Thankfulness to my father who told me regarding this weblog, this website is actually awesome.

  6. I got this site from my friend who shared with me about this website and now this time I am browsing this website and reading very informative content here.

  7. If you desire to grow your familiarity just keep visiting this website and be updated with the
    most up-to-date news posted here.

  8. It’s an amazing paragraph designed for all the web people; they will obtain advantage from it I am sure.

  9. Attractive section of content. I just stumbled
    upon your weblog and in accession capital to assert that I
    acquire actually enjoyed account your blog posts. Any way I’ll be subscribing to
    your feeds and even I achievement you access consistently fast.

  10. Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point.
    You definitely know what youre talking about, why throw
    away your intelligence on just posting videos to your weblog when you could be giving us something enlightening to read?

  11. What a material of un-ambiguity and preserveness of precious experience about unexpected feelings.

  12. I visited multiple web sites except the audio feature for audio songs existing at this website is truly fabulous.

  13. Heya i am for the first time here. I found this board and I to find It truly helpful &
    it helped me out much. I hope to offer one thing back and
    help others such as you helped me.

  14. I used to be able to find good advice from your blog posts.

  15. Wow, superb blog layout! How long have you been blogging for?
    you make blogging look easy. The overall
    look of your web site is great, let alone the content!

  16. In fact no matter if someone doesn’t understand after that its up
    to other viewers that they will assist, so here it happens.

  17. I just could not depart your website prior
    to suggesting that I actually enjoyed the usual info a
    person supply to your guests? Is going to be back incessantly in order to inspect
    new posts

  18. Very great post. I simply stumbled upon your blog and wished
    to mention that I have truly enjoyed browsing your
    blog posts. After all I will be subscribing to your feed and I’m hoping you write again soon!

  19. My partner and I absolutely love your blog and find nearly all of your post’s
    to be exactly what I’m looking for. Does
    one offer guest writers to write content for yourself?
    I wouldn’t mind creating a post or elaborating on a few of the subjects you write related to here.
    Again, awesome web site!

  20. Article writing is also a fun, if you be familiar with after that you can write if not it is complicated to write.

  21. Woah! I’m really loving the template/theme of this site.
    It’s simple, yet effective. A lot of times it’s difficult to
    get that “perfect balance” between superb usability and visual
    appearance. I must say that you’ve done a very good job with this.

    Also, the blog loads extremely quick for me on Internet
    explorer. Excellent Blog!

  22. Way cool! Some extremely valid points! I appreciate you penning
    this write-up and the rest of the website is also really good.

  23. It’s not my first time to pay a visit this site, i am browsing this site dailly and take fastidious facts from here daily.

  24. Hi there colleagues, how is the whole thing, and what you wish for to say on the topic of this piece
    of writing, in my view its really awesome in support of

  25. Saved as a favorite, I like your site!

  26. Hello there! Do you know if they make any
    plugins to assist with Search Engine Optimization? I’m trying to get my blog to rank for some targeted
    keywords but I’m not seeing very good gains. If you know
    of any please share. Thank you!

  27. Greetings from Ohio! I’m bored to tears at work so I decided
    to browse your website on my iphone during lunch break. I love the information you provide
    here and can’t wait to take a look when I get home. I’m shocked at how fast your blog loaded on my phone ..
    I’m not even using WIFI, just 3G .. Anyhow,
    fantastic blog!

  28. Hi there to every single one, it’s in fact a pleasant for me to pay a visit this website, it includes precious Information.

  29. I’m not sure where you are getting your information, but great topic.
    I needs to spend some time learning more or
    understanding more. Thanks for wonderful info I was looking for this info for my mission.

  30. First off I want to say great blog! I had a quick question which I’d like to ask if you don’t mind.
    I was interested to find out how you center yourself and clear your thoughts prior to writing.
    I’ve had a tough time clearing my thoughts in getting my ideas out there.
    I truly do take pleasure in writing but it just seems
    like the first 10 to 15 minutes are generally wasted just trying to figure out how to begin.
    Any suggestions or hints? Many thanks!

  31. Hello, i read your blog occasionally and i own a similar one and
    i was just curious if you get a lot of spam feedback?
    If so how do you stop it, any plugin or anything you
    can recommend? I get so much lately it’s driving me mad so any help is very much appreciated.

  32. I’ve learn several excellent stuff here. Certainly value bookmarking for revisiting.

    I surprise how a lot effort you place to make this type
    of great informative site.

  33. Wonderful goods from you, man. I have understand your stuff previous to and you are just too excellent.
    I actually like what you have acquired here, really like what you are saying and the way in which you say it.
    You make it entertaining and you still care for to keep it smart.
    I can not wait to read far more from you. This is
    really a great site.

  34. Thank you for every other great post. The place else may anybody get that kind of
    information in such a perfect way of writing? I’ve a presentation subsequent week, and I’m
    at the look for such information.

  35. Fantastic post however , I was wondering if you could write a litte more on this subject?
    I’d be very grateful if you could elaborate a little bit further.
    Many thanks!

  36. Woah! I’m really loving the template/theme of this blog.
    It’s simple, yet effective. A lot of times it’s difficult to get that “perfect balance” between superb usability and appearance.
    I must say that you’ve done a fantastic job with this. Also, the blog loads
    super quick for me on Opera. Excellent Blog!

  37. Simply desire to say your article is as astonishing.
    The clearness in your post is simply great and i could assume you are an expert on this subject.
    Fine with your permission let me to grab your RSS feed to keep up to
    date with forthcoming post. Thanks a million and please keep up
    the gratifying work.

  38. I do trust all of the ideas you have presented to your post.
    They’re very convincing and can definitely work. Still,
    the posts are too quick for beginners. Could you please prolong them a little from subsequent
    time? Thank you for the post.

  39. Appreciate this post. Will try it out.

  40. Hi, after reading this amazing post i am too delighted to share
    my knowledge here with mates.

  41. I am really inspired with your writing skills and also with
    the structure on your blog. Is that this a paid subject or did you
    customize it your self? Either way keep up the nice high
    quality writing, it is uncommon to look a nice blog like this one these days..

  42. I’m really enjoying the design and layout of your site. It’s a very
    easy on the eyes which makes it much more pleasant for me to come here
    and visit more often. Did you hire out a developer to create your theme?
    Excellent work!

  43. I’m really inspired with your writing abilities
    and also with the layout in your blog. Is this a
    paid subject matter or did you modify it your self? Either way keep up the nice high
    quality writing, it is uncommon to see a nice weblog
    like this one these days..

  44. Can I just say what a comfort to uncover someone who genuinely understands what they’re talking about on the web.

    You certainly know how to bring an issue to light and make it important.

    More people really need to look at this and understand this side of the
    story. I was surprised you’re not more popular since you definitely possess the gift.

  45. Thanks for one’s marvelous posting! I actually enjoyed reading it,
    you can be a great author. I will ensure that I bookmark your blog and will come back someday.
    I want to encourage continue your great posts, have a nice holiday weekend!

  46. You really make it seem so easy with your presentation but I find this matter to
    be really something that I think I would never understand.
    It seems too complex and extremely broad for me.
    I am looking forward for your next post, I’ll try to get the hang of it!

  47. Hello to every one, the contents present at this web site are actually awesome for
    people experience, well, keep up the good work fellows.

  48. Woah! I’m really enjoying the template/theme of this blog.
    It’s simple, yet effective. A lot of times
    it’s very hard to get that “perfect balance” between superb usability and appearance.
    I must say you have done a amazing job with this. Additionally, the blog loads super fast for me on Safari.
    Excellent Blog!

  49. It’s hard to come by knowledgeable people in this particular
    subject, but you sound like you know what you’re talking about!

  50. Its like you read my mind! You appear to know so much
    about this, like you wrote the book in it or something.
    I think that you could do with some pics to drive the message home
    a bit, but other than that, this is magnificent blog. An excellent read.
    I’ll definitely be back.

  51. This website was… how do you say it? Relevant!! Finally I’ve found something
    which helped me. Many thanks!

Leave a Reply

Your email address will not be published.