Speaker 1: All right, good morning everyone. I'm excited to introduce our comprehensive research project on Automatic Music Transcription or AMD. Now this fascinating endeavor explores the intersection of signal processing and artificial intelligence aiming to convert acoustic music signals into music notation. Now the project involves various subtasks such as pitch estimation, onset and offset detection, instrument recognition, beat and rhythm tracking, interpretation of expressive timing and dynamics, and score typesetting. Now AMD's significance. AMD has garnered considerable attention due to its potential applications in music production, education, and analysis. It can revolutionize the way musicians create and share their compositions while also preserving musical heritage. However, it remains a complex task due to the inherent variability in music signals. Now our research objectives were to focus on AMD systems and enhance them by incorporating high-level musical knowledge. Now we will explore two distinct approaches for monophonic and polyphonic music. Non-negative matrix factorization for monophonic music and multi-level classification using neural networks for polyphonic music. Now the project scope. Our research project focuses on the critical question, can incorporating high-level musical knowledge improve the performance of AMD systems? Now the core concept of our investigation revolves around automatic music transcription and its application for both monophonic and polyphonic music. Now monophonic music scenarios involve transcribing a single monophonic music scenarios involve transcribing a single melodic line or instrument, which can be complex due to factors like expressiveness and dynamics. On the other hand, polyphonic music represents a more intricate challenge as it involves multiple instruments playing simultaneously, leading to overlapping notes and harmonies. The main goal of our project is clear, to convert the audio waveform from a musical recording into a mid-level representation. This representation encodes the critical information, what notes were played and when they were played. Now this intermediate step is vital in the process of music transcription. So the main goal of our project is certain from all of this, to convert audio waveform from musical recording into a mid-level representation. Now this representation encodes the critical information like what notes are played and when they were played. And now we'll be investigating AMD as applied to monophonic and polyphonic music using two approaches that are non-negative matrix factorization of spectrograms and multi-label classification using neural networks respectively. So the literature review. Now we have reviewed a lot of previous work in this field and some of the key papers we found were like automatic music transcription and overview by a professor. And now this paper talks about an overview of the challenges and techniques in automatic music transcription including pitch estimation, onset detection, and polyphonic transcription. Another valuable resource is automatic music transcription challenges and future directions. And now this paper highlights the limitations of current methods and proposes tailoring algorithms to specific use cases for improved performance and knowledge transfer. Transfer of knowledge among instruments is in automatic music transcription. Now this book explores the integration of information from multiple algorithms and different musical aspects suggesting that incorporating knowledge about different musical instruments may enhance transcriptions especially in polyphonic music. Now we go on to our datasets. And for our readers we are fortunate enough to have the music net dataset at our disposal. Now this dataset comprises of 330 classical music recordings with detailed annotations allowing us to train and test our AMD algorithms. The labels in this dataset are verified by trained musicians and are an invaluable resource for our project. And a lot of this data is actually open to the public domain and has been well-maintained and annotated for a long time. And thus is a good repository for us to use as a dataset. And now Shantanu would like to cover up approaches. So I'll pass on the mic to him. Thank you. Now I would like to continue on the approach we will be taking to take on the challenge of this

Speaker 2: automatic music transcription. So we will be using the non-negative matrix factorization which is a really powerful technique particularly in the realm of music transcription. It's science in scenarios where we deal with monophonic audio sources which means there's one predominant note or sound at a monophonic audio sources which means there's one predominant note or sound at a given time. So now we will dive into how NMF is applied in music transcription and focusing on monophonic elements. It's a crucial step towards understanding how we can accurately transcribe single note melodies or sound events in audio. So to start we transform the audio signal into a spectrogram and a spectrogram is essentially a visual representation of the audio signal's frequency content over time. It gives us a two-dimensional matrix that represents how the spectral components of the audio evolve as time progresses. This representation is fundamental for further analysis. Then NMF operates by factorizing the spectrogram matrix into two non-negative matrices. These matrices are the activation matrix and the dictionary matrix. The activation matrix which is also referred as the temporal matrix encodes how the spectral components evolve over time. It captures the temporal dynamics of the music which is crucial for understanding how notes or sound events change over the duration of the audio. Now the dictionary matrix which is also known as the frequency matrix stores spectral templates describing the frequency characteristics of individual notes or sound events. The magic happens when we combine these two matrices. The activation matrix provides the temporal context while the dictionary matrix holds the frequency information. After all after the processing of these two matrices we do the template matching process. Now we do with the activation. Now what do we do with the activation and dictionary matrices? We apply them for template matching. Template matching involves comparing the spectral templates stored in the dictionary matrix to the activation matrix. Essentially we are looking at how well these templates fit the audio signal at each time frame. This process identifies which sounds or notes events are present at each time frame. It's akin to piece together a musical jigsaw puzzle. The end result of this process is the transcription of monophonic elements in the audio. We have effectively decoded the individual notes or sound events capturing their timing and frequency characteristics. This approach is incredibly useful for transcribing single melodies solo instrument performance and any situation where one sound predominates at a given time. NFF for monophonic models is a robust and efficient technique that forms the foundation for more complex polyphonic transcription methods. Now there are various applications for automatic music transcription techniques such as in music production and in education and research. One exciting application of AMT automatic music transcription is in the realm of music production. It has the potential to revolutionize the music production industry by streamlining the creative process and it allows musicians and producers to quickly convert their audio ideas into noted forms making it easier to manipulate arrangements and access high quality virtual instruments. This fosters innovation and efficiency in music composition recording and sound design. In the field of education and academic research music transcription is a gateway to deeper understanding and exploration. It enhances music education by providing practical tools for learners. A student can learn more effectively and practice with precision. Moreover transcription facilities intricate analysis in the field of music research. It helps a scholar study various aspects of music from historical compositions to contemporary pieces. While the potential of AMT is immense it's not without its challenges. Let's take a look at some of the hurdles we face in this field. First harmonic overlap. Harmonic overlap is a common challenge. It involves efficiently identifying and distributing musical notes or sounds that occur simultaneously in a piece of music. This is crucial for enhancing transcription accuracy especially in polyphonic scenarios. Second is source separation. Source separation is another significant challenge. It involves separating individual instruments or vocals from a complex audio mixture. Accurate source separation aids in a more precise transcription of each component which is particularly important in complex music pieces. The large variety of music is a challenge. Our transcription system must be versatile and adaptable capable of handling diverse genres and styles. Music spans from classical to jag rock to electronic and more each with its unique characteristic. And now timbre and instrument tracking is essential too. This involves tracking changes in instrument or sound characteristics over time. It allows us to capture the unique tonal qualities of different instruments in the transcription. Now coming to the future scope of our research and how we plan to address these challenges. So for monophonic music we will implement non-negative matrix factorization. This technique involves factorizing of the input spectrum into an activation and dictionary matrix just as we previously discussed. It's a crucial step in deciphering single note melodies and monophonic elements in audio. For polyphonic music we will be using neural networks with a multi-label approach. This approach takes a constant Q-transform as input and outputs a vector indicating the probability of each note being active in that frame. This approach is essential for handling complex music scenarios where multiple notes and instruments overlap. In conclusion while we face challenges in the field of EMT our project is forward-looking. We are implementing robust techniques to address these challenges and pave the way for more accurate and efficient music transcription. Our research aims to unlock the potential of EMT across various music domains and make it a valuable resource for musicians, educators and researchers worldwide.