Speaker 1: So, in this video we will talk about Kaldi and Kaldi from the beginning and we will also talk about how to install Kaldi and how to run a small demo speech recognition project in Kaldi. As I said we will talk about how to install Kaldi specially for Linux based environment and what are the Kaldi requirements, some there are there are some prepackages which should be installed in the Linux environment in order to install Kaldi and there are also some other requirements programming requirements such as batch programming, Perl scripting and Python and C plus plus, but most of the time we will be we will be using only the batch programming because the Kaldi core is built in C plus plus and there are also some wrappers for Python. So, for the time being we will focus on batch programming. So, we have to be prepared for an environment for Kaldi in Linux. So, there are some software components which needs to be installed in the Kaldi environment such as Arc which is a programming programming language and it is used for finding patterns and searching for the files. Another one is bash which is very familiar in Linux and it is used for scripting languages. Grip we will also use grip command which is a utility for searching and finding regular expression in the files and Mac is usually used for building binaries for Kaldi in Linux and Perl is also a programming language which is used for text processing. So, we will we should need these the software in order to install Kaldi and run Kaldi successfully. So, first of all how can we install Kaldi. So, the Kaldi repo is available here and I think it must be noted that we should also have get get install in Kaldi environment. So, first use the get clone command to download the Kaldi and then go to the Kaldi main directory and you will see some some directories such as tools source and eggs and extra. But if you if you get install in the Kaldi main directory you will you will see some helps how to install the tools the Kaldi tools in Kaldi. So, first you should get install in Kaldi in main Kaldi and you will see some install is a text file. So, you will see some commands which should be run in order to install the tools and the source directory. So, if we go to the tools directory and then get install we will also see some commands which is used for running the tools Kaldi tools and installation. So, first use the get tool install command and you will see commands which needs to be run and it must be noted that it takes some time to install Kaldi tools. And once it's done you will see a successful message in the Linux then go to the get source install command and run the commands one by one. So, it will take some while feel free to take some rest and watch the Kaldi installation. So, as I said Kaldi's have some directories which is a source directory where the Kaldi source code is available and some tools directory which contains some component and external libraries for running Kaldi successfully. And there is also one directory miscellaneous which contains different kinds of tools and other supporting and additional tools for running successfully for running Kaldi successfully. There comes x directory which is which is examples directory and it contains nearly almost 30 speech recognition recipe with the with the fully documented and bash scripts available. So, these are these are the directory structure of the Kaldi. So, next now we want to create our own recipe for Kaldi using a very small tiny data set. So, first create a create a project in the x directory and then in the project create an audio directory and then a data directory. Once the audio and data directory is created then create a train and test directory in the data directory. Once these are done then the next step is go to the train directory. In the train directory we have some Kaldi formatting style files which needs to be generated. So, the first one is wav.scp file which contains the utterance id, utterance id of the audio and the full path to the audio. An example is given here. So, here is the utterance id and the next one is the full path. It can be absolute, it can be relative path, but you can also use full path. So, the the first column is utterance id and the second column is the full path or relative path to the audio. Then the text file, the text file should contains the utterance id, utterance id is the same as in the wav file, wav file the first column. So, the the text file contains utterance id and the corresponding text transcription. So, as you can see it is the utterance id and this is the corresponding text transcription in the second column. So, we generated two files. Next is the utterance2.spk file. The utterance2.spk file is as the name as we can learn from the RAM. So, it contains the utterance with the correspondent speaker. So, it contains the utterance id and the corresponding speaker id. So, again it must be noted that the utterance id in the utterance2.spk file, the text file and the wav file should be same. So, in this utterance2.spk file, the first column is utterance id and the second column is the speaker id or speaker name. Then comes the speaker2.gender file. So, it is it contains the speaker id in the first column and in the second column it it should highlight whether the speaker is male or female. So, the speaker id and its corresponding gender. Then comes the next file is speaker2.utterance. So, the speaker2.utterance file contains in the first column the speaker id and in the second column utterance id. So, it must be noted that this file does not needs to be handcrafted because there is a tool in the utility and it automatically it automatically convert the utterance2.speaker to speaker file into speaker2.utterance files. We will we will talk about this, but it is not like it should not be manually created. So, these were for the training directory. So, in the training directory, we should have at least there are also many more files, but these are like the minimum requirements for running a speech recognition demo. So, we should have WAV file, the text file, the utterance2.id, speaker2.gender and the speaker2.utterance should be generated on the fly ok. So, the other important dictionary the other the other important directory is the dictionary directory. So, as you can see the first line, so create a directory in Kaldi with the inside the data local directory create a directory and in this directory, we should have the lexicon lexicon text file. Lexicon contains in the first column, it is the word the individual word and it corresponding phonemes. So, as you can see for example, for world 8, there are 2 phonemes a a t and similarly for 5, there are 3 phonemes. So, so in general lexicon contains words with corresponding phonemes ok. So, we have to create this file. Then comes the non-silence fonts file. So, this fonts this file contains the phonemes or fonts used in the lexicon as you can see here. So, these are the the right side fonts and these are non-silence fonts. Then comes the next part is a silence fonts which contains SIL which is used for silence and SPN which is used for spoken noise. So, these fonts must be there in the silence fonts file and the optional silence fonts contains silence fonts. So, these 2 files are pretty static, but the the main components are lexicon and non-silence fonts which contains the which contains the words with corresponding fonts and then the non-silence fonts in the in this file ok. So, this is a little about lexicon lexicon direction lexicon directory . So, in the dictionary we we have to have 4 files for example, lexicon non-silence fonts silence fonts and optional silence fonts ok. So, once this is prepared then comes the language model part and the language model the language is actually the text the text used in this the it is the text of the training data. So, as we have seen in the in the first part I will show you this is the text file. So, the text file contains the utterance id and the text transcription. So, in the corpus we will have only the text transcription. So, we will have to copy or we need the the we will remove the first column from the text and the remaining are the corpus. So, the corpus contains the text transcription in the training data for example, 1 2 3 1 2 5 6 8 3 4 4 2 and so on . This ok so, we will talk about language part later. Then comes the true file which is quite standard. So, if we want to run our Kaldi project on local machine then we use run.pl in the train and decode command and if we want to use Kaldi in the cluster machine on a server then we have to use some q.pl command which is not important here, but keep here as simple as possible to use run.pl command and then comes the path directory. So, these are some of the paths relative paths we are called the searches for, but if you run the Kaldi in the local machine then we do not need to change these ok. Then comes the run script which is the main script for running the Kaldi experiments and in the run scripts the first three lines are very static they are used for for Kaldi settings and then comes the number of jobs which in our case we we keep it as 1 and the language model order we we keep it 1. So, it means unigram language model. Then then the next commands are not important, but let us comes to ok. So, let us comes to this command the last two commands which is as I mentioned these are used for generating speaker to atoms file. So, we can generate this this file using the following utils command in the training directory and in the test directory. Then the this command is also validate command is also important for different kinds of fixing different kinds of sorting or other kinds of errors in the training or in the test directory beforehand. So, if we use these commands they will tell you whether our our data preparation process is good or not good, but this is a very good step to use because it will indicate an error if there is some bugs in our data preparation process and catching this before is very important ok. So, once the training directory and the test directory is prepared next is the time for making MFCC features for the speech recognition. So, as we know that MFCC are very well known features for extracting speech for for speech recognition and speaker recognition and many more speech recognition work. So, this command is used the make MFCC command is used to to extract features from the training data and the test data and then the compute CMV instead is used to compute the capstone mean normalization. So, it it normalize the normalize the training data. Then comes the dictionary directory which contains lexicon, non-silence forms, silence forms and optional silence forms. So, these as I mentioned these are part of the lexicon directory or dictionary directory and we have prepared this beforehand. So, once these are available we have to create a link directory using the following prepare link command. So, data slash local slash dict is already we have generated which contains the above four files and unk is an unknown phone which is used for unknown phones and it will generate data sling slash link directory ok. So, next is we want to build the language model part and the language model part is we have already the corpus right. So, for the language model this ngram count a command is used to compute the ngram of the language model. So, for example, we keep the order as it is a very simple case. So, we keep the order as one and we we use the data slash local corpus transcript in the as a input and the output is a ARFA formatted language model ok. And then this language model is converted to the FST format using ARFA to FST command as shown in the last as shown the last command shows ARFA to FST. So, this language model is converted to the FST format ok. So, once we are done the previous steps then the next step is to train a speech recognition using the train mono command which is a which is a which is a simple command and this command is used for training a hmm base speech recognition. So, this is the very simple this is very this command is for a very simple speech recognition and using some statistical methods such as hmm and Gaussian mixture model. So, once this model is generated then we use the then we make a computation graph using the using the model and the link and and and build a computation graph. And then test data is used to decode the computation graph and compute the world error rate which is used for evaluating the performance of the speech recognition model. So, the last command will will show you will come will decode the computation graph using the test data and we can evaluate our speech recognition model. So, as we know that using the above commands they generate different kinds of different kinds of directory. So, these are also important because because running every command will generate some directories and it should generate. So, the the the final the x directory which is an experiment directory they contain there should contain some mono folder and graph folder inside mono and then the decode directory inside mono and the feature directory is contains computing the features of the and some extra. So these are some directories generated and these these are some helpful materials for for beginning and studying Kaldi okay. So if we have to summarize so in summary we should have the training directory and also the test directory. The training directory and the test directory is the same except they have different kind of data. So in the training directory we should have wav file, text file, utterance to a speaker, speaker to gender and speaker to utterance. And in the dictionary directory we should have lexicon, non-silence fonts, silence fonts, optional silence font. And in the length directory we should have lexicon FST which contains the input is a input is fonts and the output is words. And the grammar which contains the input is the grammar and the output is the grammar. And the exp directory which contains different kinds of experiments with the Kaldi. So I hope you have learned a little bit about Kaldi and it is worth mentioning that all credit goes to this article which is written for Kaldi for dummies. So thank you very much. We will in the next video we will we will give a short demo how to run a Kaldi using custom data in the from the beginning okay thank you.