prepare audio data for a custom made dataset

Question

I want to create a custom made audio dataset. First I need to prepare my audio data, so that it is organised nicely. For testing and debugging my code I used a folder that contains 6 WAV files, called WAV_Folder.

I want to create 3 lists for all of these 6 WAV files. One list for the file's name, one list for the file's sample data and one list for the file's sample rate (by default 44,1 kHz).

This is the corresponding code:

import librosa
import sys, os, os.path
from os.path import isfile, join
from pathlib import Path
import glob 
import csv 
import wave

wavnames = [] # list with all of the audiofile's names 
wavsamples = [] # list of lists with all of the audiofile's sample values 
wavsamplerates = [] # list with all of the audiofile's samplerates (default: 44,1 kHz)
path = '/Users/abc/Desktop/WAV_Folder' # folder with all the data to put inside the dataset
pathlist = Path(path).glob('**/*.wav')

def sampled_audiofile(audiofile):
    list_audiosamples_for_one_file = []
    y,sr = librosa.load(audiofile,sr=44100)
    list_audiosamples_for_one_file.append(y)
    return list_audiosamples_for_one_file

for path in pathlist:
    wavnames += pathlist
    path_in_str = str(path)
    wavdata = sampled_audiofile(path_in_str)
    wavsamples += wavdata
    with wave.open(path_in_str, "rb") as wave_file:
        samplerate = []
        value = wave_file.getframerate()
        samplerate.append(value)
        wavsamplerates += samplerate 

If I then write

print(wavnames)

I get:

"/Users/abc/Desktop/Project Python Audio/dataset_creator.py" [PosixPath('/Users/abc/Desktop/WAV_Folder/WAV_NUMBER3.wav'), PosixPath('/Users/abc/Desktop/WAV_Folder/WAV_NUMBER2.wav'), PosixPath('/Users/abc/Desktop/WAV_Folder/WAV_NUMBER6.wav'), PosixPath('/Users/abc/Desktop/WAV_Folder/WAV_NUMBER5.wav'), PosixPath('/Users/abc/Desktop/WAV_Folder/WAV_NUMBER4.wav')]

Which is problematic, because the names are very long and furthermore there is always one element missing (in this case WAV_NUMBER1.wav) - How can I modify my code, so that

print(wavnames)

will result in:

WAV_NUMBER1
WAV_NUMBER2
WAV_NUMBER3
WAV_NUMBER4
WAV_NUMBER5
WAV_NUMBER6

The other two lists (wavsamples and wavsamplerates) somehow always overwrite themselves - If I run:

print(wavsamples)

I just get

[array([2.3782253e-05, 2.0563602e-05, 1.6212463e-05, ..., 0.0000000e+00, 0.0000000e+00, 0.0000000e+00], dtype=float32)]

So it's just displaying the samples of one single array of one audiofile rather than 6 arrays representing the 6 audio files.

I have the same issue for the third and last list. If I run:

print(wavsamplerates)

I get:

[44100]

It seems like I've mixed up my loops / iterations and the lists / arrays overwrite themselves along the way ... But as I'm a beginner when it comes to coding with Python I can't really decipher where I went wrong and why. Would be highly appreciated if someone can help me fix my three lists as I think the general approach of the code isn't too bad.

Answer 1

It turned out that I was overwriting my arrays because I did not integrate my .append() properly. See the code below for a proper solution to my original question from yesterday.

I also found a nice solution regarding the large file names problem. In my original code I have been printing the whole posixpath, but now it's just the file's names. Check the code below. Might be helpful for others, too.

import numpy as np 
import pandas as pd 
import librosa
import sys, os, os.path
from os.path import isfile, join
from pathlib import Path
import glob
import csv 
import wave

wavnames = [] # list with all of the audiofile's names 
wavsamples = [] # list of lists with all of the audiofile's sample values 
wavsamplerates = [] # list with all of the audiofile's samplerates (default: 44,1 kHz)
path = '/Users/abc/Desktop/WAV_Folder' # folder with all the data to put inside the dataset

def list_files(pfad):
    files = []
    for name in os.listdir(pfad):
        if os.path.isfile(os.path.join(pfad, name)):
            if name.endswith('.wav'):
                files.append(name)
    return files 

wavnames = list_files(path)

files = librosa.util.find_files(path, ext=['wav']) 
files = np.asarray(files)
for y in files: 
    wavsamples += librosa.load(y, sr = 44100)  
    ypathstring = str(y)
    with wave.open(ypathstring, "rb") as wave_file:
        samplerate = []
        value = wave_file.getframerate()
        samplerate.append(value)
    wavsamplerates += samplerate