how to interface esp32S3 with MEMS Microphone Breakout - SPH0645LM4H and stream the audio to server using http or UDP

Question

I found an example in esp-adf/examples/recorder/pipeline_raw_http. The example is intended for audio development boards such as the Krovo 2 board, and it uses the built-in analog microphone. I want to update this code for the ESP32-S3 and interface it with the MEMS Microphone Breakout - SPH0645LM4H. My question is, how can I modify the existing example?

I made same changes, first I comment out the codec part of code.

 //audio_board_handle_t board_handle = audio_board_init();
 //audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_ENCODE, AUDIO_HAL_CTRL_START);

Second I change the i2s configurations

    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();

    i2s_cfg.type = AUDIO_STREAM_READER;
    //i2s_cfg_reader.out_rb_size = 16 * 1024;
    i2s_cfg.i2s_config.mode = I2S_MODE_MASTER | I2S_MODE_RX | I2S_PDM_DSR_16S;
    i2s_cfg.i2s_config.sample_rate = 16000;
    i2s_cfg.i2s_config.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT;
    i2s_cfg.i2s_config.channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT; 
    i2s_cfg.i2s_config.communication_format = I2S_COMM_FORMAT_STAND_MSB;
    i2s_cfg.i2s_config.use_apll = false;
    i2s_cfg.out_rb_size = 16 * 1024; // Increase buffer to avoid missing data in bad network conditions
    i2s_cfg.i2s_config.dma_buf_count = 8,
    i2s_cfg.i2s_config.dma_buf_len = 200,
    i2s_cfg.i2s_port = I2S_NUM_0;
    i2s_cfg.i2s_config.fixed_mclk = 768000;
    i2s_stream_reader = i2s_stream_init(&i2s_cfg);

Third I select the AUDIO HAL as krovo-board 2 because it also have the same esp32-s3 chip therefore i checked the pin configuration of krovo-board 2 and found the following code

sp_err_t get_i2s_pins(i2s_port_t port, i2s_pin_config_t *i2s_config)
{
    AUDIO_NULL_CHECK(TAG, i2s_config, return ESP_FAIL);
    if (port == I2S_NUM_0) {
        i2s_config->bck_io_num = GPIO_NUM_9;
        i2s_config->ws_io_num = GPIO_NUM_45;
        i2s_config->data_out_num = GPIO_NUM_8;
        i2s_config->data_in_num = GPIO_NUM_10;
        i2s_config->mck_io_num = GPIO_NUM_16;
    } else if (port == I2S_NUM_1) {
        i2s_config->bck_io_num = -1;
        i2s_config->ws_io_num = -1;
        i2s_config->data_out_num = -1;
        i2s_config->data_in_num = -1;
        i2s_config->mck_io_num = -1;
    } else {
        memset(i2s_config, -1, sizeof(i2s_pin_config_t));
        ESP_LOGE(TAG, "i2s port %d is not supported", port);
        return ESP_FAIL;
    }

Therefore, i connect the GPIO_NUM_9 with BCLK, GPIO_NUM_45 with LRCLK and data pin with GPIO_NUM_10.

Code compiles properly with no errors, but unable to send the audio to the server. The data it sends to server is blank so can anybody help me out in this issue?

Answer 1

Here's my solution to make it send recorded audio to a TCP server. But this might not be exactly what you need as you need streaming. Instead of writing to PSRAM, you can try writing to the socket directly

https://gist.github.com/jayavanth/98873070da289e2a4df2971563061665

One thing you'll need to change there though is the slot config. Change it from I2S_TDM_PCM_LONG_SLOT_DEFAULT_CONFIG to I2S_TDM_PHILIPS_SLOT_DEFAULT_CONFIG

Make sure to use ESP-IDF v5.0 and above. I made it work with current master