AudioKit MultiSegmentAudioPlayer with overlapping audio clips

70 Views Asked by At

I was looking for a way to play multiple audio files seamlessly one after another in my app project, and I found the perfect solution in the MultiSegmentAudioPlayer. However, I noticed some subtle breaks between the audio clips, so I adjusted the 'playbackStartTime' to overlap them:

func createSegments() {
    guard let audio1URL = Bundle.main.url(forResource: "audio1", withExtension: "mp3") else { return }
    guard let audio2URL = Bundle.main.url(forResource: "audio2", withExtension: "mp3") else { return }

    guard let segment1 = try? MockSegment(audioFileURL: audio1URL,
                                          playbackStartTime: 0.0,
                                          rmsFramesPerSecond: rmsFramesPerSecond) else { return }

    guard let segment2 = try? MockSegment(audioFileURL: audio2URL,
                                          playbackStartTime: segment1.playbackEndTime - 1,
                                          rmsFramesPerSecond: rmsFramesPerSecond) else { return }

    segments = [segment1, segment2]
}

Ideally, the overlapped clips should have cross-faded or mixed, but they just behaved oddly. I discovered that the MultiSegmentPlayer only plays one clip at a time, and I wanted to play both parts of the overlaying audio clips. After some modifications to the original package source as an experiment, I managed to achieve the desired functionality:

public func scheduleSegments(audioSegments: [StreamableAudioSegment],
                             referenceTimeStamp: TimeInterval = 0,
                             referenceNowTime: AVAudioTime? = nil,
                             processingDelay: TimeInterval = 0) {
    // will not schedule if the engine is not running or if the node is disconnected
    guard let lastRenderTime = playerNode.lastRenderTime else { return }

    for segment in audioSegments {
        let sampleTime = referenceNowTime ?? AVAudioTime.sampleTimeZero(sampleRate: lastRenderTime.sampleRate)

        // how long the file will be playing back for in seconds
        let durationToSchedule = segment.fileEndTime - segment.fileStartTime
        
        let endTimeWithRespectToReference = segment.playbackStartTime + durationToSchedule
        
        if endTimeWithRespectToReference <= referenceTimeStamp {
            // skip the clip if it's already past
            continue
        }

        // either play right away or schedule for a future time to begin playback
        var whenToPlay = sampleTime.offset(seconds: processingDelay)
            
        // the specific location in the audio file we will start playing from
        var fileStartTime = segment.fileStartTime
            
        if segment.playbackStartTime > referenceTimeStamp {
            // there's space before we should start playing
            let offsetSeconds = segment.playbackStartTime - referenceTimeStamp
            whenToPlay = whenToPlay.offset(seconds: offsetSeconds)
        } else {
            // adjust for playing somewhere in the middle of a segment
            fileStartTime = segment.fileStartTime + referenceTimeStamp - segment.playbackStartTime
        }

        // skip if invalid sample rate or fileStartTime (prevents crash)
        let sampleRate = segment.audioFile.fileFormat.sampleRate
        guard sampleRate.isFinite else { continue }
        guard fileStartTime.isFinite else { continue }

        let fileLengthInSamples = segment.audioFile.length
        let startFrame = AVAudioFramePosition(fileStartTime * sampleRate)
        let endFrame = AVAudioFramePosition(segment.fileEndTime * sampleRate)
        let totalFrames = (fileLengthInSamples - startFrame) - (fileLengthInSamples - endFrame)

        guard totalFrames > 0 else { continue } // skip if invalid number of frames (prevents crash)

        // Create a new AVAudioPlayerNode for each segment
        let playerNode = AVAudioPlayerNode()
        engine?.attach(playerNode)
        engine?.connect(playerNode, to: mixerNode, format: nil)

        // Schedule the segment on the new playerNode
        playerNode.scheduleSegment(segment.audioFile,
                                   startingFrame: startFrame,
                                   frameCount: AVAudioFrameCount(totalFrames),
                                   at: whenToPlay,
                                   completionHandler: segment.completionHandler)

        playerNode.prepare(withFrameCount: AVAudioFrameCount(totalFrames))
        
        // Start playing the new playerNode
        playerNode.play()
    }
}

(This is a part of MultiSegmentAudioPlayer.swift, which I couldn't have modified when added as a package normally)

It worked surprisingly well, and I'm thrilled with the result. However, I'm aware that using the locally imported framework prevents me from committing changes to my project.

Is there a way to allow both audio files to play when overlapped without modifying the original package?

Thank you for your assistance. I truly appreciate AudioKit's capabilities.

0

There are 0 best solutions below