快乐炸金花无限注册教,qq德州扑克apk,少年宫棋牌室计划(中国)·官方网站

點擊藍字 ╳ 關注我們

開源項目 OpenHarmony

是每個人的 OpenHarmony

巴延興

深圳開鴻數字產業發展有限公司

資深OS框架開發工程師

一、簡介

Audio是多媒體子系統中的一個重要模塊，其涉及的內容比較多，有音頻的渲染、音頻的采集、音頻的策略管理等。本文主要針對音頻渲染功能進行詳細地分析，并通過源碼中提供的例子，對音頻渲染進行流程的梳理。

二、目錄

foundation/multimedia/audio_framework

audio_framework
├── frameworks
│ ├── js                          #js 接口
│ │ └── napi
│ │     └── audio_renderer      #audio_renderer NAPI接口
│ │       ├── include
│ │       │ ├── audio_renderer_callback_napi.h
│ │       │ ├── renderer_data_request_callback_napi.h
│ │       │ ├── renderer_period_position_callback_napi.h
│ │       │ └── renderer_position_callback_napi.h
│ │       └── src
│ │           ├── audio_renderer_callback_napi.cpp
│ │           ├── audio_renderer_napi.cpp
│ │           ├── renderer_data_request_callback_napi.cpp
│ │           ├── renderer_period_position_callback_napi.cpp
│ │           └── renderer_position_callback_napi.cpp
│ └── native                      #native 接口
│     └── audiorenderer
│       ├── BUILD.gn
│       ├── include
│       │ ├── audio_renderer_private.h
│       │ └── audio_renderer_proxy_obj.h
│       ├── src
│       │ ├── audio_renderer.cpp
│       │ └── audio_renderer_proxy_obj.cpp
│       └── test
│           └── example
│               └── audio_renderer_test.cpp
├── interfaces
│ ├── inner_api                   #native實現的接口
│ │ └── native
│ │     └── audiorenderer       #audio渲染本地實現的接口定義
│ │       └── include
│ │           └── audio_renderer.h
│ └── kits                        #js調用的接口
│     └── js
│         └── audio_renderer      #audio渲染NAPI接口的定義
│           └── include
│               └── audio_renderer_napi.h
└── services                        #服務端
  └── audio_service
      ├── BUILD.gn
      ├── client                  #IPC調用中的proxy端
      │ ├── include
      │ │ ├── audio_manager_proxy.h
      │ │ ├── audio_service_client.h
      │ └── src
      │     ├── audio_manager_proxy.cpp
      │     ├── audio_service_client.cpp
      └── server                  #IPC調用中的server端
        ├── include
        │ └── audio_server.h
        └── src
            ├── audio_manager_stub.cpp
└──audio_server.cpp

三、音頻渲染總體流程

四、Native接口使用

在OpenAtom OpenHarmony（以下簡稱“OpenHarmony”）系統中，音頻模塊提供了功能測試代碼，本文選取了其中的音頻渲染例子作為切入點來進行介紹，例子采用的是對wav格式的音頻文件進行渲染。wav格式的音頻文件是wav頭文件和音頻的原始數據，不需要進行數據解碼，所以音頻渲染直接對原始數據進行操作，文件路徑為：foundation/multimedia/audio_framework/frameworks/native/audiorenderer/test/example/audio_renderer_test.cpp

bool TestPlayback(int argc, char *argv[]) const
{
        FILE* wavFile = fopen(path, "rb");
        //讀取wav文件頭信息
        size_t bytesRead = fread(&wavHeader, 1, headerSize, wavFile);


        //設置AudioRenderer參數
        AudioRendererOptions rendererOptions = {};
        rendererOptions.streamInfo.encoding = AudioEncodingType::ENCODING_PCM;
        rendererOptions.streamInfo.samplingRate = static_cast(wavHeader.SamplesPerSec);
        rendererOptions.streamInfo.format = GetSampleFormat(wavHeader.bitsPerSample);
        rendererOptions.streamInfo.channels = static_cast(wavHeader.NumOfChan);
        rendererOptions.rendererInfo.contentType = contentType;
        rendererOptions.rendererInfo.streamUsage = streamUsage;
        rendererOptions.rendererInfo.rendererFlags = 0;


        //創建AudioRender實例
        unique_ptr audioRenderer = AudioRenderer::Create(rendererOptions);


        shared_ptr cb1 = make_shared();
        //設置音頻渲染回調
        ret = audioRenderer->SetRendererCallback(cb1);


        //InitRender方法主要調用了audioRenderer實例的Start方法，啟動音頻渲染
        if (!InitRender(audioRenderer)) {
            AUDIO_ERR_LOG("AudioRendererTest: Init render failed");
            fclose(wavFile);
            return false;
        }


        //StartRender方法主要是讀取wavFile文件的數據，然后通過調用audioRenderer實例的Write方法進行播放
        if (!StartRender(audioRenderer, wavFile)) {
            AUDIO_ERR_LOG("AudioRendererTest: Start render failed");
            fclose(wavFile);
            return false;
        }


        //停止渲染
        if (!audioRenderer->Stop()) {
            AUDIO_ERR_LOG("AudioRendererTest: Stop failed");
        }


        //釋放渲染
        if (!audioRenderer->Release()) {
            AUDIO_ERR_LOG("AudioRendererTest: Release failed");
        }


        //關閉wavFile
        fclose(wavFile);
        return true;
}

首先讀取wav文件，通過讀取到wav文件的頭信息對AudioRendererOptions相關的參數進行設置，包括編碼格式、采樣率、采樣格式、通道數等。根據AudioRendererOptions設置的參數來創建AudioRenderer實例（實際上是AudioRendererPrivate），后續的音頻渲染主要是通過AudioRenderer實例進行。創建完成后，調用AudioRenderer的Start方法，啟動音頻渲染。啟動后，通過AudioRenderer實例的Write方法，將數據寫入，音頻數據會被播放。

五、調用流程

1.創建AudioRenderer

std::unique_ptr AudioRenderer::Create(const std::string cachePath,
    const AudioRendererOptions &rendererOptions, const AppInfo &appInfo)
{
    ContentType contentType = rendererOptions.rendererInfo.contentType;
    
    StreamUsage streamUsage = rendererOptions.rendererInfo.streamUsage;
   
    AudioStreamType audioStreamType = AudioStream::GetStreamType(contentType, streamUsage);
    auto audioRenderer = std::make_unique(audioStreamType, appInfo);
    if (!cachePath.empty()) {
        AUDIO_DEBUG_LOG("Set application cache path");
        audioRenderer->SetApplicationCachePath(cachePath);
    }


    audioRenderer->rendererInfo_.contentType = contentType;
    audioRenderer->rendererInfo_.streamUsage = streamUsage;
    audioRenderer->rendererInfo_.rendererFlags = rendererOptions.rendererInfo.rendererFlags;


    AudioRendererParams params;
    params.sampleFormat = rendererOptions.streamInfo.format;
    params.sampleRate = rendererOptions.streamInfo.samplingRate;
    params.channelCount = rendererOptions.streamInfo.channels;
    params.encodingType = rendererOptions.streamInfo.encoding;


    if (audioRenderer->SetParams(params) != SUCCESS) {
        AUDIO_ERR_LOG("SetParams failed in renderer");
        audioRenderer = nullptr;
        return nullptr;
    }


    return audioRenderer;
}

首先通過AudioStream的GetStreamType方法獲取音頻流的類型，根據音頻流類型創建AudioRendererPrivate對象，AudioRendererPrivate是AudioRenderer的子類。緊接著對audioRenderer進行參數設置，其中包括采樣格式、采樣率、通道數、編碼格式。設置完成后返回創建的AudioRendererPrivate實例。

2.設置回調

int32_t AudioRendererPrivate::SetRendererCallback(const std::shared_ptr &callback)
{
    RendererState state = GetStatus();
    if (state == RENDERER_NEW || state == RENDERER_RELEASED) {
        return ERR_ILLEGAL_STATE;
    }
    if (callback == nullptr) {
        return ERR_INVALID_PARAM;
    }


    // Save reference for interrupt callback
    if (audioInterruptCallback_ == nullptr) {
        return ERROR;
    }
    std::shared_ptr cbInterrupt =
        std::static_pointer_cast(audioInterruptCallback_);
    cbInterrupt->SaveCallback(callback);


    // Save and Set reference for stream callback. Order is important here.
    if (audioStreamCallback_ == nullptr) {
        audioStreamCallback_ = std::make_shared();
        if (audioStreamCallback_ == nullptr) {
            return ERROR;
        }
    }
    std::shared_ptr cbStream =
std::static_pointer_cast(audioStreamCallback_);
    cbStream->SaveCallback(callback);
    (void)audioStream_->SetStreamCallback(audioStreamCallback_);


    return SUCCESS;
}

參數傳入的回調主要涉及到兩個方面：一方面是AudioInterruptCallbackImpl中設置了我們傳入的渲染回調，另一方面是AudioStreamCallbackRenderer中也設置了渲染回調。

3.啟動渲染

bool AudioRendererPrivate::Start(StateChangeCmdType cmdType) const
{
    AUDIO_INFO_LOG("AudioRenderer::Start");
    RendererState state = GetStatus();


    AudioInterrupt audioInterrupt;
    switch (mode_) {
        case InterruptMode:
            audioInterrupt = sharedInterrupt_;
            break;
        case InterruptMode:
            audioInterrupt = audioInterrupt_;
            break;
        default:
            break;
    }
    AUDIO_INFO_LOG("AudioRenderer: %{public}d, streamType: %{public}d, sessionID: %{public}d",
        mode_, audioInterrupt.streamType, audioInterrupt.sessionID);


    if (audioInterrupt.streamType == STREAM_DEFAULT || audioInterrupt.sessionID == INVALID_SESSION_ID) {
        return false;
    }


    int32_t ret = AudioPolicyManager::GetInstance().ActivateAudioInterrupt(audioInterrupt);
    if (ret != 0) {
        AUDIO_ERR_LOG("AudioRendererPrivate::ActivateAudioInterrupt Failed");
        return false;
    }


    return audioStream_->StartAudioStream(cmdType);
}

AudioPolicyManager::GetInstance().ActivateAudioInterrupt這個操作主要是根據AudioInterrupt來進行音頻中斷的激活，這里涉及了音頻策略相關的內容，后續會專門出關于音頻策略的文章進行分析。這個方法的核心是通過調用AudioStream的StartAudioStream方法來啟動音頻流。

bool AudioStream::StartAudioStream(StateChangeCmdType cmdType)
{
    int32_t ret = StartStream(cmdType);


    resetTime_ = true;
    int32_t retCode = clock_gettime(CLOCK_MONOTONIC, &baseTimestamp_);


    if (renderMode_ == RENDER_MODE_CALLBACK) {
        isReadyToWrite_ = true;
        writeThread_ = std::make_unique<std::thread>(&AudioStream::WriteCbTheadLoop, this);
    } else if (captureMode_ == CAPTURE_MODE_CALLBACK) {
        isReadyToRead_ = true;
        readThread_ = std::make_unique<std::thread>(&AudioStream::ReadCbThreadLoop, this);
    }


    isFirstRead_ = true;
    isFirstWrite_ = true;
    state_ = RUNNING;
    AUDIO_INFO_LOG("StartAudioStream SUCCESS");


    if (audioStreamTracker_) {
        AUDIO_DEBUG_LOG("AudioStream:Calling Update tracker for Running");
        audioStreamTracker_->UpdateTracker(sessionId_, state_, rendererInfo_, capturerInfo_);
    }
    return true;
}

AudioStream的StartAudioStream主要的工作是調用StartStream方法，StartStream方法是AudioServiceClient類中的方法。AudioServiceClient類是AudioStream的父類。接下來看一下AudioServiceClient的StartStream方法。

int32_t AudioServiceClient::StartStream(StateChangeCmdType cmdType)
{
    int error;
    lock_guard lockdata(dataMutex);
    pa_operation *operation = nullptr;


    pa_threaded_mainloop_lock(mainLoop);


    pa_stream_state_t state = pa_stream_get_state(paStream);


    streamCmdStatus = 0;
    stateChangeCmdType_ = cmdType;
    operation = pa_stream_cork(paStream, 0, PAStreamStartSuccessCb, (void *)this);


    while (pa_operation_get_state(operation) == PA_OPERATION_RUNNING) {
        pa_threaded_mainloop_wait(mainLoop);
    }
    pa_operation_unref(operation);
    pa_threaded_mainloop_unlock(mainLoop);


    if (!streamCmdStatus) {
        AUDIO_ERR_LOG("Stream Start Failed");
        ResetPAAudioClient();
        return AUDIO_CLIENT_START_STREAM_ERR;
    } else {
        AUDIO_INFO_LOG("Stream Started Successfully");
        return AUDIO_CLIENT_SUCCESS;
    }
}

StartStream方法中主要是調用了pulseaudio庫的pa_stream_cork方法進行流啟動，后續就調用到了pulseaudio庫中了。pulseaudio庫我們暫且不分析。

4.寫入數據

int32_t AudioRendererPrivate::Write(uint8_t *buffer, size_t bufferSize)
{
    return audioStream_->Write(buffer, bufferSize);
}

通過調用AudioStream的Write方式實現功能，接下來看一下AudioStream的Write方法。

size_t AudioStream::Write(uint8_t *buffer, size_t buffer_size)
{
    int32_t writeError;
    StreamBuffer stream;
    stream.buffer = buffer;
    stream.bufferLen = buffer_size;
    isWriteInProgress_ = true;


    if (isFirstWrite_) {
        if (RenderPrebuf(stream.bufferLen)) {
            return ERR_WRITE_FAILED;
        }
        isFirstWrite_ = false;
    }


    size_t bytesWritten = WriteStream(stream, writeError);
    isWriteInProgress_ = false;
    if (writeError != 0) {
        AUDIO_ERR_LOG("WriteStream fail,writeError:%{public}d", writeError);
        return ERR_WRITE_FAILED;
    }
    return bytesWritten;
}

Write方法中分成兩個階段，首次寫數據，先調用RenderPrebuf方法，將preBuf_的數據寫入后再調用WriteStream進行音頻數據的寫入。

size_t AudioServiceClient::WriteStream(const StreamBuffer &stream, int32_t &pError)
{
   
    size_t cachedLen = WriteToAudioCache(stream);
    if (!acache.isFull) {
        pError = error;
        return cachedLen;
    }


    pa_threaded_mainloop_lock(mainLoop);




    const uint8_t *buffer = acache.buffer.get();
    size_t length = acache.totalCacheSize;


    error = PaWriteStream(buffer, length);
    acache.readIndex += acache.totalCacheSize;
    acache.isFull = false;


    if (!error && (length >= 0) && !acache.isFull) {
        uint8_t *cacheBuffer = acache.buffer.get();
        uint32_t offset = acache.readIndex;
        uint32_t size = (acache.writeIndex - acache.readIndex);
        if (size > 0) {
            if (memcpy_s(cacheBuffer, acache.totalCacheSize, cacheBuffer + offset, size)) {
                AUDIO_ERR_LOG("Update cache failed");
                pa_threaded_mainloop_unlock(mainLoop);
                pError = AUDIO_CLIENT_WRITE_STREAM_ERR;
                return cachedLen;
            }
            AUDIO_INFO_LOG("rearranging the audio cache");
        }
        acache.readIndex = 0;
        acache.writeIndex = 0;


        if (cachedLen < stream.bufferLen) {
            StreamBuffer str;
            str.buffer = stream.buffer + cachedLen;
            str.bufferLen = stream.bufferLen - cachedLen;
            AUDIO_DEBUG_LOG("writing pending data to audio cache: %{public}d", str.bufferLen);
            cachedLen += WriteToAudioCache(str);
        }
    }


    pa_threaded_mainloop_unlock(mainLoop);
    pError = error;
    return cachedLen;
}

WriteStream方法不是直接調用pulseaudio庫的寫入方法，而是通過WriteToAudioCache方法將數據寫入緩存中，如果緩存沒有寫滿則直接返回，不會進入下面的流程，只有當緩存寫滿后，才會調用下面的PaWriteStream方法。該方法涉及對pulseaudio庫寫入操作的調用，所以緩存的目的是避免對pulseaudio庫頻繁地做IO操作，提高了效率。

六、總結

本文主要對OpenHarmony 3.2 Beta多媒體子系統的音頻渲染模塊進行介紹，首先梳理了Audio Render的整體流程，然后對幾個核心的方法進行代碼的分析。整體的流程主要通過pulseaudio庫啟動流，然后通過pulseaudio庫的pa_stream_write方法進行數據的寫入，最后播放出音頻數據。

音頻渲染主要分為以下幾個層次：

（1）AudioRenderer的創建，實際創建的是它的子類AudioRendererPrivate實例。

（2）通過AudioRendererPrivate設置渲染的回調。

（3）啟動渲染，這一部分代碼最終會調用到pulseaudio庫中，相當于啟動了pulseaudio的流。

（4）通過pulseaudio庫的pa_stream_write方法將數據寫入設備，進行播放。

對OpenHarmony 3.2 Beta多媒體系列開發感興趣的讀者，也可以閱讀我之前寫過幾篇文章：《OpenHarmony 3.2 Beta多媒體系列——視頻錄制》《OpenHarmony 3.2 Beta源碼分析之MediaLibrary》《OpenHarmony 3.2 Beta多媒體系列——音視頻播放框架》《OpenHarmony 3.2 Beta多媒體系列——音視頻播放gstreamer》。

原文標題：OpenHarmony 3.2 Beta Audio——音頻渲染

文章出處：【微信公眾號：OpenAtom OpenHarmony】歡迎添加關注！文章轉載請注明出處。

聲明：本文內容及配圖由入駐作者撰寫或者入駐合作網站授權轉載。文章觀點僅代表作者本人，不代表電子發燒友網立場。文章及其配圖僅供工程師學習之用，如有內容侵權或者其他違規問題，請聯系本站處理。舉報投訴