<Android Framework 之路>Android5.1 Camera Framework（二）

上一次讲解了一下CameraService的启动过程，今天梳理一下Camera预览的过程

StartPreview过程

首先，我们还是从应用层的使用入手
Camera.java (packages\apps\legacycamera\src\com\android\camera)

    Thread mCameraPreviewThread = new Thread(new Runnable() {        public void run() {            initializeCapabilities(); //初始化参数            startPreview(); //启动预览        }    });

针对相机应用，采用了单独的线程来处理预览，猜测是为了加快预览显示的速度

private void startPreview() {        ......        // If we're previewing already, stop the preview first (this will blank        // the screen).        if (mCameraState != PREVIEW_STOPPED) stopPreview();        setPreviewDisplay(mSurfaceHolder); //设置SurfaceHolder        setDisplayOrientation();       //设置显示方向        ......        setCameraParameters(UPDATE_PARAM_ALL); //设置参数        // Inform the mainthread to go on the UI initialization.        if (mCameraPreviewThread != null) {            synchronized (mCameraPreviewThread) {                mCameraPreviewThread.notify();            }        }        try {            Log.v(TAG, "startPreview");            mCameraDevice.startPreview(); //启动预览，若失败，关闭Camera        } catch (Throwable ex) {            closeCamera();            throw new RuntimeException("startPreview failed", ex);        }        ......    }

这里就是APP中启动预览的过程，过程必然会是
java -> JNI -> cpp，
然后通过Binder机制执行到CameraClient中的
CameraClient.cpp (av\services\camera\libcameraservice\api1)

status_t CameraClient::startPreview() {    LOG1("startPreview (pid %d)", getCallingPid());    return startCameraMode(CAMERA_PREVIEW_MODE);}

这里传入的是CAMERA_PREVIEW_MODE，枚举类型是在CameraClient.h中定义的
// camera operation mode
enum camera_mode {
CAMERA_PREVIEW_MODE = 0, // frame automatically released
CAMERA_RECORDING_MODE = 1, // frame has to be explicitly released by releaseRecordingFrame()
};
第一种是针对普通的预览，第二种是针对录像

// start preview or recordingstatus_t CameraClient::startCameraMode(camera_mode mode) {    LOG1("startCameraMode(%d)", mode);    Mutex::Autolock lock(mLock);    status_t result = checkPidAndHardware();    if (result != NO_ERROR) return result;    switch(mode) {        case CAMERA_PREVIEW_MODE:            if (mSurface == 0 && mPreviewWindow == 0) {                 LOG1("mSurface is not set yet.");                // still able to start preview in this case.            }            return startPreviewMode(); //开始预览模式        case CAMERA_RECORDING_MODE:            if (mSurface == 0 && mPreviewWindow == 0) {                ALOGE("mSurface or mPreviewWindow must be set before startRecordingMode.");                return INVALID_OPERATION;            }            return startRecordingMode(); //开始录像模式        default:            return UNKNOWN_ERROR;    }}

这里我们走的是预览模式

status_t CameraClient::startPreviewMode() {    LOG1("startPreviewMode"); //LOG1,一直忘记说了，这是有log开关用过setprop可以使用    status_t result = NO_ERROR;    // if preview has been enabled, nothing needs to be done    if (mHardware->previewEnabled()) { //如果已经启动预览，不必重复        return NO_ERROR;    }    if (mPreviewWindow != 0) {        //适配显示窗口的大小        native_window_set_scaling_mode(mPreviewWindow.get(),                NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);        //调整帧数据的方向        native_window_set_buffers_transform(mPreviewWindow.get(),                mOrientation);    }    mHardware->setPreviewWindow(mPreviewWindow); //设置mPreviewWindow为显示窗口    result = mHardware->startPreview();   //HAL层启动预览    return result; //返回结果}

这里面涉及到native_window_set_scaling_mode，native_window_set_buffers_transform，直接跟代码，看注释就可以理解，这部分涉及到显示的一些内容，这里暂时不做讲解，native_window_set_scaling_mode设置模式为NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW，native_window_set_buffers_transform是用来调整方向。
这里有一个问题，mPreviewWindow是从何而来的呢？
还记得我们在应用层的startPreview()方法中会有这么一个过程

        setPreviewDisplay(mSurfaceHolder);        setDisplayOrientation();

这里的setPreviewDisplay(mSurfaceHolder)，中间的过程大家可以自己跟一下，最后会执行到

status_t CameraClient::setPreviewTarget(        const sp<IGraphicBufferProducer>& bufferProducer) {    ......    sp<IBinder> binder;    sp<ANativeWindow> window;    if (bufferProducer != 0) {        binder = bufferProducer->asBinder();        // Using controlledByApp flag to ensure that the buffer queue remains in        // async mode for the old camera API, where many applications depend        // on that behavior.        window = new Surface(bufferProducer, /*controlledByApp*/ true); //这个家伙    }    return setPreviewWindow(binder, window);}

status_t CameraClient::setPreviewWindow(const sp<IBinder>& binder,        const sp<ANativeWindow>& window) {    Mutex::Autolock lock(mLock);    ......    if (window != 0) {        result = native_window_api_connect(window.get(), NATIVE_WINDOW_API_CAMERA);        if (result != NO_ERROR) {            ALOGE("native_window_api_connect failed: %s (%d)", strerror(-result),                    result);            return result;        }    }    // If preview has been already started, register preview buffers now.    if (mHardware->previewEnabled()) {        if (window != 0) {            native_window_set_scaling_mode(window.get(),                    NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);            native_window_set_buffers_transform(window.get(), mOrientation);            result = mHardware->setPreviewWindow(window);        }    }    if (result == NO_ERROR) {        // Everything has succeeded.  Disconnect the old window and remember the        // new window.        disconnectWindow(mPreviewWindow);        mSurface = binder;        mPreviewWindow = window; //这里便是赋值的操作了，后面我们操作的mPreviewWindow     } else {        // Something went wrong after we connected to the new window, so        // disconnect here.        disconnectWindow(window);    }    return result;}

是不是看的很眼熟，和startPreviewMode()的过程有点相似。这就是mPreviewWindow的赋值过程。
回到setPreviewMode()函数中，其中主要的过程是这两个
mHardware->setPreviewWindow(mPreviewWindow);
result = mHardware->startPreview();
这里都是HAL层的处理，将窗口传下去，然后启动预览，最后数据就可以投射到这个预览窗口上了。
我们继续往下看一下，
CameraHardwareInterface.h (av\services\camera\libcameraservice\device1)

    /** Set the ANativeWindow to which preview frames are sent */    status_t setPreviewWindow(const sp<ANativeWindow>& buf)    {        ALOGV("%s(%s) buf %p", __FUNCTION__, mName.string(), buf.get());        if (mDevice->ops->set_preview_window) {            mPreviewWindow = buf;            mHalPreviewWindow.user = this;            ALOGV("%s &mHalPreviewWindow %p mHalPreviewWindow.user %p", __FUNCTION__,                    &mHalPreviewWindow, mHalPreviewWindow.user);            return mDevice->ops->set_preview_window(mDevice,                    buf.get() ? &mHalPreviewWindow.nw : 0);        }        return INVALID_OPERATION;    }

   /**     * Start preview mode.     */    status_t startPreview()    {        ALOGV("%s(%s)", __FUNCTION__, mName.string());        if (mDevice->ops->start_preview)            return mDevice->ops->start_preview(mDevice);        return INVALID_OPERATION;    }

这是一个空壳，我们去看具体的实现，这里我们看下android5.1源码中Qcom的实现，由于针对HAL层的不同厂商有不同的处理方式，在这里我们就随便找个目录下的进行分析，旨在看流程，理解一些基础的内容，
QCamera2Hal.cpp （\device\asus\flo\camera\qcamera2\hal）

#include "QCamera2Factory.h"static hw_module_t camera_common = {    tag: HARDWARE_MODULE_TAG,    module_api_version: CAMERA_MODULE_API_VERSION_1_0,    hal_api_version: HARDWARE_HAL_API_VERSION,    id: CAMERA_HARDWARE_MODULE_ID,    name: "QCamera Module",    author: "Qualcomm Innovation Center Inc",    methods: &qcamera::QCamera2Factory::mModuleMethods,    dso: NULL,    reserved:  {0},};camera_module_t HAL_MODULE_INFO_SYM = {    common: camera_common,    get_number_of_cameras: qcamera::QCamera2Factory::get_number_of_cameras,    get_camera_info: qcamera::QCamera2Factory::get_camera_info,    set_callbacks: NULL,    get_vendor_tag_ops: NULL,    open_legacy: NULL,    reserved: {0}};

这里提一下HAL_MODULE_INFO_SYM这个东西，本身就是一个定义在hardware.h下的一个宏，看注释，意思很明显

define HAL_MODULE_INFO_SYM HMI //.so中将一个符号HMI,获取此符号的地址，就获取到了对应的hw_module_t地址

HAL_MODULE_INFO_SYM，这个是HAL 编译生成的so的入口，CameraService会获取这个来操作so
camera_common是针对HAL规范定义的一些内容。
Camera的open指向的是&qcamera::QCamera2Factory::mModuleMethods中的open方法，如下

struct hw_module_methods_t QCamera2Factory::mModuleMethods = {    open: QCamera2Factory::camera_device_open,};

这个方法中打开设备节点，我们可以看到HAL层中open的过程是很有讲究的，也不能这么说，应为HAL的处理方式基本上都是如此。

int QCamera2Factory::camera_device_open(    const struct hw_module_t *module, const char *id,    struct hw_device_t **hw_device){    if (module != &HAL_MODULE_INFO_SYM.common) {        ALOGE("Invalid module. Trying to open %p, expect %p",            module, &HAL_MODULE_INFO_SYM.common);        return INVALID_OPERATION;    }    if (!id) {        ALOGE("Invalid camera id");        return BAD_VALUE;    }    return gQCamera2Factory.cameraDeviceOpen(atoi(id), hw_device);}

int QCamera2Factory::cameraDeviceOpen(int camera_id,                    struct hw_device_t **hw_device){    int rc = NO_ERROR;    if (camera_id < 0 || camera_id >= mNumOfCameras)        return BAD_VALUE;    //到这里才是真正的HAL层的创建，可见HAL层的创建和open操作是相关的    QCamera2HardwareInterface *hw = new QCamera2HardwareInterface(camera_id);    if (!hw) {        ALOGE("Allocation of hardware interface failed");        return NO_MEMORY;    }    rc = hw->openCamera(hw_device);    if (rc != NO_ERROR) {        delete hw;    }    return rc;}

之前在CameraService过程中提到的CameraHardwareInterface空壳就是为这个QCamera2HardwareInterface准备的，具体实现全部都在这个类中。
关于QCamera2HardwareInterface的内容我们在后面会讲到，这里暂且先放一下，接着上面的
mHardware->setPreviewWindow(mPreviewWindow);
result = mHardware->startPreview();
经过CameraHardwareInterface后
mDevice->ops->set_preview_window(mDevice, buf.get() ? &mHalPreviewWindow.nw : 0);
mDevice->ops->start_preview(mDevice);
然后经过QCamera2HardwareInterface中的mCameraOps函数指针对应表
set_preview_window: QCamera2HardwareInterface::set_preview_window
start_preview: QCamera2HardwareInterface::start_preview
所以会调用到

int QCamera2HardwareInterface::set_preview_window(struct camera_device *device,        struct preview_stream_ops *window){    int rc = NO_ERROR;    QCamera2HardwareInterface *hw =        reinterpret_cast<QCamera2HardwareInterface *>(device->priv);    if (!hw) {        ALOGE("%s: NULL camera device", __func__);        return BAD_VALUE;    }    hw->lockAPI();    rc = hw->processAPI(QCAMERA_SM_EVT_SET_PREVIEW_WINDOW, (void *)window);    if (rc == NO_ERROR) {        hw->waitAPIResult(QCAMERA_SM_EVT_SET_PREVIEW_WINDOW);        rc = hw->m_apiResult.status;    }    hw->unlockAPI();    return rc;}

这里会经过一轮状态机，暂时先不讲，然后执行到
int QCamera2HardwareInterface::setPreviewWindow(
struct preview_stream_ops *window)
{
mPreviewWindow = window;
return NO_ERROR;
}
同理，startPreview也会执行到
int QCamera2HardwareInterface::startPreview()
{
int32_t rc = NO_ERROR;
ALOGD(“%s: E”, func);
// start preview stream
if (mParameters.isZSLMode() && mParameters.getRecordingHintValue() !=true) {
rc = startChannel(QCAMERA_CH_TYPE_ZSL);
} else {
rc = startChannel(QCAMERA_CH_TYPE_PREVIEW);
}
ALOGD(“%s: X”, func);
return rc;
}
这里开启通道，可以理解成数据通道，ZSL是之前没有的，所谓ZSL就是触发拍照后不停止预览。
这里看到会根据当前是都支持ZSL模式而进入不同的通道，我们这里就看QCAMERA_CH_TYPE_PREVIEW，startChannel

int32_t QCamera2HardwareInterface::startChannel(qcamera_ch_type_enum_t ch_type){    int32_t rc = UNKNOWN_ERROR;    if (m_channels[ch_type] != NULL) {        rc = m_channels[ch_type]->start();    }    return rc;}

m_channels是不同的通道的实例的数组，这里如果没有PREVIEW的channel就直接return，岂不是无法启动预览，这个流程感觉有点不对劲。但是这整个过程跟下来也没有看到m_channels相关的初始化过程。

这个问题出在我刚才从CameraHardwareInterface跟到QCamera2HardwareInterface的时候跳过的一个内容—–状态机，在状态机中会执行一个preparePreview()的操作

int32_t QCamera2HardwareInterface::preparePreview(){    int32_t rc = NO_ERROR;    if (mParameters.isZSLMode() && mParameters.getRecordingHintValue() !=true) {        rc = addChannel(QCAMERA_CH_TYPE_ZSL); //这里我们就添加了一个channel,当然这里是ZSL的        if (rc != NO_ERROR) {            return rc;        }    } else {        bool recordingHint = mParameters.getRecordingHintValue(); //recording        if(recordingHint) {            rc = addChannel(QCAMERA_CH_TYPE_SNAPSHOT);  //录像中是可以拍照的，需要snapshot channel            if (rc != NO_ERROR) {                return rc;            }            rc = addChannel(QCAMERA_CH_TYPE_VIDEO); //video channel            if (rc != NO_ERROR) {                delChannel(QCAMERA_CH_TYPE_SNAPSHOT);                return rc;            }        }        rc = addChannel(QCAMERA_CH_TYPE_PREVIEW); //添加preview  channel        if (rc != NO_ERROR) {            if (recordingHint) {                delChannel(QCAMERA_CH_TYPE_SNAPSHOT);                delChannel(QCAMERA_CH_TYPE_VIDEO);            }            return rc;        }    }    return rc;}

在addchannel()的过程中会根据不同的channel类型创建不同的实例，这里我们直接看从addChannel()转到的addPreviewChannel()函数

int32_t QCamera2HardwareInterface::addPreviewChannel(){    int32_t rc = NO_ERROR;    QCameraChannel *pChannel = NULL; //初始化一个QCameraChanel，后面要使用    if (m_channels[QCAMERA_CH_TYPE_PREVIEW] != NULL) {        // if we had preview channel before, delete it first        delete m_channels[QCAMERA_CH_TYPE_PREVIEW]; //如果之前preview channel存在，干掉        m_channels[QCAMERA_CH_TYPE_PREVIEW] = NULL;    }    pChannel = new QCameraChannel(mCameraHandle->camera_handle,                                  mCameraHandle->ops);     //new 一个新的channel    .....    // meta data stream always coexists with preview if applicable    rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_METADATA,                            metadata_stream_cb_routine, this);    //添加metadata stream cb    if (rc != NO_ERROR) {        ALOGE("%s: add metadata stream failed, ret = %d", __func__, rc);        delete pChannel;        return rc;    }    if (isNoDisplayMode()) { //判断是否为不需要显示的模式        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,                                nodisplay_preview_stream_cb_routine, this);    } else {        //这里添加preview stream cb到channel中        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,                                preview_stream_cb_routine, this);    }    if (rc != NO_ERROR) {        ALOGE("%s: add preview stream failed, ret = %d", __func__, rc);        delete pChannel;        return rc;    }    m_channels[QCAMERA_CH_TYPE_PREVIEW] = pChannel; //维护m_channels数据    return rc;}

这里注册的preview_stream_cb_routine回调，这之后的过程我们暂时先不去看，了解这部分之后，回到之前chanel 的start()，最后会执行到QCameraChannel::start()方法，这里往下的内容我们暂时不往下看，知道这个过程中会执行数据采集，然后返回给HAL层就行了，HAL针对底层返回的数据，我们在哪里获取，做什么对应的处理呢？找到之前注册的Callback.
QCamera2HWICallbacks.cpp (\device\asus\flo\camera\qcamera2\hal)

void QCamera2HardwareInterface::preview_stream_cb_routine(mm_camera_super_buf_t *super_frame,                                                          QCameraStream * stream,                                                          void *userdata){    ALOGD("[KPI Perf] %s : BEGIN", __func__);    int err = NO_ERROR;    QCamera2HardwareInterface *pme = (QCamera2HardwareInterface *)userdata;    QCameraGrallocMemory *memory = (QCameraGrallocMemory *)super_frame->bufs[0]->mem_info;    ......    mm_camera_buf_def_t *frame = super_frame->bufs[0];    ......    if (!pme->needProcessPreviewFrame()) {        ALOGE("%s: preview is not running, no need to process", __func__);        stream->bufDone(frame->buf_idx);        free(super_frame);        return;    }    if (pme->needDebugFps()) {        pme->debugShowPreviewFPS();    }    int idx = frame->buf_idx;    pme->dumpFrameToFile(frame->buffer, frame->frame_len,                         frame->frame_idx, QCAMERA_DUMP_FRM_PREVIEW);    //这里的注释很明显，displayer buffer而这个buffer就是我们需要投射到屏幕上的数据    // Display the buffer.    int dequeuedIdx = memory->displayBuffer(idx); //这部分涉及到显示的过程，这里不做赘述    if (dequeuedIdx < 0 || dequeuedIdx >= memory->getCnt()) {        ALOGD("%s: Invalid dequeued buffer index %d from display",              __func__, dequeuedIdx);    } else {        // Return dequeued buffer back to driver        err = stream->bufDone(dequeuedIdx);        if ( err < 0) {            ALOGE("stream bufDone failed %d", err);        }    }    //针对上层设置的datacallback过程做些处理    // Handle preview data callback    if (pme->mDataCb != NULL && pme->msgTypeEnabledWithLock(CAMERA_MSG_PREVIEW_FRAME) > 0) {        ......        qcamera_callback_argm_t cbArg;        memset(&cbArg, 0, sizeof(qcamera_callback_argm_t));        cbArg.cb_type = QCAMERA_DATA_CALLBACK;        cbArg.msg_type = CAMERA_MSG_PREVIEW_FRAME;        cbArg.data = data;        if ( previewMem ) {            cbArg.user_data = previewMem;            cbArg.release_cb = releaseCameraMemory;        }        cbArg.cookie = pme;        pme->m_cbNotifier.notifyCallback(cbArg); //封装完之后往上甩    }    free(super_frame);    ALOGD("[KPI Perf] %s : END", __func__);    return;}

这就是在addPreviewChannel的过程中添加的preview stream callback，当然还有metadata的，暂时先看preview的这个。这里面作的操作就是显示预览数据到窗口中，然后对设置下面的preview callback做对应的callback处理.

讲到这里，Camera的预览过程基本上就结束了，关于底层如果采集数据以及HAL中一些其他的内容，在这里没有讲解，主要是要理解这个过程，之后再每一个过程中在往下学习。

本文中代码使用的是Android5.1原始代码，欢迎大家留言交流。

版权声明：本文为博主原创文章，未经博主允许不得转载。