Android 4.0 Camera架构分析之preview和takePicture

时间:2020-12-03 19:49:08

上篇文章介绍了,Camera初始化的过程,完成初始化之后就可以使用Camera提供的以下功能了


1.预览preview

2.视频录制

3.拍照和参数设置


打开Camera第一键事情就是预览取景preview的动作,我们先从Camera app分析起 。所有拥有拍照功能的应用,它在预览时候都要实现SurfaceHolder.Callback接口,并实现其surfaceCreated、surfaceChanged、surfaceDestroyed三个函数,同时声明一个用于预览的窗口SurfaceView ,以下是系统自带ap的源代码

SurfaceView preview = (SurfaceView) findViewById(R.id.camera_preview);

SurfaceHolder holder = preview.getHolder();
holder.addCallback(this);

还要设置camera预览的surface缓存区 ,系统自带app实在surfaceChange()方法里面设置Camera的预览区,以供底层获取的preview数据不断投递到这个surface缓存区内。

public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {

        mSurfaceHolder = holder;

        // The mCameraDevice will be null if it fails to connect to the camera
        // hardware. In this case we will show a dialog and then finish the
        // activity, so it's OK to ignore it.

        if (mCameraDevice == null) return;

        // Sometimes surfaceChanged is called after onPause or before onResume.
        // Ignore it.

        if (mPausing || isFinishing()) return;

        // Set preview display if the surface is being created. Preview was
        // already started. Also restart the preview if display rotation has
        // changed. Sometimes this happens when the device is held in portrait
        // and camera app is opened. Rotation animation takes some time and
        // display rotation in onCreate may not be what we want.

        if (mCameraState == PREVIEW_STOPPED) {
            startPreview();
            startFaceDetection();
        } else {
            if (Util.getDisplayRotation(this) != mDisplayRotation) {
                setDisplayOrientation();
            }
            if (holder.isCreating()) {
               
// Set preview display if the surface is being created and preview
                // was already started. That means preview display was set to null
                // and we need to set it now.

               
setPreviewDisplay(holder);
            }
        }


设置好以上参数后,就可以调用startPreview()进行取景预览

startPreview()也是一层层往下调用,最后到Camera的服务端CameraService,我们看下它的过程

Camera.java(应用)-------------> Camera.java(框架)-------------->android_hardware_camera.cpp(JNI)-------------------->Camera.cpp(客户端)------------------->CameraService.cpp(服务端)--------------------->CameraHarwareInterface(HAL接口)

在CameraService端将处理preview的请求并进入HAL层

status_t CameraService::Client::startPreview() {

     enableMsgType(CAMERA_MSG_PREVIEW_METADATA);

     return startCameraMode(CAMERA_PREVIEW_MODE);
}

先是传递preview的消息到HAL层,然后执行preview

status_t CameraService::Client::startCameraMode(camera_mode mode) {
     switch(mode) {
        case CAMERA_PREVIEW_MODE:
            if (mSurface == 0 && mPreviewWindow == 0) {
                LOG1("mSurface is not set yet.");
                // still able to start preview in this case.
            }
           
return startPreviewMode();
    }
}


status_t CameraService::Client::startPreviewMode() {
    LOG1("startPreviewMode");
    status_t result = NO_ERROR;

    // if preview has been enabled, nothing needs to be done
    if (mHardware->previewEnabled()) {
        return NO_ERROR;
    }

    if (mPreviewWindow != 0) {
        native_window_set_scaling_mode(mPreviewWindow.get(),
                NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
        native_window_set_buffers_transform(mPreviewWindow.get(),
                mOrientation);
    }
    mHardware->setPreviewWindow(mPreviewWindow);
    result = mHardware->startPreview();


    return result;
}

然后就近去HAL层调用,并通过回调函数源源不断的将数据投递到surfaceview的缓存去,因为preview的数据是比较大的,所以数据不会携带着传上上层,而是直接在两个缓存区之间copy,一个是底层采集数据的缓存区,另一个是用于显示的surfaceview缓存区

我们看看preview的回调函数是怎么处理的

首先在Camera客户端与服务端连接成功的时候就会设置一个回调函数dataCallBack

CameraService::Client::Client(const sp<CameraService>& cameraService,
        const sp<ICameraClient>& cameraClient,
        const sp<CameraHardwareInterface>& hardware,
        int cameraId, int cameraFacing, int clientPid) {
    ......
   
mHardware->setCallbacks(notifyCallback,
                            dataCallback,
                            dataCallbackTimestamp,
                            (void *)cameraId);


}

在上篇有介绍到,client与server连接成功后就会new 一个client返回,在client的构造函数中,就对camera设置了notifyCallback、dataCallback、dataCallbackTimestamp三个回调函数,用于返回底层数据用于处理,看下它的处理方法

void CameraService::Client::dataCallback(int32_t msgType,
        const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata, void* user) {
      switch (msgType & ~CAMERA_MSG_PREVIEW_METADATA) {
  
      case CAMERA_MSG_PREVIEW_FRAME:
            client->handlePreviewData(msgType, dataPtr, metadata);

            break;
      .......

}


继续看handlePreviewData()

void CameraService::Client::handlePreviewData(int32_t msgType,
                                              const sp<IMemory>& mem,
                                              camera_frame_metadata_t *metadata) {
   
    sp<ICameraClient> c = mCameraClient;
    .......
     if (c != 0) {
        // Is the received frame copied out or not?
        if (flags & CAMERA_FRAME_CALLBACK_FLAG_COPY_OUT_MASK) {
            LOG2("frame is copied");
            copyFrameAndPostCopiedFrame(msgType, c, heap, offset, size, metadata);
        } else {
            LOG2("frame is forwarded");
            mLock.unlock();
            c->dataCallback(msgType, mem, metadata);
        }
    } else {
        mLock.unlock();
    }
}

copyFrameAndPostCopiedFrame就是这个函数执行两个buff区preview数据的投递


void CameraService::Client::copyFrameAndPostCopiedFrame(
        int32_t msgType, const sp<ICameraClient>& client,
        const sp<IMemoryHeap>& heap, size_t offset, size_t size,
        camera_frame_metadata_t *metadata) {
    ......
    previewBuffer = mPreviewBuffer;

    memcpy(previewBuffer->base(), (uint8_t *)heap->base() + offset, size);

    sp<MemoryBase> frame = new MemoryBase(previewBuffer, 0, size);
    if (frame == 0) {
        LOGE("failed to allocate space for frame callback");
        mLock.unlock();
        return;
    }

    mLock.unlock();
    client->dataCallback(msgType, frame, metadata);
}

将数据处理成frame,继续调用客户端client的回调函数 client->dataCallback(msgType, frame, metadata);

// callback from camera service when frame or image is ready
void Camera::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr,
                          camera_frame_metadata_t *metadata)
{
    sp<CameraListener> listener;
    {
        Mutex::Autolock _l(mLock);
        listener = mListener;
    }
    if (listener != NULL) {
       
listener->postData(msgType, dataPtr, metadata);
    }
}


还记得初始化的时候,在jni里面有设置listener吗?

static void android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
    jobject weak_this, jint cameraId)
{

    sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);
    context->incStrong(thiz);
    camera->setListener(context);

}


继续 listener->postData(msgType, dataPtr, metadata);

void JNICameraContext::postData(int32_t msgType, const sp<IMemory>& dataPtr,
                                camera_frame_metadata_t *metadata)
{
    ......
     switch (dataMsgType) {
        case CAMERA_MSG_VIDEO_FRAME:
            // should never happen
            break;
          default:
            LOGV("dataCallback(%d, %p)", dataMsgType, dataPtr.get());
          
  copyAndPost(env, dataPtr, dataMsgType);
            break;
    }
}


继续copyAndPost(env, dataPtr, dataMsgType);

void JNICameraContext::copyAndPost(JNIEnv* env, const sp<IMemory>& dataPtr, int msgType)
{
    jbyteArray obj = NULL;

    // allocate Java byte array and copy data
    if (dataPtr != NULL) {
    .......   
            } else {
                LOGV("Allocating callback buffer");
               
obj = env->NewByteArray(size);
      .......
               
env->SetByteArrayRegion(obj, 0, size, data);
            }
        } else {
            LOGE("image heap is NULL");
        }
    }

    // post image data to Java
    env->CallStaticVoidMethod(mCameraJClass, fields.post_event,
            mCameraJObjectWeak, msgType, 0, 0, obj);

    if (obj) {
        env->DeleteLocalRef(obj);
    }
}

解释一下标红的部分,先建立一个byte数组obj,将data缓存数据存储进obj数组,CallStaticVoidMethod是C调用java函数,最后执行实在Camera.java(框架)的postEventFromNative()

    private static void postEventFromNative(Object camera_ref,
                                            int what, int arg1, int arg2, Object obj)
    {
        Camera c = (Camera)((WeakReference)camera_ref).get();
        if (c == null)
            return;

        if (c.mEventHandler != null) {
            Message m = c.mEventHandler.obtainMessage(what, arg1, arg2, obj);
           
c.mEventHandler.sendMessage(m);
        }
    }

还是handler处理地

 public void handleMessage(Message msg) {
            switch(msg.what) {
                         case CAMERA_MSG_SHUTTER:
                if (mShutterCallback != null) {
                    mShutterCallback.onShutter();
                }
                return;

            case CAMERA_MSG_RAW_IMAGE:
                if (mRawImageCallback != null) {
                    mRawImageCallback.onPictureTaken((byte[])msg.obj, mCamera);
                }
                return;

            case CAMERA_MSG_COMPRESSED_IMAGE:
                if (mJpegCallback != null) {
                    mJpegCallback.onPictureTaken((byte[])msg.obj, mCamera);
                }
                return;
             case
CAMERA_MSG_PREVIEW_FRAME:
                if (mPreviewCallback != null) {
                    PreviewCallback cb = mPreviewCallback;
                    if (mOneShot) {
                        // Clear the callback variable before the callback
                        // in case the app calls setPreviewCallback from
                        // the callback function
                        mPreviewCallback = null;
                    } else if (!mWithBuffer) {
                        // We're faking the camera preview mode to prevent
                        // the app from being flooded with preview frames.
                        // Set to oneshot mode again.
                        setHasPreviewCallback(true, false);
                    }
                   
cb.onPreviewFrame((byte[])msg.obj, mCamera);
                }
                return;
             }
        }
    }

上面可以看出,这里处理了所有的回调,快门回调mShutterCallback.onShutter(),RawImageCallback.onPictureTaken()拍照数据回调,自动对焦回调等。。默认是没有previewcallback这个回调的,除非你的app设置了setPreviewCallback,可以看出preview的数据还是可以向上层回调,只是系统默认不回调,另数据采集区与显示区两个缓存区buffer preview数据的投递,以完成preview实时显示是在HAL层完成的。

takePicture()处理过程跟preview差不多,只是增加了回调函数返回时候存储图像的动作

        public JpegPictureCallback(Location loc) {
            mLocation = loc;
        }

        public void onPictureTaken(
                final byte [] jpegData, final android.hardware.Camera camera) {
              .........................
            if (!mIsImageCaptureIntent) {
                Size s = mParameters.getPictureSize();
               
mImageSaver.addImage(jpegData, mLocation, s.width, s.height);
            } else {
                mJpegImageData = jpegData;
                if (!mQuickCapture) {
                    showPostCaptureAlert();
                } else {
                    doAttach();
                }
            }
            }
        }
    }