V4L2框架分析学习

一、V4L2框架

1、v4l2_device在v4l2框架中充当所有v4l2_subdev的父设备，管理着注册在其下的子设备。以下是v4l2_device结构体原型

file: kernel include media v4l2-device.h

kernel include media v4l2-core v4l2-device.c

struct v4l2_device {

/* dev->driver_data points to this struct.

Note: dev might be NULL if there is no parent device

as is the case with e.g. ISA devices. */

struct device *dev;

#if defined(CONFIG_MEDIA_CONTROLLER)

struct media_device *mdev;

#endif

/* used to keep track of the registered subdevs */

struct list_head subdevs; //用链表管理注册的subdev

/* lock this struct; can be used by the driver as well if this

struct is embedded into a larger struct. */

spinlock_t lock;

/* unique device name, by default the driver name + bus ID */

char name[V4L2_DEVICE_NAME_SIZE]; //device 名字

/* notify callback called by some sub-devices. */

void (*notify)(struct v4l2_subdev *sd, unsigned int notification, void *arg);

/* The control handler. May be NULL. */

struct v4l2_ctrl_handler *ctrl_handler;

/* Device's priority state */

struct v4l2_prio_state prio;

/* BKL replacement mutex. Temporary solution only. */

struct mutex ioctl_lock;

/* Keep track of the references to this struct. */

struct kref ref; //引用计数

/* Release function that is called when the ref count goes to 0. */

void (*release)(struct v4l2_device *v4l2_dev);

};

可以看出v4l2_device的主要作用是管理注册在其下的子设备，方便系统查找引用到。

V4l2_device的注册和注销：

int v4l2_device_register(struct device*dev, struct v4l2_device *v4l2_dev)

static void v4l2_device_release(struct kref *ref)

2、V4l2_subdev代表子设备，包含了子设备的相关属性和操作。先来看下结构体原型：

file: kernel include media v4l2-subdev.h

kernel include media v4l2-core v4l2-subdev.c

struct v4l2_subdev {

#if defined(CONFIG_MEDIA_CONTROLLER)

struct media_entity entity;

#endif

struct list_head list;

struct module *owner;

u32 flags;

struct v4l2_device *v4l2_dev; //指向父设备

const struct v4l2_subdev_ops *ops; //提供一些控制v4l2设备的接口

/* Never call these internal ops from within a driver! */

const struct v4l2_subdev_internal_ops *internal_ops; //向V4L2框架提供的接口函数

/* The control handler of this subdev. May be NULL. */

struct v4l2_ctrl_handler *ctrl_handler; //subdev控制接口

/* name must be unique */

char name[V4L2_SUBDEV_NAME_SIZE];

/* can be used to group similar subdevs, value is driver-specific */

u32 grp_id;

/* pointer to private data */

void *dev_priv;

void *host_priv;

/* subdev device node */

struct video_device *devnode;

};

每个子设备驱动都需要实现一个v4l2_subdev结构体，v4l2_subdev可以内嵌到其它结构体中，也可以独立使用。结构体中包含了对子设备操作的成员v4l2_subdev_ops和v4l2_subdev_internal_ops。

struct v4l2_subdev_ops {

const struct v4l2_subdev_core_ops *core; //视频设备通用的操作：初始化、加载FW、上电和RESET等

const struct v4l2_subdev_tuner_ops *tuner; //tuner特有的操作

const struct v4l2_subdev_audio_ops *audio; //audio特有的操作

const struct v4l2_subdev_video_ops *video; //视频设备的特有操作：设置帧率、裁剪图像、开关视频流等

const struct v4l2_subdev_vbi_ops *vbi;

const struct v4l2_subdev_ir_ops *ir;

const struct v4l2_subdev_sensor_ops *sensor;

const struct v4l2_subdev_pad_ops *pad;

};

视频设备通常需要实现core和video成员，这两个OPS中的操作都是可选的，但是对于视频流设备video->s_stream(开启或关闭流IO)必须要实现。

struct v4l2_subdev_internal_ops {

int (*registered)(struct v4l2_subdev *sd); //当subdev注册时被调用，读取IC的ID来进行识别

void (*unregistered)(struct v4l2_subdev *sd);

int (*open)(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh); //当设备节点被打开时调用，通常会给设备上电和设置视频捕捉FMT（帧率）

int (*close)(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh);

};

v4l2_subdev_internal_ops是向V4L2框架提供的接口，只能被V4L2框架层调用。在注册或打开子设备时，进行一些辅助性操作

3、Subdev的注册和注销

当我们把v4l2_subdev需要实现的成员都已经实现，就可以调用以下函数把子设备注册到V4L2核心层：

int v4l2_device_register_subdev(struct v4l2_device*v4l2_dev, struct v4l2_subdev *sd)

当卸载子设备时，可以调用以下函数进行注销：

void v4l2_device_unregister_subdev(struct v4l2_subdev*sd)

4、video_device结构体用于在/dev目录下生成设备节点文件，把操作设备的接口暴露给用户空间。

file: kernel include media v4l2-dev.h

kernel include media v4l2-core v4l2-dev.c

struct video_device

{

#if defined(CONFIG_MEDIA_CONTROLLER)

struct media_entity entity;

#endif

/* device ops */

const struct v4l2_file_operations *fops; //V4L2设备操作集合

/* sysfs */

struct device dev; /* v4l device */

struct cdev *cdev; /* character device */ //字符设备

/* Set either parent or v4l2_dev if your driver uses v4l2_device */

struct device *parent; /* device parent */

struct v4l2_device *v4l2_dev; /* v4l2_device parent */

/* Control handler associated with this device node. May be NULL. */

struct v4l2_ctrl_handler *ctrl_handler;

/* vb2_queue associated with this device node. May be NULL. */

struct vb2_queue *queue; /* 指向video buffer队列*/

/* Priority state. If NULL, then v4l2_dev->prio will be used. */

struct v4l2_prio_state *prio;

/* device info */

char name[32];

int vfl_type; /* device type */

int vfl_dir; /* receiver, transmitter or m2m */

/* 'minor' is set to -1 if the registration failed */

int minor; //次设备号

u16 num;

/* use bitops to set/clear/test flags */

unsigned long flags;

/* attribute to differentiate multiple indices on one physical device */

int index;

/* V4L2 file handles */

spinlock_t fh_lock; /* Lock for all v4l2_fhs */

struct list_head fh_list; /* List of struct v4l2_fh */

int debug; /* Activates debug level*/

/* Video standard vars */

v4l2_std_id tvnorms; /* Supported tv norms */

v4l2_std_id current_norm; /* Current tvnorm */

/* callbacks */

void (*release)(struct video_device *vdev);

/* ioctl callbacks */

const struct v4l2_ioctl_ops *ioctl_ops; /*ioctl回调函数集，提供file_operations中的ioctl调用 */

DECLARE_BITMAP(valid_ioctls, BASE_VIDIOC_PRIVATE);

/* serialization lock */

DECLARE_BITMAP(disable_locking, BASE_VIDIOC_PRIVATE);

struct mutex *lock;

};

Video_device分配和释放，用于分配和释放video_device结构体：

struct video_device *video_device_alloc(void)

void video_device_release(struct video_device *vdev)

video_device注册和注销，实现video_device结构体的相关成员后，就可以调用下面的接口进行注册：

int __video_register_device(struct video_device *vdev, int type, int nr,

int warn_if_nr_in_use, struct module *owner)

void video_unregister_device(struct video_device*vdev);

vdev：需要注册和注销的video_device；

type：设备类型，包括VFL_TYPE_GRABBER、VFL_TYPE_VBI、VFL_TYPE_RADIO和VFL_TYPE_SUBDEV。

nr：设备节点名编号，如/dev/video[nr]。

5、v4l2_fh是用来保存子设备的特有操作方法，也就是下面要分析到的v4l2_ctrl_handler，内核提供一组v4l2_fh的操作方法，通常在打开设备节点时进行v4l2_fh注册。

file: kernel include media v4l2-fh.h

kernel include media v4l2-core v4l2-fh.c

初始化v4l2_fh，添加v4l2_ctrl_handler到v4l2_fh：

void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev)

添加v4l2_fh到video_device，方便核心层调用到：

void v4l2_fh_add(struct v4l2_fh *fh)

6、v4l2_ctrl_handler是用于保存子设备控制方法集的结构体，对于视频设备这些ctrls包括设置亮度、饱和度、对比度和清晰度等，用链表的方式来保存ctrls，可以通过v4l2_ctrl_new_std函数向链表添加ctrls。

file: kernel include media v4l2-ctrls.h

kernel include media v4l2-core v4l2-ctrls.c

struct v4l2_ctrl *v4l2_ctrl_new_std(struct v4l2_ctrl_handler *hdl,

const struct v4l2_ctrl_ops *ops,

u32 id, s32 min, s32 max, u32 step, s32 def)

hdl是初始化好的v4l2_ctrl_handler结构体；

ops是v4l2_ctrl_ops结构体，包含ctrls的具体实现；

id是通过IOCTL的arg参数传过来的指令，定义在v4l2-controls.h文件；

min、max用来定义某操作对象的范围。如：

v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BRIGHTNESS,-208, 127, 1, 0);

用户空间可以通过ioctl的VIDIOC_S_CTRL指令调用到v4l2_ctrl_handler，id透过arg参数传递。

二、ioctl框架

你可能观察到用户空间对V4L2设备的操作基本都是ioctl来实现的，V4L2设备都有大量可操作的功能(配置寄存器)，所以V4L2的ioctl也是十分庞大的。它是一个怎样的框架，是怎么实现的呢？

file: kernel include media v4l2-ioctl.h v4l2-ctrls.h

kernel include media v4l2-core v4l2-ioctl.c v4l2-ctrls.c

Ioctl框架是由v4l2_ioctl.c文件实现，文件中定义结构体数组v4l2_ioctls，可以看做是ioctl指令和回调函数的关系表。用户空间调用系统调用ioctl，传递下来ioctl指令，然后通过查找此关系表找到对应回调函数。

以下是截取数组的两项：

IOCTL_INFO_FNC(VIDIOC_QUERYBUF, v4l_querybuf,v4l_print_buffer, INFO_FL_QUEUE | INFO_FL_CLEAR(v4l2_buffer, length)),

IOCTL_INFO_STD(VIDIOC_G_FBUF, vidioc_g_fbuf,v4l_print_framebuffer, 0),

内核提供两个宏(IOCTL_INFO_FNC和IOCTL_INFO_STD)来初始化结构体，参数依次是ioctl指令、回调函数或者v4l2_ioctl_ops结构体成员、debug函数、flag。如果回调函数是v4l2_ioctl_ops结构体成员，则使用IOCTL_INFO_STD；如果回调函数是v4l2_ioctl.c自己实现的，则使用IOCTL_INFO_FNC。

IOCTL调用的流程图如下：

用户空间通过打开/dev/目录下的设备节点，获取到文件的file结构体，通过系统调用ioctl把cmd和arg传入到内核。通过一系列的调用后最终会调用到__video_do_ioctl函数，然后通过cmd检索v4l2_ioctls[]，判断是INFO_FL_STD还是INFO_FL_FUNC。如果是INFO_FL_STD会直接调用到视频设备驱动中video_device->v4l2_ioctl_ops函数集。如果是INFO_FL_FUNC会先调用到v4l2自己实现的标准回调函数，然后根据arg再调用到video_device->v4l2_ioctl_ops或v4l2_fh->v4l2_ctrl_handler函数集。

三、IO访问