Rockchip_Developer_Guide_RGA_CN-github
概念
RGA (Raster Graphic Acceleration Unit)是一个独立的2D硬件加速器，可用于加速点/线绘制，执行图像缩放、旋转、bitBlt、alpha混合等常见的2D图形操作。
版本
支持库librga.so
查看支持库的版本
$ adb shell strings vendor/lib64/librga.so | grep rga_api
rga_api version 1.10.1_[0]
源码
hardware/rockchip/librga$ tree -L 1
├── Android.bp
├── Android.go
├── Android.mk
├── buildroot.cmake
├── CHANGELOG.md
├── cmake-android.sh
├── cmake-linux.sh
├── CMakeLists.txt
├── COPYING
├── core
├── cross
├── debian
├── docs
├── im2d_api
├── include
├── meson.build
├── meson_options.txt
├── meson.sh
├── README.md
├── samples
└── version.h.template
驱动
源码
kernel-5.10/drivers/video/rockchip$ tree -L 1
├── built-in.a
├── dvbm
├── iep
├── Kconfig
├── Makefile
├── modules.order
├── mpp
├── rga
├── rga2
├── rga3
└── rve
查看驱动版本
$ adb shell cat /sys/kernel/debug/rkrga/driver_version
RGA multicore Device Driver: v1.3.3
rk3588_s:/ $ cat proc/rkrga/driver_version
RGA multicore Device Driver: v1.3.3
rk3588_s:/ $ cat proc/rkrga/load
num of scheduler = 3
================= load ==================
scheduler[0]: rga3
load = 0%
-----------------------------------
scheduler[1]: rga3
load = 0%
-----------------------------------
scheduler[2]: rga2
load = 0%
-----------------------------------
rk3588_s:/ $ cat proc/rkrga/hardware
===================================
rga3, core 1: version: 3.0.76831
input range: 68x2 ~ 8176x8176
output range: 68x2 ~ 8128x8128
scale limit: 1/8 ~ 8
byte_stride_align: 16
max_byte_stride: 32768
csc: RGB2YUV 0xf YUV2RGB 0xf
feature: 0x4
mmu: RK_IOMMU
-----------------------------------
rga3, core 2: version: 3.0.76831
input range: 68x2 ~ 8176x8176
output range: 68x2 ~ 8128x8128
scale limit: 1/8 ~ 8
byte_stride_align: 16
max_byte_stride: 32768
csc: RGB2YUV 0xf YUV2RGB 0xf
feature: 0x4
mmu: RK_IOMMU
-----------------------------------
rga2, core 4: version: 3.2.63318
input range: 2x2 ~ 8192x8192
output range: 2x2 ~ 4096x4096
scale limit: 1/16 ~ 16
byte_stride_align: 4
max_byte_stride: 32768
csc: RGB2YUV 0x3 YUV2RGB 0x7
feature: 0x205f
mmu: RGA_MMU
-----------------------------------
版本对应关系
使用RGA时需要确认保证当前的运行环境是可以正常工作的，下表为常用的librga与驱动版本对应关系。
| librga版本 | 对应驱动 | 硬件支持 |
|---|---|---|
| 无版本号 | 对应SDK内驱动 | RGA1、RGA2 |
| 1.0.0 ~ 1.3.2 | RGA Device Driver（kernel - 4.4及以上） RGA2 Device Driver（无版本号或v2.1.0） |
RGA1、RGA2 |
| > 1.4.0 | RGA multicore Device Driver（v1.2.0及以上） | RGA2、RGA3 |
| > 1.9.0 | RGA Device Driver（kernel-4.4及以上） RGA2 Device Driver（无版本号和v2.1.0） RGA multicore Device Driver（v1.2.0及以上） |
RGA1、RGA2、RGA3 |
单独升级RGA驱动后的报错日志
内核日志
[ 4.134104] [ T1] rga3 fdb60000.rga: Adding to iommu group 2
[ 4.134930] [ T1] rga3 fdb60000.rga: probe successfully, irq = 35, hw_version:3.0.76831
[ 4.135103] [ T1] rga3 fdb70000.rga: Adding to iommu group 3
[ 4.135531] [ T1] rga3 fdb70000.rga: probe successfully, irq = 36, hw_version:3.0.76831
[ 4.136448] [ T1] rga2 fdb80000.rga: probe successfully, irq = 37, hw_version:3.2.63318
[ 4.136864] [ T1] rga: Module initialized. v1.3.3
[ 3739.834327] [ T2050] rga_dma_buf: Fail to get dma_buf from fd[111], ret[-9]
[ 3739.834339] [ T2050] rga_mm: rga_mm_map_dma_buffer core[1] map dma buffer error!
[ 3739.834342] [ T2050] rga_mm: rga_mm_map_buffer map dma_buf error!
[ 3739.834345] [ T2050] rga: buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
[ 3739.856581] [ T2050] rga_dma_buf: Fail to get dma_buf from fd[111], ret[-9]
[ 3739.856595] [ T2050] rga_mm: rga_mm_map_dma_buffer core[1] map dma buffer error!
[ 3739.856598] [ T2050] rga_mm: rga_mm_map_buffer map dma_buf error!
[ 3739.856602] [ T2050] rga: buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
[ 3739.870378] [ T2050] rga_dma_buf: Fail to get dma_buf from fd[111], ret[-22]
[ 3739.870391] [ T2050] rga_mm: rga_mm_map_dma_buffer core[1] map dma buffer error!
[ 3739.870394] [ T2050] rga_mm: rga_mm_map_buffer map dma_buf error!
[ 3739.870397] [ T2050] rga: buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
[ 3739.883184] [ T2050] rga_dma_buf: Fail to get dma_buf from fd[111], ret[-22]
[ 3739.883199] [ T2050] rga_mm: rga_mm_map_dma_buffer core[1] map dma buffer error!
[ 3739.883201] [ T2050] rga_mm: rga_mm_map_buffer map dma_buf error!
[ 3739.883204] [ T2050] rga: buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
[ 3739.894799] [ T2050] rga_dma_buf: Fail to get dma_buf from fd[111], ret[-22]
[ 3739.894813] [ T2050] rga_mm: rga_mm_map_dma_buffer core[1] map dma buffer error!
[ 3739.894817] [ T2050] rga_mm: rga_mm_map_buffer map dma_buf error!
[ 3739.894820] [ T2050] rga: buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
安卓日志
rk3588_s:/ $ logcat --pid 2014
09-19 11:42:48.373 2014 2050 E YOLO_V8_TRACKING_JNI: postprocess release! ret=45500
09-19 11:42:48.373 2014 2050 D YOLO_V8_TRACK_JNI: Elapsed time ---right: 12794251 ns
09-19 11:42:48.375 2014 2050 D YOLO_V8_TRACK_JNI: size of object_size : 3
09-19 11:42:48.378 2014 2050 D RGATestAct: test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":395,"y1":116,"y2":198},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":156},{"cam_id":null,"track_id":3,"x1":206,"x2":260,"y1":155,"y2":200}],"camera_id":null,"frame_id":140515360,"frame_ts":0}---23
09-19 11:42:48.391 2014 2050 E YOLO_V8_TRACKING_JNI: postprocess release! ret=37917
09-19 11:42:48.391 2014 2050 D YOLO_V8_TRACK_JNI: Elapsed time ---right: 7180543 ns
09-19 11:42:48.391 2014 2050 D YOLO_V8_TRACK_JNI: size of object_size : 3
09-19 11:42:48.392 2014 2050 D RGATestAct: test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":395,"y1":116,"y2":198},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":156},{"cam_id":null,"track_id":3,"x1":206,"x2":260,"y1":155,"y2":200}],"camera_id":null,"frame_id":140515388,"frame_ts":0}---11
09-19 11:42:48.405 2014 2050 E YOLO_V8_TRACKING_JNI: postprocess release! ret=37042
09-19 11:42:48.405 2014 2050 D YOLO_V8_TRACK_JNI: Elapsed time ---right: 7378292 ns
09-19 11:42:48.405 2014 2050 D YOLO_V8_TRACK_JNI: size of object_size : 3
09-19 11:42:48.405 2014 2050 D RGATestAct: test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":395,"y1":116,"y2":198},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":156},{"cam_id":null,"track_id":3,"x1":206,"x2":260,"y1":155,"y2":200}],"camera_id":null,"frame_id":140515402,"frame_ts":0}---11
09-19 11:42:48.417 2014 2050 E YOLO_V8_TRACKING_JNI: postprocess release! ret=37041
09-19 11:42:48.418 2014 2050 D YOLO_V8_TRACK_JNI: Elapsed time ---right: 7123667 ns
09-19 11:42:48.418 2014 2050 D YOLO_V8_TRACK_JNI: size of object_size : 3
09-19 11:42:48.418 2014 2050 D RGATestAct: test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":395,"y1":116,"y2":198},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":156},{"cam_id":null,"track_id":3,"x1":206,"x2":260,"y1":155,"y2":200}],"camera_id":null,"frame_id":140515415,"frame_ts":0}---11
09-19 11:42:48.430 2014 2050 E YOLO_V8_TRACKING_JNI: postprocess release! ret=37917
09-19 11:42:48.430 2014 2050 D YOLO_V8_TRACK_JNI: Elapsed time ---right: 7956376 ns
09-19 11:42:48.430 2014 2050 D YOLO_V8_TRACK_JNI: size of object_size : 3
09-19 11:42:48.431 2014 2050 D RGATestAct: test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":395,"y1":116,"y2":198},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":156},{"cam_id":null,"track_id":3,"x1":206,"x2":260,"y1":155,"y2":200}],"camera_id":null,"frame_id":140515428,"frame_ts":0}---10
源码跟踪
日志分析
#define EBADF 9 /* Bad file number 错误的文件数量 */
#define EINVAL 22 /* Invalid argument 无效参数 */
kernel-5.10/drivers/video/rockchip/rga3/rga_dma_buf.c
int rga_dma_map_fd(int fd, struct rga_dma_buffer *rga_dma_buffer,
enum dma_data_direction dir, struct device *rga_dev)
{
/**********************************************************************/
dma_buf = dma_buf_get(fd); //lixiaogang
/**********************************************************************/
if (IS_ERR(dma_buf)) {
ret = PTR_ERR(dma_buf);
pr_err("Fail to get dma_buf from fd[%d], ret[%d]\n", fd, ret);
return ret;
}
}
kernel-5.10/drivers/dma-buf/dma-buf.c
/**
* dma_buf_get - returns the dma_buf structure related to an fd
* @fd: [in] fd associated with the dma_buf to be returned
*
* On success, returns the dma_buf structure associated with an fd; uses
* file's refcounting done by fget to increase refcount. returns ERR_PTR
* otherwise.
*/
struct dma_buf *dma_buf_get(int fd)
{
struct file *file;
file = fget(fd);
/**********************************************************************/
// 报错位置: rga_dma_buf: Fail to get dma_buf from fd[111], ret[-9]
if (!file)
return ERR_PTR(-EBADF); // lixiaogang
if (!is_dma_buf_file(file)) {
fput(file);
return ERR_PTR(-EINVAL);
}
return file->private_data;
}
EXPORT_SYMBOL_GPL(dma_buf_get);
kernel-5.10/drivers/video/rockchip/rga3/rga_mm.c
static int rga_mm_map_dma_buffer(struct rga_external_buffer *external_buffer,
struct rga_internal_buffer *internal_buffer,
struct rga_job *job)
{
switch (external_buffer->type) {
case RGA_DMA_BUFFER:
/**********************************************************************/
// lixiaogang
ret = rga_dma_map_fd((int)external_buffer->memory,
buffer, DMA_BIDIRECTIONAL,
map_dev);
break;
case RGA_DMA_BUFFER_PTR:
ret = rga_dma_map_buf((struct dma_buf *)u64_to_user_ptr(external_buffer->memory),
buffer, DMA_BIDIRECTIONAL,
map_dev);
break;
default:
ret = -EFAULT;
break;
}
}
static int rga_mm_map_buffer(struct rga_external_buffer *external_buffer,
struct rga_internal_buffer *internal_buffer,
struct rga_job *job, int write_flag)
{
int ret;
memcpy(&internal_buffer->memory_parm, &external_buffer->memory_parm,
sizeof(internal_buffer->memory_parm));
switch (external_buffer->type) {
case RGA_DMA_BUFFER:
case RGA_DMA_BUFFER_PTR:
internal_buffer->type = external_buffer->type;
/**********************************************************************/
ret = rga_mm_map_dma_buffer(external_buffer, internal_buffer, job); // lixiaogang
if (ret < 0) {
pr_err("%s map dma_buf error!\n", __func__);
return ret;
}
internal_buffer->size = internal_buffer->dma_buffer->size -
internal_buffer->dma_buffer->offset;
internal_buffer->mm_flag |= RGA_MEM_NEED_USE_IOMMU;
break;
case RGA_VIRTUAL_ADDRESS:
internal_buffer->type = RGA_VIRTUAL_ADDRESS;
ret = rga_mm_map_virt_addr(external_buffer, internal_buffer, job, write_flag);
if (ret < 0) {
pr_err("%s map virtual address error!\n", __func__);
return ret;
}
internal_buffer->size = internal_buffer->virt_addr->size -
internal_buffer->virt_addr->offset;
internal_buffer->mm_flag |= RGA_MEM_NEED_USE_IOMMU;
break;
case RGA_PHYSICAL_ADDRESS:
internal_buffer->type = RGA_PHYSICAL_ADDRESS;
ret = rga_mm_map_phys_addr(external_buffer, internal_buffer, job);
if (ret < 0) {
pr_err("%s map physical address error!\n", __func__);
return ret;
}
internal_buffer->mm_flag |= RGA_MEM_NEED_USE_IOMMU;
break;
default:
pr_err("Illegal external buffer!\n");
return -EFAULT;
}
return 0;
}
/*
* rga_mm_import_buffer - Importing external buffer into the RGA driver
*
* @external_buffer: [in] Parameters of external buffer
* @session: [in] Session of the current process
*
* returns:
* if return value > 0, the buffer import is successful and is the generated
* buffer-handle, negative error code on failure.
*/
int rga_mm_import_buffer(struct rga_external_buffer *external_buffer,
struct rga_session *session)
{
/**********************************************************************/
ret = rga_mm_map_buffer(external_buffer, internal_buffer, NULL, true); // lixiaogang
if (ret < 0)
goto FREE_INTERNAL_BUFFER;
}
kernel-5.10/drivers/video/rockchip/rga3/rga_drv.c
static long rga_ioctl_import_buffer(unsigned long arg, struct rga_session *session)
{
for (i = 0; i < buffer_pool.size; i++) {
if (DEBUGGER_EN(MSG)) {
pr_info("import buffer info:\n");
rga_dump_external_buffer(&external_buffer[i]);
}
/**********************************************************************/
ret = rga_mm_import_buffer(&external_buffer[i], session); //lixiaogang
/**********************************************************************/
if (ret <= 0) {
pr_err("buffer[%d] mm import buffer failed! memory = 0x%lx, type = %s(0x%x)\n",
i, (unsigned long)external_buffer[i].memory,
rga_get_memory_type_str(external_buffer[i].type),
external_buffer[i].type);
goto err_free_external_buffer;
}
external_buffer[i].handle = ret;
}
}
static long rga_ioctl(struct file *file, uint32_t cmd, unsigned long arg)
{
switch (cmd) {
case RGA_BLIT_SYNC:
case RGA_BLIT_ASYNC:
break;
case RGA_CACHE_FLUSH:
case RGA_FLUSH:
case RGA_GET_RESULT:
break;
case RGA_GET_VERSION:
break;
case RGA_IOC_GET_HW_VERSION:
break;
case RGA_IOC_GET_DRVIER_VERSION:
break;
case RGA_IOC_IMPORT_BUFFER:
rga_power_enable_all();
/**********************************************************************/
ret = rga_ioctl_import_buffer(arg, session); // lixiaogang
/**********************************************************************/
rga_power_disable_all();
break;
case RGA_IOC_RELEASE_BUFFER:
rga_power_enable_all();
ret = rga_ioctl_release_buffer(arg);
rga_power_disable_all();
break;
}
return ret;
}
const struct file_operations rga_fops = {
.owner = THIS_MODULE,
.open = rga_open,
.release = rga_release,
.unlocked_ioctl = rga_ioctl,
#ifdef CONFIG_COMPAT
.compat_ioctl = rga_ioctl,
#endif
};
librga 跟踪
hardware/rockchip/librga/im2d_api/src/im2d_impl.cpp
IM_API IM_STATUS rga_import_buffers(struct rga_buffer_pool *buffer_pool) {
int ret = 0;
ret = rga_get_context();
if (ret != IM_STATUS_SUCCESS)
return (IM_STATUS)ret;
if (buffer_pool == NULL) {
IM_LOGW("buffer pool is null!");
return IM_STATUS_FAILED;
}
ret = ioctl(rgaCtx->rgaFd, RGA_IOC_IMPORT_BUFFER, buffer_pool);
if (ret < 0) {
IM_LOGW("RGA_IOC_IMPORT_BUFFER fail! %s", strerror(errno));
return IM_STATUS_FAILED;
}
return IM_STATUS_SUCCESS;
}
IM_API rga_buffer_handle_t rga_import_buffer(uint64_t memory, int type, uint32_t size) {
struct rga_buffer_pool buffer_pool;
struct rga_external_buffer buffers[1];
memset(&buffer_pool, 0x0, sizeof(buffer_pool));
memset(buffers, 0x0, sizeof(buffers));
buffers[0].type = type;
buffers[0].memory = memory;
buffers[0].memory_info.size = size;
buffer_pool.buffers = ptr_to_u64(buffers);
buffer_pool.size = 1;
if (rga_import_buffers(&buffer_pool) != IM_STATUS_SUCCESS)
return 0;
return buffers[0].handle;
}
IM_API rga_buffer_handle_t rga_import_buffer(uint64_t memory, int type, im_handle_param_t *param) {
int format;
struct rga_buffer_pool buffer_pool;
struct rga_external_buffer buffers[1];
memset(&buffer_pool, 0x0, sizeof(buffer_pool));
memset(buffers, 0x0, sizeof(buffers));
buffers[0].type = type;
buffers[0].memory = memory;
memcpy(&buffers[0].memory_info, param, sizeof(*param));
format = convert_to_rga_format(buffers[0].memory_info.format);
if (format == RK_FORMAT_UNKNOWN) {
IM_LOGW("Invaild format [0x%x]!\n", buffers[0].memory_info.format);
return IM_STATUS_NOT_SUPPORTED;
}
buffers[0].memory_info.format = format >> 8;
buffer_pool.buffers = ptr_to_u64(buffers);
buffer_pool.size = 1;
if (rga_import_buffers(&buffer_pool) != IM_STATUS_SUCCESS)
return 0;
return buffers[0].handle;
}
App 调用部分
JAVA
package com.afei.sorting
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.os.Bundle
import android.util.Log
import androidx.appcompat.app.AppCompatActivity
import com.afei.sorting.util.MatUtils
import com.blankj.utilcode.util.GsonUtils
import com.xintian.fenJian.Tracking
import com.xintian.fenJian.Utils
import org.opencv.android.OpenCVLoader
import java.io.IOException
import kotlin.concurrent.thread
class RGATestAct : AppCompatActivity(){
private lateinit var tracking1: Tracking
private lateinit var tracking2: Tracking
private lateinit var trackingTrigger: Tracking
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
initModel()
}
private fun initModel() {
OpenCVLoader.initDebug()
tracking1 = Tracking()
val copyPath1 = Utils.modelCopy(R.raw.yolov8, "yolov8.rknn")
tracking1.initModel(copyPath1)
tracking2 = Tracking()
val copyPath2 = Utils.modelCopy(R.raw.yolov8, "yolov8.rknn")
tracking2.initModel(copyPath2)
trackingTrigger = Tracking()
val copyPath3 = Utils.modelCopy(R.raw.trigger, "trigger.rknn")
trackingTrigger.detModelInit(copyPath3)
test()
}
private fun test() {
thread {
try {
val processes = assets.list("det")
if (processes != null) {
for (fileNameAngSub in processes) {
val bitmap = getBitmapFromAsset("det/$fileNameAngSub")
if (bitmap != null) {
val rgbMat = MatUtils.bitmapToRGBMat(bitmap)
val ss = System.currentTimeMillis()
val frame = tracking1.tracking(
rgbMat.nativeObjAddr,
System.currentTimeMillis(),
0,
1,
0,
0,
400,
200
)
val ee = System.currentTimeMillis()
Log.d("RGATestAct", "test: ${GsonUtils.toJson(frame)}---${ee-ss}")
}
}
}
} catch (e: Exception) {
e.printStackTrace()
}
}
}
private fun getBitmapFromAsset(filePath: String): Bitmap? {
var bitmap: Bitmap? = null
try {
val inputStream = assets.open(filePath)
bitmap = BitmapFactory.decodeStream(inputStream)
} catch (e: IOException) {
e.printStackTrace()
}
return bitmap
}
}
MatUtils
/**
* @param srcBitmap
* @return
*/
public static Mat bitmapToRGBMat(Bitmap srcBitmap) {
Mat mat = new Mat();
Utils.bitmapToMat(srcBitmap, mat);
Mat dst = new Mat();
Imgproc.cvtColor(mat, dst, Imgproc.COLOR_RGBA2RGB);
mat.release();
return dst;
}
Yolo
package com.xintian.fenJian
class Tracking {
companion object {
init {
System.loadLibrary("rknn_yolov8_demo")
}
}
fun initModel(path: String): Int {
return modelInit(path)
}
fun tracking(matAddr: Long, frameId: Long, left: Int, showBox:Int, x: Int, y: Int, w: Int, h: Int): Frame {
return track(matAddr, frameId,left,showBox,x, y, w, h)
}
private external fun modelInit(path: String): Int
/*************************************lixiaogang**************************************/
private external fun track(matAddr: Long, frameId: Long, left: Int,showBox:Int, x: Int, y: Int, w: Int, h: Int): Frame
external fun detModelInit(path: String): Int
external fun detect(matAddr: Long, frameId: Long, left: Int,showBox:Int, x: Int, y: Int, w: Int, h: Int): Frame
}
./examples/yolov8multi/cpp/src/main.cc
extern "C"
JNIEXPORT jobject
JNICALL
Java_com_xintian_fenJian_Tracking_track(JNIEnv *env, jobject thiz, jlong mat_addr, jint frame_id, jint is_left,jint show_box,
jint roi_x, jint roi_y, jint roi_w, jint roi_h) {
if (is_left) {
/************************************lixiaogang**************************************/
ret = inference_yolov8_model(&rknn_app_ctx, &src_image, &od_results);
} else {
ret = inference_yolov8_model(&rknn_app_ctx_right, &src_image, &od_results);
}
return frame;
}
./examples/yolov8multi/cpp/src/yolov8.cc
int inference_yolov8_model(rknn_app_context_t *app_ctx, image_buffer_t *img, object_detect_result_list *od_results)
{
// letterbox
ret = convert_image_with_letterbox(img, &dst_img, &letter_box, bg_color);
if (ret < 0)
{
LOGE("convert_image_with_letterbox fail! ret=%d\n", ret);
return -1;
}
}
./utils/image_utils.c
int convert_image_with_letterbox(image_buffer_t* src_image, image_buffer_t* dst_image, letterbox_t* letterbox, char color)
{
//--------------------------------------------------------------------------------------------------------------
// alloc memory buffer for dst image,
// remember to free
if (dst_image->virt_addr == NULL && dst_image->fd <= 0) {
int dst_size = get_image_size(dst_image);
dst_image->virt_addr = (uint8_t *)malloc(dst_size);
if (dst_image->virt_addr == NULL) {
printf("malloc size %d error\n", dst_size);
return -1;
}
}
ret = convert_image(src_image, dst_image, &src_box, &dst_box, color);
return ret;
}
int convert_image(image_buffer_t* src_img, image_buffer_t* dst_img, image_rect_t* src_box, image_rect_t* dst_box, char color)
{
int ret;
printf("src width=%d height=%d fmt=0x%x virAddr=0x%p fd=%d\n",
src_img->width, src_img->height, src_img->format, src_img->virt_addr, src_img->fd);
printf("dst width=%d height=%d fmt=0x%x virAddr=0x%p fd=%d\n",
dst_img->width, dst_img->height, dst_img->format, dst_img->virt_addr, dst_img->fd);
if (src_box != NULL) {
printf("src_box=(%d %d %d %d)\n", src_box->left, src_box->top, src_box->right, src_box->bottom);
}
if (dst_box != NULL) {
printf("dst_box=(%d %d %d %d)\n", dst_box->left, dst_box->top, dst_box->right, dst_box->bottom);
}
printf("color=0x%x\n", color);
//*************************************************************************************/
ret = convert_image_rga(src_img, dst_img, src_box, dst_box, color);
if (ret != 0) {
printf("try convert image use cpu\n");
ret = convert_image_cpu(src_img, dst_img, src_box, dst_box, color);
}
return ret;
}
static int convert_image_rga(image_buffer_t* src_img, image_buffer_t* dst_img, image_rect_t* src_box, image_rect_t* dst_box, char color)
{
if (use_handle) {
if (src_phy != NULL) {
rga_handle_src = importbuffer_physicaladdr((uint64_t)src_phy, &in_param);
} else if (src_fd > 0) {
/***************************************lixiaogang****************************************/
rga_handle_src = importbuffer_fd(src_fd, &in_param);
/*******************************************************************************/
} else {
rga_handle_src = importbuffer_virtualaddr(src, &in_param);
}
if (rga_handle_src <= 0) {
printf("src handle error %d\n", rga_handle_src);
ret = -1;
goto err;
}
rga_buf_src = wrapbuffer_handle(rga_handle_src, srcWidth, srcHeight, srcFmt, srcWidth, srcHeight);
} else {
if (src_phy != NULL) {
rga_buf_src = wrapbuffer_physicaladdr(src_phy, srcWidth, srcHeight, srcFmt, srcWidth, srcHeight);
} else if (src_fd > 0) {
rga_buf_src = wrapbuffer_fd(src_fd, srcWidth, srcHeight, srcFmt, srcWidth, srcHeight);
} else {
rga_buf_src = wrapbuffer_virtualaddr(src, srcWidth, srcHeight, srcFmt, srcWidth, srcHeight);
}
}
if (use_handle) {
if (dst_phy != NULL) {
rga_handle_dst = importbuffer_physicaladdr((uint64_t)dst_phy, &dst_param);
} else if (dst_fd > 0) {
rga_handle_dst = importbuffer_fd(dst_fd, &dst_param);
} else {
rga_handle_dst = importbuffer_virtualaddr(dst, &dst_param);
}
if (rga_handle_dst <= 0) {
printf("dst handle error %d\n", rga_handle_dst);
ret = -1;
goto err;
}
rga_buf_dst = wrapbuffer_handle(rga_handle_dst, dstWidth, dstHeight, dstFmt, dstWidth, dstHeight);
} else {
if (dst_phy != NULL) {
rga_buf_dst = wrapbuffer_physicaladdr(dst_phy, dstWidth, dstHeight, dstFmt, dstWidth, dstHeight);
} else if (dst_fd > 0) {
rga_buf_dst = wrapbuffer_fd(dst_fd, dstWidth, dstHeight, dstFmt, dstWidth, dstHeight);
} else {
rga_buf_dst = wrapbuffer_virtualaddr(dst, dstWidth, dstHeight, dstFmt, dstWidth, dstHeight);
}
}
/* Configure the current thread to use only RGA3_core0 or RGA3_core1. */
imconfig(IM_CONFIG_SCHEDULER_CORE, IM_SCHEDULER_RGA3_CORE0 | IM_SCHEDULER_RGA3_CORE1); //lixiaogang add
}
./librga/im2d_api/im2d_buffer.h
/**
* Import external buffers into RGA driver.
*
* @param fd/va/pa
* Select dma_fd/virtual_address/physical_address by buffer type
* @param param
* Configure buffer parameters
*
* @return rga_buffer_handle_t
*/
IM_EXPORT_API rga_buffer_handle_t importbuffer_fd(int fd, im_handle_param_t *param);
IM_EXPORT_API rga_buffer_handle_t importbuffer_virtualaddr(void *va, im_handle_param_t *param);
IM_EXPORT_API rga_buffer_handle_t importbuffer_physicaladdr(uint64_t pa, im_handle_param_t *param);
./librga/im2d_api/src/im2d.cpp
IM_API rga_buffer_handle_t importbuffer_fd(int fd, int size) {
return rga_import_buffer((uint64_t)fd, RGA_DMA_BUFFER, (uint32_t)size);
}
IM_API rga_buffer_handle_t importbuffer_fd(int fd, im_handle_param_t *param) {
return rga_import_buffer((uint64_t)fd, RGA_DMA_BUFFER, param);
}
IM_API rga_buffer_handle_t importbuffer_fd(int fd, int width, int height, int format) {
im_handle_param_t param = {(uint32_t)width, (uint32_t)height, (uint32_t)format};
return rga_import_buffer((uint64_t)fd, RGA_DMA_BUFFER, ¶m);
}
参考资料
airockchip/librga/issues/73-github
猜测是高负载下RAM不足，buffer被分配到了swap交换分区，因此不满足RGA2只能用dma32的4GB限制，所以报错。
要么修改程序来用dma-heap/drm gem的dma32模式来分配buffer，要么修改配置图像任务的参数，使得其满足RGA3的限制（比如1/8~8缩放倍率)，然后用core flag来强制使用RGA3。后者只适用于rk3588(s)。
RGA3 修改方案
utils/image_utils.c
/* Configure the current thread to use only RGA3_core0 or RGA3_core1. */
imconfig(IM_CONFIG_SCHEDULER_CORE, IM_SCHEDULER_RGA3_CORE0 | IM_SCHEDULER_RGA3_CORE1); // lixiaoganng add
修改后的报错日志
2024-09-20 14:14:43.937 2015-2053 YOLO_V8_TRACK_JNI com.afei.sorting D RGA inference_yolov8_model right
2024-09-20 14:14:43.937 2015-2053 YOLO_V8_TRACKING_JNI com.afei.sorting E rga convert_image_with_letterbox start!
2024-09-20 14:14:45.220 2053-2053 rga_dma_buf pid-2053 E Fail to get dma_buf from fd[111], ret[-9]
2024-09-20 14:14:45.220 2053-2053 rga_mm pid-2053 E rga_mm_map_dma_buffer core[1] map dma buffer error!
2024-09-20 14:14:45.220 2053-2053 rga_mm pid-2053 E rga_mm_map_buffer map dma_buf error!
2024-09-20 14:14:45.220 2053-2053 rga pid-2053 E buffer[0] mm import buffer failed! memory = 0x6f, type = dma_fd(0x0)
2024-09-20 14:14:43.968 2015-2053 RGATestAct com.afei.sorting E test: {"boxes":[{"cam_id":null,"track_id":1,"x1":328,"x2":393,"y1":122,"y2":198},{"cam_id":null,"track_id":2,"x1":113,"x2":245,"y1":0,"y2":156}],"camera_id":null,"frame_id":236030937,"frame_ts":0}---24
2024-09-20 14:14:43.972 2015-2053 YOLO_V8_TRACK_JNI com.afei.sorting D RGA inference_yolov8_model right
2024-09-20 14:14:43.972 2015-2053 YOLO_V8_TRACKING_JNI com.afei.sorting E rga convert_image_with_letterbox start!
2024-09-20 14:14:45.255 2053-2053 rga_policy pid-2053 E invalid function policy
2024-09-20 14:14:45.255 2053-2053 rga_job pid-2053 E job assign failed
2024-09-20 14:14:45.255 2053-2053 rga_job pid-2053 E failed to get scheduler, rga_job_commit(409)
2024-09-20 14:14:45.255 2053-2053 rga_job pid-2053 E request[1] task[0] job_commit failed.
2024-09-20 14:14:45.255 2053-2053 rga_job pid-2053 E rga request[1] commit failed!
2024-09-20 14:14:45.255 2053-2053 rga pid-2053 E request[1] submit failed!
2024-09-20 14:14:43.980 2015-2053 RGATestAct com.afei.sorting E test: {"boxes":[{"cam_id":null,"track_id":1,"x1":327,"x2":392,"y1":121,"y2":199},{"cam_id":null,"track_id":2,"x1":112,"x2":242,"y1":0,"y2":155}],"camera_id":null,"frame_id":236030978,"frame_ts":0}---10
