Flutter視訊渲染系列
第一章 Android使用Texture渲染視訊
第二章 Windows使用Texture渲染視訊
第三章 Linux使用Texture渲染視訊(本章)
第四章 全平台FFI+CustomPainter渲染視訊
文章目錄
- Flutter視訊渲染系列
- 前言
- 一、如何實作?
-
- 1、定義Texture控件
- 2、建立Texture對象
-
- (1)繼承FlPixelBufferTexture
- (2)注冊Texture
- 3、關聯TextureId
- 4、寫入rgba
- 二、示例
-
- 1.使用ffmpeg解碼播放
- 三、完整代碼
- 總結
前言
flutter渲染視訊的方法有多種,比如texture、platformview、ffi,其中texture是通過flutter自己提供的一個texture對象與dart界面關聯後進行渲染,很容易搜尋到android和ios的相關資料,但是Linux上卻幾乎找不到任何資料。通過檢視一些開源庫的代碼,才找到Linux上使用flutter texture的方法,在這裡做一個簡單的介紹。
一、如何實作?
1、定義Texture控件
在界面中定義一個Texture
Container(
width: 640,
height: 360,
child: Texture(
textureId: textureId,
))
2、建立Texture對象
(1)繼承FlPixelBufferTexture
此處代碼為dart_vlc源碼,因為是一個獨立對象所有可以直接拿來用,自己繼承實作也基本差不多,是以就沒必要造輪子了。
#ifndef VIDEO_OUTLET_H_
#define VIDEO_OUTLET_H_
#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
struct _VideoOutletClass {
FlPixelBufferTextureClass parent_class;
};
struct VideoOutletPrivate {
int64_t texture_id = 0;
uint8_t* buffer = nullptr;
int32_t video_width = 0;
int32_t video_height = 0;
};
G_DECLARE_DERIVABLE_TYPE(VideoOutlet, video_outlet, DART_VLC, VIDEO_OUTLET,
FlPixelBufferTexture)
G_DEFINE_TYPE_WITH_CODE(VideoOutlet, video_outlet,
fl_pixel_buffer_texture_get_type(),
G_ADD_PRIVATE(VideoOutlet))
static gboolean video_outlet_copy_pixels(FlPixelBufferTexture* texture,
const uint8_t** out_buffer,
uint32_t* width, uint32_t* height,
GError** error) {
auto video_outlet_private =
(VideoOutletPrivate*)video_outlet_get_instance_private(
DART_VLC_VIDEO_OUTLET(texture));
*out_buffer = video_outlet_private->buffer;
*width = video_outlet_private->video_width;
*height = video_outlet_private->video_height;
return TRUE;
}
static VideoOutlet* video_outlet_new() {
return DART_VLC_VIDEO_OUTLET(g_object_new(video_outlet_get_type(), nullptr));
}
static void video_outlet_class_init(VideoOutletClass* klass) {
FL_PIXEL_BUFFER_TEXTURE_CLASS(klass)->copy_pixels = video_outlet_copy_pixels;
}
static void video_outlet_init(VideoOutlet* self) {}
#endif
(2)注冊Texture
static FlTextureRegistrar* _registrar;
//建立自定義的texture對象
auto texture=video_outlet_new();
//注冊對象
fl_texture_registrar_register_texture(_registrar,FL_TEXTURE(texture));
3、關聯TextureId
dart
int textureId = -1;
if (textureId < 0) {
//調用本地方法擷取textureId
methodChannel.invokeMethod('startPreview',<String,dynamic>{'path':'test.mov'}).then((value) {
textureId = value;
setState(() {
print('textureId ==== $textureId');
});
});
}
c++
//methodchannel的startPreview方法實作,此處略
//texure指針的位址即為textureId
g_autoptr(FlValue) result = fl_value_new_int((int64_t)texture);
//設定傳回值
response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
4、寫入rgba
static FlTextureRegistrar* _registrar;
//取得自定義的内部對象
auto video_outlet_private =(VideoOutletPrivate*)video_outlet_get_instance_private(texture);
//設定視訊幀寬高
video_outlet_private->video_width = width;
video_outlet_private->video_height = height;
//設定rgba資料
video_outlet_private->buffer = data[0];
//通知渲染
fl_texture_registrar_mark_texture_frame_available(_registrar, FL_TEXTURE(texture));
注:FlTextureRegistrar的擷取方法為:
//定義TextureRegistrar對象
static FlTextureRegistrar* _registrar;
//插件注冊代碼,這裡的插件名為ffplay_plugin,此方法為官方生成代碼
void ffplay_plugin_register_with_registrar(FlPluginRegistrar* registrar) {
FfplayPlugin* plugin = FFPLAY_PLUGIN(
g_object_new(ffplay_plugin_get_type(), nullptr));
g_autoptr(FlStandardMethodCodec) codec = fl_standard_method_codec_new();
g_autoptr(FlMethodChannel) channel =
fl_method_channel_new(fl_plugin_registrar_get_messenger(registrar),
"ffplay_plugin",
FL_METHOD_CODEC(codec));
//擷取TextureRegistrar對象
_registrar=fl_plugin_registrar_get_texture_registrar(registrar);
fl_method_channel_set_method_call_handler(channel, method_call_cb,
g_object_ref(plugin),
g_object_unref);
g_object_unref(plugin);
}
二、示例
1.使用ffmpeg解碼播放
main.dart
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
MethodChannel methodChannel = MethodChannel('ffplay_plugin');
void main() {
runApp(MyApp());
}
class MyApp extends StatelessWidget {
// This widget is the root of your application.
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key? key, required this.title}) : super(key: key);
final String title;
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
int _counter = 0;
int textureId = -1;
Future<void> _createTexture() async {
print('textureId = $textureId');
//調用本地方法播放視訊
if (textureId < 0) {
methodChannel.invokeMethod('startPreview',<String,dynamic>{'path':'https://sf1-hscdn-tos.pstatp.com/obj/media-fe/xgplayer_doc_video/flv/xgplayer-demo-360p.flv'}).then((value) {
textureId = value;
setState(() {
print('textureId ==== $textureId');
});
});
}
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
//控件布局
body: Center(
child: Row(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
if (textureId > -1)
ClipRect (
child: Container(
width: 640,
height: 360,
child: Texture(
textureId: textureId,
)),
),
],
),
),
floatingActionButton: FloatingActionButton(
onPressed: _createTexture,
tooltip: 'createTexture',
child: Icon(Icons.add),
),
);
}
}
定義一個插件我這裡是fflay_plugin。
fflay_plugin.cc
相關對象的定義
static FlTextureRegistrar* _registrar;
class PlayData {
public:
//Play中封裝了ffmpeg
Play* play;
VideoOutlet* flutter_pixel_buffer;
int64_t texture_id;
};
static std::map<int64_t, PlayData*> playMap;
擷取FlTextureRegistrar對象
//插件注冊代碼,這裡的插件名為ffplay_plugin,此方法為官方生成代碼
void ffplay_plugin_register_with_registrar(FlPluginRegistrar* registrar) {
FfplayPlugin* plugin = FFPLAY_PLUGIN(
g_object_new(ffplay_plugin_get_type(), nullptr));
g_autoptr(FlStandardMethodCodec) codec = fl_standard_method_codec_new();
g_autoptr(FlMethodChannel) channel =
fl_method_channel_new(fl_plugin_registrar_get_messenger(registrar),
"ffplay_plugin",
FL_METHOD_CODEC(codec));
//擷取TextureRegistrar對象
_registrar=fl_plugin_registrar_get_texture_registrar(registrar);
fl_method_channel_set_method_call_handler(channel, method_call_cb,
g_object_ref(plugin),
g_object_unref);
g_object_unref(plugin);
}
methodChannel部分
if (strcmp(method, "startPreview") == 0){
//擷取參數
auto arguments = fl_method_call_get_args(method_call);
auto path = fl_value_get_string(fl_value_lookup_string(arguments, "path"));
//建立texture
auto texture=video_outlet_new();
//注冊texture
fl_texture_registrar_register_texture(_registrar,FL_TEXTURE(texture));
PlayData* pd = new PlayData;
//初始化播放器
pd->play = new Play;
pd->flutter_pixel_buffer=texture;
pd->texture_id = (int64_t)texture;
playMap[pd->texture_id] = pd;
//播放視訊回調
pd->play->Display = [=](unsigned char* data[8], int linesize[8], int width, int height, AVPixelFormat format) {
//設定視訊資料
auto video_outlet_private =(VideoOutletPrivate*)video_outlet_get_instance_private(pd->flutter_pixel_buffer);
video_outlet_private->video_width = width;
video_outlet_private->video_height = height;
video_outlet_private->buffer = data[0];
//通知渲染
fl_texture_registrar_mark_texture_frame_available(
_registrar, FL_TEXTURE(pd->flutter_pixel_buffer));
};
//開始播放視訊
pd->play->Start(path, AV_PIX_FMT_RGBA);
//傳回textureId
g_autoptr(FlValue) result = fl_value_new_int((int64_t)texture);
response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
效果預覽
三、完整代碼
https://download.csdn.net/download/u013113678/87096317
包含完整代碼的flutter項目,版本3.0.4、3.3.8都成功運作,需要自行安裝ffmpeg庫版本最好為4.3或5.0.1。目錄說明如下。
注:由于筆者在Ubuntu上采用靜态加載ffmpeg so庫出現了glibc沖突問題沒有解決,是以采用了動态加載so的方式,DllImportUtils的作用隻是動态加載so,具體可檢視《C++ 使用宏加載動态庫》。
總結
以上就是今天要講的内容,flutter在linux上渲染視訊,還是有點不容易的,一是缺乏相關資料,二是flutter在Linux上的本地代碼是另外一套封裝與windows完全不相同。而且采用的是c語言自定義一套面向對象規則的方式,當然編譯器是clang++我們可以使用c++的方式編碼。總的來說,flutter是可以在Linux實作視訊渲染的,如果要進一步優化則需要用gltexture或者本地視窗渲染。