Scrcpy 投屏原理解析
|
apson Li
|
1,548
|
0
|
开源库解析

1161 字
|
20 分钟
本文最后更新于 1040 天前，其中的信息可能已经有所发展或是发生改变。

Scrcpy 分客户端和服务端，比较常见的做法是把 手机部署为服务端

 adb shell CLASSPATH=/data/local/tmp/server_test.jar app_process /  com.genymobile.scrcpy.Server 1.14 debug 0 8000000   0  -1 true - false true 0 true false -

server_test.jar 请自行编译Scrcpy 源码获取，并且 adb push 到 /data/local/tmp

补充一下吧 git 下载 Scrcpy 源码之后，进入跟目录执行以下命令，就可以生成服务端 apk，后缀改为 jar 即可
meson x --buildtype release --strip -Db_lto=true

这次解析的版本是 Scrcpy 1.14 ， 必须要传递14个参数,第七个参数写 true 就是将 手机部署为服务端

 private static Options createOptions(String... args) {
        if (args.length < 1) {
            throw new IllegalArgumentException("Missing client version");
        }

        String clientVersion = args[0];
        if (!clientVersion.equals(BuildConfig.VERSION_NAME)) {
            throw new IllegalArgumentException(
                    "The server version (" + BuildConfig.VERSION_NAME + ") does not match the client " + "(" + clientVersion + ")");
        }

        final int expectedParameters = 14;
        if (args.length != expectedParameters) {
            throw new IllegalArgumentException("Expecting " + expectedParameters + " parameters");
        }

        Options options = new Options();

        Ln.Level level = Ln.Level.valueOf(args[1].toUpperCase(Locale.ENGLISH));
        options.setLogLevel(level);

        int maxSize = Integer.parseInt(args[2]) & ~7; // multiple of 8
        options.setMaxSize(maxSize);

        int bitRate = Integer.parseInt(args[3]);
        options.setBitRate(bitRate);

        int maxFps = Integer.parseInt(args[4]);
        options.setMaxFps(maxFps);

        int lockedVideoOrientation = Integer.parseInt(args[5]);
        options.setLockedVideoOrientation(lockedVideoOrientation);

        // use "adb forward" instead of "adb tunnel"? (so the server must listen)
        boolean tunnelForward = Boolean.parseBoolean(args[6]);
        options.setTunnelForward(tunnelForward);

        Rect crop = parseCrop(args[7]);
        options.setCrop(crop);

        boolean sendFrameMeta = Boolean.parseBoolean(args[8]);
        options.setSendFrameMeta(sendFrameMeta);

        boolean control = Boolean.parseBoolean(args[9]);
        options.setControl(control);

        int displayId = Integer.parseInt(args[10]);
        options.setDisplayId(displayId);

        boolean showTouches = Boolean.parseBoolean(args[11]);
        options.setShowTouches(showTouches);

        boolean stayAwake = Boolean.parseBoolean(args[12]);
        options.setStayAwake(stayAwake);

        String codecOptions = args[13];
        options.setCodecOptions(codecOptions);

        return options;
    }

执行命令成功之后,检测 一下是否成功开启服务

adb shell lsof | grep scrcpy

有输出就代码启动成功，现在手机 已经开启了 Socket 服务端，等待 PC 连接

那么问题就来了，怎么样去连接 手机上的一个端口呢，因为Scrcpy 开启的实际上是 建立一个LocalSocket socketName 是 scrcpy ， 所以需要用 adb forward 映射手机端口到 pc ，然后pc 再访问本地端口

adb forward tcp:5005 localabstract:scrcpy

好了，重点来了，我来解释一下，Scrcpy 的 socket 通信规则

SDL_bool device_read_info(socket_t device_socket, char *device_name, struct size *size) {
    unsigned char buf[DEVICE_NAME_FIELD_LENGTH + 4];
    int r = net_recv_all(device_socket, buf, sizeof(buf));
    if (r < DEVICE_NAME_FIELD_LENGTH + 4) {
        LOGE("Could not retrieve device information");
        return SDL_FALSE;
    }
    buf[DEVICE_NAME_FIELD_LENGTH - 1] = '\0'; // in case the client sends garbage
    // strcpy is safe here, since name contains at least DEVICE_NAME_FIELD_LENGTH bytes
    // and strlen(buf) < DEVICE_NAME_FIELD_LENGTH
    strcpy(device_name, (char *) buf);
    size->width = (buf[DEVICE_NAME_FIELD_LENGTH] << 8) | buf[DEVICE_NAME_FIELD_LENGTH + 1];
    size->height = (buf[DEVICE_NAME_FIELD_LENGTH + 2] << 8) | buf[DEVICE_NAME_FIELD_LENGTH + 3];
    return SDL_TRUE;
}
  1. 首先服务端 会发一个 虚拟字节，表示 建立连接
  2. 之后会发送 64 个字节，内容是 设备信息
  3. 还有 4个字节 是 宽度和高度 也就是设备的分辨率
  4. 之后 接受的全部都是 ffmpeg 流，格式是 h265/264

下面是 demo 代码

import socket
import struct
import sys
import subprocess
import io

IP = '127.0.0.1'
PORT = 9527

print("start py")
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((IP, PORT))

b = sock.recv(1)
if not len(b):
    print("连接失败")
    exit()
else:
    print("连接成功!")

#Receive device specs
deviceName = sock.recv(64)
print("Device Name:",deviceName.decode("utf-8"))

res = sock.recv(4)
WIDTH, HEIGHT = struct.unpack(">HH", res)
print("WxH:",WIDTH,"x",HEIGHT)

#Start FFPlay in pipe mode
ffplayCmd =['ffplay', '-']
ffp = subprocess.Popen(ffplayCmd, stdin = subprocess.PIPE, stdout = subprocess.PIPE)

while True:
    data = sock.recv(10000)
    ffp.stdin.write(data)

我李某人真是太强了

但是这还不够，视频有了，还要做交互

控制流解析

  • Scrcpy 的客户端代码 写的还是很科班规范的，大概看看文件名字，点开 control_msg
static void
write_position(uint8_t *buf, const struct position *position) {
    buffer_write32be(&buf[0], position->point.x);
    buffer_write32be(&buf[4], position->point.y);
    buffer_write16be(&buf[8], position->screen_size.width);
    buffer_write16be(&buf[10], position->screen_size.height);
}

size_t
control_msg_serialize(const struct control_msg *msg, unsigned char *buf) {
    buf[0] = msg->type;
    // 第一位就是type，根据不同的type有不同的通信规则
    switch (msg->type) {
        case CONTROL_MSG_TYPE_INJECT_KEYCODE:
            buf[1] = msg->inject_keycode.action;
            buffer_write32be(&buf[2], msg->inject_keycode.keycode);
            buffer_write32be(&buf[6], msg->inject_keycode.metastate);
            return 10;
        case CONTROL_MSG_TYPE_INJECT_TEXT: {
            size_t len =
                write_string(msg->inject_text.text,
                             CONTROL_MSG_INJECT_TEXT_MAX_LENGTH, &buf[1]);
            return 1 + len;
        }
        case CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT:
            buf[1] = msg->inject_touch_event.action;
            //第一个有意义字节，代表action，一个byte 一字节
            buffer_write64be(&buf[2], msg->inject_touch_event.pointer_id);
            //write64 这是写入了一个 long 8字节 64位
            write_position(&buf[10], &msg->inject_touch_event.position);
            //write_position 函数在上面，可以看出写入了 2个 int（4字节） 2个short（2字节），总共12字节
            uint16_t pressure =
                to_fixed_point_16(msg->inject_touch_event.pressure);
            buffer_write16be(&buf[22], pressure);
             //找不到这是啥，只知道传递了 压力值
            buffer_write32be(&buf[24], msg->inject_touch_event.buttons);
             //找不到这是啥
            return 28;
        case CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT:
            write_position(&buf[1], &msg->inject_scroll_event.position);
            buffer_write32be(&buf[13],
                             (uint32_t) msg->inject_scroll_event.hscroll);
            buffer_write32be(&buf[17],
                             (uint32_t) msg->inject_scroll_event.vscroll);
            return 21;
        case CONTROL_MSG_TYPE_SET_CLIPBOARD: {
            buf[1] = !!msg->set_clipboard.paste;
            size_t len = write_string(msg->set_clipboard.text,
                                      CONTROL_MSG_CLIPBOARD_TEXT_MAX_LENGTH,
                                      &buf[2]);
            return 2 + len;
        }
        case CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
            buf[1] = msg->set_screen_power_mode.mode;
            return 2;
        case CONTROL_MSG_TYPE_BACK_OR_SCREEN_ON:
        case CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
        case CONTROL_MSG_TYPE_COLLAPSE_NOTIFICATION_PANEL:
        case CONTROL_MSG_TYPE_GET_CLIPBOARD:
        case CONTROL_MSG_TYPE_ROTATE_DEVICE:
            // no additional data
            return 1;
        default:
            LOGW("Unknown message type: %u", (unsigned) msg->type);
            return 0;
    }
}

看着这代码，我差点没把电脑砸了，大概注释了一下，勉强了解了一点发送的信息格式。

所以可以选择从终点入手，那么可以从服务端代码看看，客户端是发送，服务端总是要处理的，服务端处理代码如下

public ControlMessage next() {
        if (!buffer.hasRemaining()) {
            return null;
        }
        int savedPosition = buffer.position();

        int type = buffer.get();
        ControlMessage msg;
        switch (type) {
            case ControlMessage.TYPE_INJECT_KEYCODE:
                msg = parseInjectKeycode();
                break;
            case ControlMessage.TYPE_INJECT_TEXT:
                msg = parseInjectText();
                break;
            case ControlMessage.TYPE_INJECT_TOUCH_EVENT:
                msg = parseInjectTouchEvent();
                break;
            case ControlMessage.TYPE_INJECT_SCROLL_EVENT:
                msg = parseInjectScrollEvent();
                break;
            case ControlMessage.TYPE_SET_CLIPBOARD:
                msg = parseSetClipboard();
                break;
            case ControlMessage.TYPE_SET_SCREEN_POWER_MODE:
                msg = parseSetScreenPowerMode();
                break;
            case ControlMessage.TYPE_BACK_OR_SCREEN_ON:
            case ControlMessage.TYPE_EXPAND_NOTIFICATION_PANEL:
            case ControlMessage.TYPE_COLLAPSE_NOTIFICATION_PANEL:
            case ControlMessage.TYPE_GET_CLIPBOARD:
            case ControlMessage.TYPE_ROTATE_DEVICE:
                msg = ControlMessage.createEmpty(type);
                break;
            default:
                Ln.w("Unknown event type: " + type);
                msg = null;
                break;
        }

        if (msg == null) {
            // failure, reset savedPosition
            buffer.position(savedPosition);
        }
        return msg;
    }

  private ControlMessage parseInjectTouchEvent() {
        if (buffer.remaining() < INJECT_TOUCH_EVENT_PAYLOAD_LENGTH) {
            return null;
        }
        int action = toUnsigned(buffer.get());
        long pointerId = buffer.getLong();
        Position position = readPosition(buffer);
        // 16 bits fixed-point
        int pressureInt = toUnsigned(buffer.getShort());
        // convert it to a float between 0 and 1 (0x1p16f is 2^16 as float)
        float pressure = pressureInt == 0xffff ? 1f : (pressureInt / 0x1p16f);
        int buttons = buffer.getInt();
        return ControlMessage.createInjectTouchEvent(action, pointerId, position, pressure, buttons);
    }

    private ControlMessage parseInjectScrollEvent() {
        if (buffer.remaining() < INJECT_SCROLL_EVENT_PAYLOAD_LENGTH) {
            return null;
        }
        Position position = readPosition(buffer);
        int hScroll = buffer.getInt();
        int vScroll = buffer.getInt();
        return ControlMessage.createInjectScrollEvent(position, hScroll, vScroll);
    }

  private static Position readPosition(ByteBuffer buffer) {
        int x = buffer.getInt();
        int y = buffer.getInt();
        int screenWidth = toUnsigned(buffer.getShort());
        int screenHeight = toUnsigned(buffer.getShort());
        return new Position(x, y, screenWidth, screenHeight);
    }

读取到 socket buffer type 为touch 时，会执行 parseInjectTouchEvent 这个函数，用来解析 客户端发送的 socket

和我们上面解析的 格式差不多，

无符号的一个字节表明 action，八个字节表示Long类型的点击id（用来代表此次操作）

12 个字节 分别是 2个 int 和 short ，用来解析点击位置 和 屏幕大小

然后是 2个字节 的 short 转成无符号位，然后根据比例转换为 0~1 的数字，这个数字代表压力系数，

最后一个 buttons 固定 4个字节 是个 int 不知道什么意思，在 源码上打 log 发现，每次输入的 都是 1，所以可以先不用管这个值

再看看 大佬在源码里写的 测试用例，看看我们的源码分析结果是否正确

struct control_msg msg = {
        .type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
        .inject_touch_event = {
            .action = AMOTION_EVENT_ACTION_DOWN,
            .pointer_id = 0x1234567887654321L,
            .position = {
                .point = {
                    .x = 100,
                    .y = 200,
                },
                .screen_size = {
                    .width = 1080,
                    .height = 1920,
                },
            },
            .pressure = 1.0f,
            .buttons = AMOTION_EVENT_BUTTON_PRIMARY,
        },
    };

    unsigned char buf[CONTROL_MSG_SERIALIZED_MAX_SIZE];
    int size = control_msg_serialize(&msg, buf);
    assert(size == 28);

    const unsigned char expected[] = {
        CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
        0x00, // AKEY_EVENT_ACTION_DOWN
        0x12, 0x34, 0x56, 0x78, 0x87, 0x65, 0x43, 0x21, // pointer id
        0x00, 0x00, 0x00, 0x64, 0x00, 0x00, 0x00, 0xc8, // 100 200
        0x04, 0x38, 0x07, 0x80, // 1080 1920
        0xff, 0xff, // pressure
        0x00, 0x00, 0x00, 0x01 // AMOTION_EVENT_BUTTON_PRIMARY
    };
    assert(!memcmp(buf, expected, sizeof(expected)));

为了测试我的分析是否准确，研究了2周，把最后的 demo 代码写出来

import struct
import socket
import time
address = ('127.0.0.1', 1234)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
# data = struct.pack("@2bq2i3hi",2,0,0,100,200,1080,1920,1,0)
# data = struct.pack("x","")
# print(len(data))
# size = struct.calcsize("@2bi2i3hi")
# print(size)
i = 0
#发送点击事件
while True:
    s.send(struct.pack("2b",2,3)) # 
    s.send(struct.pack("@Q",-1))#pointid  9
    s.send(struct.pack(">2I",100,200))# x y   17
    s.send(struct.pack(">2H",1,1))# width  height  21 
    s.send(struct.pack(">h",1))#pressure 22
    s.send(struct.pack("i",1))# buttons  26
    i = i + 1
    if i == 1:
        break
    # time.sleep(1000)

s.close()

这个只是点击的demo，滑动 dmeo 差不多就不看了， 开始对接剩下的 ATX 完成最简单的自动化

暂无评论

发送评论 编辑评论 


				

			

						

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
							
													

					

				

			

			
			

				

					
				

			

				

											

							
							
						

																				
					
											
						

	

		
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
（╯‵□′）╯︵┴─┴
￣﹃￣
(/ω＼)
∠( ᐛ 」∠)＿
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ｀)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ(￣∇￣o)
ヾ(´･ ･｀｡)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò｡)
Σ(っ °Д °;)っ
( ,,´･ω･)ﾉ"(´っω･｀｡)
╮(╯▽╰)╭
o(*////▽////*)q
＞﹏＜
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
	

	

		
颜文字
Emoji
小恐龙
花!
	


									

			

			
			
		

	






上一篇
下一篇