The Android Transporter allows you to share the display contents of your Nexus 7 tablet wirelessly with other screens in real time. Now, the first tech demo of the Android Transporter is out!
Introduction The Android Transporter allows you to share display content wirelessly with remote screens in real time. Please be aware that the Transporter is still a technology study and it is missing the maturity of a full-featured product. However, we think that the Android Transporter is already exciting enough to let you play around with it. We believe that with the recently released
Miracast standard you will get a very similar technology in upcoming Android devices, and we are considering making the Transporter compliant with the Miracast specs. The Android Transporter is a custom ROM and not an app since we had to make adjustemts to various parts of the Android platform to make it happen. Be aware that you use the Android Transporter at your own risk and that you will void your tablet’s warranty by following the setup instructions below. Moreover, you are not allowed to bundle further Google apps like the Google Play Store with our firmware image. Read on for the quick start guide and some Android Transporter internals.
Quick start guide Let’s start with the Asus Nexus 7. By installing the Android Transporter firmware on your Asus Nexus 7 all data on the device will be removed including the contents of the
/sdcard directory! So maybe you want to backup some data before you begin. To start, enable USB debugging on your Nexus 7 by switching on this option under Settings -> Developer options. Next, unlock the bootloader of your Nexus 7 device. This will void your device warranty and it will also do a factory reset! Install the adb and fastboot utilities. On Microsoft Windows you will also need Google’s USB drivers. Just search the Internet for instructions how to install this tools. Unlocking the bootloader is done by getting the device into the bootloader using the adb reboot bootloader command. Now unlock it with the fastboot oem unlock command. Download the Android Transporter firmware for the Asus Nexus 7 and unpack it. Reboot your Nexus 7 device into the bootloader using adb reboot bootloader. In the Android-Transporter directory you will find the flash script, which is called flash-all.sh. Be aware that the fastboot utility has to be available in the PATH environment for the flash script. Execute the flash script. Your Asus Nexus 7 is now ready. To switch back to the original Nexus 7 firmware image you should first download it from Google’s factory images site. Then you can flash the original firmware image using the same instructions that you used to flash the Android Transporter firmware image. Let’s move on to the Raspberry Pi. Download the Android Transporter firmware for the Raspberry Pi and unpack it. Insert an SD card into your card reader and flash the firmware image using the dd utility: sudo dd bs=1M if=esrlabs-rpi-android-transporter-2012-10-02.img of=/dev/sdX. Substitute /dev/sdX with the real SD card device name. For more information on flashing the Raspberry Pi see the Embedded Linux Wiki page for the Raspberry Pi. The Raspberry Pi is now also ready. Next, you have to set up the networking between the two devices. You can either connect both devices to your home router or you can make use of the Raspberry Pi Wi-Fi hotspot. Both scenarios require the Raspberry Pi to be plugged into your home router. The Wi-Fi hotspot works with USB Wi-Fi adapters that are supported by the Realtek rtl8192cu Linux driver. We tested the Wi-Fi hotspot with the Netgear N150 Microadapter and with the Asus N13 Wi-Fi Stick. It works well with both Wi-Fi sticks except that we occasionally had some power consumption issues with the Asus device. If you want to make use of the Wi-Fi hotspot plug in the stick before the Raspberry Pi starts up. The network name of the Wi-Fi hotspot is RaspberryPiAP and the default password is E.S.R.Labs. The Wi-Fi hotspot may have the advantage that the Android Transporter has its own dedicated network to minimize the latency jitter during screen mirroring. If you connect both devices to your home router please make sure that the Wi-Fi transmitting power of your router is set to high. Otherwise you may experience high packet loss, which is bad for the Android Transporter. When you now connect your Nexus 7 to the Wi-Fi network and start the Android Transporter you should see the Raspberry Pi in the list of available media hubs. The Raspberry Pi has announced itself as media sink via service discovery. Just tap on the Raspberry Pi item to start the screen mirroring. If you want to stop the screen mirroring just pull down the notification bar and click the Android Transporter “Switch off” item. By default the Android Transporter will make use of the H.264 over RTP over UDP streaming protocols according to RFC3984. If you are in a building with a lot of Wi-Fi networks, it may be possible that the Android Transporter does not work really well because of high packet loss. If that is the case you should switch to the H.264 over RTP over TCP streaming protocols according to RFC3984 and RFC4571. You can do this in the preferences of the Android Transporter app. The latency will typically be around 20-30ms higher when using the reliable TCP transport protocol. Enjoy the Android Transporter tech demo :-).
Android Transporter We did some measurement series to show the total end-to-end screen mirroring latency. The measurements were done using two photo diodes (one on each display) and a test app that really stresses the H.264 encoder and decoder. Furthermore, we are able to measure the time periods taken by each individual step of the screen mirroring process directly on the devices. The picture below shows the box plot for a typical Android Transporter setup where a Nexus 7 is connected to the Wi-Fi hotspot of a Raspberry Pi. The screen is mirrored with a frame rate of 30 FPS, a bit rate of 8 MBit/s and an I-frame interval of 1 using the UDP transport protocol.
You see that the total latency is typically around 207ms. But there are also a few outliers of up to 257ms. The outliers are mainly introduced by the Wi-Fi network. The work that has to be done by the Asus Nexus 7 for one video frame takes around 41ms on average. It is made up of grabbing the screen, H.264 encoding it and sending it over the Wi-Fi network. The rest of the time is mainly due to decoding and displaying it on the Raspberry Pi. To further reduce the latency one needs to work with the hardware vendors in order to minimize the time that is spent in the H.264 hardware encoders and decoders and in the vendor specific OpenMAX implementations. Unfortunately, a latency of around 200ms is too high to play fast 3D games. But it is okay for apps, videos, slide shows, photo presentations, some games, etc. The next picture shows the box plot for a Samsung Nexus S, which is mirroring its screen contents onto a Raspberry Pi. The setup is the same as for the Nexus 7 except that the I-frame interval is set to 0, which means the Nexus S is doing only I-frames and no P-frames. Unfortunately this option does not work on the Nexus 7. As you see, the average end-to-end screen mirroring latency is 175ms with outliers of up to 200ms. One interesting thing is that the Nexus S needs 74ms to grab, H.264 encode and send a video frame to the Raspberry Pi, which is worse than the 41ms needed by Nexus 7. But the Raspberry Pi is a lot faster at decoding the I-frame only video stream, which additionally has a much lower resolution. The Android Transporter player on the Raspberry Pi currently does not do buffering of video frames. It simply plays them back as they arrive. This is quite good for the real-time screen mirroring but not necessarily for all kinds of applications. This strategy is also the reason why video playback on the Raspberry Pi is stuttering a bit from time to time. We will add a buffering strategy in the future. In the meantime, maybe you want to fix that since the source code of the player is freely available on GitHub :-). Since the Android Transporter does currently not support the MPEG-2 TS streaming protocol it is not compliant with the Miracast spec. We also plan to change that in the future.
Raspberry Pi The Android Transporter player supports H.264 video as well as 44.1 kHz PCM and AAC audio streams. The
source code of the Android Transporter player for the Raspberry Pi is available at GitHub under the Apache 2 license. You may try out the player as explained on its GitHub site. E.g. you can stream videos and music to the Android Transporter player on the Raspberry Pi using the VLC streaming server. To substantially improve the real-time capabilities of the Raspberry Pi we recommend applying our Raspberry Pi Linux Kernel patch. The patch is already included in the standard Linux Kernel v3.2.27 for the Raspberry Pi, which is also used by our firmware image. To configure the Wi-Fi hotspot on the Raspberry Pi please edit the /etc/hostapd/hostapd.conf file. This is the right place to change the network name (SSID) and the Wi-Fi password. If you want to change the Raspberry Pi’s overscan settings according to the needs of your television set, video projector or monitor just log in to the Raspberry Pi via SSH and adjust the /boot/config.txt file. The user name for the Raspberry Pi is pi and the password is raspberry. For more information on configuring the overscan mode please see the Embedded Linux Wiki page for Raspberry Pi video modes. Some other configuration options of the Raspberry Pi can be adjusted using the raspi-config tool. Instead of using the Raspberry Pi as viewer for the Android Transporter you can also use the VLC or GStreamer player. The only limitation is that both players only support the UDP transport protocol. Furthermore, the VLC player introduces a much higher latency than the player on the Raspberry Pi or the GStreamer player. To use the VLC or GStreamer player you have to first start the Android Transporter app on your Asus Nexus 7. Next, use the vlc rtsp://
Asus Nexus 7 The Android Transporter app allows you to change various video-encoding settings. It also enables you to announce a Nexus 7 tablet as media sink to mirror the screen contents of one Nexus 7 device to another one. Besides using the service discovery, this can also be done using the
Android Beam feature. Just start the Android Transporter on either one of your Nexus 7 devices and bump the two devices via NFC. Then touch to beam the screen contents. The total end-to-end screen mirroring latency from one Nexus 7 device to the other is around 150ms with the H.264 video decoding process consuming the biggest period of time. You will notice that the Android Transporter firmware supports the landscape mode for the Android Launcher. By default, the landscape mode is also enforced while beaming the tablet’s screen contents to the Raspberry Pi.
What’s next? We are currently working on the audio streaming support for the Nexus 7 and on the Wi-Fi Direct support for the Raspberry Pi.
Update 1 – Samsung Nexus S (GSM) We just released the
Android Transporter firmware for the Samsung Nexus S (GSM) to give you an outlook on game-playing. To flash the Android Transporter firmware please follow the instructions that you find in the quick start guide above. One problem with gaming on the Asus Nexus 7 is that the bit rate settings do not work and therefore it produces bit rates of up to 20-30 MBit/s during fast 3D games. This is way too much for a lot of WiFi routers. The high bit rate results in heavy packet loss which makes gaming on the Asus Nexus 7 nearly impossible. But thankfully, the Samsung Nexus S is pretty good at fast 3D car racing games as we already showed in our previous demo.
Update 2 – Samsung Galaxy Nexus (GSM) Due to demand, we also released the
Android Transporter firmware for the Samsung Galaxy Nexus (GSM). Keep in mind that the H.264 encoder of the Samsung Galaxy Nexus is only capable of about 20 FPS. The total screen mirroring end-to-end latency to the Raspberry Pi is typically around 270ms.
Update 3 – Android 4.2 and Miracast According to the
official Google Blog Android 4.2 will have support for wireless display so you can wirelessly watch movies, YouTube videos and play games right on your Miracast-compatible HDTV. :-) [social]