Video conferencing w/ ZipIts

Uncategorized Comments Off on Video conferencing w/ ZipIts
Jul 192012

Awhile back I was looking for a solution to send an mp3 stream out to multiple clients and then have the clients play it back synchronously.  After quite a bit of research, I decided to use the live555 library.  In the process of setting up the mp3 stream multicast, I realized the library was also the basis for a udp plugin to mjpg-streamer.  One thing led to another and pretty soon I was video conferencing with ZipIts.

Working off examples from the live555 source, I stitched together a working plugin for mjpg-streamer.  For the client I compiled mplayer to include the live555 codecs.  I now have an RTP video stream going, next up is voice.  Pretty simple right?  No, not really.  After lots of hacking on mplayer I managed to pick up an RTP speex stream only to be disappointed with a 20 second delay!  How could this be?  I have a 30 fps video feed coming in real time… WTF.  So I sat there, staring at the code, looking for that 20 second buffer….  I managed to get the delay down to about 2 secs, but that’s still way too long for holding a conversation.  Fortunately there’s the tcp side of the sockets api, and since the bandwidth consumed by the voice is really small, it works.  What I ended up with is a peer to peer video chat solution that runs on the Linux console using only the framebuffer.  The solution presented below is pretty raw.  I’ll probably write a script soon that automates the setup, but until then…

Note: The ‘client’ and ‘server’ users in the example commands represent separate ZipIts.  It’s assumed your network is configured.  The client address is 192.168.1.180, the server is 192.168.1.249

  • Load the modules needed for use of the webcam’s mic.

server@engine12:~# insmod snd-usbmidi-lib
server@engine12:~# insmod snd-usb-audio

  • Now we set-up the voice channel.  It’s a little backwards, so pay attention.  We run a program called TCPserver on what I am calling the client…

client@engine12:~# TCPserver 7080 | speexdec

server@engine12:~# arecord -D hw:1,0 -r 16000 -t raw -c 1 -f S16_LE |

speexenc | TCPclient 192.168.1.180 7080

Note: The argument “hw:1,0” is usually “hw:0,0” if your microphone is built-in, such as on a laptop

  • You should be able to hear now, using headphones on the client ZipIt.  To setup the video channel, start mjpg-streamer with the live555 output plugin and then connect to it with mplayer.

server@engine12:~# mjpg_streamer  -i “input_uvc.so -y -d /dev/video0 -r 320×240 -f 30”

-o “output_live.so -v -p 7072 -q 80”

client@engine12:~# cpu max

client@engine12:~# mplayer rtsp://192.168.1.249:7072/mjpg_streamer

-fps 30 -vo fbdev -vf scale=320:240,rotate=2

 

If all went well you should be able to see and hear the server’s webcam.  Now set up the webcam on the client the same way and start chatting with the other end, in my case it was just Bauer, but it does work with other people.  To use it over the web, add the -t switch.  You’ll need to configure your router to forward the two ports.  It’s possible to tunnel the communications using SSH and a usb ethernet dongle, but unfortunately I couldn’t get an SSH tunnel configured using the wifi interface.  I also setup a dynamic DNS account for each ZipIt in order to find each other over the web.

The live555 plugin has a lot more to offer when you add additional switches.  The -m switch multicasts the stream, you can then pick up the stream from multiple clients.  This is only going to work on your LAN, and actually works better over a B.A.T.M.A.N. mesh network.  The -a switch adds audio to the stream, albeit with a 2 sec. delay, but it’s still pretty useful for multicasting purposes.  If you add the -a switch you pick up the audio stream with my hacked version of mplayer and the TCP client/server utils aren’t needed.

The following parameters can be passed to the output_live.so plugin:

[-p | –port ]……….: port for this RTSP server
[-c | –credentials ]…: ask for “username:password” on connect
[-m | –multicast ]…: server will multicast (vs. unicast) — no args
[-q | –quality ]……: JPEG compression quality in percentn”
should match the quality value given to input_uvc
[-t | –tunnel ]……: port number for RTSP-over-HTTP tunneling
disabled if not specifiedn”
[-a | –audio ]……: stream audio
[-v | –video ]……: stream video

 

The source code for the project is located in three seperate repositories under my github account:

https://github.com/engine12/mjpg-streamer

https://github.com/engine12/mplayer

https://github.com/engine12/speex_tcp

Binaries for the ZipIt are here.

 

© 2016 engine12