{"id":507,"date":"2012-07-19T12:54:42","date_gmt":"2012-07-19T19:54:42","guid":{"rendered":"http:\/\/..\/?p=507"},"modified":"2012-07-19T12:54:42","modified_gmt":"2012-07-19T19:54:42","slug":"video-conferencing-w-zipits","status":"publish","type":"post","link":"https:\/\/blog.engine12.com\/?p=507","title":{"rendered":"Video conferencing w\/ ZipIts"},"content":{"rendered":"<p><a href=\"..\/wp-content\/uploads\/2012\/07\/bauer1.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignleft\" src=\"..\/wp-content\/uploads\/2012\/07\/bauer1.png\" alt=\"\" width=\"320\" height=\"240\" \/><\/a>Awhile back I was looking for a solution to send an mp3 stream out to multiple clients and then have the clients play it back synchronously.\u00a0 After quite a bit of research, I decided to use the live555 library.\u00a0 In the process of setting up the mp3 stream multicast, I realized the library was also the basis for a <a href=\"https:\/\/github.com\/engine12\/mjpg-streamer\">udp plugin to mjpg-streamer<\/a>.\u00a0 One thing led to another and pretty soon I was video conferencing with ZipIts.<\/p>\n<p>Working off examples from the live555 source, I stitched together a working plugin for mjpg-streamer.\u00a0 For the client I compiled mplayer to include the live555 codecs.\u00a0 I now have an RTP video stream going, next up is voice.\u00a0 Pretty simple right?\u00a0 No, not really.\u00a0 After lots of hacking on mplayer I managed to pick up an RTP speex stream only to be disappointed with a 20 second delay!\u00a0 How could this be?\u00a0 I have a 30 fps video feed coming in real time&#8230; WTF.\u00a0 So I sat there, staring at the code, looking for that 20 second buffer&#8230;.\u00a0 I managed to get the delay down to about 2 secs, but that&#8217;s still way too long for holding a conversation.\u00a0 Fortunately there&#8217;s the tcp side of the sockets api, and since the bandwidth consumed by the voice is really small, it works.\u00a0 What I ended up with is a peer to peer video chat solution that runs on the Linux console using only the framebuffer.\u00a0 The solution presented below is pretty raw.\u00a0 I&#8217;ll probably write a script soon that automates the setup, but until then&#8230;<\/p>\n<p style=\"text-align: center\"><span style=\"color: #999999\">Note: The &#8216;client&#8217; and &#8216;server&#8217; users in the example commands represent separate ZipIts.\u00a0 It&#8217;s assumed your network is configured.\u00a0 The client address is 192.168.1.180, the server is 192.168.1.249<\/span><\/p>\n<ul>\n<li>Load the modules needed for use of the webcam&#8217;s mic.<\/li>\n<\/ul>\n<p style=\"padding-left: 30px\">server@engine12:~# <span style=\"color: #ff9900\">insmod snd-usbmidi-lib<\/span><br \/>\nserver@engine12:~# <span style=\"color: #ff9900\">insmod snd-usb-audio<\/span><\/p>\n<ul>\n<li>Now we set-up the voice channel.\u00a0 It&#8217;s a little backwards, so pay attention.\u00a0 We run a program called TCPserver on what I am calling the client&#8230;<\/li>\n<\/ul>\n<p style=\"padding-left: 30px\">client@engine12:~# <span style=\"color: #ff9900\">TCPserver 7080 | speexdec<\/span><\/p>\n<p style=\"padding-left: 30px\">server@engine12:~# <span style=\"color: #ff9900\">arecord -D hw:1,0 -r 16000 -t raw -c 1 -f S16_LE | <\/span><\/p>\n<p style=\"padding-left: 210px\"><span style=\"color: #ff9900\">speexenc | TCPclient 192.168.1.180 7080<\/span><span style=\"color: #999999\"> <\/span><\/p>\n<p style=\"padding-left: 30px;text-align: center\"><span style=\"color: #999999\">Note: The argument &#8220;hw:1,0&#8221; is usually &#8220;hw:0,0&#8221; if your microphone is built-in, such as on a laptop<\/span><span style=\"color: #ff9900\"><br \/>\n<\/span><\/p>\n<ul>\n<li>You should be able to hear now, using headphones on the client ZipIt.\u00a0 To setup the video channel, start mjpg-streamer with the live555 output plugin and then connect to it with mplayer.<\/li>\n<\/ul>\n<p style=\"padding-left: 30px\">server@engine12:~# <span style=\"color: #ff9900\">mjpg_streamer\u00a0 -i &#8220;input_uvc.so -y -d \/dev\/video0 -r 320&#215;240 -f 30&#8221; <\/span><\/p>\n<p style=\"padding-left: 180px\"><span style=\"color: #ff9900\">-o &#8220;output_live.so -v -p 7072 -q 80&#8221;<\/span><\/p>\n<p style=\"padding-left: 30px\">client@engine12:~# <span style=\"color: #ff9900\">cpu max<\/span><\/p>\n<p style=\"padding-left: 30px\">client@engine12:~# <span style=\"color: #ff9900\">mplayer rtsp:\/\/192.168.1.249:7072\/mjpg_streamer <\/span><\/p>\n<p style=\"padding-left: 150px\"><span style=\"color: #ff9900\">-fps 30 -vo fbdev -vf scale=320:240,rotate=2<\/span><\/p>\n<p>&nbsp;<\/p>\n<p>If all went well you should be able to see and hear the server&#8217;s webcam.\u00a0 Now set up the webcam on the client the same way and start chatting with the other end, in my case it was just Bauer, but it does work with other people.\u00a0 To use it over the web, add the -t switch.\u00a0 You&#8217;ll need to configure your router to forward the two ports.\u00a0 It&#8217;s possible to tunnel the communications using SSH and a usb ethernet dongle, but unfortunately I couldn&#8217;t get an SSH tunnel configured using the wifi interface.\u00a0 I also setup a dynamic DNS account for each ZipIt in order to find each other over the web.<\/p>\n<p>The live555 plugin has a lot more to offer when you add additional switches.\u00a0 The -m switch multicasts the stream, you can then pick up the stream from multiple clients.\u00a0 This is only going to work on your LAN, and actually works better over a B.A.T.M.A.N. mesh network.\u00a0 The -a switch adds audio to the stream, albeit with a 2 sec. delay, but it&#8217;s still pretty useful for multicasting purposes.\u00a0 If you add the -a switch you pick up the audio stream with my <a href=\"https:\/\/github.com\/engine12\/mplayer\">hacked version of mplayer<\/a> and the TCP client\/server utils aren&#8217;t needed.<\/p>\n<p>The following parameters can be passed to the output_live.so plugin:<\/p>\n<p style=\"padding-left: 30px\">[-p | &#8211;port ]&#8230;&#8230;&#8230;.: port for this RTSP server<br \/>\n[-c | &#8211;credentials ]&#8230;: ask for &#8220;username:password&#8221; on connect<br \/>\n[-m | &#8211;multicast ]&#8230;: server will multicast (vs. unicast) &#8212; no args<br \/>\n[-q | &#8211;quality ]&#8230;&#8230;: JPEG compression quality in percentn&#8221;<br \/>\nshould match the quality value given to input_uvc<br \/>\n[-t | &#8211;tunnel ]&#8230;&#8230;: port number for RTSP-over-HTTP tunneling<br \/>\ndisabled if not specifiedn&#8221;<br \/>\n[-a | &#8211;audio ]&#8230;&#8230;: stream audio<br \/>\n[-v | &#8211;video ]&#8230;&#8230;: stream video<\/p>\n<p>&nbsp;<\/p>\n<p>The source code for the project is located in three seperate repositories under my github account:<\/p>\n<p><a href=\"https:\/\/github.com\/engine12\/mjpg-streamer\">https:\/\/github.com\/engine12\/mjpg-streamer<\/a><\/p>\n<p><a href=\"https:\/\/github.com\/engine12\/mplayer\">https:\/\/github.com\/engine12\/mplayer<\/a><\/p>\n<p><a href=\"https:\/\/github.com\/engine12\/speex_tcp\">https:\/\/github.com\/engine12\/speex_tcp<\/a><\/p>\n<p>Binaries for the ZipIt are <a href=\"http:\/\/engine12.com\/files\/videochat-zipit\/\">here<\/a>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Awhile back I was looking for a solution to send an mp3 stream out to multiple clients and then have the clients play it back synchronously.\u00a0 After quite a bit of research, I decided to use the live555 library.\u00a0 In the process of setting up the mp3 stream multicast, I realized the library was also <a href='https:\/\/blog.engine12.com\/?p=507'>[&#8230;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.engine12.com\/index.php?rest_route=\/wp\/v2\/posts\/507"}],"collection":[{"href":"https:\/\/blog.engine12.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.engine12.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.engine12.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.engine12.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=507"}],"version-history":[{"count":0,"href":"https:\/\/blog.engine12.com\/index.php?rest_route=\/wp\/v2\/posts\/507\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.engine12.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=507"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.engine12.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=507"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.engine12.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=507"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}