Sunday, October 17, 2010

Webcam Chat

Due to a reader's request I recently took some time to realize a bit bigger project, the development of a webcam chat implemented in C#.
I want to present this in this post, for understanding it some previous knowledge is needed:
- The code to use a webcam in C# I took with allowance from net-blog.
- The connection between the partners is set up via TCP / IP, I posted about that before here.
- Many want to probably connect with friends over the internet, how the connection via that works and which IP addresses are to be used, you can find here.
- How an image is "streamed" is explained in this post.
- And here finally general information on the error "A general error occured in GDI+", which occurs sporadically at graphical applications.

This is the user interface of the program:


In the left PictureBox the image of the own webcam is displayed, in the right that of the partner.
Of course the chat partner has to run the same program.
To its functionality:
With a click on the button "Connect" 2 threads are started, one of them to send the own image, the other one to receive the image of the partner.
The send thread runs on the in "eigener Port" inputted port, the receive thread on the port inputted in "Partner Port".
To connect logically both partners have to enter the IP address of the other one and matching parts, so partner A port x for "eigener Port" and Port y for "Partner Port", partner B has to input this reversed.
In the send thread a server is created, to which the client from the receive thread connects to.
So in every instance of the program there are 2 server - client connections, which communicate over their own network stream.
In the send thread an infinite loop is running, which tries to send the image of the own PictureBox via the function WriteImage().
In this the image is first written with a temporary stream to a byte array. Then the size of this array is written to a 20 character string, the remaining characters are filled up with "x".
The size and content of the image are then send over the network stream.
In the receive thread an infinite loop is running, trying to readout that stream via the function ReadImage().
First here in every iteration the first 20 bytes are read, which determine the size of the image.
If the stream can be converted to some valid number g, the next g bytes from the network stream are read and the image is reproduced by that.
If the 20 bytes cannot be converted, there was a transmission error. In order for the server and client to synchronise again, the stream has to be deleted again. For this simply bytes are readout, until the stream is empty.
In general I must say the implementation of the data transfer was pretty hard, server and client got out of sync a lot and many transmission errors occured.
In the version published here when sending and receiving images succesfully, these images are saved in variables. If an error occurs, the saved image is reused.

Okay, that should suffice with the description, as I said, when reading the above mentioned tutorials the single program parts should be good to understand.
I know, this version is just a start and was just designed for sample purposes and could be improved in any way.
For hints and advice I am always thankful.
Have fun with the program!

A setup to intall the webcam chat is available over this link.
The complete source code can be found here.
Now the source code, first the content of the file Form1.cs and then the content of the file Form1.Designer.cs, if you also copy this one in your project, you get exactly the same user interface as me.

Form1.cs:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

// for Webcam
using AForge.Video;
using AForge.Video.DirectShow;

// for Netzwerk
using System.IO;
using System.Net.Sockets;
using System.Net;

using System.Threading;

namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        #region Webcam

        // our webcam object
        VideoCaptureDevice videoSource;

        void InitWebCam(int nr)
        {
            // enumeration of all webcams / video devices
            FilterInfoCollection videosources = new FilterInfoCollection(FilterCategory.VideoInputDevice);

            // check if at least one webcam was found
            if (videosources != null)
            {
                // bind the webcam "nr" to out project
                videoSource = new VideoCaptureDevice(videosources[nr].MonikerString);

                try
                {
                    // check if webcam has technical abilites
                    if (videoSource.VideoCapabilities.Length > 0)
                    {
                        string lowestSolution = "10000;0";
                        // look for the profile with the lowest resolution
                        for (int i = 0; i < videoSource.VideoCapabilities.Length; i++)
                        {
                            if (videoSource.VideoCapabilities[i].FrameSize.Width < Convert.ToInt32(lowestSolution.Split(';')[0]))
                                lowestSolution = videoSource.VideoCapabilities[i].FrameSize.Width.ToString() + ";" + i.ToString();
                        }
                        // pass the webcam object the lowest resolution
                        videoSource.DesiredFrameSize = videoSource.VideoCapabilities[Convert.ToInt32(lowestSolution.Split(';')[1])].FrameSize;
                    }
                }
                catch (Exception e)
                {
                    MessageBox.Show(e.ToString());
                }

                // Assign the webcam object the event hanlder NewFrame
                // this event is triggered for every new image
                videoSource.NewFrame += new AForge.Video.NewFrameEventHandler(videoSource_NewFrame);

                // activate the webcam
                videoSource.Start();
            }
        }

        void videoSource_NewFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
        {
            // assign every incoming image to the picture box
            pictureBoxVideoSelf.BackgroundImage = (Image)eventArgs.Frame.Clone();
        }

        #endregion

        Image LastImageSent = null// last correctly send image
        Image LastImageReceived = null// last correctly received image

        Thread Sender; // thread for sending images
        Thread Receiver; // thread for receiving images

        bool Closing = false// true if form is to be quit
        String ClosingString = "FORM#CLOSING"// message, which is sent when closing
        byte[] ClosingBytes; // byte coding of this message
        ASCIIEncoding ByteConverter = new ASCIIEncoding(); // object to convert strings to byte arrays and other way

        private void button1_Click(object sender, EventArgs e)
        {
            // start thread for sending the own image
            Sender = new Thread(new ParameterizedThreadStart(this.Send));
            Sender.Start(int.Parse(textBox4.Text));

            // start thread for receiving the partner image
            Receiver = new Thread(new ParameterizedThreadStart(Receive));
            Receiver.Start(textBox2.Text + "-" + textBox1.Text);

            ClosingBytes = ByteConverter.GetBytes(ClosingString);
        }

        private void Send(object port)
        {
            InitWebCam(int.Parse(textBox3.Text)); // start webcam

            TcpListener Server = new TcpListener(int.Parse(port.ToString()));
            Server.Start();

            TcpClient Client = Server.AcceptTcpClient();

            NetworkStream ClientStream = Client.GetStream();

            while (true)
            {
                if (Closing)
                    break// quit
                try
                {
                    // try to send the image to the partner, then save it as backup
                    WriteImage((Image)pictureBoxVideoSelf.BackgroundImage.Clone(), ClientStream);
                    LastImageSent = (Image)pictureBoxVideoSelf.BackgroundImage.Clone();
                    Thread.Sleep(100);
                }
                catch
                {   // if the current image could not be send, take the backup
                    WriteImage(LastImageSent, ClientStream);
                }
            }

            try
            {
                ClientStream.Write(ClosingBytes, 0, ClosingBytes.Length);
            }
            catch { };
        }

        private void Receive(object portip)
        {
            // portip has the form "port-ip"
            string[] Parameter = portip.ToString().Split('-');
            System.Net.IPAddress IP = System.Net.IPAddress.Parse(Parameter[1]);

            TcpClient Exchange = new TcpClient();
            NetworkStream ExchangeStream = null;

            Image TempImage;

            // try to setup a connection every 3 seconds
            while (true)
            {
                try
                {
                    Exchange.Connect(IP, int.Parse(Parameter[0]));
                    ExchangeStream = Exchange.GetStream();
                    break;
                }
                catch
                {
                    Thread.Sleep(3000);
                }
            }

            while (true)
            {
                if (Closing)
                    break// quit

                try
                {
                    // try to display the received image
                    // in case of success
                    TempImage = ReadImage(ExchangeStream);
                    if (TempImage == null)
                        throw new Exception();

                    pictureBoxVideoPartner.BackgroundImage = TempImage;
                    LastImageReceived = (Image)pictureBoxVideoPartner.BackgroundImage.Clone();
                    Thread.Sleep(100);
                }
                catch
                {
                    try
                    {   // in case of error use backup image
                        pictureBoxVideoPartner.BackgroundImage = LastImageReceived;
                    }
                    catch { }
                }
            }
        }

        private void WriteImage(Image image, NetworkStream stream)
        {
            ASCIIEncoding Encoder = new ASCIIEncoding();
            MemoryStream TempStream = new MemoryStream();
            byte[] Buffer;

            try
            {
                // write the handed over image to the stream
                image.Save(TempStream, System.Drawing.Imaging.ImageFormat.Gif);
            }
            catch
            {
            }

            Buffer = TempStream.ToArray();

            // encode the size of the image as a 20 character string, fill up with x
            string ImageSize = Buffer.Length.ToString();
            while (ImageSize.Length < 20)
                ImageSize += "x";

            // write its size plus content to an array
            byte[] FittedImageSize = Encoder.GetBytes(ImageSize);
            byte[] ImagePlusSize = new byte[FittedImageSize.Length + Buffer.Length];
            Array.Copy(FittedImageSize, ImagePlusSize, FittedImageSize.Length);
            Array.Copy(Buffer, 0, ImagePlusSize, FittedImageSize.Length, Buffer.Length);

            try
            {
                // write the array
                stream.Write(ImagePlusSize, 0, ImagePlusSize.Length);
                stream.Flush();
            }
            catch
            {
                // if the stream cannot be written anymore, the partner has quit
                Closing = true;
            }
        }

        private Image ReadImage(NetworkStream stream)
        {
            Image Result;
            int BytesRead;

            // read the first 20 bytes of the stream, since they encode the size of the image
            byte[] ImageSize = new byte[20];
            BytesRead = stream.Read(ImageSize, 0, 20);

            /* if only 12 bytes can be read and they contain the closing string, a termination is requested */
            if (BytesRead == 12)
            {
                if (ByteConverter.GetString(ImageSize, 0, 12) == "FORM#CLOSING")
                {
                    Closing = true;
                    return null;
                }
            }

            byte[] ErrorBuffer = new byte[100000000];

            ASCIIEncoding Decoder = new ASCIIEncoding();
            string ImageSizeString = Decoder.GetString(ImageSize).Replace("x""");

            int TestSize;

            if (!int.TryParse(ImageSizeString, out TestSize))
            {
                stream.Read(ErrorBuffer, 0, ErrorBuffer.Length);
                return null;
            }

            byte[] ImageFile = new byte[int.Parse(ImageSizeString)];

            stream.Read(ImageFile, 0, ImageFile.Length);

            MemoryStream temps = new MemoryStream();

            try
            {
                temps.Write(ImageFile, 0, ImageFile.Length);
                Result = Image.FromStream(temps);
                return Result;
            }
            catch
            {
                return null;
            }
        }

        private void Form1_FormClosing(object sender, FormClosingEventArgs e)
        {
            // dispose webcam at closing
            if (videoSource != null && videoSource.IsRunning)
            {
                videoSource.SignalToStop();
                videoSource = null;
            }

            Closing = true;

            Thread.Sleep(3000);

            if (Sender != null && Sender.IsAlive)
                Sender.Abort();

            if (Receiver != null && Receiver.IsAlive)
                Receiver.Abort();
        }
    }
}


Form1.Designer.cs:

namespace WindowsFormsApplication1
{
    partial class Form1
    {
        /// <summary>
       /// Erforderliche Designervariable.
        /// </summary>
        private System.ComponentModel.IContainer components = null;

        /// <summary>
       /// Verwendete Ressourcen bereinigen.
        /// </summary>
        /// <param name="disposing">True, wenn verwaltete Ressourcen gelöscht werden sollen; andernfalls False.</param>
        protected override void Dispose(bool disposing)
        {
            if (disposing && (components != null))
            {
                components.Dispose();
            }
            base.Dispose(disposing);
        }

        #region Vom Windows Form-Designer generierter Code

        /// <summary>
       /// Erforderliche Methode für die Designerunterstützung.
       /// Der Inhalt der Methode darf nicht mit dem Code-Editor geändert werden.
        /// </summary>
        private void InitializeComponent()
        {
            this.pictureBoxVideoSelf = new System.Windows.Forms.PictureBox();
            this.button1 = new System.Windows.Forms.Button();
            this.pictureBoxVideoPartner = new System.Windows.Forms.PictureBox();
            this.label1 = new System.Windows.Forms.Label();
            this.label2 = new System.Windows.Forms.Label();
            this.label3 = new System.Windows.Forms.Label();
            this.label4 = new System.Windows.Forms.Label();
            this.textBox1 = new System.Windows.Forms.TextBox();
            this.textBox2 = new System.Windows.Forms.TextBox();
            this.textBox3 = new System.Windows.Forms.TextBox();
            this.textBox4 = new System.Windows.Forms.TextBox();
            this.label5 = new System.Windows.Forms.Label();
            ((System.ComponentModel.ISupportInitialize)(this.pictureBoxVideoSelf)).BeginInit();
            ((System.ComponentModel.ISupportInitialize)(this.pictureBoxVideoPartner)).BeginInit();
            this.SuspendLayout();
            //
            // pictureBoxVideoSelf
            //
            this.pictureBoxVideoSelf.BackgroundImageLayout = System.Windows.Forms.ImageLayout.Stretch;
            this.pictureBoxVideoSelf.BorderStyle = System.Windows.Forms.BorderStyle.FixedSingle;
            this.pictureBoxVideoSelf.Location = new System.Drawing.Point(47, 26);
            this.pictureBoxVideoSelf.Name = "pictureBoxVideoSelf";
            this.pictureBoxVideoSelf.Size = new System.Drawing.Size(331, 210);
            this.pictureBoxVideoSelf.TabIndex = 0;
            this.pictureBoxVideoSelf.TabStop = false;
            //
            // button1
            //
            this.button1.Font = new System.Drawing.Font("Microsoft Sans Serif", 12F, System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(0)));
            this.button1.Location = new System.Drawing.Point(326, 263);
            this.button1.Name = "button1";
            this.button1.Size = new System.Drawing.Size(154, 46);
            this.button1.TabIndex = 1;
            this.button1.Text = "Verbinden";
            this.button1.UseVisualStyleBackColor = true;
            this.button1.Click += new System.EventHandler(this.button1_Click);
            //
            // pictureBoxVideoPartner
            //
            this.pictureBoxVideoPartner.BackgroundImageLayout = System.Windows.Forms.ImageLayout.Stretch;
            this.pictureBoxVideoPartner.BorderStyle = System.Windows.Forms.BorderStyle.FixedSingle;
            this.pictureBoxVideoPartner.Location = new System.Drawing.Point(398, 26);
            this.pictureBoxVideoPartner.Name = "pictureBoxVideoPartner";
            this.pictureBoxVideoPartner.Size = new System.Drawing.Size(319, 210);
            this.pictureBoxVideoPartner.TabIndex = 3;
            this.pictureBoxVideoPartner.TabStop = false;
            //
            // label1
            //
            this.label1.AutoSize = true;
            this.label1.Location = new System.Drawing.Point(25, 281);
            this.label1.Name = "label1";
            this.label1.Size = new System.Drawing.Size(0, 13);
            this.label1.TabIndex = 4;
            //
            // label2
            //
            this.label2.AutoSize = true;
            this.label2.Location = new System.Drawing.Point(47, 249);
            this.label2.Name = "label2";
            this.label2.Size = new System.Drawing.Size(54, 13);
            this.label2.TabIndex = 5;
            this.label2.Text = "Partner IP";
            //
            // label3
            //
            this.label3.AutoSize = true;
            this.label3.Location = new System.Drawing.Point(47, 273);
            this.label3.Name = "label3";
            this.label3.Size = new System.Drawing.Size(63, 13);
            this.label3.TabIndex = 6;
            this.label3.Text = "Partner Port";
            //
            // label4
            //
            this.label4.AutoSize = true;
            this.label4.Location = new System.Drawing.Point(47, 321);
            this.label4.Name = "label4";
            this.label4.Size = new System.Drawing.Size(73, 13);
            this.label4.TabIndex = 7;
            this.label4.Text = "Webcam - Nr.";
            //
            // textBox1
            //
            this.textBox1.Location = new System.Drawing.Point(128, 249);
            this.textBox1.Name = "textBox1";
            this.textBox1.Size = new System.Drawing.Size(158, 20);
            this.textBox1.TabIndex = 8;
            //
            // textBox2
            //
            this.textBox2.Location = new System.Drawing.Point(128, 273);
            this.textBox2.Name = "textBox2";
            this.textBox2.Size = new System.Drawing.Size(158, 20);
            this.textBox2.TabIndex = 9;
            //
            // textBox3
            //
            this.textBox3.Location = new System.Drawing.Point(128, 321);
            this.textBox3.Name = "textBox3";
            this.textBox3.Size = new System.Drawing.Size(158, 20);
            this.textBox3.TabIndex = 10;
            this.textBox3.Text = "0";
            //
            // textBox4
            //
            this.textBox4.Location = new System.Drawing.Point(128, 297);
            this.textBox4.Name = "textBox4";
            this.textBox4.Size = new System.Drawing.Size(158, 20);
            this.textBox4.TabIndex = 10;
            //
            // label5
            //
            this.label5.AutoSize = true;
            this.label5.Location = new System.Drawing.Point(47, 297);
            this.label5.Name = "label5";
            this.label5.Size = new System.Drawing.Size(64, 13);
            this.label5.TabIndex = 11;
            this.label5.Text = "eigener Port";
            //
            // Form1
            //
            this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
            this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
            this.ClientSize = new System.Drawing.Size(758, 370);
            this.Controls.Add(this.textBox4);
            this.Controls.Add(this.label5);
            this.Controls.Add(this.textBox3);
            this.Controls.Add(this.textBox2);
            this.Controls.Add(this.textBox1);
            this.Controls.Add(this.label4);
            this.Controls.Add(this.label3);
            this.Controls.Add(this.label2);
            this.Controls.Add(this.label1);
            this.Controls.Add(this.pictureBoxVideoPartner);
            this.Controls.Add(this.button1);
            this.Controls.Add(this.pictureBoxVideoSelf);
            this.Name = "Form1";
            this.Text = "C# Webcam Chat";
            this.FormClosing += new System.Windows.Forms.FormClosingEventHandler(this.Form1_FormClosing);
            ((System.ComponentModel.ISupportInitialize)(this.pictureBoxVideoSelf)).EndInit();
            ((System.ComponentModel.ISupportInitialize)(this.pictureBoxVideoPartner)).EndInit();
            this.ResumeLayout(false);
            this.PerformLayout();

        }

        #endregion

        private System.Windows.Forms.PictureBox pictureBoxVideoSelf;
        private System.Windows.Forms.Button button1;
        private System.Windows.Forms.PictureBox pictureBoxVideoPartner;
        private System.Windows.Forms.Label label1;
        private System.Windows.Forms.Label label2;
        private System.Windows.Forms.Label label3;
        private System.Windows.Forms.Label label4;
        private System.Windows.Forms.TextBox textBox1;
        private System.Windows.Forms.TextBox textBox2;
        private System.Windows.Forms.TextBox textBox3;
        private System.Windows.Forms.TextBox textBox4;
        private System.Windows.Forms.Label label5;
    }
}

No comments:

Post a Comment