C#: WebClient Usage

May 9, 2008 by
 

Microsoft has provided a great utility since .NET 1.0 for doing quick I/O via websites.  The WebClient class provides basic functionality for downloading from and uploading to webservers.  the WebClient example on MSDN leaves a lot to be desired, especially for beginners.  I’d like to expand on that, as well as provide some example code.

C#: Using WebClient to fetch a page:

        // create a new instance of WebClient
        WebClient client = new WebClient();
 
        // set the user agent to IE6
        client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)");
        try
        {
            // actually execute the GET request
            string ret = client.DownloadString("http://www.google.com/");
 
            // ret now contains the contents of the webpage
            Console.WriteLine("First 256 bytes of response: " + ret.Substring(0,265));
        }
        catch (WebException we)
        {
            // WebException.Status holds useful information
            Console.WriteLine(we.Message + "\n" + we.Status.ToString());
        }
        catch (NotSupportedException ne)
        {
            // other errors
            Console.WriteLine(ne.Message);
        }

(This code uses DownloadString, you can also use DownloadData for a more binary-friendly version)

This is great for fetching simple pages that have data encoded in the querystring, but there are some problems with the basic DownloadString method of WebClient.  It’s synchronous, so it will block until the operation completes.  So for slow connections, or large files, this would need to run in another thread.  There is a better way.  But first, here’s another example of another basic, but important, method, DownloadFile.

        // create a new instance of WebClient
        WebClient client = new WebClient();
 
        // set the user agent to IE6
        client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)");
        try
        {
            // actually execute the GET request
            client.DownloadFile("http://www.google.com/","google_fetch.txt");
 
            // ret now contains the contents of the webpage
            Console.WriteLine("File Saved.");
        }
        catch (WebException we)
        {
            // WebException.Status holds useful information
            Console.WriteLine(we.Message + "\n" + we.Status.ToString());
        }
        catch (NotSupportedException ne)
        {
            // other errors
            Console.WriteLine(ne.Message);
        }

This example is almost identical to the above, aside from the client.DownloadFile method.  This will save the file to disk, instead of returning a string–often what you would end up doing with the string anyway.

But can I send POST data?

Yes!  Using WebClient.UploadString or WebClient.UploadData you can POST data to the server easily.  I’ll show an example using UploadData, since UploadString is used in the same manner as DownloadString.

            byte[] bret = client.UploadData("http://www.website.com/post.php", "POST",
                System.Text.Encoding.ASCII.GetBytes("field1=value1&field2=value2") );
 
            string sret = System.Text.Encoding.ASCII.GetString(bret);

UploadData returns a byte array (byte[]) which contains the contents of the page.  This can easily be converted to a string by using the System.Text.Encoding.ASCII.GetString() method.  You can also use the other encoding types in System.Text.Encoding (such as UTF8) to do the same.

Also note the use of GetBytes(), the UploadData method will only take a byte array as a buffer for the upload.  This works well if you are uploading a binary file.

What about the asynchronous methods of WebClient?

WebClient provides asynchronous methods for fetching webpages as well, they are named similarly to the synchronous methods, DownloadFileAsync, DownloadDataAsync, DownloadStringAsync, UploadFileAsync, UploadDataAsync, UploadStringAsync, and UploadValuesAsync.

In order to execute an Async request, you will need to attach event handlers before the method is executed.  Without these event handlers you will not have any sense of when the operation finishes, or what stage it is at.

C# WebClient Asynchronous call example:

void do_upload() {
   WebClient client = new WebClient();
   // add event handlers for completed and progress changed
   client.UploadProgressChanged += new UploadProgressChangedEventHandler(client_UploadProgressChanged);
 
   client.UploadFileCompleted += new UploadFileCompletedEventHandler(client_UploadFileCompleted);
  // carry out the operation as normal
   client.UploadFileAsync("http://www.daveamenta.com/up.php", @"c:\somefile.bin");
}
 
void client_UploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
{
   Console.WriteLine(e.ProgressPercentage);
}
 
void client_UploadFileCompleted(object sender, UploadFileCompletedEventArgs e)
{
   if(e.Result != null) {
      Console.WriteLine(System.Text.Encoding.ASCII.GetString(e.Result));
   }
}

The call to client.DownloadFileAsync will no longer block the thread, and will execute in the background, periodically calling the event handlers to display progress.  The other methods can be used in this same way.

Comments

16 Comments on C#: WebClient Usage

  1. Marek on Tue, 16th Sep 2008 7:43 AM

    2. Simple and wonderfull!

  2. Reza on Mon, 27th Oct 2008 11:40 PM

    3. Dave, is it possible to display the time before it the download is started? I have a situation where I have encapsulated the webclient download in a button_click event. The first item on the event is to display the datetime.now (and in the async_download_completed method) im display the time again. I seem to never be able to display the start time and the end time until the actual download is done. im running on an ASP.NET page.
    thanks

  3. WebClient usage - Baka.Blog on Thu, 4th Dec 2008 6:06 AM

    4. [...] http://www.daveamenta.com/2008-05/c-webclient-usage/ [...]

  4. Yoann. B on Thu, 4th Dec 2008 9:56 PM

    5. Great Post !

    Thanks.

  5. kimbin on Mon, 23rd Mar 2009 10:09 AM

    6. thank you for your help

  6. kimbin on Mon, 23rd Mar 2009 10:09 AM

    7. thanks

  7. mistdewfire on Wed, 29th Jul 2009 3:48 PM

    8. Hi , i need to login to a website say login.aspx and then access next page say HOMEPAGE.aspx using webclient.
    I provide username and password like:

    byte[] bret = client.UploadData(“http://www.website.com/login.aspx”, “POST”,
    System.Text.Encoding.ASCII.GetBytes(“tbxUserName=UserName&tbxPassword=Password”) );

    string sret = System.Text.Encoding.ASCII.GetString(bret);

    But sret contains only html of Login.aspx. I want the html of the page that i go to after clicking SUBMIT button on login page. So how to click submit button using webclient and get information about successful login to website.

  8. Milan on Tue, 15th Dec 2009 12:27 AM

    9. Hi, can we add cookie handling in webclient, if yes then how?

    please can you provide some links.

  9. Derrick on Thu, 11th Mar 2010 9:43 AM

    10. Very good examples here!

  10. cesar on Thu, 25th Mar 2010 1:12 PM

    11. Man how I can get the file name uploaded or something like that, for example I want show:
    xxxx uploaded. I want that the user know how file was complete

  11. Chakri on Sun, 28th Mar 2010 4:55 AM

    12. When i use “DownloadStringCompleted”, i for Login page. HOw can i extend the authentication from current request to web client’s request?

    Thanks in advance!

  12. Using WebClient with UploadValues and UploadString to simulate POST « Anhonga's Blog on Fri, 7th May 2010 5:29 PM

    13. [...] concise description of WebClient can be found here. Leave a [...]

  13. fynnbob on Thu, 15th Jul 2010 5:17 PM

    14. Being new to web dev I would love to see the other end of this call (how is the data handled on the web site).

  14. John Morgan Wight on Fri, 10th Dec 2010 12:50 AM

    15. Thank you for the great tutorial/demo. Very clearly, and throughly explained. The one on MS’s website is much less informative. Very helpful!

  15. Cáceres on Wed, 9th Feb 2011 7:28 AM

    16. Great! Simple is beautiful!!

  16. omid on Mon, 9th May 2011 7:23 PM

    17. Hi everyone,

    straight to the point, I am using WebClient.UploadData to send chunked file to my server, meanwhile on the server side, I write every received chunk to a file.Part[x], now, when I want to write some codes for Pause and resume, I am skipping the “Request.SaveAs” for those chunk parts that already there on server ,and then I get exception error on Client side which is Underlying connection was forcedly closed !!!
    Is there any way to pause and resume the data that are being sent by WebClient.UploadData() !!!!!? please help.