C#: WebClient Usage
Microsoft has provided a great utility since .NET 1.0 for doing quick I/O via websites. The WebClient class provides basic functionality for downloading from and uploading to webservers. the WebClient example on MSDN leaves a lot to be desired, especially for beginners. I’d like to expand on that, as well as provide some example code.
C#: Using WebClient to fetch a page:
// create a new instance of WebClient WebClient client = new WebClient(); // set the user agent to IE6 client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)"); try { // actually execute the GET request string ret = client.DownloadString("http://www.google.com/"); // ret now contains the contents of the webpage Console.WriteLine("First 256 bytes of response: " + ret.Substring(0,265)); } catch (WebException we) { // WebException.Status holds useful information Console.WriteLine(we.Message + "\n" + we.Status.ToString()); } catch (NotSupportedException ne) { // other errors Console.WriteLine(ne.Message); }
(This code uses DownloadString, you can also use DownloadData for a more binary-friendly version)
This is great for fetching simple pages that have data encoded in the querystring, but there are some problems with the basic DownloadString method of WebClient. It’s synchronous, so it will block until the operation completes. So for slow connections, or large files, this would need to run in another thread. There is a better way. But first, here’s another example of another basic, but important, method, DownloadFile.
// create a new instance of WebClient WebClient client = new WebClient(); // set the user agent to IE6 client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)"); try { // actually execute the GET request client.DownloadFile("http://www.google.com/","google_fetch.txt"); // ret now contains the contents of the webpage Console.WriteLine("File Saved."); } catch (WebException we) { // WebException.Status holds useful information Console.WriteLine(we.Message + "\n" + we.Status.ToString()); } catch (NotSupportedException ne) { // other errors Console.WriteLine(ne.Message); }
This example is almost identical to the above, aside from the client.DownloadFile method. This will save the file to disk, instead of returning a string–often what you would end up doing with the string anyway.
But can I send POST data?
Yes! Using WebClient.UploadString or WebClient.UploadData you can POST data to the server easily. I’ll show an example using UploadData, since UploadString is used in the same manner as DownloadString.
byte[] bret = client.UploadData("http://www.website.com/post.php", "POST", System.Text.Encoding.ASCII.GetBytes("field1=value1&field2=value2") ); string sret = System.Text.Encoding.ASCII.GetString(bret);
UploadData returns a byte array (byte[]) which contains the contents of the page. This can easily be converted to a string by using the System.Text.Encoding.ASCII.GetString() method. You can also use the other encoding types in System.Text.Encoding (such as UTF8) to do the same.
Also note the use of GetBytes(), the UploadData method will only take a byte array as a buffer for the upload. This works well if you are uploading a binary file.
What about the asynchronous methods of WebClient?
WebClient provides asynchronous methods for fetching webpages as well, they are named similarly to the synchronous methods, DownloadFileAsync, DownloadDataAsync, DownloadStringAsync, UploadFileAsync, UploadDataAsync, UploadStringAsync, and UploadValuesAsync.
In order to execute an Async request, you will need to attach event handlers before the method is executed. Without these event handlers you will not have any sense of when the operation finishes, or what stage it is at.
C# WebClient Asynchronous call example:
void do_upload() { WebClient client = new WebClient(); // add event handlers for completed and progress changed client.UploadProgressChanged += new UploadProgressChangedEventHandler(client_UploadProgressChanged); client.UploadFileCompleted += new UploadFileCompletedEventHandler(client_UploadFileCompleted); // carry out the operation as normal client.UploadFileAsync("http://www.daveamenta.com/up.php", @"c:\somefile.bin"); } void client_UploadProgressChanged(object sender, UploadProgressChangedEventArgs e) { Console.WriteLine(e.ProgressPercentage); } void client_UploadFileCompleted(object sender, UploadFileCompletedEventArgs e) { if(e.Result != null) { Console.WriteLine(System.Text.Encoding.ASCII.GetString(e.Result)); } }
The call to client.DownloadFileAsync will no longer block the thread, and will execute in the background, periodically calling the event handlers to display progress. The other methods can be used in this same way.
Simple and wonderfull!
Dave, is it possible to display the time before it the download is started? I have a situation where I have encapsulated the webclient download in a button_click event. The first item on the event is to display the datetime.now (and in the async_download_completed method) im display the time again. I seem to never be able to display the start time and the end time until the actual download is done. im running on an ASP.NET page.
thanks
[...] http://www.daveamenta.com/2008-05/c-webclient-usage/ [...]
Great Post !
Thanks.
thank you for your help
thanks
Hi , i need to login to a website say login.aspx and then access next page say HOMEPAGE.aspx using webclient.
I provide username and password like:
byte[] bret = client.UploadData(“http://www.website.com/login.aspx”, “POST”,
System.Text.Encoding.ASCII.GetBytes(“tbxUserName=UserName&tbxPassword=Password”) );
string sret = System.Text.Encoding.ASCII.GetString(bret);
But sret contains only html of Login.aspx. I want the html of the page that i go to after clicking SUBMIT button on login page. So how to click submit button using webclient and get information about successful login to website.
Hi, can we add cookie handling in webclient, if yes then how?
please can you provide some links.