Digsby ToGo, and Digsby URL resolution!
I’ve been too busy the last couple of days to update, but I would like to showcase two small projects I have been working on.
First, is Digsby ToGo:
Digsby ToGo is a remote access solution, similar to Trillian Anywhere. It allows you to access your current conversations from a web browser, and to interact exactly as how you would at the desktop. This solution is seemless to the chat partner.
And the other project, is to have Digsby resolve URL’s in conversations. So a link to youtube: http://www.youtube.com/watch?v=MgpzUo_kbFY – which carries no meaning, will replaced with the title of the page, far more descriptive. This all happens in the background.
More to come!
Windows XP SP3: Give Microsft a break!
I ran into this article today over at TechSpot, and became immediately annoyed at the way they are nitpicking at the decisions Microsoft made on the release of SP3.
I put SP3 on my laptop last week, it installed much the same way that XP SP1 and SP2 did, it took around 25 minutes to install, and everything feels the same as before the update. I have had problems in the past updating computers to SP2 and getting a blue-screen due to unexpected winlogon.exe termination. Perhaps it’s just that I do like Microsoft, and I cut them slack and don’t expect everything to be perfect, or perhaps it’s because I work in the industry and know how complicated it can be to push an update of this magnitude.
Windows is really complicated. It’s not as simple as it looks from a user perspective, there are very complex systems in Windows (this does apply to any operating system–OSX and Linux are just as complex), pushing an update can have an adverse affect on any one of those systems.
The Microsoft Windows operating system supports thousands of devices. More than 30,000 [as of October 2006] drivers have been released; more are introduced daily.
It is not a secret that the success of Windows in the recent years has been largely due to the massive hardware and software that are compatible with Windows, and only Windows. Microsoft has certainly put a substantial amount of effort into ensuring that more software and more hardware works with Windows than any other platform. When Apple releases an update to OSX, things break. Users expect software to break. Many applications weren’t compatible with 10.4 Tiger to 10.5 Leopard. Apple sees this as acceptable, and perhaps necessary to build a rock solid OS over time. I’m quite sure that the API for OSX has changed and evolved over the years, and is far cleaner and simple to use than Windows win32 API. This backwards compatibility–something that Microsoft often gets flack for, is one of their strongest selling points, and the track record of compatibility is one that businesses can depend on. Something written for Windows XP will still likely work on Windows 95, unless it is using advanced graphics, or has a dependency on the .NET CLR.
For these reaons, a service pack–or even an update can and eventually will break something. It’s a delicate system, if you want massive hardware and software support, you will need to deal with some of the shortcomings.
The aforementioned article also mentions how SP3 won’t be installed on any computer with IE8 Beta–and that if you install on a computer with IE7, it will no longer be able to roll back to IE6. This is because the components in IE (mainly the Trident Layout engine) are used in other parts of Windows. If the service pack allowed IE8b installed, or to roll back to a time before the service pack, file version inconsistencies would be created. There are many ways that this could happen, and it would have had to have been extensively tested. This costs more, and benefits very very few users. Most would not complain about the limitation.
C#: WebClient Usage
Microsoft has provided a great utility since .NET 1.0 for doing quick I/O via websites. The WebClient class provides basic functionality for downloading from and uploading to webservers. the WebClient example on MSDN leaves a lot to be desired, especially for beginners. I’d like to expand on that, as well as provide some example code.
C#: Using WebClient to fetch a page:
// create a new instance of WebClient WebClient client = new WebClient(); // set the user agent to IE6 client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)"); try { // actually execute the GET request string ret = client.DownloadString("http://www.google.com/"); // ret now contains the contents of the webpage Console.WriteLine("First 256 bytes of response: " + ret.Substring(0,265)); } catch (WebException we) { // WebException.Status holds useful information Console.WriteLine(we.Message + "\n" + we.Status.ToString()); } catch (NotSupportedException ne) { // other errors Console.WriteLine(ne.Message); }
(This code uses DownloadString, you can also use DownloadData for a more binary-friendly version)
This is great for fetching simple pages that have data encoded in the querystring, but there are some problems with the basic DownloadString method of WebClient. It’s synchronous, so it will block until the operation completes. So for slow connections, or large files, this would need to run in another thread. There is a better way. But first, here’s another example of another basic, but important, method, DownloadFile.
// create a new instance of WebClient WebClient client = new WebClient(); // set the user agent to IE6 client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)"); try { // actually execute the GET request client.DownloadFile("http://www.google.com/","google_fetch.txt"); // ret now contains the contents of the webpage Console.WriteLine("File Saved."); } catch (WebException we) { // WebException.Status holds useful information Console.WriteLine(we.Message + "\n" + we.Status.ToString()); } catch (NotSupportedException ne) { // other errors Console.WriteLine(ne.Message); }
This example is almost identical to the above, aside from the client.DownloadFile method. This will save the file to disk, instead of returning a string–often what you would end up doing with the string anyway.
But can I send POST data?
Yes! Using WebClient.UploadString or WebClient.UploadData you can POST data to the server easily. I’ll show an example using UploadData, since UploadString is used in the same manner as DownloadString.
byte[] bret = client.UploadData("http://www.website.com/post.php", "POST", System.Text.Encoding.ASCII.GetBytes("field1=value1&field2=value2") ); string sret = System.Text.Encoding.ASCII.GetString(bret);
UploadData returns a byte array (byte[]) which contains the contents of the page. This can easily be converted to a string by using the System.Text.Encoding.ASCII.GetString() method. You can also use the other encoding types in System.Text.Encoding (such as UTF8) to do the same.
Also note the use of GetBytes(), the UploadData method will only take a byte array as a buffer for the upload. This works well if you are uploading a binary file.
What about the asynchronous methods of WebClient?
WebClient provides asynchronous methods for fetching webpages as well, they are named similarly to the synchronous methods, DownloadFileAsync, DownloadDataAsync, DownloadStringAsync, UploadFileAsync, UploadDataAsync, UploadStringAsync, and UploadValuesAsync.
In order to execute an Async request, you will need to attach event handlers before the method is executed. Without these event handlers you will not have any sense of when the operation finishes, or what stage it is at.
C# WebClient Asynchronous call example:
void do_upload() { WebClient client = new WebClient(); // add event handlers for completed and progress changed client.UploadProgressChanged += new UploadProgressChangedEventHandler(client_UploadProgressChanged); client.UploadFileCompleted += new UploadFileCompletedEventHandler(client_UploadFileCompleted); // carry out the operation as normal client.UploadFileAsync("http://www.daveamenta.com/up.php", @"c:\somefile.bin"); } void client_UploadProgressChanged(object sender, UploadProgressChangedEventArgs e) { Console.WriteLine(e.ProgressPercentage); } void client_UploadFileCompleted(object sender, UploadFileCompletedEventArgs e) { if(e.Result != null) { Console.WriteLine(System.Text.Encoding.ASCII.GetString(e.Result)); } }
The call to client.DownloadFileAsync will no longer block the thread, and will execute in the background, periodically calling the event handlers to display progress. The other methods can be used in this same way.
C#: Send text to a pastebin (HttpWebRequest POST example)
I’m working on a small application that hooks into the clipboard monitor chain and displays a list of options whenever text is saved to the clipbaord. One of the options I wanted was to be able to send data to pastebin.com, a popular text/code sharing service. This is a relatively mundane task, faking the postdata for a webpage, and sending the requests as a browser. There were some hicups though, mostly with Finding the redirect URL using HttpWebRequest/HttpWebResponse.
A little background:
HttpWebRequest and HttpWebResponse live in the System.Net namespace, and provide the ability to quickly send HTTP requests to a webserver. This is what a simple GET request would look like:
try { HttpWebRequest request = (HttpWebRequest) WebRequest.Create("http://davux.pastebin.com/pastebin.php"); HttpWebResponse res =(HttpWebResponse) request.GetResponse(); if (res.StatusCode == HttpStatusCode.OK) { StreamReader reader = new StreamReader(res.GetResponseStream()); Console.WriteLine(reader.ReadToEnd()); } else { // handle errors } res.Close(); } catch (Exception ex) { Console.WriteLine("Error: " + ex.Message); }
Note: For most simple tasks, it is wise to use System.Net.WebClient, which is not suitable for this task, because it does not expose the ability to disable automatic redirection.
To send new data to pastebin, it needs to be sent via HTTP POST. Using Firefox LiveHTTPHeaders, I determined what string needs to be sent in order to create a new entry. This can also be determined by reading the source of the page, and looking at each of the fields contained in the <form> block.
string post = "&parent_pid=&format=text&code2=" + code_text_to_send + "&poster=" + poster_name + "&paste=Send&expiry=" + expiry + "&email=";
- code_text_to_send – The data which should be used to create a new page.
- poster_name – The author of the post
- expiry – How long the data should be retained for. Options are ‘d’ for one day, ‘m’ for one month, and ‘f’ for indefinitely.
Note: Make sure to URLEncode each field that is sent!
After further examination of the headers and data sent back, I decided that the best way to extract the link was to catch the HTTP 302 Document Found response.
HTTP/1.x 302 Found Date: Thu, 08 May 2008 01:06:08 GMT Server: Apache/1.3.33 (Debian GNU/Linux) mod_python/2.7.10 Python/2.3.4 PHP/4.3.10-22 mod_perl/1.29 X-Powered-By: PHP/4.3.10-22 Location: http://davux.pastebin.com/m10a794d6 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1
HttpWebRequest, by default, has AllowAutoRedirect set to true, which means that, internally, it will resolve this message, and fetch the new URL. This extra operation is needless in this circumstance, so it is best to disable it and extract the URL from the header. The headers associated with each request or response are found in the Headers object for each class. We wish to look at the Location field to find our URL.
C# WebClient HTTP POST Example:
try { HttpWebRequest request = (HttpWebRequest) WebRequest.Create("http://davux.pastebin.com/pastebin.php"); request.AllowAutoRedirect = false; request.Method = "POST"; string post = "&parent_pid=&format=text&code2=" + HttpUtility.UrlEncode(text) + "&poster=Dave&paste=Send&expiry=m&email="; byte[] data = System.Text.Encoding.ASCII.GetBytes(post); request.ContentType = "application/x-www-form-urlencoded"; request.ContentLength = data.Length; Stream response = request.GetRequestStream(); response.Write(data,0,data.Length); response.Close(); HttpWebResponse res =(HttpWebResponse) request.GetResponse(); res.Close(); // note that there is no need to hook up a StreamReader and // look at the response data, since it is of no need if (res.StatusCode == HttpStatusCode.Found) { Console.WriteLine(res.Headers["location"]); } else { Console.WriteLine("Error"); } } catch (Exception ex) { Console.WriteLine("Error: " + ex.Message); }
(Some error checking and extra code have been removed for the sake of clarity)
The values for expiry and author have been hard-coded in order to save space. This basic example will create a new Pastebin entry on my not so private pastebin, and print the URL of the entry to the console. If there is an error, it will print the exception message. If the HTTP code returned is not 302/Found, it will print a generic error. (If the server is not returning a 302 for this request, there is nothing codewise that can be fixed, and thus the only option for error is generic.)
What is P/Invoke (PInvoke)?
P/Invoke, or Pinvoke stands for Platform Invocation Services. PInvoke is a feature of the Microsoft .NET Frameowrk that allows a developer to make calls to native code inside Dynamic Link Libraries (DLL’s). When Pinvoking, the .NET framework (or Common Language Routine) will load the DLL and handle the type conversions automatically. The most common use of P/Invoke is to use a feature of Windows that is only contained in the Win32 API. The API in Windows is extremely extensive and only some of the features are encapsulated in .NET libraries. For example, Form.Show(); is really a wrapper for the ShowWindow() API found in shell32.dll.
ShowWindow Declaration API for C#:
[DllImport("user32.dll")] static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
Note the use of the DllImport attribute. This attribute, coupled with extern tells the .NET CLR where it can find the native declaration for ShowWindow. shell32.dll must be registered on the system for this to work properly.
Note: Remember import the System.Runtime.InteropServices namespace! This is where the [DLLImport] attribute is found.
using System.Runtime.InteropServices;
How are P/Invoked methods called?
Methods that are loaded in using DLLImport are Pinvoked by simply treating them as native .NET functions, and calling them as you would call any other. Win32 API do tend to use data types that are not used often in .NET, for example – the IntPtr datatype, which is actually a pointer to an integer–in this case, a Window Handle. nCmdShow is an integer that represents the show command. But what values of nCmdShow mean what? These can either be found on MSDN, or PInvoke.net.
C# Constants for ShowWindow in shell32.dll:
const int SW_HIDE = 0; const int SW_SHOWNORMAL = 1; const int SW_NORMAL = 1; const int SW_SHOWMINIMIZED = 2; const int SW_SHOWMAXIMIZED = 3; const int SW_MAXIMIZE = 3; const int SW_SHOWNOACTIVATE = 4; const int SW_SHOW = 5; const int SW_MINIMIZE = 6; const int SW_SHOWMINNOACTIVE = 7; const int SW_SHOWNA = 8; const int SW_RESTORE = 9; const int SW_SHOWDEFAULT = 10; const int SW_FORCEMINIMIZE = 11; const int SW_MAX = 11;
These constants can be used to call ShowWindow(). So in our origional example of Form.Show(), the P/Invoked way of doing this would be:
ShowWindow(Form.Handle, SW_SHOW);
This isn’t nearly as clean as the Object-Oriented way of using Form.Show(), but this is what is actually happening under the hood of .NET.
Why would I ever want something so overly complicated?
ShowWindow is an example of an API that has been wrapped nicely by the .NET CLR, it is unlikely that you will ever need to import and call it. But something more useful might be: Deleting a file to the recycle bin.
What resources are out there for PInvoking?
The best resource I have found is PInvoke.net, which has declarations for both VB.NET and C# for commonly used Win32 API.
C#: URLEncode and URLDecode?
Often one needs to make a call to a webserver in .NET, and just as often there is Querystring data that needs to be sent along with this request. Thanksfully .NET has a great WebClient class in System.net.WebClient. But you can’t just add the querystring data into your URL like such:
Wrong: WebClient.DownloadFile("http://www.daveamenta.com/file.aspx?mydata=this is a string that I need to send to the server?");
The data needs to be encoded so that the server will not misinterpret the data. This is an example of using an encoded string:
Right: WebClient.DownloadFile("http://www.daveamenta.com/file.aspx?mydata=this+is+a+string+that+I+need+to+send+to+the+server%3F");
The second string is encoded properly, this way the webserver will properly parse that last ?. This was done using the URLEncode function, .NET has built-in classes for this–but they are a bit hidden. They’re kept in the System.Web namespace. This means that if you are working with .NET on the desktop side, as opposed to ASP.NET, you’ll need to Add a Reference to System.Web.dll. Once you do, you’ll see new classes in System.Web that you can use for many things.
Using URLEncode in C#:
WebClient.DownloadFile( "http://www.daveamenta.com/file.aspx?mydata=" + System.Web.HttpUtility.UrlEncode( "this is a string that I need to send to the server?" ) );
It is unlikely you will have much use for URLDecode in a C# application, but it is applied the same way.
What exactly is URLEncode?
URLEncode returns a string in which all non-alphanumeric characters other than – (hyphen) _ (underscore), and . (period) have been replaced with a percent (%) sign followed by two hex digits that correspond to the charactor code and spaces encoded as plus (+) signs. Spaces can alternatively be encoded as %20. 20 is (16*2)+(1*0)=32 in decimal, which is the ASCII code for space.
URLDecode will reverse this operation.
C#: Find location of My Documents, Application Data, Program Files and other Windows directories
I’ve seen the (perhaps VB-style) way of getting the MyDocuments folder, by using %HOMEDIR%, a batch-style variable. Some of these environment variables can be modified by navigating to Control Panel -> System -> Advanced (tab). In Vista, you’ll need to select Advanced System Settings at the System page in order to invoke the System Properties dialog.
These environment varibles work as pathes, but they are not really the accepted way of doing things. What you should do, is use System.Environment.GetFolderPath(). This method takes a value from the System.Environment.SpecialFolder enum. GetFolderPath will return a string for your location. This is supported on all versionf of the .NET framework.
Below is an excerpt from MSDN of the possible values for SpecialFolder:
| Name | Description | |
|---|---|---|
| ApplicationData | The directory that serves as a common repository for application-specific data for the current roaming user. | |
| CommonApplicationData | The directory that serves as a common repository for application-specific data that is used by all users. | |
| LocalApplicationData | The directory that serves as a common repository for application-specific data that is used by the current, non-roaming user. | |
| Cookies | The directory that serves as a common repository for Internet cookies. | |
| Desktop | The logical Desktop rather than the physical file system location. | |
| Favorites | The directory that serves as a common repository for the user’s favorite items. | |
| History | The directory that serves as a common repository for Internet history items. | |
| InternetCache | The directory that serves as a common repository for temporary Internet files. | |
| Programs | The directory that contains the user’s program groups. | |
| MyComputer | The “My Computer” folder. | |
| MyMusic | The “My Music” folder. | |
| MyPictures | The “My Pictures” folder. | |
| Recent | The directory that contains the user’s most recently used documents. | |
| SendTo | The directory that contains the Send To menu items. | |
| StartMenu | The directory that contains the Start menu items. | |
| Startup | The directory that corresponds to the user’s Startup program group. | |
| System | The System directory. | |
| Templates | The directory that serves as a common repository for document templates. | |
| DesktopDirectory | The directory used to physically store file objects on the desktop. | |
| Personal | The directory that serves as a common repository for documents. | |
| MyDocuments | The “My Documents” folder. | |
| ProgramFiles | The program files directory. | |
| CommonProgramFiles | The directory for components that are shared across applications. |
C#: Delete a file to the recycle bin
I had a friend ask me yesterday what the easiest/best way to delete a file to the recycle bin was, in C# that is. I knew about SHFileOperation, but I didn’t know if .NET 2.0 had native support for sending files to the recycle bin. As it turns out, there are two ways in which you can move files to the recycle bin. Microsoft has included the support in the Microsoft.VisualBasic namespace, and referencing the Microsoft.VisualBasic assembly will allow you to use that function directly.
Deleting a file using Managed code in C# or VB.NET:
(Remember to add a reference to Microsoft.VisualBasic)
using Microsoft.VisualBasic; string path = @"c:\myfile.txt"; FileIO.FileSystem.DeleteDirectory(path, FileIO.UIOption.OnlyErrorDialogs, .RecycleOption.SendToRecycleBin);
The other alternative is to use COM Interop and P/Invoke the SHFileOperation from the Windows shell32.dll.
Deleting a file using P/Invoke in C#:
I have written a class that provides methods to easily specify how the file should be sent to the recycle bin. SHFileOperation can take flags, which I have put into an enum and commented. These flags define how the operation should be executed.
- FOF_SILENT – Do not show the “Recycling…” dialog.
- FOF_NOCONFIRMATION – Do not prompt the use for confirmation of the operation.
- FOF_ALLOWUNDO – This is the option that specifies that the file should be sent to the recycle bin. It is added by default in each Send method.
- FOF_SIMPLEPROGRESS – This option will surpress the display of the names of the files that are being deleted.
- FOF_NOERRORUI – This option will suppress any errors encountered during the operation.
- FOF_WANTNUKEWARNING – Adding this flag will allow a dialog to allow the user to cancel removing large files permanently, if they are too large for the recycle bin.
Any one of these flags can be added to change the operation.
Methods provided by the RecycleBin class:
- bool Send(string path)
- bool Send(string path, FileOperationFlags flags)
- static bool SendSilent(string path)
The first will simply send a path to the bin while showing the dialog, but surpressing the confirmation.
The second will allow you to customize the flags for the file operation, the only flag set for you is FOF_ALLOWUNDO. Flags can be combined with the | operator. e.g. FOF_NOERRORUI | FOF_SIMPLEPROGRESS.
The third, and probably most commonly used, will surpress all errors and warnings, and not show the user any operation is in progress.
Code Sample:
SendToRecycleBin.zip [ZIP] [Visual Studio 2008] (43KB)
Update – 04/18/2010: Updated sample for x64 compatibility.
HowTo: Save and retrieve C# objects in XML
I’ve spent a bit of time tweaking this XML Serialization class I wrote to make use of the System.Xml.Serialization objects Microsoft provides in .NET 2.0.
XML Serialization is a great way to store complex data types, and also a great alternative to binary serialization. Unfortunately, using these classes is not as straightforward as I would like, there isn’t any simple Save or Load methods, I’ve bridged that gap with a a small class that provides this functionality.
Basic Methods:
These are the two basic methods you would need to save and load an object from a file.
- public static T Load<T>(string filename)
- public static void Save<T>(string filename, T cls)
Additional Methods:
These methods are included for those that would like to bypass the files, and send objects directly over a different medium.
- public static String SerializeObject<T>(T pObject)
- public static T DeserializeObject<T>(String pXmlizedString)
Important Note:
Remember to wrap all of these methods in try-catch blocks and catch any exceptions thrown. If the file cannot be accessed, or there is an error during serialization, it will be passed on.
Code Example:
Load an object from file:
List MyList = null; try { MyList = XSerial.Load<list>("MyList.xml"); } catch (Exception e) { Console.WriteLine("Error: " + e.Message); // make sure the object won't cause errors. MyList = new List(); } Save an object to file: try { XSerial.Save</list><list>("MyList.xml", MyList); } catch (Exception e) { Console.WriteLine("Error Saving: " + e.Message); // nothing can be done about recovery. }</list>
Download Demo Project:
XMLSerialTest [ZIP] [Visual Studio 2008]
Tweet from mIRC | mIRC Twitter
Twitter is great, and it’s nice that there is more than one way to send a tweet (message) out, you can use Web, SMS or Google Talk (or even Digsby!). But what if you’re a heavy mIRC user, or if you just like to have more ways to tweet? Clint, on DALnet/#system was working on this, and I’ve added a few things that I think are worth sharing.
How can I use twitter through mIRC?
This would be entered into the Remote script section in mIRC:
alias tweet { ; use /tweet set %tw.username twitter_username_here set %tw.password twitter_password_here ; exit if the message is too long for twitter to take. if ($len($1-) > 140) { echo -a Sorry, that was $calc($len($1-)-140) characters too long! halt } ; connect to twitter on HTTP port 80 set %authentication $encode($+(%tw.username,:,%tw.password),m) sockclose twitter sockopen twitter twitter.com 80 ; set a 2-second timeout in case there is a problem ; this could be changed to 5 or 10 .timertwitter 1 2 twitter_timeout set %tweet $$1- ; store the text } ; encoding is important for sending data to a web server ; the decode function is not used, but is here for the sake of ; completeness alias urlencode return $regsubex($1-,/\G(.)/g,$iif(($prop && \1 !isalnum) || !$prop,$chr(37) $+ $base($asc(\1),10,16),\1)) alias urldecode return $replace($regsubex($1-,/%(\w\w)/g,$chr($iif($base(\t,16,10) != 32,$v1,1))),$chr(1),$chr(32)) ; when the socket accepts our connection on *:sockopen:twitter:{ sockwrite -n twitter POST /statuses/update.json HTTP/1.1 sockwrite -n twitter Host: twitter.com sockwrite -n twitter User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 sockwrite -n twitter Content-Length: $calc($len($urlencode(%tweet)) + 9) sockwrite -n twitter Authorization: Basic %authentication sockwrite -n twitter $crlf sockwrite twitter status= sockwrite -n twitter $urlencode(%tweet) sockwrite twitter $crlf sockwrite twitter $crlf ; send the postdata } ; the data wasn't returned, kill the socket ; and tell the user. alias twitter_timeout { echo -a Message Failed to Send - Socket Timeout sockclose twitter } ; we have data back on *:sockread:twitter: { .timertwitter off sockread -f %string echo -a Message Returned ( $+ %string $+ ) sockclose twitter }
Hopefully this is useful to someone else, I often find myself updating twitter this way, because I always have mIRC open, albeit in the background.






