« Previous Entries

Category

Software

• Security, Software, Technology

  WinHasher 1.6

  February 8th, 2010 (1 month ago) by Jeff | Permalink | Dump Core

  Wow! A non-Twitter digest post! Amazing!

  This is a quickie to let you guys now I’ve just released WinHasher 1.6. This is a minor release containing a few cosmetic and minor functional changes, so there’s no need to upgrade unless the features or bug fixes listed below seem worth the effort.

  • Added a progress indication message for all the console applications. Previously, these programs simply went off into lala land when you kicked them off, kind of like the original GUI version did, giving you no indication how far along things have come.  Now the program displays what it’s doing (“Computing SHA-1 of…”) and then a percentage complete as it works.  Once complete, it displays the resulting hash as it did before. I thought about making the progress indicator “graphical” with little periods, hyphens, or equal signs, but the percentage was a lot easier to do (and I’m lazy).
  • Added a “case kludge” for comparing computed hashes against reference hashes (i.e. digests copied from a website). Typically, most sites will display hashes as a lower-case hexadecimal string, which is why I made that the default. However, I and other users have run into occasions where a download site presents the hash as upper-case hexadecimal. Unfortunately, if you paste this value into the comparison field on either the main WinHasher form or the “Send To” result dialog, the comparison will fail simply because the case does not match, even if the actual hash values do. Technically, you can tweak the command line of the “Send To” shortcut or change the output type on the main form, but either can be a pain when you know what you expect to see. So I’ve added a little bit of code that looks at what the comparison expects (i.e. “I’m looking for a lower-case hex string”) and adjusts the case of the comparison field if necessary. This means lower-case hex and Bubble Babble comparisons are forced to lower-case, while upper-case hex is forced to upper-case, no matter what case the original pasted-in string was. In the case of Base64, which includes mixed case characters, the comparison string is compared unaltered. After any case adjustment, the hash comparison occurs as previously designed.
  • Fixed a bug that wasn’t displaying the icon correctly within the application itself. Oopsy.

  For those who don’t know, WinHasher is a cryptographic hash generator for Microsoft .NET. It is roughly analogous to digest programs on other platforms (such as “openssl dgst” from OpenSSL) but designed for Windows and other .NET platforms. It lets you verify the integrity of downloads and determine whether changes have been made to files. It does NOT guarantee the authenticity of a file; for that, use cryptographic signatures such those produced by PGP or GnuPG. It also lets you create hashes of arbitrary text, which is handy for generating strong “passwords”, although I’m working on a different project that will do a much better job of this particular task. [Looks around shifty-eyed.]

  

• Software, Technology

  WinHasher 1.5 Released

  July 9th, 2009 (8 months ago) by Jeff | Permalink | Dump Core

  Just posting a quick note to let you guys know I’ve bumped good ol’ WinHahser to version 1.5. This is both a bug and feature release, so both of you using it will probably want to upgrade. Here’s a quick list of the changes:

  • Bug fix: When I wrote the hashing engine, I made sure to use a 64-bit integer when computing the total sum of the size of all files in a multi-file comparison. This allowed for a whopping total of 8.05EB (exabytes, or millions of gigabytes) of data to be compared. Unfortunately, I used the wrong data type when running this calculation on a single file, using a 32-bit integer instead. That put the cap on a single file at around 2GB, which can easily be exceeded by a decent-sized DVD ISO. The really embarrassing thing here is that .NET’s System.IO.FileStream object uses a 64-bit integer for its Length attribute, meaning this was totally my mistake, not Microsoft’s. The end result here is that WinHasher would crash on files larger than 2GB since it would end up trying to calculate its percent complete on an overflowed negative value. I’ve updated the code so that the single-file length calculations also use 64-bit integers and now I can finally validate that Fedora 11 DVD ISO download. Note that there’s still a hard cap at 8.05EB whether your hashing a single file or you sum up multiple files together. While it’s possible to bump this up to an unsigned 64-bit integer and go for even more ridiculous large numbers, I seriously doubt anyone is going to be running a SHA-1 hash that large any time soon.
  • The “Send To” shortcuts call the GUI app via the command line to hash a file and throw the result up into a dialog box, making it convenient to right-click a file and quickly get a hash. Unfortunately, I was using a standard MessageBox object for this, meaning the hash was displayed in a read-only form that couldn’t be copied and pasted elsewhere to be compared. (It’s much easier to copy and paste two hashes into a text editor, for example, and visually scan the two lines for differences.) Well, I wasn’t the only one to find this annoying.  WinHasher user Todd Henry had issues with this too and suggested that I either put the hash result in a text box that could be copied and pasted elsewhere, or add a box where an externally produced hash (say from a Web site) could be pasted into the dialog and have WinHasher compare them. Interestingly enough, I was already planning to make that change when he wrote me, and now it’s there. Once WinHasher is done, it will display a new result dialog with both a copyable hash result field and a new “compare to” field that will take an external hash string and tell you if it matches or not.
  • The Hash Single File tab in the main GUI app also has a new “compare to” field that works in an identical fashion to the one in the result dialog described above. I didn’t see a need to add it to the Hash Text tab, and it would be redundant on the Compare Files tab.
  • And in the completely useless trivia department, I’ve added hooks in to my Subversion clients to add the SVN revision number as the last part of the version number. This is more for my purposes than anything else, but now if you have a problem and can give me the full version number of the app, I know exactly what revision of the code you’re using.

  I realized after I updated the files and the site that I forgot to make any changes to the documentation to reflect these updates. Oh, well. I don’t think they’re major enough to sweat over, so I’ll leave those alone for now and make sure they get updated by the next release.

  

• Personal, Software, Technology

  Bill Gates now inhabits my dreams…

  September 9th, 2008 (1 year, 6 months ago) by Jeff | Permalink | Dump Core

  So I was listening to this week’s edition of TWiT, during which Leo Laporte and the usual band of miscreants psychoanalyze Microsoft’s new ad campaign featuring Bill Gates and Jerry Seinfeld. I had not seen the ad yet myself—apparently it debuted during an NFL opening game, and considering that I don’t watch professional sports and the overwhelming majority of my television watching now consists of shows containing magic backpacks and talking monkeys that wear red boots, it hadn’t come to my attention yet—so the discussion naturally raised my morbid curiosity. So I dug around a little on YouTube and found this. I must admit, it’s as surreal as I was led to believe. I won’t attempt to try and mine this thing for hidden meaning like Ryan Block did; the only comment I think I can really make about it is that it tells me absolutely nothing about Microsoft, Windows, or any other product they may have in the pipeline, and after watching it I am no more inclined to pick Microsoft options over the competition than I was before. I thought that was the point of advertising….

  But that’s not the weirdest part. Last night, I dreamed about Bill Gates. Maybe it was exhaustion, maybe it was a prescription-drug fueled haze (I’m currently in the middle of my quarterly bout with bronchitis), but it was not something I was particularly expecting. There’s nothing really interesting to say about the dream, though. In what little I remember, Mr. Gates was there, tying his shoes. He wasn’t necessarily trying on new ones, nor was there any indication that the shoes were noticeably old. They were shiny, brown leather dress shoes, so they could have been either new or well maintained. Mr. Seinfeld was nowhere in sight. The setting was unclear; I can’t say that it was a shoe store, a men’s locker room, or any other recognizable setting. I know only that I was seated on a wooden bench which I believe was painted a dark green and that Bill Gates stood next to me, lifted one leg, and set the foot on the bench, then proceeded to tie his shoe laces. Then he left without saying a word and the dream moved on to wherever it went after that. I remember nothing else about the dream, and to my knowledge Mr. Gates appeared nowhere else within it.

  I have no desire to do any research on what kind of Fruedian analysis can be drawn from watching a billionare-CEO-turned-philanthropist from one of the world’s largest and most reviled software companies tying his shoes next to me. I’d be afraid of what I’d find. So I’ll just say it was the prescription cough syrup working its magic and go back to talking to the pink elephant and the green roast beef sandwich on either side of me. It’s a conversation about world politics and an economy built entirely around edible golf balls will solve the world’s energy crisis. It’s very enlightening. Maybe, somehow, some way, we’ll figure out exactly what makes Windows “delicious” while we’re at it. Drug-enduced hysteria is about the only way I can think of in my current semi-lucid state to make an operating system taste delicious. It makes me begin to wonder, though… what would other OSes taste like? Would Mac OS be crunchy? Would Linux be spicy? Would my Treo’s PalmOS be light in calories? I certainly hope so… I am trying to lose weight….

  

• GPF, Internet, Software, Technology

  Generating XML Sitemaps for MediaWiki

  August 22nd, 2008 (1 year, 6 months ago) by Jeff | Permalink | 3 Core Dumps

  Not long ago, I took advantage of a nifty WordPress plugin to enable XML sitemaps for the blog. For those who’ve never heard of XML sitemaps (I hadn’t for quite a while), they are little XML files in a specific format that give search engines like Google hints on how to index your site. They don’t necessarily improve your search rankings per se, but they help the search engine better decide what to index, when it was last updated, relative priorities of different pages, etc. You then throw a special line into your robots.txt file or directly submit the file to the search engine to let it know the file is available. Once the engine knows about it, it will check it periodically to optimize how the site is indexed.

  The plugin, of course, makes this ridiculously easy for WordPress. However, GPF gets orders of magnitude higher traffic than the blog does, so finding a way to generate sitemaps there would be ideal. I toyed with the idea for a while until I finally sat down, examined the sitemap specification, and figured out how to roll my own code. It now successfully runs via cron each morning and gives a pretty thorough census of what’s available on the GPF server. The problem is that the GPF site is divided into several parts that are largely autonomous and self-contained:

  • The archive, of course, is the main bread and butter.  This is what people come to see, so it is vitally important to get everything represented, especially since the archive is entirely dynamic PHP. I count the main index page as part of the archive, as it displays the latest strip.
  • The forum is self-contained because it uses a third-party application: phpBB. I decided long ago to discourage spamming by blocking search engines from indexing the forum, relying on the forum’s internal search capabilities for users who want to find things. Since search engines were being blocked anyway, I decided the forum didn’t need a sitemap; that would just be a waste of processing time.
  • The wiki, however, is slowly becoming a vital part of the site. It replaces the old cast pages and adds tons more GPF metadata in a convenient, easily searchable form. I wanted to make sure that got included too.  The GPF Wiki runs MediaWiki, the same software that powers Wikipedia.  As far as I know, MediaWiki does not include any internal mechanism for generating XML sitemaps nor are there any extensions to do so.  (I could easily be wrong, however; I’ll admit my research on this was extremely limited.)
  • Then there’s everything else. The GPF site beyond the two pre-packaged items above is completely custom-built PHP, so there’s no one to blame for that mess but myself.

  Ignoring the forum, that left me three major sub-projects for creating sitemaps. It’s easy enough to segregate these into separate files and tie them together using a “sitemap index” file, so that wasn’t a problem.  The archive would just be a formatted dump of the archive database, deriving approximate update times from the posting date. The bulk of the rest of the site could be done by stepping through the file structure of the site and taking note of every HTML or PHP file and its last modification time (conveniently ignoring certain files and directories that don’t need to be counted, like access-restricted Premium pages). And that leaves the wiki.

  I managed to come up with a decent wiki sitemap routine that I thought I’d share, just in case someone else might be interested. Of course, it’s not likely to be useful for massive wikis like Wikipedia—sitemaps are restricted to 10MB in size and 50,000 URLs—but something small like the GPF Wiki would be easy to submit and index. It was built using MediaWiki 1.12.0; I am uncertain what database changes may be needed for older or newer versions. Here’s my current process:

  I only want to index relevant pages, including category pages. The relevant database table for this is “page”. (How… convenient). Unfortunately, this table also contains things like redirects and images. Each image has its own “page” assigned to it; try clicking on an image in Wikipedia or in the GPF Wiki to see what I mean. The time stamp of the latest revision, however, is stored in the “revision” table, joined to the page table by the latest revision ID number. So a good starting bit of SQL would be:

  select p.page_title, r.rev_timestamp from page p, revision r where p.page_latest = r.rev_id and p.page_is_redirect = 0 and p.page_title not like '%.gif' and p.page_title not like '%.png' and p.page_title not like '%.jpg';

  Unfortunately, this also returns a few meta pages like the sidebar and editing pages. Before selecting, I define a look-up hash of titles I want to avoid and as I loop through the results I just skip those.

  The title, of course, is both the displayed title and the input portion of the URL that uniquely identifies the page. Thus, knowing the base URL (http://www.gpf-comics.com/wiki/) I can easily reconstruct the public URL of any article from the title. As with Wikipedia links, spaces have already been converted to underscores, but the rest of the string needs to be be URL encoded. This is easy enough, so we can quickly build the full URL as required by the XML schema.

  The time stamp is a little bit tougher. MediaWiki stores time stamps as a 14-digit number in YYYYMMDDHHMMSS format, always in UTC time. In Perl (in which almost all my crons are coded) this is easy enough to break apart and turn into a UNIX time stamp. I then output the date in W3C ISO 8601 format as required by the schema. A sample of a resulting entry would be:

  <url>
     <loc>http://www.gpf-comics.com/wiki/Nick</loc>
     <lastmod>2008-08-22T06:00:07Z</lastmod>
     <changefreq>monthly</changefreq>
     <priority>0.3</priority>
  </url>

  Change frequency and priority are purely guesses and fudges for mine. According to the sitemap specification, priorities are purely relative to other parts of the site. I rated the wiki pages as relatively low since the wiki at GPF is considered a “supporting” page and subordinate to things like the archive. As for change frequency, the sitemap specification includes a number of predefined choices (hourly, daily, weekly, monthly, etc.). Monthly was a purely off-the-cuff guess; some pages may update more or less frequently, but monthly would be a good average. It is entirely possible to rate select pages as higher priority or frequency than others, but I decided to take the easy route and rate everything the same. To apply different values, you just need to pay special attention to the title and assign a non-default value when that title crops up.

  Well, I hope someone out there might find this helpful. I’m not sure if it really helps anyone find anything at GPF, but it was a fun little exercise nonetheless.

  

• Software, Technology

  WinHasher 1.3

  June 30th, 2008 (1 year, 8 months ago) by Jeff | Permalink | Dump Core

  For both of you out there who care, WinHasher has now been bumped to version 1.3. The changes are very minor, so there’s no need to upgrade unless you find the following two new features useful:

  • There’s now a drop-down box to choose between hexadecimal output and Base64 (RFC 2045, although without line wrapping). I personally like using Base64 because it’s more compact than hex and more “obscure-looking”; even non-coders are familiar with hex, seeing it in error messages all the time, but are less familiar with Base64, which looks more like “random noise”. The default output is still hex, so you only need to switch modes if you really find Base64 more useful.
  • There’s now a tab in the GUI to hash arbitrary text. I added this feature because there was an extension to Firefox that I used all the time to do this—which I used frequently to generate “random-looking” passwords by hashing simpler plain text and using the hash as the “real” password—but the extension hasn’t been updated in forever and is no longer working in Firefox 3. So to keep myself from getting locked out of the accounts where I used hashed passwords, I threw this feature in WinHasher. The backbone was already there, so all that needed to be added was a GUI.  There’s also an encoding drop-down box that lets you change the text encoding if you so desire/need. To be honest, I don’t think it’s really necessary because I think .NET uses Unicode internally all the time, but it does have a nice side-effect: if you enter text in one encoding and change the encoding setting to something radically different (say you enter Western European Windows text but change the encoding to IBM EBCDIC Japanese), you’ll end up with a very different hash than you would have had if you used the “right” encoding. This isn’t encryption by any means as it could be easily reverse engineered if you knew what encodings were used, but it does significantly scramble things to the point that it makes it much harder to figure out exactly what you did.

  I had originally started adding support for HMAC signed hashes but have abandoned that for now. If there’s anyone out there who might actually find that useful, drop me a line and I’ll revisit the code to see what I might be able to add. Downloads can be found at the first link above.

  

• GPF, Internet, Software, Technology

  Praise for XCache

  March 18th, 2008 (1 year, 11 months ago) by Jeff | Permalink | Dump Core

  The new GPF site has been running live for half a month now, and I’m proud to say things have been running incredibly smoothly. That is, at least, from my perspective; I haven’t seen any major glitches, and aside from a few typos in the comic (which are obviously independent of the site code), nobody has written me about any problems. This is especially heartening because the new site was pretty much entirely coded by hand by me, sans a few bits and pieces. (I can’t take credit for the OS, the web server software, the database engine, or the forum. But everything else… yep, that was me.)

  There were a lot of motivations for writing my own archiving system, but the primary one was efficiency. While I considered trying something off-the-shelf, so to speak, like ComicPress or Drupal, I really wanted something that would be blazingly fast yet still dynamically generated to let me do things like GPF Premium on the server side, primarily for security reasons. (Server-side processing means no messy JavaScript is required by the users, thus exposing them to less risks, while Premium content doesn’t even get sent to the browser at all if Premium isn’t enabled.) So the GPF site is optimized out the wahzoo, with certain high-volume pages built once by nightly crons while others that require more interactivity reduce database queries to simple selects as much as possible. I’m never one to brag and toot my own horn, but I’m actually pretty proud of the new site and how responsive it is.

  Of course, I can’t really take all the credit. I do have to give some serious props to XCache.

  For those unfamiliar with PHP, it is one of many server-side, interpreted scripting languages commonly used for dynamic Web site development. The caveat, however, to any interpreted language is that on each request the source script must be read, parsed, compiled, and executed before anything is set back to the end user’s browser. This is one reason why dynamic sites are and will always be slower than serving purely static HTML files. Static HTML just needs to be read and regurgitated; anything that requires the Web server to actually think takes more time. Add to that the fact that there could be hundreds or even thousands of requests all competing at once for content and it’s a miracle anything get served at all.

  XCache is one of several opcode caching extensions for PHP. Essentially, when the first request for a script is made, the script is parsed and compiled as usual. However, XCache stores the compiled code so subsequent requests can skip the parsing and compilation steps and go directly to executing the code. This significantly increases the speed of execution by eliminating one of the costliest parts of the process (except perhaps database connections). In addition, XCache also includes the ability to cache variables and objects, so commonly repeated and expensive variable generation–such as the cryptographic hashes I use for salting cookie hashes or database look-ups for common elements like the Premium subscription levels–can be stored in the cache rather rebuilt on each request.

  I was first introduced to XCache by the XCache for WordPress plugin, which was probably mentioned in one of the development feeds built into the WordPress dashboard. I’ve been running this combination here on the blog for a little while with moderate success; I’m still trying to find a good balance of configuration settings to get the best results, but I’ve been happy with the results so far. Without putting much thought into it, I went ahead and installed XCache on the GPF server, hoping that it would help even if I never got a chance to optimize it. Fortunately, it has helped, and now that I’ve optimized the settings it’s exceeded most of my expectations. I’m not sure if there’s something about my code that caches better than WordPress, but GPF has done much better with XCache than the blog has.

  Admittedly, I haven’t compared it to any other opcode cachers, nor have I benchmarked it against any of the competition. That said, however, I heartily recommend it to anybody running PHP applications. To get the greatest benefit, you may need to modify some code (or install a plugin if you’re using a prepackaged application) to take advantage of the variable/object caching. But even without modification the opcode caching alone makes for a vast improvement.

  

• Software, Technology

  Vista first impressions

  December 10th, 2007 (2 years, 3 months ago) by Jeff | Permalink | 7 Core Dumps

  I had my first brush with Microsoft Windows Vista this weekend. Like most hard-core geeks who are skeptical of just about anything Microsoft, I’ve read all the hype and negative press and have thus avoided it like the plague. I recently bought a new tablet PC (which just arrived today, woohoo!) and made sure to “downgrade” it to Windows XP. But this weekend as I was performing a Good Samaritan deed I was inadvertently forced to directly interact with Microsoft’s latest and “greatest” OS. And while there’s probably nothing new in this post to anyone who’s used Vista already, I’m sad to report most of what I’d heard and feared are true.

  First, a little background. This past week, my sister-in-law’s notebook died. Exactly what happened is still uncertain; we know for certain that the video subsystem is on the fritz, which likely means that something is up with the motherboard (since the video is on-board). The LCD occasionally looks like a black light lava lamp, if that makes any sense, although I was surprised to have it actually work off and on with any given reboot. Windows XP crashes on boot on the NVIDIA video driver, which might (or might not) be consistent with a video hardware problem. Throw into the mix the fact that the system spontaneously reboots or locks up after a indeterminable period of time, sometimes as long as several hours or as short as ten minutes. I pulled out ever trick and tool in my geek arsenal and haven’t been able to completely diagnose the problem, let alone fix it. So now the task has become one of data recovery, and with a creative combination of a Knoppix “live” CD, a USB flash drive, and a USB external hard drive this has gone off without much of a hitch.

  Now we introduce the new machine. Like its predecessor, it’s an HP Pavilion “media center” notebook. I put “media center” in quotes because while the old machine actually ran Windows XP Media Center Edition, the new machine runs Vista Home Premium. Other than the OS, it’s obvious both machines are built for one thing: to be a portable home theater system. Both have massive widescreen LCDs, dual huge hard drives, several gigs of RAM, and the latest processors for their time. Needless to say, both machines are meant to be powerful multimedia workhorses and they have the muscle to prove it. Thus, there’s no reason to expect the new machine to be sluggish or slow.

  And yet, it occasionally was. HP, like many manufacturers, loads its new machines with tons of useless garbage software. That said, I was surprised to see how little junk was really pre-installed on this thing. So the only thing I can think of that was really bogging it down was Vista itself. I can’t be 100% certain of this as I didn’t take the time to really investigate (most of my time was spent extracting data from the old machine), but there were plenty of times Vista seemed to drag and stutter, sometimes becoming unresponsive for a few seconds.

  The culprit, I expect, is the new “Aero” interface. Sure, it looks pretty. I’ll give it that. Compared to XP’s default Crayola-inspired interface (which is one of the first settings I turn off on a new XP machine), it looks slick and modern. But it also seems bulky and bloated. The moment I turned it off and went back to the Windows 95-ish “classic” interface the machine become much more responsive and easier to use. While it was cute watching windows “pop” into existence (something that will probably smell suspiciously like copyright infringement to any Mac OS X user) and the translucent window borders are a nice aesthetic trick, the performance cost is pretty high and not really worth it.

  Then there’s the security model. I’d like to applaud Microsoft for finally taking security seriously and making a concerted effort to be responsible with its market dominance by forcing users to be more secure. But boy howdy, is it a bear to work with. Apple has been running attack ads against Vista in their “PC vs. Mac” campaign where the nerdy PC character has to ask permission from a Secret Service inspired man-in-black for every single thing he does. I thought that was funny at the time, but I didn’t really realize how true it was. Having now been trained and used to doing things as an administrator in XP, it’s a real shock to be stopped at every other mouse click with a warning that what I’m about to do has serious security implications. It’s not just a pop-up box, either; the entire screen flashes, dimming everything else and forcing to acknowledge the pop-up before you can continue. Yes, I’m aware of the serious security implications; I’m stepping outside the box and doing advanced things outside what a normal user is likely to do. (For example, moving the contents of the old machine’s “Application Data” and “Local Settings” folders, normally hidden, to their new home.) But do you have to warn me every single blasted time? Really. What’s worse is that this extends beyond some of the obscure, funky guru work I’m currently doing. Simple configuration changes are challenged with the same severity as drastic, devastating, and potentially damaging attacks. Where 95/98 was blatantly promiscuous (or more properly naive) and XP (post SP2) was cautious, Vista is downright paranoid. I half expect it to call in the FBI and the National Guard every time I change my wireless SSID.

  Maybe there’s someone out there who can help. If you have experience with Vista and you know how to turn these security pop-up off, just for my login, at least until I’m done doing arcane geek magic to finish restoring this machine, please let me know. I think I’d be done in a fraction of the time if I didn’t have to babysit these prompts all the time. Even if it’s a setting that lets me check a box that says “don’t show this again” so I only get it once per action will be a big help.

  After all that complaining, let me mention one thing I did like about Vista: parental controls. As a parent who is faced with a future where my young son will be a few clicks away from all the porn and identity theft of the Internet, I’ve been looking hard at third-party (as well as home grown) filtering and monitoring solutions. Vista apparently has this built in. Unfortunately, I have no idea how effective it is. My guess is that workarounds to bypass it are now just a Google search away. But still, just like XP’s firewall is more of an afterthought than a real security measure, it’s got to be better than nothing, and it will probably be easier to train my non-tech-savvy sister-in-law in how to use it than to explain about proxies and packet filtering. Depending on how long this machine is in my possession, I might try to experiment and see just how effective these parental controls really are.

  Again, nothing necessarily new here that you probably haven’t seen everywhere else, but I thought I’d share my experiences to anyone interested in listening. I’m leaning more and more toward ditching Microsoft completely and going with a completely FLOSS setup, and Vista is helping push me in that direction. Then again, I had huge reservations about XP when it came out, too, so who knows what the future will bring?

  

• Software, Technology

  WinHasher 1.2

  November 30th, 2007 (2 years, 3 months ago) by Jeff | Permalink | Dump Core

  I just can’t leave well enough alone. I’ve been mildly annoyed with the “hash in progress” and progress dialogs in WinHasher 1.1. The original idea was to use System.ComponentModel.BackgroundWorker to easily multi-thread very large hashes (say of CD or DVD ISOs or uncompressed video files). This had two benefits: (1) it allows the user to cancel a hash in progress and (2) gives us an opportunity to update the GUI while the hashing takes place in the background, meaning we can inform the user of the progress. Unfortunately, I couldn’t find a method right away to determine the progress of an individual hash. System.Security.Cryptography.HashAlgorithm.ComputeHash() by default takes a byte array or file stream and chugs the whole thing at once, spitting out the hash as a result. There’s no way with this method to determine how far along you are.

  However, if you look at the guts of ComputeHash(), you’ll find it reads in chunks of bytes into a buffer, then calls two methods: TransformBlock() for every chunk but the last, and TransformFinalBlock() to hash the last chunk and finalize the hash. The result can then be obtained from the HashAlgorithm.Hash property. If we bypass the convenience of the single ComputeHash() method call, you can read chunks of bytes from the buffer, feed it to the Transform...() methods, and keep track of how many bytes have been read so far. Since we already know how big the file is from the start (System.IO.FileStream.Length), it’s trivial to calculate a percentage complete. Want the progress of a multi-file comparison? Sum the lengths of all files in the batch, then keep track of the total number of bytes hashed along the way.

  I’ve bumped WinHasher to version 1.2. It should be available on the official site by tomorrow morning.

  

• Software, Technology

  WinHasher 1.1

  November 9th, 2007 (2 years, 4 months ago) by Jeff | Permalink | Dump Core

  I was downloading Fedora 8 yesterday with every intention of (eventually) upgrading Demeter, our Linux box, currently running Fedora Core 5. I say “eventually” because finding time to back everything up, install the new OS, restore all the files and third-party apps, and then spend hours debugging what went wrong is the hardest part of the whole process.

  Anyhoo, I thought I’d put good ol’ WinHasher to use to verify the SHA-1 hash and make sure the download was successful. That was, after all, the primary reason I wrote the thing. But then I noticed something incredibly annoying. The DVD ISO for Fedora 8 weighs in at around 3.18GB. WinHasher has no problems hashing such a file as it hashes the file stream as it reads it, so it can hash any file of arbitrary size. But while it’s doing so, there’s absolutely no feedback to the user. In fact, if you use the right-click “Send To” shortcut, unless you go into the Windows Task Manager and check the process list, you won’t even know the thing is running. It seems to go off into la-la land until mysteriously, a minute or two later, it comes back with a mysterious pop-up box with a cryptographic hash in it.

  Bad programmer! Bad!

  Well, this annoyed me enough to fix it, so I’ve tweaked it now so it actually runs multi-threaded. This is ridiculously easy to do when you use the System.ComponentModel.BackgroundWorker object in .NET. You just create the object, create a few event handlers for it, and tell it to go do its thing. The hardest part is if you want it to report its progress; then you have to tweak your working code to generate a percentage complete so it has something to display. I learned this trick when I was working on Mandelbrot Madness! 2.0; I have no idea why it never occurred to me to incorporate it here. The annoying part about this addition, though, is that there’s no way to report on the progress of an individual hash. System.Security.Cryptography.HashAlgorithm doesn’t provide a way for you to know how far it’s gone, so there’s nothing to feed to BackgroundWorker.ReportProgress(). The comparison feature does, however, let you know how many of the files have been processed so far in the batch.

  So if you’re one of the few brave souls who downloaded WinHasher 1.0, please grab 1.1 at the official site. It officially goes up tomorrow (a delay caused by Keenspot’s parsing to insert the advertising codes). If you haven’t already downloaded it, why not give it a try?

  

• Software, Technology

  WinHasher 1.0 and Mandelbrot Madness! 2.0

  September 26th, 2007 (2 years, 5 months ago) by Jeff | Permalink | Dump Core

  I’m not sure if anyone cares, but I’ve been doing a tiny bit of dabbling in releasing Open Source software lately. Since I don’t particularly care to announce them on the GPF News (it isn’t, after all, GPF news), I’ll announce them here. For those of you who might complain that working on these has taken precious time away from the comic, fret not. The tiny bit of time I’ve been able to squeeze in here and there to work on these have been during periods when working on the comic would be impossible, so there’s no way for there to be any conflict.

  The first one I’ll announce is the most recent. WinHasher is a Microsoft .NET Framework 2.0 application for generating cryptographic hashes of files. It is both a Windows GUI applet and a console (command line) program, and it operates in two possible modes. The first mode generates the hash of a single file, which you can then use to verify a download or check to see if a file has been tampered with. The second mode takes the hashes of multiple files and compares them; in this way, you can see if two or more files have the same binary contents regardless of their names, locations, and time stamps. The Windows app supports drag-and-drop functionality, and the installer lets you also build shortcuts in your Windows Explorer “Send To” context menu so you can just right-click a file and get its hash.

  So why did I build this? Well, the full details are on the site, but the quick version is that I’ve grown tired of not being able to validate the hashes of downloaded files because Windows doesn’t have a built-in hashing program. Linux and the other free UNIX clones have OpenSSL; heck, even Mac OS has OpenSSL under the hood. Not Windows… of course. So instead of downloading a file on a Windows machine, copying it to the Linux box, validating the hash, and moving it back (or worse, just not even validating the hash at all and taking my chances), I hacked together this little program. Then I thought it might be useful enough to share, so I did. If you find it useful, please let me know.

  For the really technically inclined out there, most of the hashes are built-in to .NET 2.0, so this was obscenely easy to implement. In fact, 2.0 has an abstract hash algorithm class (System.Security.Cryptography.HashAlgorithm) that all of the built-in hashes implement. The two non-standard hashes, Whirlpool and Tiger, were taken from the Legion of the Bouncy Castle Crypto API, which is actually .NET 1.1 based. I (rather crudely) ported these classes to be subclasses of HashAlgorithm, so they can technically be used as a drop-in replacement of any .NET 2.0 hash. I plan to add additional hashes over time, provided that (a) the original source code is free and (b) I can port it to be a HashAlgorithm subclass.

  The second program to mention is actually a bit of an oldie now. (I actually released it back in June.) The Windows version of Mandelbrot Madness! is back, also in a .NET 2.0 edition. I wrote the original in Visual C++ 4.x, but have long since lost the source code. Then the Java version came into being and rapidly surpassed the Windows version, leaving it to suffer from bit rot. Both eventually languished as I lost time to work on them. Well, in December of 2005, I released the not-quite-complete-but-close-enough 4.0 version of Mandelbrot Madness JAVA!, declaring it abandoned. It always bugged me that I never went back and revisited the Visual C++ code, but without the source I was stuck.

  Well, to make a long story short, my new job had me learning a new programming language: C#. That had me programing Web sites, but I knew you could also do Windows GUI apps in it as well. So somewhere down the line I got a wild hair and started the agonizing work of porting the Java code from the last version of MMJ! to C#. Actually, Java and C# are similar enough that the porting work wasn’t all that hard. Not only is MM! 2.0 now pretty much identical in functionality to MMJ! 4.0, but I introduced a number of new features that I hope to eventually port back to the Java version. While I still prefer the platform independence of Java, I’ll readily admit that the .NET version is a lot faster on Windows. I think that anyone on that platform that has actually bothered to play with the Java version (both of you) should make the switch. Anyone still using the decrepit old 1.0 version of Win32 MM! should enter the 21st century and upgrade too.

  Both programs have been released under version 2 of the GPL (haven’t had time to really review version 3 yet), so the sources are also available. If you have any suggested changes, feel free to pass them along and I might incorporate them into the official builds (giving you credit, of course).

  

« Previous Entries

About

Jeffrey T. Darlington is the creator of the popular online comic strip General Protection Fault. He is also the creator of a number of other inconsequential things, but seeing as they are inconsequential, he didn't think you should be bothered by mentioning them. More...

Search

Capitalism

Recent

• Weekly Twitter Updates for 2010-03-06
• Weekly Twitter Updates for 2010-02-27
• Weekly Twitter Updates for 2010-02-20
• Weekly Twitter Updates for 2010-02-13
• WinHasher 1.6

Tag Cloud

Twitter GPF Writing Recycle Bin Security humor Technology Software TWiT WinHasher Webcomics .NET Linux Star Wars server meta podcast WordPress Microsoft cookie Steve Gibson Security Now virtual machine Wii Fedora mod_rewrite Doctor Who Windows Apache schedule blog advertising C# Star Trek PHP Help Desk Christmas authentication hard drive DoubleClick comments anniversary gifts To Thine Own Self PayPal